AnchorHead · Regression
Atlas Support Agent
Tier-1 customer support · billing, refunds, account access
Source: Datadog trace export · postgres.agent_runs
Production model: Claude Sonnet 4.5
Best candidate · last 30 days of production traffic
Claude Sonnet 5 would have prevented 62% of your retries — an estimated $4,150/mo net.
203 correction/retry sequences detected across 1,284 runs, replayed on 4 candidate models and scored by an LLM judge. Savings combine 18% lower token spend (of $14,200/mo) with 18 avoided human escalations ($38 each, fully loaded).
Runs analyzed
1,284
updating live
Retry sequences detected
203
user corrected or re-asked the agent
Est. net savings
$4,150/mo
Claude Sonnet 5 · recommended
Regressions caught
68
across all candidate replays
Retry sequences per day
Production vs. the same trajectories replayed on candidate models
· click to show/hide
Cost vs. quality
Every candidate, plotted against your current model
Dashed line marks the Pareto frontier — nothing above-left of it exists. ★ recommended.
Regression inbox
New failures introduced by candidates, grouped by root cause
6 open
- Acted before querying3
- Premature deflection2
- Missing required evidence2
Candidate models
Every production trajectory replayed with identical context, per turn
| Model | Retries prevented | Improved | Regressed | Token cost | Median latency | Est. net $/mo |
|---|---|---|---|---|---|---|
Claude Sonnet 5 Anthropic | 62% | +126 | 7 | -18% | -9% | +$4,150 |
Claude Opus 5 Anthropic | 71% | +144 | 4 | +42% | +21% | −$980 |
Claude Haiku 4.5 Anthropic | 31% | +63 | 38 | -64% | -41% | +$6,920 |
GPT-5.1 OpenAI | 48% | +97 | 19 | -6% | +4% | +$2,140 |
Recent runs
Production trajectories with replay verdicts · click a run to inspect the trace
| Run | Task | Started | Outcome | Sonnet 5 | Opus 5 | Haiku 4.5 | GPT-5.1 |
|---|---|---|---|---|---|---|---|
| run_9f2c1a Brightline Health | Refund duplicate charge on annual plan | Sep 12, 2:22 PM | Corrected | Improved | Improved | Failed | Improved |
| run_4d81be Northwind Logistics | Restore admin access after SSO migration | Sep 12, 11:03 AM | Retried | Improved | Improved | Regressed | Failed |
| run_c07e55 Juniper Labs | Downgrade seats and prorate invoice | Sep 12, 8:47 AM | Success | Matched | Matched | Matched | Matched |
| run_1aa9d0 Fairview Capital | Export SOC 2 report for auditor | Sep 11, 10:15 PM | Success | Matched | Matched | Regressed | Matched |
| run_b3341f Osprey Robotics | Dispute chargeback on hardware order | Sep 11, 7:38 PM | Retried | Improved | Improved | Failed | Improved |
| run_77c2e8 Lakeshore Dental | Update payment method across 3 locations | Sep 11, 4:51 PM | Corrected | Improved | Improved | Improved | Failed |
| run_5e90aa Meridian Freight | Cancel account and export data | Sep 11, 1:29 PM | Failed | Improved | Improved | Failed | Failed |
| run_e2d47b Copper Canyon Coffee | Apply nonprofit discount retroactively | Sep 11, 9:12 AM | Success | Matched | Matched | Matched | Regressed |
Demo with mock data — live feed is simulated. All AnchorHead demos