METR Finds Anthropic's Model Progress Runs Twice as Fast, Thanks to Its Own AI

TL;DR

METR, the evaluator that keeps score on frontier agents, spent 11 weeks inside Anthropic and published its findings on November 5. Benchmark progress on Anthropic models has been running 1.9x faster since October 2025 (95% CI 1.5x to 2.8x), which works out to roughly two years of the old pace inside one calendar year. METR's best guess attributes perhaps 80% of the current speedup to Anthropic researchers using Anthropic's own models, and it is confident the cause is not faster growth in training compute. One condition attaches to everything: Anthropic redacted a majority of the evidence METR found most informative, so you are leaning on METR's synthesis more than usual, and METR says exactly that in its own preamble.

What METR got to see

The engagement ran from August 12 to October 29, stretched from a planned 4 weeks to 11, staffed at 2.5 full-time equivalents and authored by Tom Cunningham, Nate Rush, Parker Whitfill and Hjalmar Wijk. They received internal benchmark scores, pretraining and RL metrics, training-compute data, lines-of-code counts, agent inference spend, internal surveys, and about 15 interviews. No fees moved in either direction; METR worked on Anthropic-supplied machines and used free Anthropic tokens, mostly for analysis. One team member had declined an Anthropic job offer during the project, so an unconflicted colleague re-checked that person's work.

What METR never got was query access to unreleased models. The assessment is document and interview forensics.

The trend, in points per year

The spine of the report is AECI, Anthropic's internal version of the Epoch Capabilities Index, computed with Epoch's public code. METR fit the trend with a single breakpoint, estimated at October 7, 2025. Before it, Anthropic models gained about 15 AECI points a year. After it, about 29. Same models, same yardstick, double the slope.

Acceleration here means the slope, not the score: the odometer did not double, the speed at which it spins did. In model terms, Anthropic made as much benchmark progress in the past year as it made between October 2023 and October 2025, roughly Claude 2.1 to Sonnet 4.5.

The number moves around under stress. Swap the statistical link function and estimates range from 1.1x to 2.9x; swap which snapshots, benchmarks or regularizer you use and they sit tight at 1.9x to 2.2x. Across all frontier labs together the slope is up 1.4x, from 17 to 24 points a year. Anthropic's own Haiku 5.5 system card had reported 1.7x; METR's higher figure comes mostly from counting internal-only models the system card never saw.

Who gets the credit

METR ruled out the boring explanation first: it received convincing evidence that the acceleration is not an acceleration in training-compute scaling. The details behind that sentence are redacted. Its best guess attributes perhaps 80% of the current acceleration to AI usage, with the late-2025 part of the bump likely helped along by non-AI factors. Anthropic's own staff disagree with each other about the split.

The interviews are the readable part. METR had Claude code the interview notes by whether each researcher supported a greater-than-2x AI-driven speedup in their area: two clear yes, three leaning yes, four unclear, two flat no. The yes camp is quotable. One researcher: "I would definitely endorse >2 years of progress in 1 year and I don't think 4 is crazy." Another: "It is hard to see how I am not being 3x more productive. I feel like I have postdocs and grad students." A third pins the mechanism: "Now only bottlenecked on ideas & compute, no longer bottlenecked on code."

Then there are the Slack polls, where staff vote with emoji reactions on how much the newest internal model speeds them up. Average self-reported uplift was about 1.2x in January 2025, the Claude 3.7 Sonnet era, and about 12x by October 2026. METR flags these as very loose evidence: low-effort votes from an unrepresentative slice of staff, and self-reports tend to overshoot measured productivity. Read the 12x as a vibe with a chart attached.

For anyone tracking autonomy, the report draws a boundary: the acceleration runs primarily through AI assisting researchers, and fully autonomous contributions are not yet doing heavy lifting in the most important parts of training. Researchers described "signs of life" that autonomous contributions could arrive soon. Today's 2x is leverage on humans, not a pipeline that runs itself.

Model 4 and the end of the yardstick

The report names the internal-only models it counted: Model 2 finished training in July 2026, Model 3 in August or September, Model 4 in September or October. On all 15 benchmarks Model 4 was tested on, it scores above 50%. Nine months earlier, Opus 4.6 still had plenty of benchmarks where it scored below half. METR says this leaves it "somewhat nervous" about measuring progress this way at all, now and going forward: when the best internal model clears everything, the index stops discriminating.

Two details widen the picture. Anthropic spends real inference money on its own evals, about 2.5M input and 72k output tokens per task on internal benchmarks and about 10M input on agentic coding ones, where Epoch's public runs use roughly 0.5M input per task. And METR dated models by when most Anthropic employees could use them rather than public release, which shows the internal frontier running slightly ahead of anything you can call.

The part you cannot read

Anthropic held unilateral redaction rights and used them on what METR calls a majority of the evidence most informative for its conclusions: algorithmic-progress estimates, the detailed attribution numbers, the bottleneck analysis, forecasts of future models. Anthropic described the redactions as "aimed at removing confidential information that could accelerate other AI developers." METR kept the right to disclose that the redactions happened and that its findings depend on redacted material, which is what the first screen of the report does.

On the AI Evaluator Forum's minimum-standards checklist, METR marks the engagement as failing on independent review of its methods, impossible under the redactions, and on the item asking whether the provider lacked authority to redact concerning findings. It did not lack that authority. Separately, Anthropic cut a contracted piece verifying its compute split between capabilities and safety midway through, before METR reached any conclusions, saying it wanted the engagement focused on risk assessment for now.

A final disclosure: METR reports that Claude did the majority of its data analysis for this project, including classifying the interview quotes, and that METR checked the machine-classified results against the human interviews and found no systematic bias. The audit of AI-accelerated AI research was, in large part, assembled by the AI. The report says so itself.

Why a builder should care

• Model churn is the design constraint. A doubled slope with no identified 12-month bottleneck in compute, data, algorithms or experiment capacity means the model you integrated in Q1 is stale by Q3, on schedule. Keep the provider layer thin and your eval harness fast.
• Public leaderboards keep losing signal. When an internal model clears 50% on all 15 of its own benchmarks, the saturating public ones rank less and less. Your task suite, with your pass bar, is the only scoreboard that still discriminates between models for your work.
• Assistance did this, and assisted researchers are what the tooling sells today. Anthropic researchers got their 2x and 3x the same way you can: agents that turn an idea into a running experiment for near zero overhead. The quote "no longer bottlenecked on code" is the whole playbook.

Caveats

• One lab. The all-frontier slope is up 1.4x, and METR offers two readings: Anthropic's historical baseline was unusually low, or its current benchmark gains are partly catch-up.
• AECI measures well-specified, gradeable tasks. METR warns benchmark acceleration may not map to real-world impact, in either direction.
• The headline number carries a 1.5x to 2.8x interval, and on the subjective bottom line METR would not be surprised by 1.3x or 3x.
• Redactions mean the public cannot check the attribution or bottleneck claims. You are taking METR's word, and METR flags that plainly.

Key Takeaways

• METR's 11-week, high-access assessment, published November 5, finds Anthropic's benchmark progress up 1.9x since October 2025 (95% CI 1.5-2.8x): about two years of the old pace inside one year.
• Perhaps 80% of the current speedup is attributed to Anthropic's own AI tools; faster training-compute scaling was ruled out. Coded interviews split 5 for a greater-than-2x AI effect, 4 unclear, 2 against.
• Internal Model 4 scores above 50% on all 15 benchmarks used to measure it, where Opus 4.6 nine months earlier missed half on plenty; METR is nervous the index is saturating.
• Slack-poll self-reported uplift climbed from about 1.2x in January 2025 to about 12x in October 2026, which METR treats as very loose evidence.
• Anthropic redacted a majority of the most informative evidence and cut the capabilities-vs-safety compute verification midway; METR kept only the right to disclose that it happened.
• Practical read: model churn stays fast, headline leaderboards keep fading, and your own eval suite is the scoreboard that still works.

Sources: METR assessment, Unite.AI, Epoch Capabilities Index