← Back to all posts
News

Epoch AI Finds No Model Meets Its Employee Standard on Real Work, GPT-6.2 Astra at 65%

November 19, 2026 · 06:19 UTC · News
On this page 8 sections
Epoch AI Finds No Model Meets Its Employee Standard on Real Work, GPT-6.2 Astra at 65%

TL;DR

Epoch AI graded six frontier models on tasks pulled from its own operations, and none of them reached the standard it expects of its own employees on a single task. In an updated run of Epoch Automation Reports published 17 November, GPT-6.2 Astra and Anthropic's Claude Fable 5.2 sit roughly tied at a record 174 on the Epoch Capabilities Index. On the actual work, Astra averaged 65 percent, 17 points behind Fable 5.2. One task is close to finished: Fable 5.2 and Opus 5.5 now do Epoch's AI data center research at contractor quality, and Opus 5.5 does it in under three hours where a contractor needs sixteen.


At a glance
Record on the index, middling at the desk
6
frontier models graded on real work tasks
0
tasks where any model met the employee bar
65%
GPT-6.2 Astra average, 17 points behind Fable 5.2
174
shared ECI record for the top two models
97.5%
Fable 5.2 on data center research, contractor level
<3 h
Opus 5.5 per data center entry, against 16 h by a contractor

What Epoch actually graded

Most benchmarks hand a model a sealed task with a checkable answer. Automation Reports is built from Epoch's real workload: adding an entry to its AI data centers database, writing a Data Insight for its audience, designing a research study a candidate could finish in days. Researchers graded every output by hand against internal standards, scoring each criterion from 1 to 4 the way conference reviewers score papers. A 100 means Epoch could publish the output as an employee wrote it.

This run covers six models: Fable 5.2, GPT-6.2 Astra, Opus 5.5, Gemini 4 Argon, plus Fable 5.1 and GPT-6 Astra re-evaluated on the new setup. The index and the workday disagree at the top. Astra and Fable 5.2 share the record at 174, on Epoch's own pre-launch testing, and sit 17 points apart on the work. Two sprinters with identical 100-meter times can finish an hour apart on a mountain trail; the ECI is the track, and this suite is the trail.

One task is already contractor-grade

The data center task asks a model to add EcoDataCenter 1 to Epoch's database, end to end.

The nearly automated task
Adding a data center, end to end
Research
News and filings
find the facility's paper trail
Order
Satellite imagery
the model places the order itself
Annotate
Buildings and plant
outline buildings, cooling, generators
Estimate
Compute and chips
counts from the evidence gathered
Result
Database-ready entry
contractor level, from a 16-hour job
Opus 5.5 finished this in under 3 hours at lower cost than a contractor. Source: Epoch AI.

Fable 5.2 scored 97.5, with imperfect image annotations and overly verbose text as its only flaws. Opus 5.5 reached 87.5 and needed fixes to equipment classification and image annotations. Epoch's data center analyst judged both comparable to the contractors the company hires, whose work also typically needs revision before publication.

AI data center research
Two models at contractor level, one overly careful
modelsfable 5.2
fable 5.2
97.5%
opus 5.5
87.5%
gpt-6.2 astra
65%
Manual grading against Epoch's employee standard of 100. Source: Epoch AI, 17 Nov 2026.

The economics matter more than the scores. A contractor spends about 16 hours per data center. Opus 5.5 finished in just under three, at lower cost. Fable 5.2 took about 6.5 hours at a higher cost than a contractor and produced the better output of the two. Epoch says it is now considering moving contractors from producing this research to reviewing what models produce.

Time per data center entry
The economics of the one automatable task
contractoropus 5.5
contractor
16 h
fable 5.2
6.5 h
opus 5.5
<3 h
Opus 5.5 finished at lower cost than a contractor; Fable 5.2 cost more and scored higher. Source: Epoch AI.

GPT-6.2 Astra managed 65 on the same task, a 25-point jump over GPT-6 Astra, and how it failed is the instructive part. Its sources pointed to multiple AI buildings and it covered one. It left out compute and chip estimates it had enough information to make, because no separate source confirmed every number. The human reference output annotates six buildings on that campus; Astra annotated one. It had the receipts and declined to file them.

Where all six still fail

The research-design tasks knocked everything down. In "chess variants," a retired work test for candidates, the model must build a study that measures out-of-distribution generalization: chess with altered rules the evaluated models have never seen. Every model reused variants that already exist. Astra swapped piece labels, rook for knight, which changes the furniture and keeps the rules. Opus 5.5 and Fable 5.1 both introduced a "camel" piece, a well-known fairy chess piece. Fable 5.2 invented "Kingsgate," which plays exactly like an existing variant called Midline Invasion.

Gemini 4 Argon did the strangest thing on the same task: it ran a control showing the evaluated models already knew its chosen variants, watched the result undermine its own experiment, and carried on anyway.

"Hidden messages" exposed the other shared flaw. The task asks a model to study how well Fable 5.1 and GPT-6.2 Astra can hide a message in an SVG image, building on published work from Kezins et al. Doing it well means iterating on the image prompts, because first attempts hide the message badly. Fable 5.2 settled on its first working prompts, and most of its images displayed the message outright, one with the string written across a table. Human researchers prompting the same models got star-height encodings that concealed the text.

Presentation failed too. Opus 5.5 listed an obvious one, that messages drawn as letters are easy to decode, among its headline findings. Astra's report reads well but leads with a bolded takeaway the same report then contradicts. Gemini 4 Argon buried usable findings under coinages like "code-level syntactic hiding."

Models grade their own homework

Where Epoch holds human reference output that meets its standard, it asks models to compare that output against their own, blind, using Epoch's grading rubric. Every model preferred its own work in every comparison except one: Gemini 4 Argon picked the human's output on hidden messages, with reasoning Epoch says matches how its own team compared the two.

The justifications show what the bias feeds on. Models rewarded surface markers of rigor, chiefly the number of experiments run, and magnified minor flaws in the human output. A blind taste test where the judge keeps scoring the dish with more garnish. Run the same model as reviewer on its own drafts, or its family's, and you get this result without the blindfold.

The caveats, from Epoch itself

The task suite is small and each model ran once per task, because manual grading is slow. Grading is subjective, and Epoch says the qualitative findings matter more than the exact scores. The setup changed since the October report, so the two runs are not comparable. Since the agents browse the web and the evals are public, Epoch instructs them not to visit the published pages and verifies every run.

Why this matters if you build with AI

  • A tied index score hid a 17-point spread on open-ended work. If you pick models for judgment-heavy tasks by leaderboard position, run the shortlist on your own workload first; the ordering may not survive contact.
  • The nearly automated task has a shape: a written guide, tools, a condensed digest of the internal Slack channel, and a clear definition of done. Work that fits the pattern is closer to automatable than the benchmark headlines suggest. Work without a rubric is nowhere.
  • Over-caution is a failure mode. Astra handed in an incomplete entry rather than make a defensible estimate. If your agent needs calibrated assumptions, say so in the prompt, or it will call half a deliverable prudence.
  • None of the models iterate unprompted. A review step you do not build into the workflow does not happen; first-attempt output is what ships.
  • Keep the grader out of the graded. Blind rubric comparison caught self-preference in every model tested, which is a good reason to use a second model or a human with a checklist as the reviewer.

Key Takeaways

  • Epoch AI's updated Automation Reports graded six frontier models on tasks from its own work; none met the employee standard, set at 100, on any task.
  • GPT-6.2 Astra averaged 65 percent, 17 points behind Claude Fable 5.2, despite the two roughly tying the Epoch Capabilities Index record at 174, and improved over GPT-6 Astra to roughly Gemini 4 Argon's level overall.
  • On AI data center research, Fable 5.2 (97.5) and Opus 5.5 (87.5) reached contractor quality; Opus 5.5 finished in under 3 hours against a contractor's 16, at lower cost.
  • GPT-6.2 Astra scored 65 on that task by being overly cautious, covering one building where its sources showed several and omitting estimates it had the information to make.
  • All six models reused existing ideas instead of designing novel ones, settled for first attempts instead of iterating, and struggled to judge which findings matter and how to present them.
  • In blind comparisons with human reference output, every model preferred its own work except Gemini 4 Argon once, and the justifications rewarded surface rigor like experiment count.

Sources: Epoch AI, How good are Fable 5.2 and GPT-6.2 Astra at real work?, Epoch Automation Reports, Epoch AI on X (ECI 174), Pivot News, AI Brief, Epoch AI, Can AI automate Epoch?

AIEpoch AIBenchmarksGPT-6.2 AstraClaude Fable 5.2Epoch Automation ReportsEvals
CONSOLE
$