GPT-6 Astra Aced 19 of 20 Robot Trials. A Puzzle Piece Beat It.

TL;DR

On September 4, independent robotics evaluator Robocurve published GPT-6 Astra on robotic manipulation, a follow-up to its September 3 report on Claude Fable 5 and Fable 5.1. Same arms, same harness, same two tasks, 20 trials per model per task. On "put the red block in the bowl," GPT-6 Astra finished 19 of 20 runs against Fable 5.1's 8 and Fable 5's 1, in 2.5 minutes a run instead of 6.8, at an estimated $0.94 instead of $2.12. Then the second task, seating a puzzle piece in a matching groove, flattened everyone: Astra 2 of 20, Fable 5.1 2 of 20, Fable 5 zero. Every trial ships with video, a transcript and a downloadable log, which is why this is worth your time and most robot demos are not.

The setup, and why it is not a demo reel

The rig is a bimanual pair of I2RT YAM arms, 6 degrees of freedom each, with parallel-jaw grippers. The list price on a single YAM 6-DoF is $2,999, which puts this in reach of a university lab rather than a national one.

The model does not output joint angles. Each turn it receives three camera views (top, left wrist, right wrist) plus proprioceptive state, and replies with an absolute end-effector pose: x, y, z, yaw, pitch, roll and a gripper value, per arm. The robot's inverse kinematics turns that into joint angles. The agent gets a budget of 20 model calls, medium thinking effort, a 25% speed cap and the default safety guardrails.

Picture talking someone through parallel parking over the phone. They send you three photos, you shout one steering correction, they send three more photos. You get twenty shouts. That is the control loop, and the surprising part is that it works at all.

The harness is Inspect Robots 0.58.0, MIT-licensed, which Robocurve pitches as Inspect AI for robots: run any model on any embodiment against any benchmark, with full trace logs and Rerun visualization. Robocurve is a Public Benefit Corporation whose stated job is measuring physical AI in the open rather than shipping a robot, which is a materially different set of incentives from the labs whose models it grades.

The bowl task was a rout

Every trial was graded by a human on the furthest stage it reached, so a failure still records how far it got: 0 no purposeful approach, 1 contact, 2 lifted clear of the table, 3 positioned above the deposit point, 4 placed.

Mean stage tells the same story: Astra 3.95, Fable 5.1 2.40, Fable 5 1.30. The Fable 5 detail is the bleak one. In 13 of its 20 bowl trials it reached the block and then never closed the gripper, which is an expensive way to point at something.

Then the puzzle piece happened

Task two: pick up a round blue puzzle piece by the knob at its center and seat it in the matching circular groove. Astra completed it 2 times in 20. Fable 5.1 also completed it 2 times in 20. Fable 5 never did.

Note the wrinkle: on the puzzle task Astra's mean stage was 2.00, lower than Fable 5.1's 2.35, despite matching it on completions. Astra either got the piece in or bailed early. Fable 5.1 crawled further up the ladder more often and still could not finish. Two models can tie on the headline number and be failing in completely different ways, which is an argument for reading the stage histogram rather than the leaderboard.

The token collapse is the other headline

Robocurve priced all three models identically, at list rates of $10 and $50 per million input and output tokens, so the cost gap here is a token gap, not a discount. On the bowl task Astra produced about 2.1k output tokens per run against Fable 5.1's 12.9k and Fable 5's 19.2k. It also finished in fewer turns, which means fewer image payloads on the way in.

That is the number a builder should care about more than the success rate. A policy that deliberates for 19,000 tokens and then declines to close the gripper is not merely slow, it is paying premium rates to be indecisive.

Read the limitations before you quote any of this

Robocurve publishes its caveats, and they are not decorative:

• Not interleaved. Astra's trials ran two days after the Fable trials.
• Different rig on the bowl task. The puzzle comparison is same-rig. The bowl comparison is not: the Fable bowl runs used rig-3, which was unavailable when Astra ran, so Astra used rig-1. That is the exact task with the biggest reported gap.
• Unblinded grading. A human operator scored every trial knowing which model produced it.
• Caching asymmetry. Anthropic requests went out without prompt caching. OpenAI automatically cached roughly a fifth of Astra's input, and that discount is not applied, so Astra's stated cost is if anything too high.
• One effort setting. Everything ran at medium reasoning effort, and objects were reset by hand between trials.

The bowl result is large enough (19 versus 8 out of 20) that a rig swap is unlikely to be the whole story, but "unlikely to be the whole story" is not the same as controlled. Treat the bowl gap as strongly indicative and the puzzle result, which is same-rig, as the cleaner comparison.

What this actually means for you

Three things worth taking away.

The generalist is now a plausible robot policy for coarse tasks. No robot-specific pretraining, no VLA, no teleoperation dataset: a general model with three cameras and an inverse kinematics solver put a block in a bowl 19 times out of 20. If your application is "move an identifiable object from roughly here to roughly there," the off-the-shelf model is a real baseline now, not a joke.

Precision remains the wall, and it is a different wall. Seating a piece in a matching groove is a contact-rich, sub-centimeter, force-feedback problem. Discrete pose commands over vision, twenty of them, are the wrong instrument for it, and no amount of model scaling in this harness fixed that. The 10% completion rate is the honest state of the art for an LLM driving this class of arm.

Independent, reproducible robot evals barely exist, and that is the actual news. The robotics field runs on cherry-picked demo videos. Here you get 120 graded trials, a published rubric, per-trial video and transcripts, downloadable logs, and an MIT-licensed harness you can point at your own arm. Whatever you make of the scores, that is the format every one of these announcements should ship in.

Key Takeaways

• GPT-6 Astra completed the block-into-bowl task 19 of 20 times, against Fable 5.1's 8 and Fable 5's 1, on the same YAM arms and the same Inspect Robots harness.
• On the precise puzzle-into-groove insertion, Astra and Fable 5.1 tied at 2 of 20 and Fable 5 scored zero. Coarse manipulation is close, fine manipulation is not.
• Astra used about 2.1k output tokens per bowl run against Fable 5.1's 12.9k, and finished in 2.5 minutes versus 6.8, at $0.94 versus $2.12 with identical list pricing applied to both.
• Astra's puzzle mean stage (2.00) was lower than Fable 5.1's (2.35) despite equal completions, so identical headline scores hid different failure shapes.
• The bowl comparison was run on different rigs two days apart with unblinded grading. The puzzle comparison was same-rig and is the cleaner number.
• The whole harness is MIT-licensed and every trial ships with video, transcript and logs, which is the reproducibility bar robot announcements should be held to.

Sources: Robocurve, GPT-6 Astra on robotic manipulation (Sept 4, 2026), Robocurve, Comparing Fable 5.1 to Fable 5 on robotic manipulation (Sept 3, 2026), Robocurve, Inspect Robots on GitHub, I2RT YAM 6-DoF arm, Hacker News discussion