The Same Coding Task Costs $5.54 or 32 Cents

TL;DR

The OpenHands Index drives every model it scores through the same agent harness on five suites of real software work, and publishes the column every other leaderboard leaves out: what each run cost. The spread is not subtle. Anthropic's Claude Fable 5 tops the board at 81.0 average and $5.54 per task. MiniMax M3 scores 57.2 at 32 cents. DeepSeek V4 Pro scores 40.7 at 15 cents.

If you run coding agents at any volume, the score column is the wrong column to shop in. Here is what the money column says instead.

One harness, five suites, and a bill

The index scores five categories, one benchmark each: Issue Resolution (SWE-bench), Frontend (SWE-bench Multimodal), Greenfield (Commit0), Testing (SWT-bench) and Information Gathering (GAIA). Every model runs inside the same agent, the OpenHands Software Agent SDK, rather than each vendor's own tool. It is the same agent, not the same build of it: the rows span SDK versions, which is a caveat worth holding on to and one we come back to below. Each row carries three numbers: score, mean cost per instance in dollars, and mean runtime.

That cost figure is the interesting one. A price per million tokens is a rate, and nobody can convert a rate into next month's bill without already knowing their own token volume. Buying inference by the token is like buying fuel by the litre with no idea what your car does to the gallon. Cost per task is the mileage. Read it as cost per task attempted, because the board divides by tasks run rather than tasks solved, so the failures are paid for too.

What the extra $5.22 buys

Fable 5 beats MiniMax M3 by 23.8 points, which is a real gap and not a rounding error: 81.0 against 57.2. It also costs 17 times as much per task. Per point of score, M3 is roughly twelve times cheaper, and it reaches 71% of the leader's result for 6% of the leader's money.

The middle of the board is where this gets uncomfortable for anyone signing the invoice. GPT-5.5 scores 65.9 at $2.31. OpenAI's own GPT-5.4 scores 64.3 at $1.44. That is 1.6 points of score for 60% more money, and both numbers came off the same harness.

The cheap seats bill you in wall clock

Nothing is free. The models that cost the least also sat on the tasks the longest. Fable 5 averaged 459 seconds per task; MiniMax M3 averaged 1,150. The board publishes runtime, not turn counts, so treat that as a correlation between price and patience rather than a mechanism.

For a nightly batch of refactors nobody is watching, patience is free. For an agent sitting in front of a developer, or one holding a CI lane open, it is the most expensive resource on the list.

The harness is worth up to 8.6 points

Buried in the same dataset is the result almost nobody cites. The index also runs a handful of models through other harnesses, and publishes those side by side: Claude Code, Codex, Gemini CLI, and OpenHands' own sub-agent mode. Same weights, same benchmarks, different agent code wrapped around them.

A model is the engine and the harness is the driver. Swap drivers and the same car posts a different time, in both directions. Here the swing is measured in score points, and it is not small.

Read that in both directions. Claude Opus 4.7 drops 8.6 points moving from the OpenHands SDK to Claude Code. Claude Sonnet 4.5 gains 5.2 points making the same move, and Gemini 3 Flash gains 5.8 on Gemini CLI while getting cheaper, from $0.90 to $0.57 per task. GPT-5.4 gives up 1.9 points on Codex and comes back 44% cheaper, $0.81 against $1.44.

The bias here is not hidden. The people publishing this benchmark also make the harness that wins it. That does not make the numbers wrong, and publishing rows where their own agent loses to Gemini CLI on two models is a point in their favor. It does mean the alternative-harness rows deserve the same skepticism you would apply to any vendor's own bake-off, and each one is a single run at a single agent version, not an average of many.

The column that does not say what you think

The rows are less controlled than "one harness" suggests, and the leaderboard's own SDK column understates it. Go to the per-benchmark results in the results repository and the versions that produced the scores span v1.8.3 to v1.28.0, eighteen distinct releases across the 33 models. 27 of those 33 mix versions inside their own five-benchmark average: Qwen3.5 Flash's five results were produced by v1.11.5, v1.16.1 and v1.17.0.

The displayed version is a model-level field, and for 14 of the 33 it is not among the versions that actually ran: Claude Fable 5 shows v1.18.1 and its five results record v1.28.0, DeepSeek V4 Pro shows v1.0.0 and ran v1.22.1. By this dataset's own evidence the code around a model is worth up to 8.6 points, so an average stitched from three releases carries a confounder the size of the gaps people quote from it. Every board has one of these. Most do not publish enough to let you find it, which is the reason this one is worth reading closely.

How to actually spend this

• Shop by cost per point, not by rank. For bulk work, batch refactors, test generation, triage, the top of the board is a luxury purchase. MiniMax M3 and Kimi K2.6 land within a point of each other at 32 and 72 cents.
• Route by category, because the money is not evenly spread. Fable 5 spent $1.43 per instance on SWE-bench issue resolution and $12.49 on Commit0 greenfield builds. Greenfield is the priciest category for 24 of the 33 rows the leaderboard API serves, and front-end work takes it for most of the rest.
• Pick the harness with the model, not after it. Claude Code and Gemini CLI each beat the reference agent on at least one model, and both did it for less money.
• If you self-host, this is the bill you delete. An open-weight model at 32 cents a task on somebody's API is electricity on your own box. Our self-host page now lists these measured per-task costs, open and paid models in one ranking, under the table that pairs each open model with its closest paid one.

What this does not measure

• Five benchmarks are not your codebase. Nothing here measures your framework, your test suite, or your reviewers.
• Single runs, no error bars. Agent runs are noisy, and a point or two of difference is not a result.
• The dollar figures are per task attempted, not per task solved. A model that fails expensively looks the same in this column as one that succeeds expensively.
• The costs are whatever those APIs charged on the day of the run. Prices move, and the cheapest rows depend on providers that discount aggressively.
• The board is aging. The newest model on it was released 9 June 2026, so everything shipped since is absent, and a benchmark that has not scored a current model cannot rank the current field.

Key Takeaways

• Claude Fable 5 leads the OpenHands Index at 81.0 with a bill of $5.54 per task attempted. MiniMax M3 gets 57.2 for 32 cents, roughly twelve times more score per dollar.
• The cheap models pay you back in wall clock: 1,150 seconds per task for M3 against 459 for Fable 5.
• Swapping the agent harness around the same model moves the score by up to 8.6 points, up as well as down.
• Greenfield work is where agents burn money: $12.49 per Commit0 instance for the leader, against $1.43 on SWE-bench.
• The runs behind it span 18 harness releases (v1.8.3 to v1.28.0), and 27 of the 33 models mix versions inside their own average, a confounder the size of the differences people quote from it.
• Nothing released after 9 June 2026 has been measured, so treat it as evidence about cost, not as a current ranking.

Sources: OpenHands Index, its public leaderboard API, the openhands-index-results data repository, the OpenHands Software Agent SDK, and the benchmark homes for SWE-bench, Commit0, SWT-bench and GAIA. Figures retrieved 8 September 2026.