OpenAI Just Hit 750 Tokens a Second Without Nvidia

TL;DR

On August 13, OpenAI previewed Ultrafast, a new API service tier that runs GPT-5.6 Sol at up to 750 output tokens per second, which it bills as up to 14x faster than Standard processing. The speed does not come from a smaller model or a cheaper router. It comes from Cerebras wafer-scale silicon, which means OpenAI's most capable model is now served, for at least some customers, on hardware that is not a GPU at all.

What actually shipped

Ultrafast is a tier, not a model. Both companies are explicit that it is the same GPT-5.6 Sol weights with the same intelligence, just executed on different silicon. That distinction is the entire story. Until now, the way you bought latency was by buying less model: drop to a smaller checkpoint, drop to a distilled variant, drop your expectations. OpenAI's own framing in the announcement is that "getting real-time speed typically meant choosing a smaller or more specialized model."

The published multipliers, from OpenAI's post and Cerebras' technical writeup, compare against three baselines.

Note that the 14x is measured against GPT-5.6 Sol on the Standard tier. The 11x and 5x are against Anthropic models, Fable 5 and Opus 4.8 running in Fast mode respectively. Three baselines, three numbers, one headline. Standard press-release practice, and worth untangling before you put any of it in a capacity plan.

The number that matters is 5.6, not 14

Peak token rate is the least useful benchmark in inference. Real agent workloads spend time on prefill, tool calls, retrieval hops, and waiting on your own slow database. A 14x generation speedup gets amortized against all of that, which is why the end-to-end figures are the honest ones, and to its credit Cerebras published them.

On GDPval, OpenAI's benchmark of real economically valuable knowledge work, Ultrafast delivered a 5.6x end-to-end speedup with no quality degradation. That is the number to plan around. It is less than half the headline, and it is still a genuinely large speedup for a workload made of actual deliverables rather than token streams.

The other end-to-end result is more visceral. Cerebras ran the full 2,500-question Humanity's Last Exam set and reported comparable accuracy at nearly 7x the pace.

Reframe that: an eval sweep that used to be an overnight-and-then-some job becomes something you can start after breakfast and read before you log off. If your loop is "change the prompt, rerun the suite, look at the diff," collapsing that cycle from days to hours changes what you are willing to try, not just how fast you finish.

Why a wafer beats a card here

Frontier inference is not compute-bound, it is memory-bandwidth-bound. Every generated token requires streaming the model's weights past the arithmetic units, and on conventional accelerators those weights live in HBM, off to the side of the compute die. The chip spends most of its life waiting for parameters to arrive, not multiplying them.

Cerebras' Wafer-Scale Engine sidesteps that by keeping the weights on the chip itself, with 44 GB of SRAM on each wafer-sized die, and pipelining the model's layers across multiple wafers so tokens flow through rather than looping back to external memory.

The kitchen version: a GPU is a brilliant chef who keeps every ingredient in a pantry down the hall, so the cooking speed is really the walking speed. Cerebras built a counter big enough to hold the entire pantry, then hired a second chef for the next course. The chopping was never the bottleneck.

That last clause matters more than the throughput. This is not a quantized model, not a speculative-decoding trick with a draft model that occasionally drifts, and not a distilled sibling wearing the flagship's name. Both parties state it is the same weights at the same quality, and the GDPval result is the evidence offered for it.

The part you cannot use yet

Ultrafast is a limited preview in the OpenAI API, restricted to a small group of customers, expanding "as capacity grows." Businesses can request access by submitting their workload profile and latency requirements, which is a polite way of saying wafers are scarce and OpenAI is triaging.

Neither company published Ultrafast pricing. For a tier whose entire value proposition is a latency-for-money trade, that is the single most load-bearing missing number, and its absence should temper any architecture decision you are tempted to make this week.

The named target workloads are the latency-sensitive ones you would expect: voice, customer support, commerce, developer agents, financial research, and incident response. OpenAI cited developers using it to compress research cycles that previously needed overnight runs into several same-day iterations, plus chewing through incident logs.

The quotes, and what is under them

"GPT-5.6 Sol on Ultrafast is proof that speed and intelligence are no longer mutually exclusive." Andrew Feldman, CEO and co-founder, Cerebras

Feldman has been making a version of that argument since the first wafer taped out, and this is the first time the counterparty is the lab with the most valuable inference workload on earth. OpenAI's Sachin Katti, VP of Compute Strategy and GPT-Infra, framed it as exploring what becomes possible, which is executive for "pilot."

The most telling line came from OpenAI researcher Jeffrey Wang, who said the model "now finishes for me before I even have the opportunity to context-switch." Somewhere between a productivity claim and a threat, that is the actual product: the latency floor drops below the point where you go check Slack, and the tab-switch habit of an entire profession quietly loses its excuse.

Why this is bigger than one tier

Strip the marketing and the structural fact is this: OpenAI's flagship reasoning model is being served in production, to paying customers, on silicon from a company whose entire pitch is that the GPU memory hierarchy is the wrong shape for inference. It is a preview, the capacity is small, and it changes nothing about who trains what on which cluster. But the list of frontier labs willing to put their best model on non-GPU inference hardware just went from zero to one, and that is the kind of number that only moves in one direction.

Key Takeaways

• 750 output tokens per second on GPT-5.6 Sol, billed as up to 14x the Standard tier, via a new OpenAI API service tier called Ultrafast.
• Plan against 5.6x, not 14x. That is the end-to-end GDPval speedup with no reported quality loss, and it is the figure that survives contact with a real workload.
• Same weights, not a smaller model. Both companies state Ultrafast runs GPT-5.6 Sol at Standard-tier intelligence, no distillation or quantization in the pitch.
• The mechanism is memory, not math. 44 GB of on-chip SRAM per wafer plus layers pipelined across wafers removes the HBM round-trip that caps GPU inference speed.
• Humanity's Last Exam in 11h 11m versus 78h 27m for Claude Fable 5, at comparable accuracy: a 67-hour swing on a 2,500-question sweep.
• No pricing, no general access. Limited preview, waitlist gated on workload and latency profile, and the cost per token is still unpublished.

Sources: OpenAI, Previewing Ultrafast mode, Cerebras, Accelerating GPT-5.6 Sol Ultrafast with OpenAI, GlobeNewswire, Cerebras Powers Ultrafast Mode for OpenAI's GPT-5.6 Sol, TechCrunch, 9to5Mac, GDPval paper