AMD Launched Its 72-GPU Answer to Nvidia, Then Handed the Token Generation to Cerebras

TL;DR

On July 23, AMD and Cerebras announced a joint inference product that splits one request across two companies' silicon. AMD's Helios rack does the prompt processing; Cerebras' Wafer-Scale Engine does the token generation. The claim is up to 5x higher tokens per second per watt against a Cerebras-only configuration, and it ships through Cerebras Cloud in the second half of 2026. The sequencing is the story: hours earlier at Advancing AI 2026, AMD launched Helios as its most complete rack-scale answer to Nvidia yet, and then conceded that the half of inference users actually feel runs better on somebody else's chip.

What they actually announced

The two press releases are short and say the same thing. AMD Helios provides "a high-performance, scalable throughput engine." Cerebras Wafer-Scale Engine technology provides "ultra-fast, ultra-low latency decode and token generation." They run as one integrated workflow, with Cerebras deploying Helios racks in its own data centers and reselling the combination through Cerebras Cloud.

Why inference splits in half

If you have never had to care about the difference: an LLM request has two phases with almost opposite hardware appetites. Prefill reads your entire prompt at once and computes the KV cache for it. That is a big, embarrassingly parallel matrix job, and it is compute-bound. Decode then emits one token, feeds it back in, and emits the next, which means streaming the model's weights through the arithmetic units once per token. That is memory-bandwidth-bound, and no amount of extra FLOPS fixes it.

Think of prefill as unloading a shipping container: the more hands you throw at it, the faster it clears. Decode is the drive-through window at the other end of the lot. It serves one car at a time, and the only number that matters is how fast the window opens and closes.

Sizing one machine for both jobs means overpaying for whichever half it is bad at. Splitting them lets you buy the right silicon twice.

Why the wafer gets the decode half

Cerebras builds a processor the size of a dinner plate. Its third-generation Wafer-Scale Engine packs 4 trillion transistors and 900,000 AI cores on 5nm, with 44 GB of SRAM sitting on the same silicon as the cores and no HBM stacks at all. AMD's release does not name a WSE generation, so treat the specifics as the shipping part, not a confirmed bill of materials.

That 44 GB is tiny next to a GPU. It is also on-die, which is the point: reading a byte from SRAM next to your cores is far cheaper than crossing a package to an HBM stack. Decode does nothing but read bytes, over and over, so it is exactly the workload where that geometry pays.

The 5x number, read carefully

Here is the footnote, verbatim from Cerebras: "Based on modelling by AMD Performance Labs and Cerebras in July 2026 to determine tokens per second per kilowatt (TPS/kW) at a comparable interactivity point with Kimi 2.6 1T Model comparing an AMD Helios rackscale solution with Cerebras WSE to a Cerebras WSE-only configuration."

Unpack that. It is modeling, done in July by the two companies selling the product, on a 1-trillion-parameter Moonshot model, at a "comparable interactivity point" that nobody defines. The headline says per watt and the footnote says per kilowatt, which does not change the ratio but does tell you how fast this went out the door.

That denominator is worth staring at. The 5x is not measured against Nvidia, and not against AMD alone. It is Cerebras beating its own current configuration. That is a real result for anyone already buying Cerebras inference, and a very carefully chosen baseline for everyone else.

You probably already run disaggregated inference

Splitting prefill from decode is not new. vLLM, SGLang, NVIDIA Dynamo, and llm-d all ship some form of it, and Dynamo's disaggregated serving docs lay out the standard three steps: prefill workers compute the KV cache, transfer it, decode workers take over.

What is new here is that the two halves now sit on different vendors' architectures. Inside a homogeneous fleet, the KV cache transfer is an RDMA hop between machines that share a memory layout. Across a GPU rack and a wafer, it is a hop between two different worlds, and the cache for a long context is not small. If the handoff costs more than the wafer saves, the whole pitch collapses at exactly the long-context, agentic workloads it is aimed at. Neither release says a word about transfer latency, which is the number to demand first when this hits Cerebras Cloud.

Who this is for

Not you, if you are serving a fine-tune on two GPUs. This is aimed at real-time agentic products where a human or another agent is blocked on the token stream: voice, coding loops, anything with a tool-call round trip in the middle. Lisa Su framed it as extending AMD's reach "into the most latency-sensitive applications." Andrew Feldman's angle was distribution: "Partnering with AMD gives us an incredible opportunity to bring that performance to even more customers."

What Helios actually is

Worth knowing, since half this product is now the newest rack in the market. Per AMD's own spec page, a full Helios rackscale solution is 72 Instinct MI455X GPUs on the CDNA 5 architecture, 6th Gen EPYC "Venice" CPUs, and Pensando Vulcano AI NICs, in a double-wide rack built on Meta's Open Rack Wide standard submitted to the Open Compute Project.

• 31 TB of HBM4 memory per rack, 432 GB per GPU.
• 2.9 exaFLOPS of FP4 compute and 1.4 exaFLOPS of FP8 per rack.
• 260 TB/s aggregate scale-up bandwidth over UALink-over-Ethernet, plus 43 TB/s of Ethernet scale-out.
• Up to 40 PFLOPS of FP4 per GPU, at 19.6 TB/s of per-GPU memory bandwidth by AMD's figure.

AMD's Advancing AI 2026 release names OpenAI, Anthropic, Meta, Microsoft, Oracle, HUMAIN, Tensorwave, Vultr and Cirrascale as choosing or deploying Helios, with OpenAI bringing systems online beginning in Q4 2026 and deployments accelerating through 2027. That follows AMD's Microsoft expansion on July 20 and the 2-gigawatt Anthropic deal on July 22. The order book is having a week.

The caveats

Nothing here is shipping today. Availability is "second half of 2026" through Cerebras Cloud only, with no on-premises path announced, no pricing, and no named launch customer for the joint system. The 5x is modeled rather than measured, on a single model, against a self-selected baseline. No transfer-latency, time-to-first-token, or end-to-end throughput figures were published. And a two-vendor serving stack means two software stacks, two failure domains, and a scheduler that has to keep both fed.

Key Takeaways

• AMD and Cerebras announced a disaggregated inference system on July 23: Helios racks run prefill, the Wafer-Scale Engine runs decode, sold as one workflow.
• The headline claim is up to 5x tokens per second per watt, modeled by both vendors in July on Kimi 2.6 1T, versus a Cerebras-only configuration.
• The split follows real physics: prefill is compute-bound, decode is memory-bandwidth-bound, and on-wafer SRAM beats off-package HBM for the second job.
• Prefill/decode disaggregation already ships in vLLM, SGLang, Dynamo and llm-d. Crossing vendor architectures with the KV cache is the genuinely new part, and the unpublished transfer cost is the risk.
• Helios itself launched the same week: 72 MI455X GPUs, 31 TB of HBM4, 2.9 exaFLOPS FP4, with OpenAI, Anthropic, Meta, Microsoft and Oracle named as customers.
• Available through Cerebras Cloud in H2 2026. No pricing, no on-prem option, and no measured benchmarks yet.

Sources: AMD Investor Relations (AMD and Cerebras announcement), Cerebras (press release), AMD (Advancing AI 2026), AMD Helios Rackscale Solution, Cerebras (WSE-3 announcement), NVIDIA Dynamo (disaggregated serving)