7 of 8 Agent Stages Run on the CPU. Intel Just Sold Out.

TL;DR

The scarce chip in AI infrastructure is no longer the one with HBM stacked on it. IEEE Spectrum reports that agentic workloads have made ordinary server CPUs the new bottleneck: AWS leadership told engineers this spring to conserve capacity "by any means necessary" as wait times for CPU servers exploded, Intel has sold out of server CPUs through at least the end of the year, and AMD has doubled its server CPU forecast. The mechanism is simple and quantified: in AMD's testing, seven of the eight stages in a realistic agent pipeline run entirely on the CPU, and two recent papers show CPU starvation alone can inflate time-to-first-token by up to 5x. After three years of GPU-shortage discourse, the chip you cannot buy this year is the one your desktop has had since 2006.

"Conserve capacity by any means necessary"

The AWS detail comes from an August 7 report in The Information: in a May meeting, AWS leadership instructed engineers to cut compute usage wherever possible to keep EC2 capacity available for customers, and the directive explicitly covered plain CPU servers, not just AI accelerators. Engineers reportedly saw wait times for CPU server capacity balloon as agent workloads spread.

AWS pushed back on the framing, calling efficiency drives longstanding practice rather than a response to a new crunch. But the supply signal predates the leak: Andy Jassy's 2025 shareholder letter had already conceded unmet demand, including two large customers who asked for AWS's entire 2026 Graviton allocation and could not be fully accommodated. Meta has reportedly contracted for tens of millions of Graviton cores specifically for agent deployments. Graviton5 ships 192 cores per chip, about 25 percent faster than Graviton4, and apparently that still is not enough.

Your agent barely touches the GPU

Why would chat-era infrastructure buckle under agents? Because an agent is mostly not inference. Madhu Rangarajan, AMD's VP of compute and enterprise AI, put a number on it: "in our testing, seven of the eight stages in realistic agentic AI pipelines run entirely on the CPU." Planning tokens come off the GPU in bursts, but everything between those bursts, tool execution, retrieval, response parsing, sandbox management, API orchestration, is CPU work.

If the mechanism feels backwards, think of a restaurant. The GPU is the star chef; the CPU is everyone else, the host, the waiters, the dishwasher, the person answering the phone. Chatbots were a fixed menu: one order in, one dish out, and the kitchen was always the constraint. Agents turned every meal into a forty-course tasting menu with a phone call between courses, and now the chef stands idle while the front of house drowns.

The scale of "between courses" is easy to underestimate. When an OpenAI evaluation agent famously broke out and probed Hugging Face's infrastructure this summer (OpenAI's incident report has the timeline), IEEE Spectrum notes the model fired off as many as 300 actions an hour. Every one of those actions is a process spawned, a request tokenized, a container scheduled, a network call awaited. On the CPU.

The measurements

Two recent papers put numbers on the anecdotes. A Georgia Tech study by Euijun Chung and colleagues, Characterizing CPU-Induced Slowdowns in Multi-GPU LLM Inference, finds that multi-GPU serving frequently degrades not because GPUs saturate but because starved CPUs fail to keep them fed: delayed kernel launches, stalled communication, slow tokenization. Under moderate load, CPU-starved configurations frequently timed out entirely, and giving the same GPUs adequate CPU cores cut time-to-first-token by 1.47x to 5.15x across configurations. No additional GPUs required.

The second paper, a CPU-centric characterization of agentic AI execution from Intel's Souvik Kundu with Georgia Tech collaborators, goes a step further and reschedules the whole pipeline around the CPU. Their CPU-aware micro-batching scheme (COMB) cuts service latency up to 3.9x and end-to-end latency up to 1.8x under sustained load, and a mixed-workload scheduler (MAS) cuts P90 latency 2.49x for minority request types that would otherwise starve behind bulk traffic.

The economic kicker in the Chung paper is the part worth reading twice: the marginal cost of extra CPU cores is small next to GPU instance pricing. Which means a meaningful slice of the industry's "we need more GPUs" spend is actually a CPU allocation bug with a cheap fix.

The supply side is already reacting

Per IEEE Spectrum, Intel's server CPUs are sold out through at least year-end, and AMD has doubled its server CPU forecast. Arm and Qualcomm have both announced CPUs pitched specifically at agentic AI, and even Nvidia, the company that spent three years telling you the CPU era was over, has prioritized Vera, its Arm-based CPU for agent workloads. Distributor reports describe server CPU lead times stretching from weeks into multiple quarters, though the exact figures vary by SKU.

For the self-hosting crowd this is a spec-sheet correction. If your agent box was planned as maximum VRAM bolted to whatever leftover cores were lying around, the research says you built the bottleneck in on purpose: tokenization, tool sandboxes, retrieval, and scheduler threads all queue on those leftover cores while your expensive GPU idles. Core count and memory bandwidth just got promoted from footnote to line one, and secondhand server CPUs are unlikely to get cheaper while two chipmakers are allocation-constrained.

The caveats

Straight-faced section. First, the AWS mandate is single-outlet reporting from The Information, and AWS disputes the framing on the record. Second, "sold out" means OEM and hyperscaler allocation is spoken for, not that CPUs have vanished from retail. Third, notice who is loudest here: an AMD VP and an Intel researcher have an obvious interest in a CPU renaissance, and one of the two papers is partly Intel-authored. The independent Georgia Tech measurements point the same direction, which is why this story clears the sniff test, but the 7-of-8 figure is vendor testing, not a peer-reviewed benchmark.

Key Takeaways

• The bottleneck moved. Agent workloads spend most of their time in CPU-bound stages; AMD's testing puts it at seven of eight pipeline stages running entirely on the CPU.
• AWS is feeling it. The Information reports a May directive to conserve capacity "by any means necessary," covering plain CPU servers, with EC2 CPU wait times ballooning.
• Supply is tight. Intel is sold out of server CPUs through at least year-end, AMD doubled its forecast, and Arm, Qualcomm, and Nvidia are all shipping or prioritizing agent-focused CPUs.
• The numbers are real. CPU starvation inflates time-to-first-token 1.47x to 5.15x, and CPU-aware scheduling cuts service latency up to 3.9x, per two Georgia Tech-affiliated papers.
• Cheap fix first. Extra CPU cores cost a rounding error next to GPU instances; audit your CPU allocation before buying another accelerator.
• Caveat emptor. The loudest voices sell CPUs, the AWS report is single-outlet, and "sold out" describes allocation, not empty shelves.

Sources: IEEE Spectrum, Chung et al., arXiv:2603.22774, Raj, Kundu et al., arXiv:2511.00739, XenoSpectrum on The Information's AWS report, OpenAI incident report