Sakana AI Trained a Network Without Backpropagation and Kept the Brain's Rules. It Tops Out at 61.7% on CIFAR-10.

TL;DR

Sakana AI put out a paper, Diffusing Blame, that trains a neural network without backpropagation while forcing every neuron to stay purely excitatory or purely inhibitory, a rule real brains obey and almost no artificial network does. The method is called Error Diffusion, and it is the first to satisfy that biological constraint, skip backpropagation's notorious "weight transport" step, and still carry the whole thing into reinforcement learning. It is also refreshingly honest about the bill: 96.7% on MNIST, 61.7% on CIFAR-10. If that second number makes you wince, good. The accuracy was never the headline. The headline is that a network learns at all under rules this strict.

Backprop's Forty-Year-Old Skeleton in the Closet

Backpropagation is the engine under nearly every model you touch, and it hides a fact that has irritated neuroscientists since the 1980s: it almost certainly is not how the brain learns. To push an error signal backward through a network, backprop needs the exact transpose of every forward weight matrix. In plain terms, each connection has to know, precisely, the strength of the connection that fed into it, and read that number back perfectly on the return trip.

Picture an assembly line where, to fix a defect, every worker has to phone the exact upstream worker who caused it and read back their machine settings to the decimal. Backprop assumes that phone line exists and is noise-free. Your cortex has no such wiring. This is the weight transport problem, and it is why "biologically plausible learning" has spent decades as a graveyard of clever workarounds that never quite scaled.

Two Rules Nobody Had Satisfied at the Same Time

There are two separate demands a truly brain-like learning rule has to meet, and the field has historically been able to hit one or the other, never both.

The first is the weight transport problem above. The best-known escape hatch is Direct Feedback Alignment, which throws away the transposed weights and broadcasts the error through fixed random feedback connections instead. It works, but a brain wired with frozen random noise is its own kind of implausible.

The second is Dale's principle: a real neuron is either an accelerator or a brake for everyone it talks to, never gas for one partner and brake for another. Standard networks cheerfully ignore this, letting a single weight flip sign whenever training pleases. Most methods that respect Dale's principle still lean on backprop; most backprop-free methods quietly break Dale's principle. Error Diffusion is the first to hold both lines at once, and then it does the thing nobody had bothered to try under these constraints: it runs reinforcement learning.

How Error Diffusion Assigns Blame

Error Diffusion is an old idea (first floated by Kaneko in 2000) that Sakana dragged into the deep-learning era. Each weight update leans on just three local signals: how active the input neuron was, the slope of the receiving neuron's activation, and a single global sign that says whether the network was too high or too low. There is no transposed backward pass and no random feedback matrix. The error's blame is diffused outward from one shared signal rather than routed back along a mirror-image network.

To get past simple yes/no tasks, the authors add "modulo error routing," a scheme that fans the single error signal out across many output classes by assigning class i to channel i mod C. It sounds like a hack because it is one, but it is the hack that lets the rule handle ten-way classification instead of stalling at binary.

The Numbers, Stated Plainly

On MNIST, Error Diffusion reaches 96.7%, which is respectable for a handwritten-digit task. On CIFAR-10, it lands at 61.7%, the kind of score you would normally bury in an appendix. Sakana put it in the paper's headline results. That candor is the tell: this is a demonstration that the rules can be obeyed, not a bid for a leaderboard.

Obeying biology is not free, and the paper is unusually specific about the invoice. Enforcing Dale's principle costs between 0.9 and 7.4 accuracy points versus Direct Feedback Alignment on classification. The dual-stream design, which needs separate non-negative pathways for the excitatory and inhibitory halves, roughly quadruples the parameter count, from about 8 million to about 32 million for the same job.

There is a deeper wrinkle too. Because the receiving neuron's activation slope gates the error, the signal decays roughly 25 times over from the output layer to the first hidden layer. Credit assignment through depth, the exact thing backprop is freakishly good at, is where these biological rules hurt most.

It Also Does Reinforcement Learning

The part that lifts this above a neat classification stunt is ED-PPO, which bolts Error Diffusion onto Proximal Policy Optimization, the workhorse RL algorithm. Tested on Brax, Google's differentiable physics simulator for continuous control, ED-PPO matches a Direct Feedback Alignment baseline. On Craftax, an open-ended survival-and-crafting benchmark, it beats it. Learning a control policy from reward, with no backprop and every neuron kept sign-pure, is a genuinely new data point.

The Ablation Nobody Should Skip

Buried in the results is the most useful methodological point in the paper. The three tricks that make Error Diffusion work (layer-specific sigmoid widths, batch-centered class error signals, and asymmetric initialization) do not have a fixed pecking order. Which one matters most flips between MNIST and CIFAR-10. The authors call these "task-dependent credit-assignment bottlenecks invisible to single-benchmark evaluation," which is a mouthful for a sharp warning: test your learning rule on one dataset and you will draw exactly the wrong conclusion about why it works.

Why Builders Should Care

First, keep PyTorch installed. Nothing here replaces backprop for anything you are shipping; 61.7% on CIFAR-10 is not coming for your production classifier. This is a research signal, not a tool drop.

Second, the signal points somewhere real. Local, backprop-free learning with no global synchronous backward pass is precisely the shape of computation that neuromorphic and analog chips want, because you never have to store a forward pass and replay it in reverse. If on-device continual learning and energy-efficient edge training are on your roadmap, the constraints Sakana is wrestling with are the constraints the hardware imposes.

Third, the framing is worth stealing regardless of the subject. A paper whose main product is a precise accounting of what you give up (accuracy, parameters, depth) to gain a property you want (biological plausibility) is more useful than the thousandth "we beat the baseline" result. Quantifying the tax is the contribution.

Caveats

• This is toy scale. MNIST and CIFAR-10 are a long way from ImageNet, and a galaxy from a transformer. There is no evidence yet that Error Diffusion scales to large models or language.
• The costs are real: roughly four times the parameters, a 25-fold error-signal decay through depth, and up to 7.4 points of accuracy surrendered to Dale's principle.
• The paper went up on arXiv on June 30, 2026 and surfaced in the AI press this week. It is a preprint, not peer-reviewed.
• "Without backpropagation" refers to the credit-assignment mechanism. Error Diffusion still uses local activation derivatives; gradients did not vanish from the universe, they just stopped taking the mirror-image trip home.

Key Takeaways

• Sakana AI's "Diffusing Blame" trains networks without backpropagation while keeping every neuron purely excitatory or inhibitory, satisfying Dale's principle.
• Its method, Error Diffusion, is the first to skip both weight transport and random feedback matrices and to extend into reinforcement learning.
• Results are deliberately modest: 96.7% on MNIST, 61.7% on CIFAR-10, with ED-PPO matching a DFA baseline on Brax and beating it on Craftax.
• Biology's bill is quantified: up to 7.4 accuracy points, about 4x the parameters, and a 25x error-signal decay through depth.
• The practical value is directional, pointing at neuromorphic and on-device learning, not at replacing backprop in anything you ship today.

Sources: Diffusing Blame (arXiv), MarkTechPost, Sakana AI