Qwenputer Compiles a RISC-V CPU Into Qwen3.5-4B's Weights and Uses Its RNN State as RAM

TL;DR

Qwenputer is Qwen3.5-4B with a 32-bit RISC-V CPU living inside its weights. Divyansh Agrawal did not train it in: a compiler sets every weight of 11 new layers placed in front of Qwen's 32, and the model's Gated DeltaNet layers hold the registers, memory and program counter as exact, addressable RAM. Qwen asks for a calculation in the tool-call format it already knows, the compiled layers run the program during the forward pass, and the model writes the tool response itself. No interpreter runs outside the forward pass. The CPU passes the official riscv-tests, its programs match QEMU, and 230,000 replayed output positions matched a reference machine with zero mismatches. On DeepMind's big-multiplication set, Qwen goes from 55% to 97.5% correct. Code is MIT, weights are Apache-2.0, and it runs on one 24 GB GPU.

What he built

The project is a week old as a public artifact (the repo was created on October 19 and the write-up went up on October 22) and landed on Hacker News on October 29. The CPU is RV32IM with the Zba extension and a small bignum unit for long arithmetic. It runs C and MicroPython inside the model's forward passes.

The part that makes it more than a party trick is how little of Qwen changed. The base model's own weights are untouched. The config is widened (hidden size 3968, up from 2560) to make room for the computer's dimensions, the 11 computer layers sit in front, and the vocabulary gains 230,324 trace tokens the CPU uses to narrate its own execution. The checkpoint on Hugging Face loads as a stock Qwen3_5ForCausalLM. The computer itself is about a million nonzero weights.

Agrawal's own summary: "it is not a plugin, a tool or a sandbox." The README adds: "none of their weights are learned: a compiler computes every one."

Why the RNN layers can be exact RAM

Qwen3.5 is a hybrid. Per the model card, its 32 layers come in eight blocks of three Gated DeltaNet layers followed by one full-attention layer, so three of every four mixing layers are a linear RNN. Each DeltaNet head keeps a fixed-size state matrix that the delta rule updates on every token: it multiplies the whole state by a gate, then writes a new value along the direction of the key, with a write strength beta that sets how much of the old value survives.

Agrawal's observation is that three settings turn that fuzzy associative memory into a precise one. Set the gate to 1 and nothing ever fades. Set beta to 1 and the old value along the key is replaced rather than blended. Use one-hot keys, and every write lands on exactly one row of the state, and every one-hot query reads that row back exactly. Picture the state as a wall of post-office boxes: the one-hot key is the box number, beta of 1 means you swap the letter inside instead of averaging it with the old one, and gate of 1 means nothing in the wall yellows with age. The CPU keeps its registers, memory and program counter in those rows. Gate values are also used for side effects: a gate of 0 clears a head, and a gate of one half makes a value halve each step, which works as a counter.

He credits Schlag et al. (2021) for showing that linear attention with the delta rule acts as an associative memory. The new part is making it exact and addressable, and then compiling a whole CPU against it. A nice property follows for free: a DeltaNet head's state stays the same size however long the conversation runs, unlike a KV cache.

How Qwen uses it

Every instruction the CPU executes becomes a few generated tokens. An ADDR token names the memory row to write, a VAL token carries the value, and a STEP token closes the instruction. The stream of those tokens is the machine's execution log, and the CPU advances as the model decodes.

From Qwen's side, nothing new was taught. It writes a <tool_call> block for a tool named calc in its normal format. The computer layers read the call, run the program in DeltaNet RAM, and the model writes the <tool_response> itself. A small driver called RvChat boots the firmware into the DeltaNet state after the prompt and keeps the trace tokens out of Qwen's layers, so the language model never sees the CPU's inner monologue.

Does it work?

The correctness claims are consistent across the blog, the README and the model card. The CPU passes the official riscv-tests suites, its program output matches QEMU, and it has run 18 million instructions free-running (the CPU keeps going on its own outputs for the whole run) without an error. Agrawal replayed about 230,000 of the model's output positions against a reference machine and found zero mismatches. The README puts it more bluntly: "The computer has never printed a wrong result."

The arithmetic benchmarks use DeepMind's mathematics dataset and the long-integer tasks from NUPA, 40 problems per set, greedy decoding. The base model got no tools but a system line telling it to work each step exactly; the Python-tool column gives Qwen a real interpreter on the host.

The other sets follow the same pattern. Synthetic integer arithmetic goes from 75% to 97.5%, matching the Python tool. On mul_div_multiple_longer Qwenputer scores 100% against 97.5% for the tool and 90% for base. On add_or_sub_big it ties the tool at 97.5%, with base at 87.5%. NUPA's long-integer arithmetic jumps from 48.6% to 100%, again level with Python. And bolting a computer onto Qwen does not change how it talks: ordinary text stays within noise of base Qwen, a KL divergence of 6e-4 against a noise floor of 5e-4.

Where it does not help is also documented. GSM8K and GSM-Hard come out as ties because reasoning, not arithmetic, is the bottleneck there, base Qwen still leads on long DeepMind chains, and Qwen has a habit of re-verifying an exact result until it exhausts its token budget. A calculator that is always right turns out to be hard for a language model to trust.

A hard drive too

"Because a CPU inside a language model clearly wasn't cursed enough," Agrawal writes, "I decided the prudent thing would be to give it a filesystem too." The filesystem model (a separate checkpoint, code on the rv-store branch) adds put, append, get, list and delete beside calc, with the same no-tool-call trick.

When Qwen saves a file, the compiled layers hash its name into one of 1,024 directory slots kept in the DeltaNet state and store the text in other heads, eight tokens per row, chained together. A collision just checks the next slot, so finding a file takes two or three steps rather than a scan. Capacity is 512 files, about 53,000 tokens of text. Because it all lives in RNN state, you can save the state at the end of one conversation, load it into the next, and the files are still there.

Agrawal reports that with the filesystem, Qwen gets facts that changed between chats right 90% of the time, against 30% when you paste the old chats back into context. He calls this one "still experimental."

Limits, and what the skeptics said

• The extra layers and wider hidden size make the checkpoint 18.8 GB in bf16, roughly double base Qwen, though most of it is zero padding. It runs on a single 24 GB GPU.
• It needs RvChat to boot the firmware and filter the trace tokens. It does not run from a plain generate() call yet.
• Every CPU instruction costs a few decoded tokens, so a big calculation takes seconds rather than milliseconds. The batched Triton executor used for the benchmarks runs about 3,200 CPU tokens a second on an NVIDIA L4.
• It makes a 4B model exact at arithmetic and gives it a tiny persistent disk. It does not make it smarter at word problems, as the GSM8K tie shows.

The Hacker News thread had the expected "but why" contingent. One commenter called it a Rube Goldberg machine and pointed to Neural Turing Machines from 2014 as the same idea with less ceremony. Fair, up to a point: NTMs learned to use an external memory, which is the opposite of this. Here nothing is learned, the memory is the model's own layers, and every output can be checked against a reference CPU.

Why a builder should care

The cheapest takeaway is practical: you can give a local 4B model exact arithmetic and a small persistent memory without running a sandbox, which matters when the model lives somewhere you cannot ship an interpreter. The quickstart is short. Clone the repo, uv sync --extra hf inside experimental/riscv, pull the weights with the Hugging Face CLI, and run the chat driver with the calc-bridge.elf firmware and an --ask prompt. An earlier CHIP-8 version is also published, with 3.9 million replayed positions at zero mismatches and MMLU within 0.2 points of base.

The more interesting takeaway is what the technique says about hybrid models. If a compiler can store exact, addressable state in DeltaNet layers, a model could learn to do the same. Agrawal has two follow-ups built on Qwenputer: one on how hybrid models can hide reasoning in their RNN state and evade chain-of-thought monitoring, and one on building oversight tools into a model's weights, where compiled layers act as a monitor the model cannot switch off, or as an answer key for interpretability work. Both are worth your time if you ship Qwen3.5-class hybrids in production and have been assuming the KV cache is the only place state can hide.

Next on his list: having Qwen write Python and run it on the MicroPython that already lives in its weights, through a python() tool that works the same way.

Key Takeaways

• Qwenputer compiles a 32-bit RISC-V CPU (RV32IM plus Zba and a bignum unit) into 11 layers in front of Qwen3.5-4B's 32, with no training; Qwen's own weights are unchanged.
• Gated DeltaNet layers become exact RAM by setting the gate to 1, beta to 1 and using one-hot keys, so every write hits one row and every read returns it exactly.
• Qwen calls the CPU through its normal tool-call format and writes the tool response itself; no runtime, sandbox or executor runs outside the forward pass.
• It passes riscv-tests, matches QEMU, ran 18 million instructions free-running without error, and 230,000 replayed positions showed zero mismatches.
• Arithmetic accuracy rises from 55% to 97.5% on DeepMind's big multiplication and from 48.6% to 100% on NUPA, with text behavior within noise of base (KL 6e-4).
• Costs: an 18.8 GB mostly-zero checkpoint, a driver requirement, and about 3,200 CPU tokens a second on an L4. Code is MIT, weights Apache-2.0.

Sources: Qwenputer: a functioning, deterministic RISC-V CPU inside Qwen3.5 (Divyansh Agrawal), divagr18/qwenputer on GitHub, Qwen3.5-4B-Qwenputer model card, Qwen3.5-4B-Qwenputer-Filesystem model card, Qwen3.5-4B model card, Gated Delta Networks (arXiv), Linear Transformers Are Secretly Fast Weight Programmers (arXiv), Hacker News discussion