Tinyworldformer Generates and Plays Five Games With Neural Nets on a Raspberry Pi Pico 2 W

TL;DR

Tinyworldformer, by GitHub user SheepSeb, runs five small games on a Raspberry Pi Pico 2 W with no game code on the chip. An int8 world model of 26k to 131k parameters predicts every next frame and reward from the current frame and a button press. A second network, an actor of 9k to 16k parameters, picks the buttons. The board can also train a fresh agent from scratch inside that generated world. At two transformer passes per frame, Flappy Bird reproduces held-out frames exactly 100% of the time and Breakout 99.99%. Doom-lite manages 83.6% and Pac-Man lite 86.4%. Every game fits in the chip's 520 KB of SRAM, and the README records each exploit its agents found in the model. The project drew more than 260 points on Hacker News on October 29.

Two networks, no game engine

The model running on the board is a world model in the sense of Ha and Schmidhuber's World Models and Google's GameNGen. The real games exist only at training time, as a few hundred lines of numpy each. On the chip, neither network sees them. The world model gets actions only from the actor, and the actor sees only frames the world model drew.

• World model: a looped transformer with int8 weights, 26k to 131k parameters depending on the game, and 2 to 8 million multiply-adds per frame. It maps frame plus action to next frame plus reward.
• Actor: a float32 MLP with two hidden layers, 9k to 16k parameters, and 9k to 16k multiply-adds per frame. It maps frame to action.

Both run on core 1 of the RP2350 while core 0 handles USB and an optional 320x240 Display Pack. Press any button and you take over from the actor, so you can play the generated game yourself. The firmware is no_std Rust on embassy, and nothing is allocated at run time. The weights are static arrays in flash, and every buffer is a fixed-size static in RAM.

How accurate the generated games are

Each game is a token grid, from 20x16 cells for Breakout to 48x48 for Pac-Man lite, and one frame always carries the full game state. Hidden state gets drawn into the frame. The ball's heading is part of its token, and Doom-lite has a minimap.

The author's metric is strict: a frame counts only if every cell matches the real game. Frames right after a random event, like a serve or a ghost respawn, are left out because those can only be sampled, not predicted. Doom-lite's raycast view has 768 cells, so one wrongly shaded wall pixel fails the whole frame. The README says its game state is almost always right even when the pixels aren't.

One block, reused

The world model is a single transformer block, attention plus a ReLU MLP with one set of weights, applied one to four times with the input added back before each pass. The output after every pass is trained against the next frame, so at run time the loop count trades speed for accuracy. The Pico uses two loops.

Think of it as one editor proofreading the same draft four times instead of four different editors each taking one pass. You pay for one editor's memory, and on a chip with 520 KB of SRAM, memory runs out long before patience does. In the README's Doom-lite example, the first pass draws an enemy that isn't there, and the wrong-cell count then falls 92, 35, 12, 0 across the four passes.

Decoding mattered almost as much as the model

The model outputs a score for every cell and token, and turning that into a frame took three rules. Background cells like walls, bricks and pellets each take their most likely token. Singletons like the ball, the player or a ghost are placed with one choice across all cells, so exactly one of each always exists, and after a random event that choice is sampled among close candidates. Pac-Man and the ghosts are 2x2 sprites where only one cell carries state and the rest are stamped from it.

The singleton rule came from a bug. The first Breakout model drew two balls at once. Forcing one global choice took exact frames from 91.6% to 99.7%.

The agents found every crack

Recorded-data accuracy wasn't enough, because an agent steers the generated world into states the training data never had, and any mistake there is something it can exploit. So the author's main check runs during play: read the game state back off each generated frame, apply the real rules to it and the agent's action, and compare with the next generated frame. Over 5,000 steps at two loops, Flappy Bird had zero wrong game states and zero wrong rewards. Pac-Man had 742 wrong states and 38 wrong rewards.

The exploits read like a speedrunner's notes:

• Breakout: an agent scored 154 points with 26 bricks left, because the model failed to remove corner bricks when they were hit.
• Doom-lite: agents learned to snipe the enemy from 5 cells away.
• Dream-only training: agents trained only inside the world model scored 210 there, against a real maximum of 56, and 28 in the real game. That matches what World Models predicts.
• Doom-lite's vanishing enemy: the model sometimes lost the enemy sprite and never redrew it, copying the previous view instead of reading the minimap. Input noise, as in GameNGen, fixed it.

The fix each time was to record real games with the exploiting agent and fine-tune the world model on them. Each collect-and-fine-tune round closed the hole that agent had found. Somewhere in there, a 26k-parameter network got patched for a brick glitch, which is the most authentic game-development experience a microcontroller has ever had.

Learning on the board

The agents use Stream AC(λ) from "Streaming Deep Reinforcement Learning Finally Works" (Elsayed, Vasan and Mahmood). It learns from one transition at a time: no replay buffer, no batches, no target network. That is what makes on-chip learning possible, since the board can update and then forget each step.

The author wrote the same algorithm in no_std Rust with a hand-written backward pass, and it matches 200 PyTorch updates to within 3.5e-6. On a host machine, a fresh agent inside the Rust int8 Flappy Bird dream averages 6 pipes per episode after 25k steps, 121 after 100k and more than 300 after 250k. The board boots with a fresh learner by default. Press l over USB, or hold B on the Display Pack, to switch to the pretrained actor.

On Hacker News, the author said the world models were trained on an RTX 4060. Training data was 1 to 3 million recorded transitions per game, and the int8 rounding switches on after the first 3 to 4k of 30k training steps, so the int8 model is the one that was trained. A cargo test run checks that the Rust engine reproduces the Python integer reference bit for bit.

What fits in 520 KB

The RP2350 has two cores at 150 MHz, 520 KB of SRAM and 4 MB of flash on the board. Breakout's firmware uses 361 KB of flash and 299 KB of SRAM, including USB, the display driver and the on-device learner. Pac-Man lite is the largest at 600 KB of flash and 460 KB of SRAM, and its per-cell output scores alone take 184 KB.

Most of the compute is int8 dot products. One 32-bit load fetches four weights, and the chip's SMLAD instruction does two multiply-adds at once. Norms and the softmax exp lookup stay in float32 because the RP2350 has an FPU.

You can try it without hardware. The same engine compiles to WebAssembly with no imports. It generates the same episodes as the native build for the same seed, at 1 to 5 ms per frame, with stream-x learning in the browser. On the Pico itself, the README lists the commands to flash any game and agent pair:

cd tinyworldformer-rs
./flash.sh --list
./flash.sh --env doomlite --agent doom_agent_roam_p0.3
./flash.sh --env pong --host

If you build your own tiny models, the useful piece may be src/tinyworldformer/rustexport/. It imports only numpy and torch, and the README walks through training a looped model with quantization-aware training, converting it and writing Rust source plus golden test vectors.

Caveats

• No on-device frame rate yet. The README gives 0.7 to 1.5 ms per frame on a desktop CPU and says the author hasn't measured the Pico's frame rate. In play mode the board deliberately slows to 66 to 150 ms per frame.
• One model per game. Each game has its own world model. The author said on Hacker News that merging them should be easy, but that isn't shipped.
• Tiny worlds only. These are grid games with full state drawn in every frame. The author said open-world games would be hard to keep free of hallucinations.
• No license file. GitHub reports no license on the repository, so treat the code as look-and-learn until one is added.

Key Takeaways

• Tinyworldformer runs an int8 world model (26k to 131k parameters) and an actor (9k to 16k) on a Raspberry Pi Pico 2 W, with no real game code on the chip.
• At two loops, exact held-out frames are 100% for Flappy Bird, 99.99% for Breakout, 99.45% for Pong, 86.4% for Pac-Man lite and 83.6% for Doom-lite.
• A single reused transformer block lets loop count trade speed for accuracy without adding memory, and the Pico stops at two loops.
• Agents trained in the generated world exploited its bugs, scoring 210 in a dream where the real maximum is 56, until recording and fine-tuning on their play closed each hole.
• Stream AC(λ) lets a fresh agent learn on the board one transition at a time. Every game fits in 520 KB of SRAM, Pac-Man lite at 460 KB.
• The Pico frame rate is unmeasured and the repo has no license yet, but a WebAssembly build lets you try it in a browser.

Sources: tinyworldformer on GitHub (README), Hacker News discussion with the author, Raspberry Pi Pico 2 product page, Elsayed, Vasan and Mahmood, Streaming Deep Reinforcement Learning Finally Works, Ha and Schmidhuber, World Models, GameNGen