MIT Paper Claims GPT-6 Astra Fixed Qwen by Editing Weights; Peers Call It Misleading

TL;DR

Four MIT CSAIL researchers published Sparks of Weight-Space Programming on November 2 with a claim that reads like a typo: OpenAI's GPT-6 Astra can read a neural network's raw weights and rewrite them usefully, no training loop involved. It named the arcade game behind a 17,253-weight policy, repaired a stuck reinforcement-learning agent by editing 3.5% of its weights, and lifted Qwen3.5-0.8B from 7.9% to 48.1% on MATH-500 by subtracting one misbehaving direction from 131,072 weights, 0.016% of the model. The Hacker News thread passed 2,900 points. Public review then arrived in under a day, faster than most venues manage: spaCy creator Matthew Honnibal called the framing misleading, and Simon Willison replicated the raw-weights half with a 27B model on his laptop.

What the paper claims

Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab call the technique Weight-Space Programming: an LLM reads and writes weights directly, guided by natural language, with no optimizer in the loop. Weights are the machine code of a neural network. The source code was the training run, and nobody ships that, so reading the floats means reading the binary without a disassembler.

The reading experiments escalate in size. Shown the raw 17,253 weights of four PPO-trained policies, with no tools at all, Astra named the right game for all four (the authors' own 10x10 grid versions of Breakout, Pong, Pac-Man and Freeway), 99% confident on Breakout. Given only the middle four transformer blocks of a 7.2M-parameter character model, input and output layers deleted so the network cannot even be run, it identified the training language in all three cases (Python, romaji, English) and located every copying head, the induction circuit interpretability researchers have mapped by hand for years. It had Python to analyze the tensors, but no way to execute the model.

The writing experiments are the startling half. Astra repaired a Kinetix car policy stuck driving straight into a ramp, 0% to 100% success across 1,024 episodes in a single try, editing 3.5% of 171,664 weights. Asked to author MinAtar game networks from scratch, weights initialized to zero and no rollouts allowed, its one-shot networks beat PPO agents trained for 100 million steps on all three games, 517.4 to 15.8 on Breakout. Against PPO trained five times longer, it won Breakout by 20x, lost Seaquest, and came in just under on Asterix. Hold that asterisk; the rebuttals certainly did.

The Qwen edit, in numbers

Experiment 8 is the headline. Astra got a Python kernel over Qwen3.5-0.8B's weights, about 14,700 practice problems from the GSM8K and MATH training sets, and a budget of forward passes. No gradients, no fine-tuning. The base model loops badly: 85 of its first 96 practice answers ran to the 32,768-token limit. Astra traced the looping to a single direction written by the MLPs of layers 15 and 19, then subtracted it from 64 output columns in each. This is unplugging the one appliance tripping the breaker instead of rewiring the house.

With the full 50-pass budget, MATH-500 went from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval, which measures instruction following, from 21.0% to 62.3%. Answers hitting the token limit fell from 93% to 33%. The budget matters: at 10 and 25 passes, MATH-500 only reached 12.3% and 14.1%. The paper also carries its own control. On Gemma 3 270M, a model that does not loop, Astra found ordinary reasoning errors with no single component behind them and moved MATH-500 by about a point at most, a gain it reported itself as not established. Removing an obstacle is not the same as adding capability, and the authors say so.

The rebuttal: one paper, two toolkits

Honnibal's November 3 post is the critique that stuck. His table of the eight experiments shows the split: the raw-floats-in-context setting, no code, no running the network, appears only in experiments on 17K to 172K weights. Every experiment involving a real language model uses Python, and the Qwen result, the headline one, also ran the model on up to 50 batches of practice problems. His metaphor is a stage magician doing the same trick three different ways while the audience assumes there was one method.

His verdict is gentler than the title. What the Qwen result shows, he writes, is that Astra can devise and carry out tasks a human ML engineer could do, and that models of a year or two ago would have failed all of them. Sean Goedecke walked the same experiment step by step and landed in the same place: competent, genuinely impressive, and recognizably the work of a human researcher running in a loop rather than a new gestalt of weight space. What both reject is the packaging, the suggestion of one deep capability that spans authoring tiny networks and repairing a large one.

A 17GB replication on a laptop

Willison took the part of the claim he found most interesting, the no-tools reading, and tested it with prompts taken verbatim from the authors' public logs. Qwen 3.8 27B, quantized to 17GB, running through LM Studio on his own machine, identified Pac-Man at 72% confidence after 46 minutes and 43,506 reasoning tokens, Pong at 75% after just over an hour, and Breakout at 88% after 52 minutes. It missed Freeway, guessing Pac-Man. Kimi K3.1 Pro got Breakout right in under 10 minutes.

His conclusion cuts both ways. The raw-weights result is real, reproducible, and within reach of a 17GB model on consumer hardware, which makes it more solid and less magical at once. As he put it, he is "not ready to declare that the age of source code is over and our future involves incomprehensible machines editing their own model weights in incomprehensible ways."

Why a builder should care

The reusable part is the harness. A Python kernel over tensors plus a metered budget of forward passes is the same shape as any agent over an API, and the design questions are the usual ones: what tool surface to expose, what to meter, what counts as a verifiable result. The authors are now building a benchmark and harness for exactly this, and they released what you need to check them: full logs of every prompt, reply and evaluation in the lm_understand-logs repo, the edited Qwen3.5-0.8B on Hugging Face, re-evaluated by Ahsen Khaliq with a live demo.

The security angle is the one worth watching, in the authors' own pitch: if a model can audit weights for failure modes or backdoors without running benchmarks, detection stops depending on having the right test set. Nobody has shown that at scale here. The sycophancy experiment, the closest thing to an audit task, failed from weights alone and only worked once Astra could see attention patterns. But the direction is on the table, with code attached.

Caveats

This is a blog post, not peer review, and the evaluations are the authors' own; Khaliq's re-run covers the Qwen edit only. One frontier model was tested, so harness sensitivity is unknown. The beats-PPO-500M claim holds for Breakout alone. The reading tasks that were independently reproduced were within reach of a laptop model, and the one large-model repair targeted a looping model with a documented, known failure. On the other side of the ledger: the failures are published alongside the wins, the logs are public, and the argument got sharper within 24 hours of landing.

Key Takeaways

• MIT CSAIL's Sparks of Weight-Space Programming (Nov 2) claims GPT-6 Astra reads and writes neural network weights directly: 4/4 naming a policy's game from raw weights, 0% to 100% on a stuck RL policy, and one-shot MinAtar networks beating PPO-100M on all three games.
• The headline result: editing 131,072 weights (0.016%) of Qwen3.5-0.8B, with 50 practice batches and no gradients, raised MATH-500 from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21.0% to 62.3%, by removing the looping direction in layers 15 and 19.
• Matthew Honnibal's rebuttal (Nov 3): only the 17K-172K weight experiments truly read raw weights; every language-model experiment used Python, and the Qwen fix also ran 50 practice batches. One paper, two toolkits, stage-magician framing.
• The controls matter: the edit did not improve non-looping Gemma 3 270M beyond noise, and sycophancy detection from weights alone failed until attention patterns were shown.
• Simon Willison replicated the no-tools reading with Qwen 3.8 27B on his laptop (3 of 4 games, 46 to 65 minutes each), so the reading skill is real, reproducible, and not exclusive to frontier models.
• Full logs, the edited model, and a live demo are public; the authors are building a benchmark and harness for Weight-Space Programming next.

Sources: Sparks of Weight-Space Programming (Takehi, Teoh, Zhang, Khattab, MIT CSAIL), Honnibal: the Astra weights post is misleading, Goedecke: LLMs are not superhuman at reading and writing neural network weights, Willison: Qwen 3.8 27B on my laptop can interpret model weights, too, Experiment logs, Edited Qwen3.5-0.8B on Hugging Face, Hacker News discussion