PrismML Squeezed a 27B Model to 3.9GB. It Runs on an iPhone and Keeps 90% of Its Smarts.

TL;DR

PrismML just did something that sounds like a rounding error: it took Qwen3.6-27B, a 27-billion-parameter model that eats about 54GB of memory in full precision, and shrank it to a 3.9GB file that runs on a phone. Bonsai 27B is not a new model. It is an extreme quantization of an existing open-weight one, published on Hugging Face under Apache 2.0. Two builds ship: a ternary version at 5.9GB that keeps 94.6% of the original's benchmark average, and a 1-bit version at 3.9GB that keeps 89.5% and fits inside an iPhone 17 Pro's memory budget at roughly 11 tokens a second. The interesting part is not the marketing milestone. It is the tradeoff underneath it.

What PrismML actually shipped

PrismML is selling one idea: to run a big model locally, you do not need a smaller model, you need fewer bits per weight. Bonsai 27B keeps every one of Qwen3.6-27B's roughly 27.8 billion weights (about 24.8B in the language layers, a 0.46B vision tower, the rest embeddings). What changes is how each weight is stored. A full-precision weight is a 16-bit number. Bonsai rounds nearly all of them down to one of three values, or in the aggressive build, one of two.

• Ternary: each weight becomes -1, 0, or +1, with a shared scale factor per group of weights, for a true 1.71 bits per weight. 5.9GB on disk, 94.6% of the full-precision benchmark average retained.
• 1-bit: each weight is just -1 or +1 (no zero), same group scaling, 1.125 bits per weight. 3.9GB, 89.5% retained. This is the build that fits on a phone.

For reference, even a competent 4-bit quantization of this model lands around 18GB, comfortably past what a phone will hand to a single app. Both builds carry the full 262K-token context and keep a 4-bit vision tower, so they stay multimodal.

How you fit 27 billion weights in 1.1 bits each

The number to sit with is 1.125 bits per weight. That is barely more than a coin flip of information for each of 27.8 billion parameters, and the model still writes code and does math. The reason it survives is that the sign of a weight matters far more than its exact magnitude, and the magnitude can be recovered approximately from a single scale value shared across a block of weights.

Think of each weight in the original model as a precise elevation on a map, stored to several decimal places. Ternary throws away the decimals and keeps only whether each point is uphill (+1), flat (0), or downhill (-1), then writes one steepness number for each small block of the map. The 1-bit build is harsher still: every point is just uphill or downhill, no flat. You lose the fine contours, but the overall shape of the terrain, the part the model actually navigates by, mostly survives.

That per-group scale is the load-bearing detail. "Is this weight positive or negative" plus "how loud is this block" reconstructs something close to the original activations. To claw speed back, PrismML ships a speculative-decoding drafter it calls DSpark, which lets the model draft its own tokens and verify them in a batch, worth about 1.37x on the CUDA serving path.

The catch: 90% of the brain, not 100%

Ninety percent is a real number, and the missing tenth is not spread evenly. On PrismML's own 15-benchmark suite (math, coding, knowledge, agentic tasks, and vision), run in thinking mode and including names like LiveCodeBench and MATH-500, the ternary build's 94.6% average sits close enough to full precision that most people will not feel it. The 1-bit build's 89.5% is where you start negotiating.

Early hands-on reports line up with that. In the Hacker News thread, testers found Bonsai 27B beats Gemma 4 12B on math and coding, comes in slightly behind on knowledge and tool calling, and drops off noticeably on vision, which tracks with a vision tower squeezed to 4 bits. A few users hit the classic Qwen failure mode where the model spirals into a reasoning loop or emits a wall of punctuation instead of an answer. Quantize hard enough and the seams show up exactly where you would predict: the fuzzy, low-margin tasks, not the crisp ones. Independent benchmarks are still pending, so treat the vendor's averages as a starting point, not a verdict.

Speed: an iPhone and a 5090 run the same 3.9GB file

The headline is portability, not raw throughput. The same 3.9GB 1-bit file runs across a startling range of hardware, because once it fits in memory the rest is arithmetic. An iPhone 17 Pro Max generates around 11 tokens a second. An Apple M5 Pro does about 44, an M5 Max about 66, and an NVIDIA RTX 5090 hits up to 163. It runs on llama.cpp (CUDA and Metal) and MLX, the two runtimes most local setups already have installed.

Your iPhone can now hold a 27-billion-parameter model. Whether it can hold a charge while running one is a separate research program. Eleven tokens a second is not fast, it is conversational-if-you-are-patient. But it is a 27B-class model answering entirely on-device, with no network, no API key, and no per-token bill.

Why this matters for builders

For anyone building local or self-hosted agents, this moves the floor. Until now, "runs on a laptop" mostly meant a 7B to 14B model and the quality hit that comes with it. Bonsai 27B puts a 27B-class reasoner, with tool use and a long context, on hardware you already own, under a license that lets you ship it inside a product. On a laptop it clears roughly 26 to 66 tokens a second depending on the chip, enough to drive a real agent loop rather than a demo.

The caveats are the standard ones for aggressive quantization: verify on your own tasks, because the 10% you lose might be exactly the 10% you needed, and watch vision-heavy workloads in particular. But the direction is hard to miss. The distance between "the model I can call over an API" and "the model I can hold in my hand" just got a lot shorter.

Key Takeaways

• Bonsai 27B is an extreme quantization of Qwen3.6-27B, not a new pretrain: 1-bit (3.9GB, 1.125 bits/weight, 89.5% retained) and ternary (5.9GB, 1.71 bits/weight, 94.6% retained), both Apache 2.0.
• The 1-bit build fits a 27B-class model into an iPhone 17 Pro's memory budget and runs at about 11 tokens a second, a first for this model class on a phone.
• The trick is storing only each weight's sign plus one shared scale per block; the sign carries most of the signal, the magnitude is approximated back.
• You pay for it on fuzzy tasks: vision (4-bit tower) and knowledge slip most, math and coding hold up, and early testers report occasional Qwen-style reasoning loops.
• The same 3.9GB file runs on llama.cpp and MLX from an iPhone (~11 tok/s) to an RTX 5090 (~163 tok/s), with speculative decoding for extra speed.
• For local and self-hosted builders, a shippable 27B-class agent now fits on hardware you already own. Verify on your own workload before trusting the vendor averages.

Sources: PrismML: Bonsai 27B, MarkTechPost, Hugging Face (1-bit MLX), Hugging Face (Ternary GGUF), Hacker News discussion