Unsloth Shrank Qwen3.8-27B by 89%. 72% of It Survived.

TL;DR

Unsloth Dynamic 3.0 shipped on August 19 as a new generation of GGUF quantization, and the first model to get it is Qwen3.8-27B. The headline claim from Unsloth: more than 10% better top-1 accuracy at the same file size than any other quant provider. The number people will repeat is the 1-bit build, UD-IQ1_S at 6.19GB against a 54.66GB BF16, about 89% smaller, which Unsloth says still picks the same next token as full precision roughly 72% of the time. None of this uses quantization-aware training. It is pure post-training quantization, a better calibration set, and smarter per-layer choices. There is a real catch, and it is in the file listing rather than the blog post.

What actually changed

Dynamic quantization has always been Unsloth's pitch: instead of hammering every layer down to the same bit width, pick a different width per layer based on what that layer can survive. Dynamic 3.0 keeps that idea and rebuilds the inputs to it.

Three concrete changes, per the docs. A much higher-quality imatrix calibration dataset drawn from diverse sources and deliberately refined for agentic coding, chat, and multilingual work. Improved layer selection. And a pile of additional quantization techniques applied on top.

The part worth pausing on is what they did not do. Unsloth is explicit that they do not train on the calibration dataset, and that there is no QAT or QAD anywhere in the pipeline. Everything is post-training quantization, and the imatrix file itself is published for anyone to test or reuse.

That matters because the reference point for "good small quant" has been Google's quantization-aware trained Gemma 3, where the low-precision behavior is baked in during training. QAT is the expensive path and only the lab that owns the weights can walk it. PTQ is the path anyone can run after the fact, on a model somebody else released. Unsloth is arguing the cheap path has caught up.

The 1-bit number, in context

The specific claims for the new small quants, straight from the docs. UD-IQ1_S is 6.2GB without the multi-token prediction head and retains around 72% top-1 accuracy while being 89% smaller. UD-Q2_K_XL, at 9.83GB, is around 8% more accurate on top-1 than the next best provider, and Unsloth notes it now produces a working HTML program with one small JavaScript bug where the previous generation simply broke.

For a sense of the ceiling, the docs cite their Kimi-K3 work, where a Dynamic 1-bit build reached roughly 78.9% top-1 while being 62% smaller. Different model, different starting point, so do not stack the two numbers next to each other.

One honest caveat before anyone starts downloading 1-bit weights to their laptop: on the Hacker News thread, a commenter who ran 1-bit, 2-bit, and bonsai quants against closed eval sets called them "essentially useless" for their case, with small errors accumulating until the output went off the rails. Their suggestion, and it is a reasonable one, is that a 9B-class model at a sane precision will beat a 27B lobotomy for most workloads. The 1-bit builds are the drag-racing numbers. The 3-bit and 4-bit builds are the daily driver.

Divergence-300, or why top-1 was never enough

Unsloth also introduced a new metric with this release, and it is the most interesting technical thing in the announcement.

Top-1 accuracy asks a narrow question: for a given prompt, does the quantized model's single highest-probability next token match what BF16 would have picked? That is an argmax on one prediction. It is a bit like grading a sat-nav on whether it announces the same first turn as the reference route, then never checking whether you arrived. One matching turn tells you almost nothing about the trip.

Divergence-300 @32 extends the check. Unsloth built a set of 300 held-out prompts, explicitly not in the calibration dataset, pulled from Terminal-Bench 2.1, DeepSWE, Harbor, MathArena 2025-26, and a batch of non-Latin and long-document prompts. Then they run greedy argmax decoding for 32 tokens on BF16 and on every quant from every provider, and compare the trajectories. In the thread, Unsloth confirmed a @512 variant is coming.

The reason to build this at all is the argument in Accuracy is Not All You Need, which Unsloth cites directly: benchmark scores can hold steady through quantization while individual answers "flip" in both directions, so a stable MMLU number can hide a model that behaves quite differently from the original. KL divergence correlates with those flips. Divergence-300 is the same instinct pushed out to a multi-token horizon.

It is also, conveniently, a metric Unsloth designed, running on a dataset Unsloth assembled, scored by Unsloth. The methodology is public and the held-out sourcing looks careful, but nobody outside has reproduced it yet. Treat the numbers as vendor-reported until someone does.

The catch: your big quant probably did not change

Here is the part the headline number papers over. Dynamic 3.0 did not replace the whole ladder. The docs say that against the older UD-2 on unseen Wikitext and code, the improvement is large for the small quants and "the bigger ones not so much, so we still use our old UD-2 for the larger quants."

So the repo is now a mix of two generations, and Unsloth did not publish where the line falls. A commenter on Hacker News checksummed their UD-Q8_K_XL from four or five days earlier against the freshly announced one and got an identical SHA-256, which is a very efficient way to discover you were already running the good stuff. Filenames carry no version, so the only way to know what you have is to hash it.

The MTP head is gone from the small files

Second gotcha, and this one will bite people mid-download. Unsloth stripped the multi-token prediction module out of the quants at 8.37GB and below, saving roughly 500MB of disk. On an 8GB machine, which is exactly who those files are for, half a gig is the difference between loading and not loading.

If you pull UD-IQ2_XXS and your runtime throws an error about a missing MTP head, that is why. The module ships separately as a Q4_0 file in the same repo, 1.37GB, and you can attach it if you want speculative decoding back. Quants above that threshold still have it embedded.

What this means for your box

The new files run on llama.cpp and most other GGUF engines, plus Unsloth's own desktop app. Practical placements from the release and the thread: Unsloth suggests UD-IQ3_XXS at 10.93GB or UD-Q2_K_XL for a 16GB machine, and one user reports 14 tokens per second at full context on a 16GB card with UD-Q3_K_XL. Another running a 2-bit build for light coding measured about 15 tokens per second over the first 32k of context.

The demand is not theoretical. Unsloth says the Qwen3.8 GGUFs took over 5.1 million downloads in five days, and the 27B repo alone shows more than 4.3 million downloads in the last month against a repo that was only created on August 13.

Key Takeaways

• Dynamic 3.0 is out, Qwen3.8-27B first. Unsloth claims more than 10% better top-1 accuracy at the same file size than any other provider, from a new imatrix calibration set and improved per-layer selection.
• No QAT, no QAD, pure post-training quantization. The imatrix is published, and Unsloth states it does not train on the calibration data. That is the cheap path beating the expensive one on its own turf.
• The 1-bit build is 6.19GB against a 54.66GB BF16 and retains roughly 72% top-1 accuracy per Unsloth. Independent testers on Hacker News found sub-2-bit quants unusable for real work, so treat 1-bit as a benchmark flex, not a daily driver.
• Divergence-300 @32 is the metric to watch. Three hundred held-out prompts from Terminal-Bench 2.1, DeepSWE, Harbor and MathArena, scored over 32 greedy tokens against BF16 instead of a single argmax. A @512 version is coming.
• Larger quants are still UD-2. Unsloth kept the old methodology where the new one did not help, did not publish the cutoff, and does not version filenames. Hash your file if you need to know which you have.
• MTP is stripped below 8.37GB. Roughly 500MB saved, available as a separate 1.37GB Q4_0 module if you want speculative decoding back.

Sources: Unsloth Dynamic 3.0 GGUFs documentation, unsloth/Qwen3.8-27B-GGUF on Hugging Face, Hacker News discussion, Accuracy is Not All You Need (arXiv 2407.09141), Google Developers Blog on Gemma 3 QAT