A 284B Model on Your MacBook: antirez's ds4 Fits a Frontier LLM in One C File

TL;DR

Salvatore Sanfilippo (antirez), the creator of Redis, got the frontier running on a laptop. His new project ds4 is a single-file C inference engine that runs DeepSeek V4 Flash, a 284-billion-parameter open-weight model, locally on a 128GB Mac over Apple Metal. It leans on a custom 2-bit quantization to squeeze the whole model into roughly 70GB, keeps a 1-million-token context (with the KV cache persisting across restarts), and on a 128GB M3 Max clocks about 26 tokens/sec generation at 50 watts, per early reviews. No Python, no GGML, no 40-dependency install that breaks on a Tuesday. One file.

The model: DeepSeek V4 Flash

DeepSeek V4 Flash is the small, fast member of the V4 family, but "small" is relative: 284B total parameters with only 13B active per token in a Mixture-of-Experts design spread across hundreds of experts. Its trick is a hybrid attention architecture that cuts the KV cache to roughly a tenth of the previous generation, which is exactly what makes a million-token context survivable on consumer memory. It is open-weight on Hugging Face, and DeepSeek's own API serves it at around 72 tokens/sec in reasoning mode. The point of ds4 is that you no longer need DeepSeek's API to touch it.

The 2-bit magic (and why it usually fails)

The reason a 284B model normally cannot live on a laptop is arithmetic: at 16 bits per weight, the weights alone want around 568GB. Two bits per weight cuts that by 8x, down to the ~70GB ds4 needs. The catch is that 2 bits is only four possible values per weight, and naive 2-bit quantization usually turns a model into confident nonsense.

Here is the analogy. Imagine recording a full orchestra but only allowing four volume levels per instrument. Do it carelessly and you get noise. But if you pick those four levels carefully, tuning them against the original recording bar by bar, the symphony still comes through. That is what ds4 does: instead of a stock GGUF Q2 format, it uses a purpose-built quantization mix validated against DeepSeek's own logits, so the compressed model's outputs stay close to the real one's. The compression is aggressive; the calibration is what keeps it honest.

The engine: one C file, no ceremony

The part that will make systems programmers grin is ds4 itself. It is a single C file that talks directly to Apple Metal, with no GGML, no PyTorch, no runtime abstraction layers. antirez wrote a bespoke engine for exactly one model family rather than a general framework, and that focus is the whole advantage: fewer moving parts, less overhead, and code you can actually read in an afternoon. The repo also targets CUDA and ROCm, so it is not Mac-only, but Apple Silicon's unified memory is the natural home because the GPU can address all 128GB directly.

Running it: the numbers and the steps

On a 128GB MacBook Pro M3 Max, reviewers measured roughly 58 tokens/sec prefill and 26-27 tokens/sec generation at a 32K context on the 2-bit build, drawing about 50 watts at peak. That is genuinely usable for a frontier-class model on battery-class hardware. The weights sit near 70GB, a full million-token KV cache adds roughly 26GB, and the cache survives restarts, so you are not re-ingesting a giant prompt every session. Practically, you get 64K to 100K tokens comfortably in RAM before the engine leans on SSD for the very long contexts.

Setup is refreshingly short: clone github.com/antirez/ds4, download the DeepSeek V4 Flash weight files, compile ds4 (you need the Metal toolchain), and launch its OpenAI-compatible API server. Reviewers put the whole thing at about 30 minutes. Because the server speaks the OpenAI API, anything you already point at a local endpoint just works.

The honest caveats

This is a frontier model wearing a very tight corset. Two-bit quantization, even a well-calibrated one, gives something up versus the full-precision model, and "validated against logits" is not the same as "identical on your hard prompt"; benchmark it on your own tasks before trusting it for anything load-bearing. You also need a 128GB Mac, which is roughly a $7,500 machine, so "runs on a laptop" means a very expensive laptop. And ds4 is a focused personal project, not a hardened product: a single C file is a feature until you hit an edge it does not handle. But as a proof that a 284B frontier model can run, fast, on hardware you can buy at a store, it is a genuine milestone for local AI.

Key Takeaways

• ds4, by Redis creator antirez, is a single-file C engine that runs the 284B DeepSeek V4 Flash locally on a 128GB Mac over Metal, no GGML or Python.
• Custom 2-bit quantization (validated against DeepSeek's logits, not stock GGUF Q2) shrinks the weights from ~568GB at FP16 to ~70GB.
• It keeps a 1M-token context with a KV cache that persists across restarts, and runs ~26 tok/s generation on an M3 Max at about 50 watts.
• Setup is ~30 minutes and it exposes an OpenAI-compatible API, so existing local-endpoint tooling works unchanged.
• Caveats: 2-bit costs some quality, it needs a ~$7,500 128GB Mac, and it is a focused personal project, not a general framework. Test it on your own prompts.

Sources: Flowtivity, antirez/ds4 (GitHub), andrew.ooo (review), DeepSeek V4 Flash (Hugging Face)