Can You Actually Run GLM-5.2 at Home? The 744B VRAM Math for RTX and Mac

TL;DR

GLM-5.2 is a 744B-parameter mixture-of-experts model (40B active per token) with a 1M-token context window. In full BF16 it is 1.51TB. Unsloth's dynamic GGUFs crush that to 217GB at 1-bit and 239GB at 2-bit, but this is still a model you fit across memory, not on a single gaming card. The realistic home build is one GPU plus a pile of system RAM with MoE offloading, not a wall of 3090s. If you want it fully resident, the answer is a 512GB Mac Studio or a small stack of RTX Pro 6000 Blackwells. Here is the full math.

What GLM-5.2 Actually Is

Per the Unsloth docs, GLM-5.2 is a sparse mixture-of-experts model: 744B total parameters, but only 40B active on any given token. That split is the whole story for hardware. The active count (40B) sets how fast it runs. The total count (744B) sets how much memory you need, because every expert has to be loaded somewhere even though only a few fire per token. You do not get to skip the cold experts.

The context window is 1 million tokens, which sounds great until you price the KV cache. More on that below.

The Quants and What They Weigh

Unsloth's dynamic GGUFs do not quantize every layer to the same bit width. They keep the sensitive layers high-precision and squeeze the rest, which is why a "2-bit" build is actually usable. Here is the disk footprint and the realistic memory you need to run each one (weights plus a working KV cache), straight from the model card:

Quant            On disk     RAM+VRAM to run
Full BF16        1.51 TB     ~1.6 TB
1-bit (IQ1_S)    217 GB      223 GB
2-bit (IQ2_M)    239 GB      245 GB
3-bit            ~270 GB     290 to 360 GB
4-bit            ~340 GB     372 to 475 GB
5-bit            ~470 GB     570 GB
8-bit            ~710 GB     810 GB

The 2-bit dynamic quant is the sweet spot: it keeps most of the model's quality while fitting in a quarter of a terabyte. The ranges on the 3-bit and 4-bit rows are not noise, that spread is the context budget. The low end is the model with a short context, the high end is the model with room to think.

How Many RTX Cards Does It Take?

This is the question everyone asks, so here is the blunt answer. If you wanted every weight resident in GPU VRAM with nothing offloaded to system RAM, you would need this many cards (counting weights plus a modest KV cache, rounding up):

Card                     VRAM    2-bit(245GB)  4-bit(~440GB)  8-bit(810GB)
RTX 3090                 24 GB       11x            19x            34x
RTX 4090                 24 GB       11x            19x            34x
RTX 5090                 32 GB        8x            14x            26x
RTX Pro 6000 Blackwell   96 GB        3x             5x             9x

Read those consumer-card numbers as a reality check, not a shopping list. Eleven 3090s is a 3.3kW space heater that needs server PCIe risers, a dedicated circuit, and a tolerance for pain. Nobody should build that to run one model.

The RTX Pro 6000 Blackwell (96GB GDDR7) is the only card here where small multi-GPU is sane: 3 of them hold the 2-bit quant fully in VRAM with room for context, and 5 to 6 handle the 4-bit at long context. That is a real, if expensive, workstation. Everyone else should stop thinking about all-VRAM and start thinking about offload.

The Build Most People Should Actually Do

Because only 40B parameters are active per token, you do not need the whole model on the GPU. With MoE offloading in llama.cpp, you keep attention and the active compute path on the GPU and park the cold experts in system RAM. Unsloth's own guidance: GLM-5.2 runs on a single 24GB GPU plus 256GB of system RAM.

So the realistic homelab recipe is not eleven GPUs. It is:

• One RTX 3090, 4090, or 5090 for the active path and KV cache.
• 256GB of DDR5 system RAM to hold the offloaded experts (for the 2-bit quant).
• llama.cpp with the experts routed to CPU/RAM.

It will not be fast. You are bottlenecked by system RAM bandwidth for the offloaded experts, so expect single-digit to low-double-digit tokens per second depending on your memory speed. But it runs a 744B frontier-class model on a machine you can actually build, and the 5090's 32GB gives you the most headroom for KV cache of the consumer options.

Every Mac Option, Honestly

Apple Silicon's unified memory is the cleanest way to hold a model this big, because the GPU can address all of it. The catch is that the smaller Macs simply cannot fit GLM-5.2, and pretending otherwise wastes your money. Leaving roughly 10 to 15 percent for macOS, here is what each tier can do:

Mac (max unified memory)        Usable    GLM-5.2?
Mac mini / M4 Pro      64 GB    ~57 GB    No (1-bit alone is 223GB)
M4 Max (Studio / MBP) 128 GB   ~115 GB    No
Mac Studio M2 Ultra   192 GB   ~172 GB    No (1-bit needs 223GB)
Mac Studio M3 Ultra   256 GB   ~245 GB    Yes: 2-bit, tight, short context
Mac Studio M3 Ultra   512 GB   ~470 GB    Yes: 3-bit big ctx, or 4-bit

The honest takeaway: only the 256GB and 512GB M3 Ultra Mac Studios can hold this model. The 64GB, 128GB, and even the 192GB M2 Ultra cannot fit the smallest 1-bit quant without spilling to SSD, which turns "running" into "technically loading." On a 256GB machine you raise the Metal wired-memory limit and run the 2-bit at a modest context. On a 512GB machine you get real breathing room: 3-bit with a large context window, or 4-bit at a tighter one. Memory bandwidth (around 800GB/s on M3 Ultra) means you will see better tokens per second than a RAM-offloaded PC, with far less assembly.

Realistic Context Sizes for Your VRAM

The 1M context number is real but aspirational. The KV cache is what eats your remaining memory after the weights load, and at full precision it is brutal on a model this size. GLM-5.2 uses grouped-query attention, which keeps the KV cache smaller than a naive estimate, but long context is still measured in tens of gigabytes. Two levers matter:

• Headroom. Whatever memory is left after the weights is your context budget. That is exactly why the 4-bit row spans 372 to 475GB: roughly 100GB of that is KV cache for long context.
• KV quantization. Running the cache at q4_1 instead of fp16 stretches usable context by about 3.2 to 3.5x for the same memory. It is the single highest-leverage flag for long context.

Pragmatic targets, treating these as estimates that depend on your exact build:

Setup                                  Realistic context
256GB Mac, 2-bit (weights eat it all)  8K to 32K (quantize KV to reach top)
~50-100GB headroom (512GB Mac 3-bit)   128K to 256K with q4/q8 KV
5-6x Pro 6000, 4-bit, 100GB+ for KV    256K to ~1M with q4 KV cache

Translation: if you bought a 256GB Mac to run the 2-bit, do not expect 1M tokens. Plan for tens of thousands and quantize the cache. Full 1M context on GLM-5.2 is a thing you reach with a 512GB Mac or a Pro 6000 stack and aggressive KV quantization, not on a single 24GB card.

How To Run It

The path of least resistance is llama.cpp with the Unsloth GGUFs. Pull the 2-bit dynamic quant, offload experts to RAM if you are GPU-poor, and quantize the KV cache for context:

llama-cli -hf unsloth/GLM-5.2-GGUF:UD-IQ2_M   --cache-type-k q4_1 --cache-type-v q4_1   --ctx-size 65536   --temp 1.0 --top-p 0.95   --n-gpu-layers 99 --override-tensor "exps=CPU"

The --override-tensor "exps=CPU" flag is the magic: it keeps the expert tensors in system RAM while everything else runs on the GPU. Drop it if you have enough VRAM to hold the whole quant. Unsloth recommends temperature 1.0 and top_p 0.95, and thinking mode is on by default (disable it with --chat-template-kwargs '{"enable_thinking":false}' if you want faster, terser answers). For a friendlier path, Unsloth Studio auto-detects your GPUs and handles the RAM offload for you.

The Other Path: Full FP8 on a Wall of 4090s

Everything above assumes the Unsloth route: quantize hard, park the cold experts in system RAM, accept a quality hit and slow speed. There is a second answer that just landed, and it is worth knowing even if you will never build it. A developer (renning22) got the full GLM-5.2-FP8 running across a cluster of RTX 4090s with no quantization compromise at all. Not a 2-bit GGUF. The complete ~753GB FP8 weights, the same build a datacenter would serve.

The trick is not memory, it is kernels. GLM-5.2 uses DeepSeek-Sparse-Attention (DSA), and the stock sglang and vLLM stacks gate those sparse-attention kernels to Hopper (H100/H200) and Blackwell (B200) only. On an Ada card like the 4090 (sm_89) there is no fallback, so the stack hard-crashes. This project ships a drop-in ada_dsa.py that ports the whole DSA kernel stack down to Ada: the lightning-indexer GEMM, the top-k and page-mapping, and the MLA sparse decode, all rewritten in Triton plus a non-WGMMA tilelang path, and patched in only on sub-Hopper GPUs. The author claims it is the first time a DSA model has run correctly on Ada, validated kernel by kernel down to roughly 1e-6, with 0.999999 cosine similarity against the reference on the live model's real tensors.

Here is the honest hardware bill, because this is the wall of GPUs the rest of this article told you to skip:

GPU         VRAM    GPUs    Layout                Status
RTX 4090    48 GB    24     TP=8 x PP=3 (3 nodes)  proven
RTX 4090    24 GB   ~40-48  TP=8 x PP=5-6          estimate
RTX 5090    32 GB   ~32     TP=8 x PP=4 (4 nodes)  estimate

The only tested config is 24 modded 48GB 4090s across three nodes. Note the 48GB: those are the aftermarket memory-doubled 4090s, not the stock 24GB cards. On stock 24GB 4090s you would need roughly 40 to 48 of them, and the 5090 number is an estimate too because consumer Blackwell (sm_120) is not covered by the stock DSA kernels either and needs the same port. Either way, this is a multi-node cluster, not a desktop.

And the payoff for all that iron? About 10 tokens per second single-stream with CUDA graphs, around 2.5 in eager mode. That is interactive, and it is the full-precision model, but it lands in the same ballpark as a single 24GB GPU plus 256GB of RAM doing MoE offload. So the real lesson is not "go build 24 4090s." It is that the wall on this model is never raw card count, it is memory bandwidth and attention kernels. The project is a genuinely impressive piece of kernel engineering and a clean reference for anyone trying to run DSA models on consumer silicon. But for actually using GLM-5.2 at home, the single-GPU offload path or a big Mac is still the sane move.

Key Takeaways

• 744B total, 40B active. The total sets your memory bill, the active count sets your speed. You must hold every expert even though only a few fire per token.
• Forget the wall of consumer GPUs. Fitting the 2-bit quant purely in VRAM needs 11 RTX 3090s or 8 RTX 5090s. That is a science project, not a build.
• One GPU plus 256GB RAM is the real home build. MoE offloading runs GLM-5.2 on a single 24GB card. A 5090's 32GB just buys you more KV headroom.
• RTX Pro 6000 Blackwell is the only sane all-VRAM card. Three hold the 2-bit, five to six handle the 4-bit at long context.
• Only big Macs qualify. The 256GB M3 Ultra runs the 2-bit; the 512GB runs 3-bit at large context or 4-bit. Anything 192GB or smaller cannot fit it.
• 1M context is not free. Quantize your KV cache to q4_1 for roughly 3x the tokens per gigabyte, and size your context to the memory left after the weights load.
• Full FP8 on 4090s is possible, but it is a kernel feat, not a home build. Porting GLM-5.2's sparse-attention kernels to Ada runs the complete 753GB model on 24 modded 48GB 4090s at about 10 tok/s, the same speed as single-GPU offload. Impressive engineering, wrong tool for home use.

Sources

• Unsloth GLM-5.2 docs and dynamic GGUF model card: unsloth.ai/docs/models/glm-5.2 and huggingface.co/unsloth/GLM-5.2-GGUF
• NVIDIA RTX Pro 6000 Blackwell (96GB GDDR7) specifications
• Apple Mac Studio M3 Ultra unified memory configurations (up to 512GB)
• GLM-5.2 on RTX 4090, full FP8 via ported DSA kernels (renning22): github.com/renning22/glm-5.2-4090