Baidu's Unlimited-OCR Reads Dozens of Pages in One Pass, and the KV Cache Never Grows

TL;DR

Baidu open-sourced Unlimited-OCR on June 22, an MIT-licensed document-parsing model that does something the current crop of OCR models cannot: it reads a whole multi-page document in one forward pass instead of slicing it into pages and running the model once per page. It is a small model (3B total parameters, roughly 500M activated in an MoE setup), it scores 93% on OmniDocBench v1.5 (about 6 points above the DeepSeek-OCR baseline it builds on), and the headline architecture trick, Reference Sliding Window Attention, keeps the KV cache constant across the entire decode. It ships with weights, a paper, and ready-made paths for vLLM, SGLang, Ollama, and llama.cpp.

Why "one-shot" is the part that matters

Every production OCR pipeline you have ever shipped does the same ugly thing: it chops a PDF into pages, runs the vision model on each page independently, then tries to stitch the results back together. Tables that span a page break get mangled. Footnotes lose their anchors. A heading on page 3 has no idea what section it belongs to. You spend most of your engineering effort on the glue, not the model.

Unlimited-OCR's pitch is that you feed it the whole stack and it reads it in one pass, with one continuous context. The paper reports good fidelity at 20 pages and still-usable results past 40. That means the model sees the document the way a human reader does, in order, with memory of what came before, so cross-page structure survives instead of getting shredded at every page boundary.

The trick: a KV cache that does not grow

Reading dozens of pages in one pass should be impossible on a small model, because the standard transformer KV cache grows linearly with sequence length. Forty pages of dense text is a lot of tokens, and a linearly-growing cache is exactly what blows up your VRAM and tanks your throughput on long inputs. This is the wall that forces everyone to chunk in the first place.

Baidu's answer is Reference Sliding Window Attention (R-SWA), which replaces the standard attention layers in the decoder. The claim from the technical report is that R-SWA holds the KV cache at a constant size for the entire decode, no matter how long the document runs. Combine that with a high-compression vision encoder inherited from the DeepSeek-OCR lineage, and you get a model that can keep reading without the memory cost climbing page after page.

The numbers worth quoting

• 3B total parameters, ~500M activated (mixture-of-experts), per the technical report. Small enough to be interesting for self-hosting.
• 93% on OmniDocBench v1.5, roughly 6 points over the DeepSeek-OCR baseline. The report also cites 93.92% on the newer v1.6.
• 32,768-token max length, which is what makes dozens of pages fit in a single context.
• A reported 12.7% throughput gain over the DeepSeek-OCR baseline (around 5,580 tokens/sec in the paper's measurement), thanks to the constant-cache decode.
• MIT license. Download, self-host, modify, ship commercially, no per-token bill.

What you can actually run it on

This is the part that should make homelab and indie builders sit up. Baidu did not ship a research curiosity that only runs on their stack. The release lists support for Hugging Face Transformers, vLLM, SGLang, Ollama, llama.cpp, and Docker, plus a batch infer.py for local runs. At 3B parameters in BF16 it is a comfortable fit on a single consumer GPU, and the llama.cpp path opens the door to quantized CPU and Apple-silicon inference for anyone who wants to parse documents without a datacenter.

The honest caveats

Two things to keep in mind. First, the splashy benchmark numbers and the R-SWA mechanism live in the arXiv technical report, not on the model card, so treat them as the authors' claims until the community reproduces them on its own document sets. Second, "acceptable past 40 pages" is the authors' framing, and OCR quality on long, table-heavy, multi-column documents is exactly where these models tend to fray. The right move is to throw your own worst PDFs at it before you rip out a pipeline that works.

Key Takeaways

• Unlimited-OCR (MIT, released June 22) parses whole multi-page documents in a single forward pass instead of chunking page by page.
• It is a small model: 3B total parameters with roughly 500M activated via MoE.
• Reference Sliding Window Attention keeps the KV cache constant across the entire decode, which is what makes long-document, single-pass reading feasible.
• Reported accuracy is 93% on OmniDocBench v1.5, about 6 points above the DeepSeek-OCR baseline it extends.
• Ships with weights and ready paths for vLLM, SGLang, Ollama, and llama.cpp, so it self-hosts on a single consumer GPU.
• Benchmarks and the architecture trick come from the technical report, so verify on your own documents before trusting it in production.

Sources: Baidu Unlimited-OCR (GitHub), arXiv 2606.23050 (technical report), Hugging Face paper page, AI Weekly