Cohere's New 2.4B VLM Loses 17 of 22 Rows in Its Own Table

TL;DR

Cohere Labs released North Micro Vision Instruct on August 12: a 2.4B Apache 2.0 vision language model built specifically for documents. It pairs a 400M native-resolution vision encoder with a 2B language model, and it accepts a full A4 page at 200 dpi as a single image instead of chopping it into tiles. It scores 0.921 on DocVQA and 0.732 on RefCOCO grounding, which is roughly nine times Gemma 4 E2B's grounding score. It also loses 17 of the 22 benchmark rows in Cohere's own published comparison table to Qwen3.5-2B, a smaller Apache 2.0 model sitting two columns over. And it does not do tool calling.

What actually shipped

North Micro Vision is Cohere's smallest VLM to date. The split is 2B of language model plus a 400M vision encoder custom-trained from SigLIP 2 SO400M, checkpointed in bfloat16 with a 262,144-token vocabulary. Context window is 128K tokens, though the model card is careful to note the validated multimodal range is 8K.

Input is interleaved text and images, output is text. The advertised jobs are visual question answering, captioning, grounding, OCR, and chart and document understanding, across English, German, French, Spanish, Italian, Portuguese, Hindi, Japanese, Korean, Chinese, Arabic, and more. License is Apache 2.0 with no additional use restrictions, which after the past month of open-weight releases arriving wrapped in bespoke license text is worth saying out loud.

Native resolution is the actual feature

The encoder handles up to 1654 x 2339 pixels, which is exactly an A4 page at 200 dpi, and it uses 2D RoPE alongside learned 1D positional embeddings to keep spatial structure intact across variable input sizes.

That matters more than the parameter count. Most small VLMs resize or tile an image into fixed square patches before the encoder sees it, which is a bit like photocopying a contract on a machine that only accepts 4x6 prints: you get every piece of the page, but a table that spans a seam arrives as two unrelated fragments. Native resolution feeds the whole page in one go, so a wide table stays a wide table and 8-point footnote text stays legible instead of getting resampled into mush.

The training curriculum backs that up. Stage 1 trained only the encoder and projector at 384x384 on 10M examples split 60% dense captions and 40% OCR. Stage 2.1 pushed to 1024x1024 with the full model unfrozen on 13M examples at an even caption/OCR split. Stage 2.2 raised the ceiling to the A4-at-200-dpi limit on another 10M. Stage 3 was instruction tuning across 50M examples of OCR, charts, grounding, and VQA. Stage 4 was preference optimization with the vision tower frozen.

The column Cohere left in

Here is where the release gets interesting. Cohere published a 22-row evaluation table against seven other small VLMs, and one of those columns is Qwen3.5-2B: 2.1B parameters, Apache 2.0, general purpose, with vision fused in rather than bolted on.

North Micro Vision wins five of those 22 rows. It takes GQA, ChartQA, AI2D, IFEval, and one MMLU variant. Qwen3.5-2B takes the other 17, including DocVQA (0.926 to 0.921), OCRBench (0.861 to 0.792), and RefCOCO grounding (0.785 to 0.732). Those are the three rows a document-specialist model is supposed to own.

Cohere put that column in the table themselves, which deserves credit. Plenty of launch posts would have quietly benchmarked against the four models they beat and called it a leaderboard.

Where it genuinely separates

Against the models most people would actually shortlist for a local document pipeline, North Micro Vision holds up. It takes six of the seven document, chart, OCR, and grounding rows against Ministral 3 3B, a model 58% larger. On RefCOCO it more than doubles Ministral's score and beats Gemma 4 E2B by nearly nine times, which is the difference between a model that can point at the right cell in a form and one that cannot reliably point at anything.

Two rows undercut the specialist story, and they are worth knowing before you commit. On OCRBench v2 (English) it scores 0.367, third-lowest in the entire table, below Gemma 4 E2B at 0.435 and Qwen3-VL-2B at 0.417. And on MMMU it scores 0.329, which is dead last across all eight models listed.

That MMMU number is the honest shape of this thing. Cohere spent the parameter budget on the eyes, not the brain. It will read your page beautifully and then have noticeably less idea what the page is talking about than a general-purpose model of the same size.

The deployment fine print

A few things will bite you on day one, all of them stated plainly in the model card rather than buried:

• No tool calling. The card says tool calling and agentic workflows are not supported. It will read your invoice. It will not go pay it.
• Not a reasoning model. Cohere explicitly flags limited math and code generation. Do not put it at the end of a chain that expects arithmetic on the numbers it just extracted.
• No vLLM yet. Support is listed as coming soon. Today you run it through Transformers 5.16.0 with accelerate and Pillow, which is fine for a prototype and awkward for a serving tier.
• 8K validated multimodal context. The 128K number is the language model's. Plan your multi-page batching around the smaller figure.
• Sampling defaults matter. Cohere recommends temperature 0.7, top_p 0.8, top_k 20. Small VLMs are unusually sensitive to this, and greedy decoding on a doc model is not automatically the safe choice.

Who should actually care

If you are building a local document extraction pipeline and you want a base to fine-tune rather than a model to call as-is, this is a reasonable pick. It is small enough to fine-tune on a single consumer card, Apache 2.0 with no strings, and the native-resolution encoder is a real architectural advantage for full-page inputs that no amount of prompt engineering gets you from a tiling model.

If you want the best zero-shot small VLM for documents today, the table says the answer is the other Apache 2.0 model in it. That is an unusual thing for a launch post to demonstrate, and it is the most useful thing about this one.

Key Takeaways

• Cohere Labs released North Micro Vision Instruct on August 12: 2.4B parameters (2B language model plus a 400M vision encoder), Apache 2.0, no use restrictions.
• The encoder handles native resolution up to 1654 x 2339 pixels, an A4 page at 200 dpi, with no tiling and preserved aspect ratio.
• It scores 0.921 on DocVQA and 0.732 on RefCOCO, beating Ministral 3 3B on six of seven document, chart, OCR, and grounding rows.
• In Cohere's own 22-row table, Qwen3.5-2B wins 17 rows including DocVQA, OCRBench, and RefCOCO, while North Micro takes five.
• It ranks last of eight on MMMU (0.329) and third-lowest on OCRBench v2 English (0.367), so the specialist framing has real edges.
• No tool calling, no agentic workflows, limited math and code, vLLM support still pending. Treat it as a fine-tuning base, not a drop-in agent component.

Sources: Cohere Labs launch post on Hugging Face, North-Micro-Vision-Instruct model card, Qwen3.5-2B model card, SigLIP 2 SO400M, DocVQA