Six Fully Open Models, 375B Down to 0.9B. The Data Is a 401.

TL;DR

On September 3, MBZUAI's Institute of Foundation Models released K2 Horizon: six Apache 2.0 models at 0.9B, 3.7B, 7B, 32B, 36B-A4B and 375B-A23B, sharing one architecture, one vocabulary and one training recipe. The selling point is not the weights, it is the training lifecycle around them: data, code, intermediate checkpoints, logs. Today that part is largely an IOU. The two dataset repos every model card points at return HTTP 401, the pretraining code is not up, and the 32B is a stage 1 checkpoint that trails Qwen3.8-27B by 43 points on Terminal-Bench. The small models, on the other hand, are the most interesting thing anyone has open-sourced this month.

What actually shipped

Six models went up on Hugging Face on September 1 and 2, with the announcement following on the 3rd. Every model above 0.9B carries a native 524,288-token context window from midtraining onward; the 0.9B gets 131,072 via YaRN RoPE scaling. All are Apache 2.0.

• 0.9B and 3.7B and 7B: dense, aimed at watches, glasses, phones and cheap serving.
• 32B: dense, for a local workstation. Shipped as K2-Horizon-32B-Stage1, with the final checkpoint still pending.
• 36B-A4B: sparse, 4B active per token, using IFM's new Mixture-of-Value-Attention design.
• 375B-A23B: the flagship MoE, 23B active per token.

GGUF builds exist for the five smaller models. The 375B has an FP8 repo and no GGUF, which is the polite way of saying it is not for your desk. Serving goes through vLLM and SGLang, with Cerebras, AWS, Nebius and Compass named as inference partners.

The small models are the story

Look at what IFM claims a 7B does on SWE-bench Verified. These are self-reported numbers from the model card, and they are not close.

The 3.7B reports 68.6 on the same benchmark against 41.2 for Qwen3.5-4B, and 70.5 on HMMT Feb 2026 against 61.6. The 7B adds 59.0 on BrowseComp, above the DeepSeek V4 Flash and GPT-5 figures IFM lists for comparison. The 0.9B posts 48.5 on AIME 2026 and 79.9 on HumanEval+ against 0.2 and 16.5 for Qwen3.5-0.8B.

Treat all of that with the skepticism any self-reported table deserves, especially the sub-4B SWE-bench results, where harness choices move the number more than the weights do. IFM at least publishes its harness caveats: strict means no internet, BrowseComp uses a discard-all-at-95k context strategy, WildClawBench and Apex-Agents use text-only subsets. That is more disclosure than most labs bother with, and it is also a reminder that you cannot compare these rows to somebody else's blog post.

MoVA, and why 4B active parameters punch above their weight

The 36B-A4B is the architecturally novel one. A normal mixture-of-experts routes each token to a different feed-forward block. MoVA pushes that routing into the value heads of attention, creating experts over the values the model retrieves rather than over the block that processes them. If a standard MoE swaps out the workbench a token gets sent to, MoVA swaps out the filing cabinet the model reaches into while it is still reading. It stays compatible with FlashAttention, grouped-query attention and sparse attention, so it is not an exotic runtime.

The result: 26.8 on tau3-Banking and 58.6 on Terminal-Bench 2.1 at 4B active parameters, against 14.2 and 53.9 for Nemotron 3 Ultra, a 550B MoE with 55B active. It loses on GPQA Diamond (80.8 against 86.7) and on long-context reasoning. Capability per active parameter is the axis where it wins, which is exactly the axis that matters if you are paying for tokens.

Separately, IFM shipped Uno, a discrete diffusion decoder that generates token blocks in parallel instead of one at a time, with Uno variants of the 0.9B and 7B. Moor Insights reports 2.5x to 3x faster inference without quality loss. The repo tagline claims lossless speedups; nobody outside IFM has checked that yet.

Now the part the whole release is built on

IFM's own framing is that open weights are not open science. Eric Xing, the institute's founder, put it this way in the press release: "Science works when others can see the data, follow the method, reproduce the result, and improve on it." The blog promises training data or recipes, training code, configs, intermediate checkpoints, fine-grained logs and evaluations for every model.

Here is what you can actually pull today.

Every model card lists IFM/K2-Horizon-Pretrain-Data and IFM/K2-Horizon-Midtrain-Data in its frontmatter. Both return HTTP 401 to an anonymous request, which is what Hugging Face sends for a repo that is private or gated. The five datasets IFM did publish on August 24, including TxT360-v2 at 5.29 TB under CC BY 4.0, return 200 and download fine. So this is not a hosting outage; the K2 Horizon corpora specifically are not open yet.

Code is in the same state. The post-training repo is up, alongside xllm, a lightweight pretraining infrastructure library. The actual pretraining recipe is not.

To be fair to IFM, they never claimed otherwise in the fine print. The blog says the fleet will release "training data or detailed data-construction recipes," and that datasets ship under their applicable licenses with construction methods disclosed "when redistribution is not possible." The word doing the work in that sentence is "or." The 375B and 36B cards use plain future tense: data and training code "will be released." Only the 3.7B and 7B cards claim, present tense, that data and code are already public, and they point at the same two 401s.

One more artifact of a fast release: the 0.9B card declares license: apache-2.0 in one frontmatter line and license_name: internal-only two lines later, with a license_link aimed at a LICENSE file that returns 404. Somebody's release script had a long night.

The 32B is a work in progress, and says so

The mid-size dense model is the weak link, and IFM labels it clearly: stage 1 of the final training run, with stage 2 results promised. Against the obvious open competitor at that scale, it is not close.

The flagship fares better. On agentic work the 375B posts 65.3 on Toolathlon Verified against 67.5 for GPT-5.6 Luna and 71.6 for Claude Sonnet 5, 34.0 on tau3-Banking against 37.3 for Sonnet 5, and 1,441 Elo on GDPVal-AA against 1,584. Beating a 550B open MoE while running 23B active is the real claim, and the table supports it. On frontier physics reasoning it collapses: 8.6 on CritPt against 20.9 for GLM 5.2 and 22.9 for GPT-5.6 Terra. Agentic parity, reasoning deficit, roughly the shape you would expect from a lab that spent its post-training budget on tool use.

What to do with this

If you want a small local model for coding or tool use, download the 3.7B or the 7B GGUF today and run your own evals; the claimed numbers are extreme enough that verifying them is cheap and worth doing. If you want a workstation model, wait for stage 2 of the 32B. If you are serving agents at volume, the 36B-A4B is the one to benchmark, because 4B active parameters changes your unit economics more than two points of GPQA does.

And if you came for the open science, set a reminder. LLM360, the namespace IFM grew out of, has an actual track record here going back to Amber and the original TxT360, so the promise is credible. It is just not a thing you can clone this week.

Key Takeaways

• Six Apache 2.0 models, one recipe. 0.9B through 375B-A23B, 512K context on everything above the 0.9B, GGUF for the five smaller models and FP8 for the flagship.
• The small models carry the release. Self-reported SWE-bench Verified of 70.6 at 7B and 68.6 at 3.7B, both far above the Qwen, Gemma and Granite comparisons IFM lists. Verify before you believe.
• The openness is partly a promissory note. Both training-data repos referenced by every model card return HTTP 401, and pretraining code is not published. The post-training code and TxT360-v2 are genuinely public.
• MoVA is the interesting architecture. Expert routing pushed into attention value heads gives 36B of capacity at 4B active, beating a 550B MoE on agentic tool use while losing on knowledge benchmarks.
• Skip the 32B for now. It ships as a stage 1 checkpoint and trails Qwen3.8-27B 36.6 to 79.8 on Terminal-Bench 2.1, which IFM discloses rather than hides.

Sources: IFM: Introducing K2 Horizon, IFM on Hugging Face, PR Newswire announcement, Moor Insights and Strategy, BigDATAwire, MBZUAI