Tencent's 770B Model Is Apache 2.0. Only 6% Fires Per Token.

TL;DR

On August 28 Tencent released Hy4 preview, a mixture-of-experts model with 770B total parameters and 49B activated per token, a 1M-token context window, and an Apache License 2.0 covering code and weights. It posts 92.3 on GPQA Diamond and 65.7 on SWE-bench Pro. In Tencent's own blind evaluation, 163 experts scored it 2.99 out of 4.00 across 203 engineering tasks, against 2.94 for Kimi K3 and 2.92 for GLM-5.3. The interesting numbers are not the benchmark ones. They are 6.4%, eight, and zero: the fraction of the model that runs per token, the GPUs you need to serve it, and the amount of training data or recipe that came with it.

What actually shipped

Two repositories, both public: tencent/Hy4-preview at full precision and tencent/Hy4-preview-FP8 quantized. The announcement aims the model at four areas of work: software engineering, office tasks, game development, and scientific research. Tencent is also serving it through its own CodeBuddy and WorkBuddy products, free for two weeks from launch, with API access via Tencent Cloud TokenHub and OpenRouter.

The architecture, from the model card:

• 78 layers. The first uses a dense feed-forward network; the remaining 77 are MoE.
• 256 routed experts plus 1 shared expert per MoE layer, with the top 8 routed experts and the shared expert active per token.
• Hidden size 6144, vocabulary 120,832, context length 1,048,576.
• Gated DeepSeek Sparse Attention with IndexCache, plus identity Hyper-Connections on the residual pathways.
• A native MTP layer: 10B parameters, 0.7B activated, for speculative decoding.

That last item is why the FP8 repository reports 804B parameters rather than 770B. The draft model ships inside the box.

6.4% of the model does the work

Divide 49 by 770 and you get 6.4%. That is the share of Hy4 that fires on any given token, and it is the whole economic argument for a sparse MoE at this scale. You get a 770B model's knowledge with a 49B model's per-token compute.

The bill splits in an awkward way, though. Compute scales with the active parameters. Memory scales with all of them. It is like renting a 770-desk office to run a company where 49 people clock in on any given morning: your payroll is small and your lease is not, and the 721 empty desks still have to physically exist.

The draft head is the practical win

Speculative decoding normally means finding or training a small model that predicts what the big one would say, then verifying its guesses in a batch. Sourcing that draft model is a real chore: too weak and the acceptance rate collapses, too strong and you have paid twice.

Hy4 ships a multi-token-prediction layer trained alongside the main model, 10B parameters with 0.7B active. Tencent's prebuilt vLLM image (vllm/vllm-openai:hy4-preview) and SGLang image (lmsysorg/sglang:hy4-preview) both enable MTP speculative decoding by default, with a FLASHMLA_SPARSE attention backend for the sparse-attention path. You are not assembling a serving stack from parts; you are pulling a container.

The scores

Published results from the model card, cross-checked against DataLearner's record of the same release:

The shape is consistent: this model is tuned for agentic software work. Terminal-Bench 2.1 at 85.4 and MCP-Atlas at 83.7 say it holds a tool loop. SWE-bench Pro at 65.7 and SWE-bench Multilingual at 82.9 say it lands patches. Humanity's Last Exam at 43.4 says it is not a research oracle, which is fine, because nobody deploys 770B parameters to answer trivia.

The blind eval margin is thinner than the press release

Tencent ran a blind evaluation with 163 experts across 203 engineering tasks on a 4-point scale. Hy4 preview averaged 2.99. Kimi K3 averaged 2.94. GLM-5.3 averaged 2.92.

Read that chart the way it is drawn, not the way it is headlined. A 0.05 to 0.07 gap on a 4-point scale, from a rubric designed by the vendor and graded by the vendor's own experts, is a statement that three models are interchangeable for this workload. It is not a coronation. Treat it as evidence that Hy4 belongs in the same tier, and pick between them on price, license, and how each one behaves on your codebase.

What it costs, and what it costs you

Tencent's API is $0.834 per million input tokens, $2.501 per million output, and $0.042 per million on cache hits. In China the same tiers are 6.00, 18.00, and 0.30 yuan. A 20x discount for cached input is aggressive, and with a 1M-token window it is clearly aimed at people who park a whole repository in the prompt and then keep asking questions.

Self-hosting is the other story. The official launch commands specify --tensor-parallel-size 8 for vLLM and --tp-size 8 for SGLang. At one byte per parameter, 804B parameters in FP8 is roughly 800 GB of weights before you allocate a single token of KV cache, which at eight GPUs is about 100 GB each. That is an H200-or-better shelf. Apache 2.0 grants you the right to run this on your own metal; physics grants you the right to buy the metal first.

The license is the quiet part

The model card says it plainly: Apache License 2.0, covering both the code and the weights, with commercial use permitted and no separate agreement to sign. No acceptable-use addendum, no monthly-active-user threshold, no clause that flips terms once you get big.

What is not in the box: no pretraining data, no training recipe, no data-mixture documentation. This is an open-weights release with a permissive license, which is a genuinely useful thing and is not the same as an open-source model. You can run it, fine-tune it, embed it in a product, and redistribute your derivative. You cannot reproduce it, and you cannot audit what went into it.

Caveats worth holding

• Preview means preview. The name is not decoration. Expect the numbers, the serving images, and the defaults to move.
• Every benchmark figure here is vendor-published. There is no independent third-party reproduction of the Hy4 scores yet, on any of these suites.
• The 1M context is a maximum, not a promise of quality. Gated sparse attention with an IndexCache is exactly the kind of mechanism that trades recall at depth for throughput. Test retrieval at your actual working length before you design around it.
• Tencent's claimed 31.8% end-to-end throughput gain is measured on its own training and inference stack against its own baseline, per TechNode. It is not a portable number.

Key Takeaways

• 770B total, 49B active, 1M context, Apache 2.0. Compute scales with the 49B; VRAM scales with the 770B. Budget for both separately.
• The MTP draft head ships in the weights. 10B parameters, 0.7B active, wired into the official vLLM and SGLang images. Speculative decoding without sourcing a second model is the most immediately useful thing here.
• It is an agentic coding model. Terminal-Bench 2.1 at 85.4, MCP-Atlas at 83.7, SWE-bench Pro at 65.7, HLE at 43.4 without tools. Deploy it accordingly.
• The blind-eval win is 0.07 points on a 4-point scale, vendor-graded. Hy4, Kimi K3, and GLM-5.3 are one tier. Choose on license, price, and your own evals.
• Eight GPUs, roughly 800 GB of FP8 weights. "Open" here means open to anyone with a rack, not open to your desk.
• Open weights, not open source. Apache 2.0 on the artifact, nothing on the data or the recipe.

Sources: tencent/Hy4-preview model card, tencent/Hy4-preview-FP8, Tencent announcement, TechNode, DataLearner model record, Apache License 2.0