Zyphra Open-Sources ZONOS3-Mini, a TTS Model That Clones Voices in the Browser

TL;DR

ZONOS3-Mini, released Nov 9 by Zyphra, is an open text-to-speech model under Apache 2.0 with 100 million active parameters out of 800 million total. It clones a voice from up to 60 seconds of reference audio, speaks 43 languages, and steers emotion, pace and accent down to single words. The part that should worry TTS API vendors: it runs real-time in a browser tab through WebGPU, with Zyphra reporting under 570 ms to first audio on a Pixel 9 and under 150 ms on an M3 MacBook Air. Training took 112 H100s and 2.5 days.

What shipped

The weights sit on Hugging Face: a bf16 checkpoint, GGUF files, WebGPU shards and 27 default voices grouped by language. The runtimes repo ships a CUDA server for NVIDIA GPUs, an MLX server for Apple Silicon, a C++/ggml server and CLI covering CPU, CUDA, Metal and Vulkan, and the in-browser WebGPU build. On the server side there is day-0 support in SGLang through SGLang-Omni, which exposes an OpenAI-compatible /v1/audio/speech endpoint with streaming and cloning from a reference clip; the release post also names vLLM. No install is needed to try it: Zonos Voice Studio and its Lite sibling run the model on your own GPU through the browser, cached after the first visit.

The lineage explains the name. The original Zonos v0.1 landed in February 2025, and ZONOS2 in June 2026 proved that a sparse mixture of experts helps TTS, at the cost of 8 billion parameters riding on an 86 Hz codec that limited where it could deploy. ZONOS3-Mini is the attempt to keep the quality and lose the deployment tax.

How 800M parameters fit in a phone

The efficiency starts at the representation, before any architecture tricks. ZONOS2 modeled speech with the Descript Audio Codec, which runs at 86 Hz and which Zyphra calls "incredibly hard to model" despite its fidelity. ZONOS3-Mini's own ZAC-1 codec runs at 15 Hz with ten codebooks of 4,096 entries: one semantic codebook distilled from a Wav2Vec2-BERT teacher, plus nine acoustic codebooks. Under similar data constraints, Zyphra measured a 10x reduction in word error rate against a DAC-trained model, with better acoustic similarity.

A speech codec at 86 frames a second is video at 240 fps: faithful, and hopeless to stream live from a phone. Fifteen frames a second, encoded well, still reads as motion. Same trick, smaller pipe.

The backbone spends the savings. Eleven layers at model width 1,024, a latent MoE with 64 experts and top-2 routing from layer one, and a 9:2 mix of Gated DeltaNet layers and global attention over a 32,768-position context. Text enters through an 8,192-entry SuperBPE tokenizer; the speaker arrives either as a reference clip encoded into a token prefix or as a 192-dimension MECT-B2 embedding. A causal multi-resolution iSTFT head renders each finished frame to 44.1 kHz as it completes, which is what makes streaming work, and a single generation can run up to ten minutes.

Fast enough to interrupt

Latency is where the size pays off. Zyphra reports under 570 ms to first audio in a browser on a Pixel 9 and under 150 ms on an M3 MacBook Air. On an H100, the model card's evaluation logs a median 80 ms to first audio at 8 concurrent requests and peak throughput of 457 seconds of audio per second, measured against the 55 language subsets of the benchmark below.

Steering, down to the word

Identity controls work through the speaker representation: age, gender presentation, accent and vocal texture, with accent steering handled by task vectors. The release samples hold one narrator and walk the accent from General American to Southern English to Australian, and the age target from 25 to 75, while the voice stays recognizable as the same person.

Delivery controls run on activation steering: pace, emotion, projection and liveliness. The interesting part is granularity. Learned alignment heads are monitored during streaming generation, so steering can target a span of words inside a sentence: the demo applies whisper +4 to "I didn't say" in one take and to "she took the keys" in another, same voice, same seed. A separate edit model, built on a VoiceCraft recipe, rewrites a sentence afterward with the same controls as a fresh generation.

Post-training ran in two stages, direct preference optimization and then GRPO, for robustness. Zyphra is careful about what the sliders promise: the emotion values "do not guarantee a specific emotion", they are targets.

The scoreboard, and who keeps it

Alongside the model, Zyphra released ZTTS2-Eval: read speech across 35 languages from FLEURS-R and in-the-wild recordings across 20 languages from VoxBlink2, scored with a mix of ASR word error rates and prosody metrics, transcribed by Qwen3-ASR with Whisper Large v3 filling in lower-resource languages. The headline number is a new TTS Capability Index v1, built with item response theory over 420 items (metric x test set x language). ZONOS3-Mini ranks first among open models, ahead of Fish S2 Pro and MOSS TTS v1.5.

Those listening samples put ZONOS3-Mini next to ElevenLabs, Cartesia, Qwen, Fish and OmniVoice on fixed texts, including a Dune-style litany about overfitting and a Parks and Recreation monologue about sending data scientists to jail. Worth your ears, and worth remembering they are curated by the vendor.

Running it

Fastest path is the browser studio above. For a real service, SGLang-Omni gives you the OpenAI-compatible endpoint in two commands from the model card:

sgl-omni serve --model-path Zyphra/ZONOS3-Mini --port 8000

curl -X POST http://localhost:8000/v1/audio/speech -H "Content-Type: application/json" -d '{"input": "Hello from ZONOS3-Mini!"}' --output output.wav

Add a references field with an audio path to clone a voice, no transcript required, or "stream": true with "response_format": "pcm" to pipe 44.1 kHz audio as it generates. The ggml build covers machines WebGPU cannot, and MLX covers Apple Silicon natively.

The catches

• The benchmark, the capability index and every listening sample come from Zyphra itself. No blind third-party arena has scored ZONOS3-Mini yet, and the open-model ranking is a field of one vendor's choosing.
• All latency numbers are company-reported. The browser figures are ceilings on tested hardware and need a WebGPU-capable browser.
• Forty-three languages, but full NeMo text normalization covers only nine: English, Mandarin, Japanese, Korean, German, Spanish, French, Italian and Portuguese. The rest pass through as written, so the long tail will vary.
• Cloning needs 60 seconds of anyone's voice and Apache 2.0 attaches no consent conditions. The license tells you what you can run; a lawyer tells you what you can publish.
• Against the closed leaders, ElevenLabs and Cartesia, the evidence is curated audio samples rather than a controlled comparison. Treat parity as unproven either way.

Why builders should care

If you pay per character for speech, this is the exit ramp. An OpenAI-compatible endpoint means SGLang-Omni slots into an existing stack with a base-URL change, and from there your marginal cost per word is electricity. For anything sensitive, health, legal, enterprise support, the audio path can stop at the user's own GPU, which is a compliance story you cannot buy from an API.

If you ship product, the browser build is the headline feature. A voice in the page with no backend, no keys and no per-user bill changes what a demo, a game or an accessibility feature can cost.

If you build models, two ideas transfer. Sparse MoE on a deliberately slow representation is why 800M total parameters behave like 100M and fit a phone, and that recipe is not TTS-specific. And monitoring alignment heads to steer single words mid-stream is the kind of control API TTS vendors do not sell you at any price.

Key Takeaways

• Zyphra released ZONOS3-Mini on Nov 9 under Apache 2.0: weights, GGUF files, WebGPU shards and 27 voices on Hugging Face, with CUDA, MLX, ggml and in-browser runtimes on GitHub.
• It is a latent MoE with 100M active parameters out of 800M total, trained on 112 H100s in 2.5 days; its ZAC-1 codec runs at 15 Hz against DAC's 86 Hz and cut word error rate 10x under similar data constraints.
• On-device latency: under 570 ms to first audio in a Pixel 9 browser and under 150 ms on an M3 MacBook Air, versus a median 80 ms on an H100 at 8 concurrent requests with peak throughput of 457 seconds of audio per second.
• It clones voices from up to 60 seconds of reference audio, speaks 43 languages, steers age, gender, accent, texture, pace, emotion, projection and liveliness, targets individual words via alignment heads, and ships a VoiceCraft-recipe edit model.
• It ranks first among open models on Zyphra's own ZTTS2 Capability Index v1, ahead of Fish S2 Pro and MOSS TTS v1.5; benchmark, index and samples are all vendor-run, with no independent blind comparison yet.
• SGLang-Omni serves it day-0 behind an OpenAI-compatible /v1/audio/speech endpoint with streaming and reference-clip cloning.

Sources: Zyphra, ZONOS3-Mini release post (Nov 9, 2026), ZONOS3-Mini model card, Hugging Face, ZONOS3-Mini runtimes, GitHub, SGLang-Omni, GitHub, ZTTS2-Eval, Zyphra, ZONOS2 (June 2026)