Kyutai Open-Sources AURA, Real-Time Talking-Head Avatars on a Single GPU
On this page 9 sections
TL;DR
AURA is Kyutai's new audio-driven talking-head family, open-sourced November 20 with the paper out the day before. It generates lip-synced video chunk by chunk as the speech arrives, instead of waiting for the multi-second audio look-ahead that bidirectional models need. The AURA-1.3B model streams 576p at 20.7 fps end-to-end on a single H100, faster than the 16 fps playback rate, so the mouth keeps up with the voice. Weights, inference code, and a 30K-video dataset are all public. The weights carry a non-commercial license, which is the one catch.
Why streaming was the hard part
Talking-head video diffusion is already good. The catch is the shape of the computation: dozens of denoising steps over multi-second chunks, with attention that sees the whole chunk at once, so the model needs several seconds of future audio before it draws your first frame. Fine for a rendered clip, useless for a live avatar. A dubbing studio watches the entire scene and then matches the mouths; AURA has to work like a simultaneous interpreter, moving lips with only the audio it has heard so far.
The mechanism is chunked causal generation. Video comes out in chunks of 4 latent frames, about 0.64 seconds each, with block-wise causal attention and a rolling KV cache. Only the current chunk's audio is needed to start generating. Each chunk takes 4 denoising steps (timesteps 1000, 745, 498, 255), with no classifier-free guidance at inference.
What real time means here
Two sizes shipped. AURA-1.3B is the one the model card calls the only real-time member of the family: 576p at 20.7 fps end-to-end on one H100. AURA-5B streams 720p at 9.7 fps, under the playback rate, so it is the tier for nicer offline video rather than live interaction.
The reference code in the repo runs at about 16 fps on one H100 with --compile, which is exactly real time for 16 fps playback; the 20.7 fps figure comes from the paper's optimized pipeline. Output resolutions are 576x576 or 768x512.
How they made a four-step student
The authors are Nathaniel Cohen, Nicolas Dufour, Amélie Royer, Alasdair Newson, and Patrick Pérez, from Kyutai and Sorbonne Université's ISIR (Pérez's work was done while at Kyutai); the paper is accepted at BMVC 2026. Their base for the 1.3B model is Wan2.1-T2V-1.3B, fine-tuned with LoRA adapters (rank 128) plus a few trained audio modules around Mimi, Kyutai's audio tokenizer.
Speed comes from two distillation stages, and every training artifact is on the Hub. First a bidirectional teacher, AURA-1.3B-Teacher-TalkVerse, generates 30K full-trajectory solutions. The causal student trains on them with an MSE loss, learning to match the teacher's trajectories in fewer diffusion steps. Stage two applies Distribution Matching Distillation with backward simulation, with a frozen audio LoRA, to close the remaining quality gap. The student drills the teacher's worked solutions until it lands nearly the same answer in four steps instead of dozens.
The payoff, measured against each model's own teacher: 23x faster for the 1.3B at quality comparable to TalkVerse (at lower resolution), and 17x faster for the 5B, which matches its teacher at 720p.
Quality, with receipts
On the standard talking-head test sets, AURA-1.3B posts the best identity preservation (CSIM) of all compared methods on both HALLO3 test (0.84, 100 clips) and CelebV-HQ test (0.77, 150 clips). Full FID, FVD, JEDi, and sync tables are in the paper. Treat those as research-leaderboard numbers; the demo clips on the project page are the faster sanity check.
Run it tonight
The code lives at kyutai-labs/aura. Clone it, run uv sync, then uv run generate.py --image portrait.jpg --audio speech.wav --output out.mp4. The script pulls everything from the Hub: the merged checkpoint (a ~6.3 GB fp32 safetensors with the LoRA updates and Mimi modules baked into Wan2.1-T2V-1.3B), the VAE and T5 text encoder from Wan2.1, and Mimi for audio. Flags include --prompt, --num_chunks, --seed, and --compile.
The intermediate artifacts are public too: the stage-one ODE-distilled student, LoRA-only forms of every checkpoint, training metrics in JSONL, and launch configs with the seed and code commit. If you want to watch a two-stage distillation converge, the logs are there.
The catch: non-commercial weights
The weights are CC BY-NC 4.0. That restriction is inherited honestly: the distillation trajectories were conditioned on reference images and prompts from the Hallo3 training data, itself CC BY-NC-ND 4.0, and Kyutai passed the terms through instead of laundering them. The VAE and T5 encoder from Wan2.1 keep their Apache 2.0 terms. Practical reading: the avatar can charm users, just not paying ones.
The model card's responsible-use section is blunt. Do not impersonate people, do not create non-consensual content, animate only portraits you have consent for, and disclose that the video is AI-generated. A model that renders realistic people saying whatever audio you feed it needs that paragraph, and this one has it.
What it unlocks for small builders
A live avatar on one GPU changes the bill of materials. A voice assistant with a face, a VTuber rig with no tracker and no artist, a lip-synced host for a product page: each is a portrait photo and an audio stream away, on hardware that fits a homelab rather than a render farm. The 720p 5B is the tier for when you can wait for the render.
The dataset matters as much as the weights. 30K synthetic videos with precomputed teacher trajectories is exactly the artifact you would need to reproduce the distillation or train your own student, and few labs publish that layer at all.
Key Takeaways
- Kyutai open-sourced AURA on November 20 (paper November 19): two talking-head models, 1.3B and 5B, that generate video causally, chunk by chunk with a rolling KV cache, needing only the current chunk's audio rather than multi-second look-ahead.
- AURA-1.3B streams 576p at 20.7 fps end-to-end on one H100, past the 16 fps playback rate that defines live; AURA-5B does 720p at 9.7 fps for offline use. The reference code holds about 16 fps on one H100 with torch compile.
- The recipe: LoRA fine-tuning on Wan2.1-T2V-1.3B with Mimi audio tokens, then two distillation stages (MSE trajectory matching on 30K teacher runs, then Distribution Matching Distillation with backward simulation). The student runs 23x (1.3B) and 17x (5B) faster than its teacher at comparable quality.
- Best identity preservation of all compared methods on both test sets: CSIM 0.84 on HALLO3 test, 0.77 on CelebV-HQ test.
- Everything is public: both models' weights, inference and training code, the 30K-video dataset with teacher trajectories, intermediate checkpoints, and training metrics with seeds and commit hashes.
- Weights are CC BY-NC 4.0 because the distillation data (Hallo3) is non-commercial; the Wan2.1 VAE and T5 encoder stay Apache 2.0.
Sources: Kyutai AURA project page, AURA paper (arXiv 2611.26540), AURA-1.3B model card, AURA-5B model, kyutai-labs/aura repo