← Back to all posts
News

Pika's Enact Research Preview Directs AI Video in Real Time at 1.8x Playback

November 21, 2026 · 03:14 UTC · News
On this page 7 sections
Pika's Enact Research Preview Directs AI Video in Real Time at 1.8x Playback

TL;DR

Pika Enact, a research preview that Pika published on November 18, swaps the prompt box for something closer to a working film set. Video and sound generate as a live stream, in chunks shorter than a second, while you lift an actor with the mouse and set them down somewhere better, talk to the cast and hear them answer in character, or push a virtual camera around the room. It is built on Pika H3 Flex, the company's post-trained open-weights video model (itself a post-training of MiniMax's H3), distilled to a few denoising steps for the live loop. Pika publishes measured throughput for all three modes: staging generates at 1.8x playback on a single server, spoken direction at 1.75x, and camera work at 1.50x on an 8x B300 node. You cannot touch any of it yet: the only door is an enterprise email address.


At a glance
A film set that renders while you direct it
1.8x
generation vs playback, live staging, one server
1.75x
picture and sound, live actor direction
1.50x
camera mode, 8x b300 at 640x480 24 fps
<0.4s
picture response to a drag, h100, decode included
0.17s
low-latency chunk length; 0.7s standard
Nov 18
research preview, enterprise contact only

Three ways to grab the scene

Live staging is the mode to watch first. While the clip plays, you press the mouse on a person or a chair and move them; the subject reacts to being carried, shifts weight mid-air, then settles where you let go, with pose, lighting and shadows following the room. No cut-out edges, and no compositing pass afterwards. In the low-latency setting the picture responds in under 0.4 seconds on an H100, decoding included.

Live actor direction makes the cast conversational. Type or say a note, "move closer to her", "make it a kitchen at night", and the actors play it out while the stream continues, answering in their own voices and remembering every note you gave. Each one is an agent with its own personality and session memory. When the rehearsal feels right you call Action, and the script model rewrites the whole exchange into one clean take, shot fresh from the first frame.

Live camera control needs no existing video. An appearance image anchors the cast and set, a lightweight 3D motion proxy (an untextured Blender viewport) guides the camera path, and you pan, dolly and orbit with the mouse while picture and spatial audio keep generating.

Live direction pipeline
From your note to a running take
Input
You type or speak a note
session memory carries every prior note
Script model
Notes become structured beats
action, dialogue, emotion, gaze, lighting
Student model
Few-step chunks of picture and sound
0.7s standard, 0.17s low latency
Action
One clean take, first frame onward
rehearsal chatter rewritten out
The performance never stops while you talk; idle scenes hold with breathing and blinking.

The numbers, with their hardware attached

All three speed figures are Pika's own, measured on different hardware, so read them as existence proofs rather than a leaderboard. Staging generates 1.8x faster than playback on a single server. Direction, which adds the script model and native voices, holds 1.75x. Camera control runs at 1.50x on an 8x NVIDIA B300 node at 640x480 and 24 fps, rendering initial frames in 0.51 seconds and spending 0.473 seconds of compute per 0.708-second chunk.

Vendor-measured throughput
Every mode generates faster than it plays
playbackenact modes
staging
1.80x
direction
1.75x
camera
1.50x
playback
1.00x
Generation speed versus playback, per mode. Hardware differs by mode: one server, unspecified, and 8x B300 at 640x480 24 fps. Source: Pika Enact blog.

How the latency got under half a second

Two ideas do the work. First, the model does not generate clips; it streams short causal chunks, 0.7 seconds standard and 0.17 in low-latency mode, through a sliding key-value cache. The cache keeps the prompt, the source video, the first chunk and the four most recent chunks, with positional embeddings re-based so a session can run forever at constant cost. The first chunk is the anchor that stops faces and sets from drifting, which is why it stays cached while everything else rolls. An actor who keeps the first read and the last few lines fresh, letting the middle blur, plays longer runs than one who insists on the whole script word for word.

Second, distillation. Enact applies self-forcing distribution matching distillation: a few-step student generates each chunk from its own previous outputs, corrected by a trainable critic and the full-clip teacher with cursor signals emphasized, which cuts sampling from twenty steps to a few. Interaction itself enters through lightweight, zero-initialized modules that inject 24 fps mouse states, a spatial grab heatmap, and the untouched source video, frame-aligned.

Then there is the data problem, the most copyable idea in the post. Dragging requires before-and-after pairs synchronized with cursor trajectories at frame level, and no film archive on earth ships with mouse telemetry. Pika built a physics-driven synthetic engine instead: animate a crude 3D stand-in along the drag path with simulated weight and momentum, have the video model render photoreal targets, and auto-log mouse positions and button states. Zero human labeling. The camera mode gets the same treatment through a Blender-to-Video pipeline that records multi-angle trajectories, untextured 3D motion references, RGB frames and spatial audio together, then filters the pairs with optical-flow and identity checks.

The catches, in order of size

  • You cannot use it. "Research preview, unavailable for consumer use" is Pika's own wording; the only contact is an enterprise email address. No pricing, no date.
  • The numbers are vendor-measured on vendor hardware, and the camera figure is at 640x480, a rough draft of a resolution.
  • Direction at 1.8x playback still costs a server per creator, and the camera mode costs eight B300s. The economics fit a studio pipeline, not a browser tab.

What you can poke today is the foundation. Pika H3 Flex ships with open weights, generates a 15-second clip in as little as 7 seconds, and sits seventh on DesignArena's text-to-video board, the highest-ranked entry with open weights. The interactive layer is closed; the model under it is not.

What to steal

  • Budget for sub-half-second feedback. Under about 0.5 seconds, an interface stops feeling like a form you submit and starts feeling like something you steer. That threshold, not model quality, separates Enact from a prompt box.
  • Distill for the loop, keep the teacher offline. The few-step student handles the live stream; the expensive full-clip model only grades training. The same split fits any interactive generative tool.
  • Anchor the first chunk. A sliding cache that keeps the opening state plus recent context buys infinite sessions at constant memory, a pattern any streaming agent can borrow.
  • Synthesize the pairing you cannot find. When an interaction needs data no corpus contains, cursor paths, camera proxies, synchronized spatial audio, generate it synthetically with physics underneath and photorealism on top.
  • Rehearse, then reshoot. Logging a messy interactive session and regenerating a clean artifact from it is a UX pattern that extends well past video, to any live agent session whose output must look deliberate.

Key Takeaways

  • Pika Enact, published November 18 as a research preview, is a live-streaming directing environment built on distilled Pika H3 Flex: drag actors mid-play, speak notes the cast answers in character and remembers, and steer a virtual camera, all while generation outruns playback.
  • Measured throughput per mode: 1.8x playback for staging on a single server, 1.75x for spoken direction, and 1.50x for camera control on an 8x B300 node at 640x480 24 fps, with picture response under 0.4 seconds on an H100 in low-latency mode.
  • The latency comes from streamed causal chunks (0.7s standard, 0.17s low-latency) through a sliding KV cache anchored on the first chunk, plus self-forcing distribution matching distillation that cuts sampling from twenty steps to a few.
  • Training data for interaction was synthesized: a physics-driven 3D stand-in renders drag pairs with auto-logged cursor telemetry, and a Blender-to-Video pipeline yields camera trajectories with spatial audio, filtered by optical flow and identity checks.
  • It is enterprise-contact only, with no consumer access, pricing, or date; the base H3 Flex model, by contrast, ships with open weights and ranks seventh on DesignArena text-to-video.
  • Reusable pattern for builders: rehearse live with a cheap student, then regenerate the clean artifact from the session log with the expensive path.

Sources: Pika Enact research preview post, Pika H3 Flex model page, Distribution Matching Distillation, arXiv 2311.18828, AlphaSignal

AIPikaPika EnactAI VideoReal-Time GenerationFilmmakingDirectingOpen Weights
CONSOLE
$