Your Prompt Now Compiles to a File. It Takes 51 Seconds.

TL;DR

On September 3, Yuntian Deng and Pengyu Nie of the University of Waterloo and Stuart Shieber of Harvard posted Compile by Training, an EMNLP 2026 system demonstration. You write an English description of a text-to-text function. Teacher models synthesize training examples for exactly that function. Roughly 51 seconds later you get a .paw file: a LoRA adapter plus a prompt scaffold that specializes a frozen Qwen3-0.6B interpreter. After that the function runs on its own, offline, with no calls back to the big model. On FuzzyBench-Hard, the slice where the previous one-shot compiler produced zero exact matches, the trained programs score 0.836 semantic accuracy versus 0.224. The compiler is deployed as a public service you can point at right now.

The pitch: treat the frontier model as a build tool

The paper opens on an email triage function. "Signature needed by EOD" maps to immediate. A newsletter maps to wait. Trivial to describe, miserable to write as rules, and far too small to justify a remote API call on every one of ten thousand messages.

That gap is the whole thesis. A lot of what people currently spend tokens on is not reasoning, it is a narrow recurring classifier they never got around to training. The authors call the alternative compile by training: invoke the large model once, at build time, to manufacture a small artifact, and then stop paying it.

In this view, adaptation becomes a software build step, and large language models act as tool builders rather than run-time dependencies.

This is a direct extension of Program-as-Weights (PAW), from July, where a 4B compiler predicted an adapter in a single forward pass. That paper's headline claim is worth restating: a 0.6B Qwen3 interpreter running a PAW program matched direct prompting of Qwen3-32B on their benchmark, at roughly one fiftieth of the inference memory and 30 tokens per second on a MacBook M3. PAW also released FuzzyBench, a 10M-example dataset, which is what makes any of this checkable.

What the extra minute buys

PAW's fast compiler spends the same fixed compute on every function, whether the spec is "lowercase this" or a nested extraction rule. Compile by training keeps that prediction as a warm start, then invests real gradient steps on top.

The analogy the paper does not make: the fast compiler is autocomplete for model weights, one forward pass and a confident guess. Compile by training takes that guess and makes it rehearse against worked examples for a minute before the gig. Same piece of music, very different performance.

Two things to hold onto about that chart. FuzzyBench-Hard is defined as the specs where the fast compiler produced no exact matches, so it is adversarial by construction and these are not general-purpose accuracy numbers. And the metric is LEM, LLM Exact Match, where a GPT-5.5 judge decides whether a prediction is semantically correct given the spec, input and reference. The authors validated the judge against 128 hand labels: 0.977 accuracy, Cohen's kappa 0.946. That is a defensible grader, and it is still a grader.

How the compile actually runs

The public "Finetuned Standard" configuration is spelled out, which is rarer than it should be: a quantized Qwen3-0.6B interpreter, mixed teachers, a rank-64 LoRA with alpha 16, a warm start from the amortized compiler, and a 100-step cosine schedule. Teacher requests come back as structured JSON, and the compiler validates and rejects malformed batches before they reach the trainer.

Supervision quality moves the number more than you might expect. Mixing teachers beats piling on more of the cheap one: 3600 pairs from the small teacher alone gave 0.746, while a 2:1 split of 2400 small and 1200 large reached 0.851. Straight data scaling is gentler, 0.821 at 1440 unique pairs, 0.836 at 2400, still 0.836 at 3600, and 0.866 at 7200. If you are going to copy one design decision from this paper, copy the teacher mix, not the batch size.

Fifty seconds is a build, so they built it like one

A minute is an eternity for a web request and completely normal for make. The service leans into the second framing. Compiles are persistent background jobs with queue position and training progress, surviving page navigation and reloads, so you can keep browsing while yours runs.

Under that, teacher synthesis, model loading and training all start concurrently. Training begins as soon as the first batch has enough examples and only blocks if it catches up with synthesis, because teacher latency is the slow, variable part of the critical path. Workers also check a cache before requesting examples they have already paid for.

They also load tested it: four concurrent compile jobs finished with a mean queue wait of 1.01 second and even utilization across workers. That is a small number, on a small service, and it is still more operational detail than most demo papers bother to publish.

Three things they shipped on top

The composition argument is the part builders should read twice. paw-helper is an open repo that splits a generic pipeline executor from a site-specific content pack, and the deployed pack holds 30 compiled programs, 28 of them in live routing. One backend serves four sites, including the first author's CS 486 course page at Waterloo. When a student asks what changed about Assignment 1, a router picks a branch, one compiled function drafts from the course page, ordinary BM25 retrieval hits Piazza in parallel, another compiled function answers from the best post, and a fourth decides which answer to return.

That division of labor is the reusable idea: compiled neural functions make the fuzzy decisions, plain deterministic code handles retrieval, caching and control flow. Nobody has to hand a whole agent loop to a frontier model and hope.

Avatar Director pushes on output shape rather than labels. A compiled program turns "jump twice, then dance" into a small action DSL supporting sequences, durations, repetition and compatible parallel motions, and the browser validates and executes it. On 44 hand-authored validation instructions it produced the expected action structure 43 times.

Then there is the Claudish translator, which converts prose in both directions between plain English and the house style of Claude Code. The paper's characterization of the target dialect is delivered completely straight:

The style often uses explicit contrasts; metaphors of gates, boundaries, and load-bearing structure; and hyphenated constructions such as X-shaped and X-gated.

Two specifications, one per direction. It is simultaneously a joke and the cleanest possible demo of a task that is easy to describe and impossible to write as rules.

What this is not

• Not a general accuracy claim. 0.836 is on a benchmark subset deliberately chosen to be the hard cases for the previous method, graded by an LLM judge rather than string equality.
• Not free compute. You still pay teacher tokens and GPU time at every compile. The bet is that you compile a function far less often than you call it.
• Not local end to end. Compilation is a hosted process: your spec goes to the service and comes back as an artifact. Only execution is offline.
• Not a reasoning model. The target is narrow recurring text-to-text functions, the stuff currently wasting a frontier call per row.

Key Takeaways

• Compile by training converts an English spec into a rank-64 LoRA over a frozen quantized Qwen3-0.6B interpreter in about 51 seconds on a B300, packaged as a versionable .paw file.
• On FuzzyBench-Hard it reaches 0.836 mean LEM against 0.224 for PAW's one-shot amortized compiler, a 0.612 absolute gain, judged by a GPT-5.5 grader validated at 0.977 accuracy on 128 human labels.
• Teacher mix beats teacher volume. A 2:1 split of small and large teachers hit 0.851 where the small teacher alone hit 0.746 at the same example count.
• The deployment pattern is the takeaway. paw-helper routes 28 live compiled functions alongside plain BM25 retrieval and ordinary control flow, one backend across four sites.
• The economics flip, not the capability. Teachers and GPUs become build dependencies; the shipped function calls neither, which is what makes per-row work cheap and offline.

Sources: Compile by Training (arXiv 2609.04199), Program-as-Weights (arXiv 2607.02512), the public PAW playground, programasweights/paw-helper on GitHub, EMNLP 2026