Tools

A 125M Model Finishes Your Piano Phrase. More Data Made It Worse.

August 22, 2026 · 00:10 UTC · Tools

TL;DR

Simon Edwardsson trained a 125M-parameter decoder-only transformer from scratch to continue a piano performance in real time, running entirely on a phone, and shipped it as a free iOS app called RollTab. Play a phrase on a MIDI keyboard, pause, and it keeps going at roughly 108 notes per second on an iPhone 15. The app is fun. The build log is the actually useful artifact: the single biggest speedup came from changing how a note is spelled as tokens, scaling the dataset 5x made the model measurably worse, and a short preference-tuning pass moved pairwise win rate from 24.55% to 69.05%. The weights are not released.

What it does

Connect a USB or Bluetooth MIDI keyboard, play something, stop playing. The model continues your performance, on device, offline, with no network round trip and no subscription. It is autocomplete for a piano roll: the same interaction shape as an inline code completion, except the prompt is eight to thirty-two notes you played with your hands.

Measured on an iPhone 15. Nothing leaves the device, which is why latency is a property of the phone and not of somebody's inference queue.

The representation was the optimization

Most MIDI language models emit a note as a little sequence of separate events: a time shift, then a note-on, then a duration, then a velocity. That means four autoregressive passes to advance the music by one note, and four chances to derail.

Edwardsson collapsed all of it into one compound event, NOTE(pitch, delta_onset, duration, velocity), where silence is encoded implicitly as the gap since the previous onset rather than as its own token. Each field gets its own embedding, the embeddings are summed, and separate output heads with a small nested decoder predict the fields. The transformer now advances the music one complete note per forward pass. His words on Hacker News: "roughly 5x fewer autoregressive passes per note."

Think of it as the difference between texting a friend one letter per message and just sending the word. Same information, a fifth of the notifications, and far less opportunity for the conversation to go sideways halfway through.

That reframing, not a kernel, not a quantization trick, is what made real-time generation on a phone possible. Worth remembering the next time you reach for a profiler before you reach for the tokenizer.

Five times the data, worse results

Here is the least fashionable sentence in machine learning right now: "I tried scaling the dataset to roughly 5x the size, hoping it would improve performance, but the resulting models were worse."

What worked instead was curation. The final corpus was a few hundred thousand MIDI files, roughly 300 million note events, mostly public-domain classical piano. The pipeline selected piano-focused material, stripped or reduced multi-track mixtures, filtered by note density and pitch and time coverage, and deduplicated using fingerprints that ignore transposition and tempo, so the same sonata in a different key does not sneak into both train and validation. Versions of the same composition were forced into the same split.

Three sizes were trained. The 125M is the one that shipped, and it is small enough that the whole app is under 200MB.

Two other training details are worth stealing. Augmentation was global transposition, uniform tempo scaling, jitter on duration and velocity, and randomly dropped prompt notes. And scheduled sampling between note fields, ramped from 0% to 50% over training, raised validation loss and improved the continuations anyway. If your eval metric and your ears disagree, your metric is the thing that is wrong.

The base model knew music. It had no taste.

Pretraining produced a model with a plausible internal grammar of piano writing and a weak sense of what makes a good continuation of a specific phrase. The fix was Direct Preference Optimization: generate several continuations per prompt, pick winners and losers pairwise, tune on the pairs.

Evaluation is where this gets interesting for anyone building generative tools without a benchmark to hide behind. There is no SWE-bench for "does this sound like music." Absolute 1-to-10 scoring from a judge model is noise. So the pairwise comparisons were scored by Gemini 3.5 Flash, split into a continuation score (does it follow the prompt) and a sounds-good score (is it musical on its own), with continuation weighted for the DPO signal.

Preference tuning nearly tripled the win rate. The pretrained model already understood music, it just did not know which continuation you wanted.

Note the shape of that curve: beta=0.10 helps a little, beta=0.03 and beta=0.01 help a lot, and the best result comes from consensus sampling at beta=0.03. Preference strength is a dial with a peak in it, not a slider you push to the end.

Prompt length mattered too. Four-note prompts were the hardest case, eight notes worked better, and 16 to 32 notes were substantially more reliable. Which is an honest way of saying the model needs a bar or two of context before it can guess where you were going, same as a human accompanist.

Getting it onto the phone

The PyTorch model was exported to Core ML with INT8 weight quantization. Training context is 512 notes; longer sessions are handled by keeping the most recent 384 notes and rebuilding the cache. The shipped app is 194.1MB, requires iOS 18 or iPadOS 18 (or macOS 15 on Apple silicon), needs a MIDI keyboard for live input, and costs nothing. First launch is, in the author's own words, "still annoyingly slow" while Apple's runtime optimizes the model for the device.

Training ran on four RTX 4090s the author already had access to, which is the other quiet lesson here. This entire project sits inside the compute budget of one enthusiast's desk.

What it is not

No open weights. Multiple commenters in the Hacker News thread asked some version of "can I have the model," and the question is still sitting there unanswered. If you do not own an iPhone, iPad, or Apple silicon Mac, you cannot run this.
The judge is a model. Every win rate above comes from Gemini comparing pairs, not from listeners in a room. It is a reasonable proxy and it is not the same claim.
The training data has an accent. Public-domain classical piano in, classical-sounding continuations out. Commenters on Hacker News flagged exactly the kind of tell you would expect from that diet, including a passage resolving to full cadences in the tonic where the idiom calls for a half cadence, which one described as a musical run-on sentence.
It is a continuation engine, not a composer. It extends what you played. It does not have a plan for your piece, and 384 notes of memory is not a plan.

Key Takeaways

A 125M-parameter transformer trained from scratch generates piano continuations at roughly 108 notes per second on an iPhone 15, fully offline, shipped free as RollTab.
Collapsing four separate MIDI events into one compound note token bought roughly 5x fewer autoregressive passes per note, and was the single biggest performance win in the project.
Scaling the training set about 5x made the models worse. Curation, deduplication that ignores transposition and tempo, and split hygiene beat raw volume.
DPO moved pairwise win rate from 24.55% to 69.05%, with the best result at beta=0.03 with consensus sampling. Preference strength has an optimum, not a maximum.
For generative work with no benchmark, pairwise judging by an LLM beats absolute scoring, and a rising validation loss can coexist with output that sounds better.
Four consumer GPUs, a few hundred thousand MIDI files, and INT8 Core ML export is the whole stack. No frontier lab required.

Sources: Training a 125M-Parameter Model to Autocomplete Piano, SimEdw's Blog, 20 August 2026, Show HN discussion (item 49373456), RollTab on the App Store, Direct Preference Optimization (arXiv:2305.18290), Scheduled Sampling (arXiv:1506.03099), Apple Core ML documentation

AImusicMIDIon-devicesmall modelsDPOCore MLcreative tools