Demoscener-7B Fine-Tunes Qwen2.5-Coder on 26K Dwitter Demos, Its Maker Rating 40K Clips

TL;DR

A maker publishing as datavorous has released Demoscener-7B, a post-trained Qwen2.5-Coder-7B that completes 140-character JavaScript animations from Dwitter. The write-up, dated October 31, lays out a three-stage recipe: continued pretraining on 26,318 scraped dweets, a curriculum that over-samples the 5,000 most "awesome"-voted programs, and a final GRPO stage in which the author was the reward model, rendering eight candidate endings side by side and clicking the one they liked, about 40,000 animations in total. Perplexity on held-out dweets fell from 28.63 to 8.57 after the first stage and from 7.42 to 5.65 after the second. The adapters and the dataset are on Hugging Face under CC BY 4.0, with no training code. The Hacker News thread was mostly demosceners asking the author to leave their hobby alone.

Why Dwitter

Dwitter is a creative-coding site built by lionleaf, open source on GitHub, that first ran at the Solskogen demoparty in 2016. A dweet is a JavaScript function u(t) called 60 times a second, limited to 140 characters, with a 1920x1080 canvas c, its 2D context x, and shorthands S, C and T for sine, cosine and tangent plus R() for rgba strings. Readers vote dweets "awesome" and can remix any of them.

The author came to it through HellMood's Memories, the 256-byte MS-DOS intro that won the 256-byte compo at Revision 2020, and wanted to understand "how the author had arrived at some of its visual effects." Dwitter's remix culture made it a tidy testbed. Chop the last 30 characters off a 140-character dweet and, with a 64-character alphabet, there are almost 1.5 × 10⁵⁴ possible endings. The author wanted an autocomplete that proposes plausible ones, then bent it toward their own taste.

A first attempt used synthetic dweets generated with Claude Sonnet 5.5. The author found them unappealing and dropped them, so every program in the corpus was written by a person.

Stage one: continued pretraining on 26,318 dweets

The dataset card lists 26,318 unique programs pulled through Dwitter's public API, split 25,002 train and 1,316 validation with no code overlap. Code is kept verbatim, Unicode and code-golf tricks included, and each row carries the dweet id, URL, author, awesome_count, rank_by_awesome and a remix_family_id. The whole thing is a 3.68 MB Parquet file. The base checkpoint is the plain Qwen2.5-Coder-7B; the card spells out that it is the base model and that Instruct was not used.

After continued pretraining, perplexity on 30 held-out dweets dropped from 28.63 to 8.57. Perplexity is roughly the number of equally likely next tokens the model is choosing among, so picture a typist hovering over the keyboard: at 28.63 the base model is unsure across most of it, and at 8.57 it has narrowed to a handful of keys. That measures predictability of code, which is a different thing from beauty, hence the two stages that follow.

Stage two: a curriculum weighted by awesome votes

The second stage continues training on only the top 5,000 dweets by awesome count, split into five rank buckets of 1,000. At step 1 each bucket gets 20 percent of the sampling probability. Over 400 steps the schedule tilts toward the top: by step 400 the top 1,000 take 67.23 percent, the next 1,000 take 24.99 percent, and ranks 4,001 to 5,000 are down to 0.03 percent. Perplexity measured across 7,000 dweets fell from 7.42 to 5.65, and the author reports gains at every awesomeness rank, including dweets the curriculum never sampled.

"This turned out to be a breakthrough!" the author writes, adding that the continuations "became much more visually interesting" afterward. The reusable idea is that a platform's own votes are a free quality label, and a sampling schedule lets you spend them without training a reward model first.

Stage three: the maker is the reward model

GRPO, Group Relative Policy Optimization, samples a group of completions for one prompt, scores them, and nudges the policy toward the ones that beat the group average. The scoring is normally a verifier or a learned reward model. Here it was a person with a mouse. For each unfinished dweet the model sampled eight completions, a side-by-side tool rendered them, and the author clicked the one they found most pleasing, or skipped the group if none appealed. Those choices updated the policy, the tool drew fresh continuations, and the loop repeated for roughly 9 to 10 rounds of 500 groups each. The author puts the total at around 40,000 animations inspected by eye.

Somewhere around the 10,000th animation, the post admits, it dawned on them that vision-language models and preference scorers such as HPSv3 exist and could have done some of the clicking. "Sadly, it was too late." Forty thousand clicks is an expensive way to learn that lesson, and also the reason the model's taste is one specific person's.

Evaluation was a blind test: a friend judged 400 sets of demos drawn from four randomly chosen checkpoints without knowing which model made which, and checkpoint 4 came out ahead. The post gives the result as a figure, with no numeric breakdown. Compute came out of the author's GitHub Sponsors payout, spent on a rented GPU. No GPU type, hours or dollar figure is disclosed, and later inference ran on the author's Mac.

What you can download

The model repo holds three independent LoRA adapters, cpt/, curriculum/ and grpo/, each continuing from the stage before. You load the base with transformers, attach an adapter with PEFT, and switch between stages with set_adapter(). The card's example prompt is c.width=1920;for(i=0;i<80;i++)x.fillRect(, which is the kind of half-written dweet the model was built to finish. One caveat sits on the card: later stages were trained on quantized bases, so their probabilities on a shared BF16 base may differ from the original runs. Everything is CC BY 4.0. At the time of writing the model had 7 likes, no inference provider hosting it, and the dataset showed 40 downloads in the last month.

Two things are missing. There is no training code, so the recipe exists only as prose, and the side-by-side comparison tool the author built for the GRPO stage has not been released. The dataset card notes that "scores and ranks reflect the scrape, not current Dwitter counts." The post does not discuss the licence terms of the individual dweets it scraped.

The demoscene says no

The Hacker News thread, 48 points and 34 comments at the time of reading, was not a warm reception. "Trust me when I say there are few things the demoscene despises more than using AI to make demos," wrote velo_aprx. "This is oil and water." InvisibleUp argued the scene "is about mathematical prowess and the joy of puzzle-solving moreso than pretty pictures" and added: "Nobody in the scene wants a magic box that solves the puzzle for them." psenough offered "congratulations on rediscovering genetic algorithms in procedurally generated art." A scene whose founding sport is doing more with 256 bytes was never going to swoon at 7 billion parameters.

The defence came from GaryBluto, who asked "Who is this hurting, exactly? It's just a project designed for writing short JavaScript animations," and ChildOfChaos, who called AI "just the latest tech" and therefore "entirely what the scene is about." The sharper point in the thread came from InvisibleUp as well, who said an acceptable use would be "to trace the history of demoscene effects, or to explain how a particular scene works." That is close to the motivation the author gives for the project, and the post does include a remix trace back to the parallax checkerboard effect from Memories. The author had not replied in the thread when we read it.

Caveats

• The reward signal in stage three is one person's taste. The model will prefer what datavorous prefers, which is the point and also the limit.
• The blind test is one friend and 400 sets, with no statistical breakdown in the text.
• Perplexity measures how well the model predicts code and says nothing about whether the result looks good.
• Compute, cost, hyperparameters and the LoRA ranks are not disclosed, and there is no training code, so an exact reproduction is not possible.
• The dweets were scraped through Dwitter's public API; the post and dataset card say nothing about the terms under which that code was written.

Key Takeaways

• Demoscener-7B is Qwen2.5-Coder-7B post-trained to complete 140-character Dwitter animations, written up on October 31 and shipped as three stacked LoRA adapters plus a 26,318-dweet dataset, both CC BY 4.0.
• Stage one, continued pretraining on the scrape, cut perplexity on 30 held-out dweets from 28.63 to 8.57.
• Stage two over-sampled the 5,000 most awesome-voted dweets over 400 steps, ending with 67.23 percent of samples from the top thousand, and cut perplexity on 7,000 dweets from 7.42 to 5.65.
• Stage three ran GRPO with the author as the reward model: eight completions per prefix, one click or a skip, roughly 9 to 10 rounds of 500 groups, about 40,000 animations viewed.
• No training code, no cost figures, and a blind test of 400 sets by a single friend. The numbers describe one experiment and are not comparable across models.
• The demoscene's reaction on Hacker News was largely hostile, which matters if you plan to point a model at a craft community that defines itself by human constraint.

Sources: datavorous, "Post training a model to explore Demoscenes", Hugging Face: datavorous/Demoscener-7B, Hugging Face: Demoscener-Dweets-25k dataset, Hacker News discussion, Dwitter, Dwitter on GitHub, Qwen2.5-Coder-7B, DeepSeekMath (GRPO) on arXiv, HPSv3 on arXiv, Hackaday on HellMood's Memories