Qwen3-4B De-Slop Model Trains Out AI Writing Tics on a MacBook Air; Em Dashes Hit Zero

TL;DR

Erik Markus Kannike released qwen3-4b-deslop-v4, a version of Qwen3-4B-Instruct-2507 that writes without em dashes, "not X, but Y" or reflexive lists of three, and needs no system prompt to do it. He built it on an M2 MacBook Air by flipping the sign of four neurons and then running three rounds of filtered self-distillation into a rank-16 LoRA. On 40 held-out prompts, his slop score dropped from 121 to 12.5 per 1,000 words, and 68% of answers landed in the range he measures for human writing. The weights are Apache 2.0, and the 2.5 GB Q4_K_M build runs in Ollama. His Show HN post, submitted October 19, had 146 points and 108 comments when we checked. The result is a cleaner writer with a flat voice, and the detector that scores it is one person's style guide written as regexes.

What shipped

The write-up on 100toolkit.com and the model card describe the same release. The repo has merged bf16 weights with the adapter and neuron edit already applied, so it loads like stock Qwen3-4B. If you want the pieces separately, it also has the 126 MB LoRA adapter on its own and a small JSON file listing the four neuron edits.

Quantized builds sit in a separate GGUF repo with Q4_K_M (2.5 GB) and Q8_0 (4.3 GB) files and an Ollama params file:

ollama run hf.co/erikkannike/qwen3-4b-deslop-v4-GGUF:Q4_K_M
llama-cli -hf erikkannike/qwen3-4b-deslop-v4-GGUF:Q4_K_M

The card recommends temperature 0.7, top-p 0.9, a repetition penalty of about 1.1 and no system prompt. Keep the penalty on: every training answer was under about 250 words, so without it the model can fall into repeating one sentence pattern once an output gets long. One Hacker News commenter reported that the Q4 build looped after 16k generated tokens.

The scorecard

Kannike's slop score is a weighted count of 26 traits per 1,000 words, taken from a style guide of AI writing habits. The traits include the em dash, "it's not X, it's Y", lists of three, rhetorical questions answered on the spot, signposts like "here's the thing", short fragments for emphasis, and words such as "delve" and "seamless". He says careful human prose usually scores under 15. Stock Qwen3-4B averages 121.

A regex cannot tell good writing from bad, so he tracked damage separately. Here is the original model against the released one on the 40 held-out prompts, which cover ten topics never seen in training, each in four genres (blog post, LinkedIn post, explainer, opinion piece):

• Answers at human level (score under 15): 0% to 68%.
• Answers that fit the requested genre and length: 78% to 80%.
• A 30-question factual and arithmetic quiz: 97% to 100%.
• Drift on 22 passages of human prose: a KL divergence of 0.061 nats per token from the original.
• Answers flagged as damaged (lowercase starts, broken punctuation, off topic): 0% to 5%, or 2 of 40.

How he built it

Step one: steering, which worked but needed a harness

An earlier experiment found the "assistant register" as a direction in the residual stream. He got it by contrasting the model's own answers in an exaggerated-slop persona against a plain one. Subtracting that direction during generation, together with a plain-style system prompt, took the score to about 15. That needs hook code running at inference time, though, and it shifts the model's predictions on ordinary text.

Step two: four neurons

He then read MLP neurons one at a time, following Nick Cammarata's method from Goodfire's essay Neural Networks Are Already Clean, and found a small circuit that writes "not". The patch file names the four neurons it flips: an em-dash writer, a dash/ellipsis neuron, a "not" writer and an anti-"rather" neuron (layer 32 neuron 4852, layer 33 neuron 348, layer 28 neuron 1121, layer 32 neuron 5418). Flipping their signs removed 70% of "not X, but Y" at runtime, with a KL of 0.001.

The trick that makes this shippable is that scaling a neuron's output is the same as scaling one column of that layer's mlp.down_proj weight matrix. Negate the column and the edit lives in the checkpoint, with no hook required. Think of each neuron as a fader on a mixing desk that pushes the next token toward a habit. Kannike wired four of them backwards, so the same signal now pushes away from the habit.

Step three: three rounds of filtered distillation

Steering and the neuron flip each fixed part of the problem, and only the flip lived in the weights. To get the rest into the weights he used distillation. A teacher assembled from whatever worked best so far answers 288 training prompts. Answers that pass the checks become training data, and a rank-16 LoRA on every attention and MLP projection learns to produce them from the bare prompt, with no system prompt. Each round trained for two epochs, with loss on the response only, and took one to three hours on the MacBook Air.

Round one got the score to about 20 with no prompt and no hooks. For round two, the teacher was the round-one model plus the neuron patch and the plain-style prompt. Adding direction steering on top produced the lowest score he measured, 9.5, from text with no capital letters that wandered off topic, so he removed it. The teacher then averaged 7.4, and a student trained on its 171 best answers scored 9.9.

Why the best score was the wrong model

Round two looks like the winner on the chart, but Kannike read the output and found the score was misleading. An opinion piece about a company's new CRM never took a position. A blog post about moving to a new city became a third-person story about a family with a leaking apartment. A piece about houseplants cited "a study from the University of Technology Sydney", which the model made up.

Almost every sentence also ran into the next with ", which" or ", so", 28 times as often as in the original model. He traced that habit to his own teacher prompt, which told the model to join sentences "with because, so, but, which". Two rounds of imitation amplified one instruction into a new tic. Ask for "which" and you get a classroom of whichers.

Kannike's fix was a second set of rule-based checks for whether an answer does what was asked. An opinion piece has to state a position early, a personal reflection has to use the first person, an email has to open with a greeting, and the length has to roughly match the request. On those checks, round two did what was asked only 60% of the time, against 78% for the original.

The round-three teacher got a prompt that put the task first. It wrote three answers per prompt, and an answer survived only if it scored under 12 and passed both the damage and task checks. About 30% did, covering 161 of the 288 prompts. The resulting model scores 12.5 instead of 9.9. Task fit went back to 80%, ", which" joins fell from 1.4 to 0.3 per 100 words, and invented studies nearly disappeared.

The lesson carries over to anything trained against a metric. A detector for bad habits will reward a model that drops the habits and the job together, so you need a second check for whether the answer still does what was asked.

The caveats, from the author

• Flat voice. The card says the model writes "runs of short declarative sentences", personal pieces read like diary entries, and it is "closer to a careful junior writer than a good one."
• It still invents specifics. A study or percentage appears in about one answer in thirty, and made-up names are more common. His example is a coffee shop on Maple Street run by a woman named Maria, which, judging by small-model fiction, makes her the busiest barista on the internet.
• It runs long. Requests for about 150 words often come back at 180 to 230.
• Narrow testing. Testing covered only short English prose in four genres. Code, long conversations, other languages and tool use are untested and may be worse than the base model.
• An unvalidated metric. No blind human test has checked the score yet, and the task-fit numbers flatter the model because the training data was filtered on the same checks. The same model scored 10.2 through the adapter and 12.5 as merged weights, which he puts down to sampling noise.

Hacker News readers were split. One commenter said the output is "still rather obvious" as AI-written. Another said the cleaned-up CRM post "looks like a millennial student who rambled on a page," while the slop version read like a draft you could send to corporate managers. A third pointed to earlier work: Sam Paech's gemma-3-27b-it-antislop model and his antislop sampler attacked the same habits at 27B.

What comes next

Kannike's diagnosis is that each student ended up about as good as the best answers its teacher could write. For the next teacher, he plans to give a base model, one that never went through assistant tuning, a curated set of human writing to imitate: public-domain essays and letters plus modern pieces. After that he plans the same pipeline on a 7-8B model and one around 30B, then a blind test where people pick the better text. The method write-ups behind this release are Where AI slop comes from and Slop, neuron by neuron.

Key Takeaways

• qwen3-4b-deslop-v4 cuts a 26-trait slop score from 121 to 12.5 per 1,000 words on 40 held-out prompts, with 68% of answers in the human range and em dashes at zero.
• The recipe combines four sign-flipped MLP neurons (baked into down_proj) with three rounds of filtered distillation into a rank-16 LoRA, each round run on an M2 MacBook Air in one to three hours.
• The lowest-scoring checkpoint (9.9) was worse: it ignored genre, invented studies and picked up a ", which" habit from the teacher prompt. Task-fit checks fixed that.
• Quiz accuracy and genre fit held (100% and 80%), and drift on human prose is 0.061 nats per token, but 2 of 40 answers were flagged as damaged.
• Apache 2.0 weights and a 2.5 GB Q4_K_M GGUF run in Ollama or llama.cpp. Use a repetition penalty of about 1.1, because the model can loop on long outputs.
• The metric is a regex style guide with no blind human test yet. Read 12.5 as fewer tics, and wait for a reader test before calling the output human.

Sources: 100toolkit: Qwen3-4B with the AI writing tics trained out, Hugging Face: qwen3-4b-deslop-v4 model card, Hugging Face: qwen3-4b-deslop-v4-GGUF, Hacker News: Show HN thread, Goodfire: Neural Networks Are Already Clean, Hugging Face: Qwen3-4B-Instruct-2507