Loop Engineering: Why Anthropic's Best Code Comes From Agents That Run in Circles

TL;DR

Anthropic now says over 80% of its merged code comes from Claude Code agents, up from low single digits before 2025. The engineers behind that number, including Claude Code creator Boris Cherny, describe their job as writing loops rather than prompts: cycles where an agent sets a goal, generates work, tests it, reads why it failed, and runs again. That is the whole trick, and it is not specific to code. A loop without a scoring function and an out-of-sample gate does not find answers faster, it finds prettier garbage faster. Here is what loop engineering actually is, why one prompt stalls, and the one honest step the hype skips, with quant trading as the worked example.

The Claim, And What Is Real About It

The headline making the rounds: at Anthropic, the share of merged code authored by Claude Code agents went from single digits to more than 80% in roughly a year, and the framing is that engineers shifted from manual prompting to running 100+ agents on self-improving loops. Treat the exact percentage as a directional company stat, not a peer-reviewed result. The part that is genuinely useful is the method, not the metric.

The method is this: stop hand-writing one prompt and grading the answer. Instead, write the loop that lets an agent grade its own answer against a goal, learn from the failure, and try again, dozens of times, while you watch the architecture instead of the keystrokes. Andrej Karpathy and others have been pointing at the same shape: git worktrees so parallel agents do not stomp each other, a CLAUDE.md of hard rules to cut error rates, tight test feedback so each pass is cheap. The human moves up a level, from author to director.

Why One Prompt Never Finds The Answer

A single prompt is a single guess. The model returns something generic, you run it, it usually fails, because the first idea is rarely the right one. Without a loop, every attempt starts from zero and nothing compounds. You are not searching, you are sampling once and hoping.

The whole edge lives in the iteration. A loop turns one attempt into a search: each round keeps what scored well and feeds the rest back in for refinement. That is the difference between asking a model to write a function and giving an agent a failing test suite and the autonomy to keep going until it passes. One is a guess. The other is a process that converges.

What A Self-Improving Loop Actually Is

Strip away the branding and a loop is a closed cycle with five beats: perceive, reason, act, observe, repeat. Concretely:

• Generate. The agent produces a candidate: code, a fix, a plan, a strategy.
• Test. It runs that candidate against reality: a test suite, a compiler, a benchmark, a backtest.
• Score. It grades the result against a clear objective, not a vibe. A number it is trying to move.
• Read the failure. It feeds the reason for failure back in, so the next attempt is informed, not random.
• Repeat. It runs again, and the search narrows because every result rules something out.

Each pass is cheap, so you can afford dozens. The system gets sharper because the scoring function is doing the selecting, not your patience. This is why "self-improving" is accurate but oversold: the model is not getting smarter, the loop is throwing away bad candidates faster than you ever could by hand.

The Step The Hype Skips

Here is the part nobody puts in the viral thread. A loop that optimizes on the same data it was built on does not find a real answer faster. It finds noise that looks like an answer faster. Every extra iteration is another chance to fit the quirks of the exact problem in front of it instead of the general case.

In code, this is the agent that hard-codes its way to a green test suite, or special-cases the one input the test checks. In research, it is a model that nails the training set and faceplants on anything new. The loop is a search engine, and a search engine with a sloppy objective will happily find you the wrong thing at high speed.

The only thing that turns a loop into a real engine is a held-out gate: every surviving candidate gets tested on data, inputs, or cases it never saw during the loop, and it has to hold up there too. In software that is integration tests and review the agent did not write to. In quant it is out-of-sample data. Skip the gate, and loop engineering is just automated overfitting with a nicer dashboard. Count your attempts honestly, because the more variants you try, the higher your bar has to climb to mean anything.

The Same Loop, Running A Trading Desk

The cleanest proof that this is a method and not a code-specific trick: it is exactly how serious quants hunt for alpha. The story is familiar to anyone who has done it. Your backtest looked flawless. You went live. Two weeks later the strategy was bleeding. That strategy died because it was one guess with nothing iterating on it.

The fix is the same loop: generate a strategy, test it, score it, feed the result back, run it again until one survives out-of-sample. What makes the quant version instructive is that the scoring function is explicit math, so you can see exactly what the loop optimizes toward.

The metric that closes the loop

A loop without a scoring function just wanders. In factor research the standard objective is the information coefficient, the correlation between a factor's value today and the return that follows:

IC = corr(factor_t, return_t+1)

A single IC reading is noisy, so the number that actually matters is its consistency over time, the ICIR:

ICIR = mean(IC) / std(IC)

A factor with a modest but steady IC beats a flashy one that works once and breaks. The ICIR is what the loop drives toward, the same way a code loop drives toward "all tests green and still green tomorrow."

The decay check most people skip

Even a real factor has a shelf life. You can measure it: model the signal's persistence as an AR(1) process and read its half-life.

t_half = -ln(2) / ln(rho)

where rho is how strongly the factor carries from one period to the next. A half-life of 50 days is a slow, tradeable edge. A half-life of two days means you are paying transaction costs to chase noise. A good loop rejects short-half-life factors automatically, the way a good test suite rejects a fix that only works for one input.

The gate, again

Same honest step as the code case. A surviving strategy gets tested on data it never saw, and the ICIR has to hold there too. The more variants the loop tried, the higher that out-of-sample bar climbs, because you have given yourself that many more chances to get lucky. This is the line between a research engine and a curve-fitting machine, and it is identical whether the candidate is a Python patch or a momentum factor.

This framing comes from the team building Horizon, a platform that runs the generate-test-score-refine-deploy loop for plain-English trading strategies. You do not need their product to use the idea. The loop is the point.

How To Run A Loop Without Fooling Yourself

The failure modes are the same across domains, and they are worth memorizing because they are how loops lie to you:

• Looping on the data you built on. Faster iteration on the same history, or the same test, just overfits faster. The loop needs fresh data at the gate.
• No scoring function. Without a clear objective, the loop optimizes for nothing and drifts. "Looks good" is not a metric.
• Chasing the spike, ignoring stability. One great month of IC, or one green CI run, is a costume. Consistency is the edge.
• Mistaking the model for the method. A better model speeds up generation. The loop, the scoring, and the held-out gate are what actually produce a result. Swapping in a smarter model with no loop just gets you a smarter single guess.

The Checklist

• Define the objective before you start, and make it a number on a held-out window.
• Generate a handful of variants each pass, not one.
• Score every candidate, read the failures, feed them back in.
• Check for decay, in quant via half-life, in code via "does it still pass on cases I did not write for."
• Gate every survivor on data or inputs it never saw, and raise the bar as your attempt count grows.
• Keep only what holds. Throw away the rest without sentiment.

Key Takeaways

• The shift is real, the number is directional. Anthropic crediting 80%+ of merged code to Claude Code agents is a company stat, but the underlying move from prompting to loop engineering is the genuine lesson.
• One prompt is a sample, a loop is a search. The edge is in the iteration, because the first attempt is almost never the answer.
• A loop is generate, test, score, read the failure, repeat. Each pass cheap, the scoring function doing the selecting.
• "Self-improving" means the loop discards faster, not that the model gets smarter.
• The held-out gate is the whole game. Without it, loop engineering is automated overfitting. With it, it is a research engine. Same truth in code and in quant.
• It is a method, not a code trick. The exact same loop hunts alpha on a trading desk, scored by ICIR and half-life instead of a test suite.

Sources

• Anthropic and Claude Code agent-authored code share, as discussed by Boris Cherny (Claude Code creator) and echoed by Andrej Karpathy on agent loops, git worktrees, and CLAUDE.md rules.
• Loop engineering framework for quant factor research (IC, ICIR, AR(1) half-life, out-of-sample gating), as articulated by the team behind Horizon (horizon.trade).