Harvard Study of 718 Firms: AI Coding Agents Add 30% More Code, No Measurable Extra Software

TL;DR

Two Harvard economists, Fiona Chen and James Stratton, got access to the engineering telemetry that Jellyfish collects for its customers: roughly 300 million GitHub, Jira and Google Calendar events from 725,938 workers at 718 firms between January 2021 and March 2026. Their working paper finds that once a firm adopts AI coding agents, lines of code per worker rise 30%, commits rise 20% and pull requests rise 23%. Resolved Jira issues do not move by a statistically detectable amount, and neither does employment. The extra code piles up in review: time from pull request to merge rises 49%, the share of PRs with changes requested nearly doubles, and comments per PR climb 35%. Ars Technica covered it on October 9 and the Hacker News thread hit the front page on October 12 with roughly the ratio of code-review opinions to data you would expect.

The dataset nobody else has

Most AI-productivity studies measure one of two things: how fast an individual finishes a toy task in a lab, or what a vendor's own usage dashboard says. This paper measures firms. Jellyfish sells an analytics platform to engineering organizations and, with customer opt-in, let the authors see the whole production pipeline: HR records, version control activity, issue trackers, calendars, and seat data from the GitHub Copilot, Cursor and Claude Code licensing APIs.

Agent activity is detected two ways: through the Claude Code API, and by fingerprinting commits and comments for bot signatures and trailers (the paper lists Cursor Agent, Claude Code, Devin and GitHub Copilot Agent on the authoring side, and CodeRabbit, Greptile, Copilot Reviewer, Cursor Bug Bot, Graphite and Bito on the review side). Firm-level agent adoption went from near zero in October 2024 to over 95% of the sample by January 2026. Assistant adoption started earlier and had reached about 45% of firms by April 2024.

The identification is a staggered difference-in-differences: firms that adopted earlier are compared with firms that adopted later or not at all, under the assumption that absent adoption they would have tracked each other. The authors lean on the fact that enterprise procurement (security review, legal, budget) makes adoption timing noisy relative to any firm's underlying trajectory, and they check the usual robustness suspects in an appendix.

More code, the same number of finished tickets

The agent effect on lines of code is 1,494 extra lines per worker-month on a baseline of 5,017, hence the 30% figure. Commits rise 4.57 on a baseline of 22.58, and pull requests rise 1.22 on a baseline of 5.34. The authors call these estimates consistent with, but at the low end of, the existing literature, and they say so because they measure adoption at the firm level, which averages over engineers who never touched the tool.

Then they look at output. Their output measure is Jira issues resolved, which at these firms happens when the work is deployed, plus Jira epics for whole features. For agents, resolved issues move 0.12 per worker-month on a baseline of 3.67, with a standard error of 0.17. The 95% confidence interval rules out an increase larger than 12% of baseline. Epics show nothing either. The obvious objection is that tickets got bigger, so they trained a model to predict how long each issue would take an engineer and found no change in average predicted size.

Employment is the second null. Measured by LinkedIn headcount, the agent effect on total firm employment is a precise zero that rules out a decline greater than 2.9%. Measured by active engineers inside Jellyfish, it is a noisier zero that rules out a decline greater than 13.8%. Splitting LinkedIn profiles into junior and senior produces nothing either. The authors are careful to call this a short-run, partial-equilibrium estimate: a firm that keeps its engineers while winning business from a competitor still shows up as a zero here, and the competitor's layoffs are not in the sample.

Where the code goes: the review queue

The paper's explanation is a bottleneck. Writing code is one stage in a pipeline, and the authors estimate that engineers spend only 30% to 42% of their time on it, depending on whether you trust a survey of 100 engineers on Prolific or the time allocation imputed from the Jellyfish signals. Speed up that stage alone and the next stage fills up. It is the kitchen sink problem: turn the tap up and the basin fills faster, but the drain is the same diameter, and the drain here is a senior engineer with approval rights and a calendar.

After agent adoption, the time from PR submission to merge rises 3.45 days on a baseline of 7.03, the 49% figure. The share of PRs with at least one formal request for changes rises from 13% to 25%. Review comments per PR go from 1.66 to 2.24, a 35% increase. PR size, measured in lines added, does not change, so this is not simply bigger diffs taking longer. Assistants, the autocomplete generation of tools, move none of these review metrics at all.

Firms do react, just not by hiring. The share of workers who comment on or approve at least one PR in a month rises from 29% to 33%, a 14% increase, while the share who write code stays flat. People are being drafted into review without being released from coding. One junior engineer the authors interviewed put the texture on it: on their project only one senior engineer was allowed to approve changes because he was the expert on the codebase, and he was so bandwidth constrained he could review their code once a week. Your agent writes the PR in two seconds. Your tech lead reads it the following Thursday.

AI reviewers have not drained it yet

The natural rebuttal is that the bottleneck is temporary and AI review tools will clear it. The data says firms have tried. By March 2026, nearly 80% of the sample had used an AI tool in code review. But on the intensive margin the machines are a minority voice: AI generated 23.3% of all review comments, and only 10.8% of pull requests received even one AI-generated comment. Humans still do most of the reviewing, and a senior engineer in the interviews explained why: the tools lack comprehension of the overall feature and what it is supposed to do, so "you absolutely still need humans involved in the review process."

Caveats, straight-faced

• This is a working paper and a job market paper, current version dated August 4, 2026, not a peer-reviewed publication. Jellyfish reviewed the manuscript only to confirm it discloses no proprietary information, not for scientific content.
• The sample is Jellyfish customers who opted in to research use. These are firms that already pay to measure their engineering teams, which is not a random draw of the software industry.
• Treatment is firm-level adoption, so the 30% coding gain is diluted by engineers who never used the tool. Individual-level gains are larger in other studies, and the authors say so.
• The data ends in March 2026. As one HN commenter pointed out, the agent results mostly reflect the 2025 generation of models and the first months of each firm's deployment, before review processes had a chance to adapt. The paper does say the bottleneck persists across its window, but it cannot speak to what current models do.
• The employment null is partial equilibrium. Adopting firms keeping their engineers is compatible with job losses elsewhere.

What a builder does with this

If you run an engineering team, the paper hands you a metric and a target. The metric is PR-to-merge time, not lines of code or commits, because the latter two are exactly what inflated without producing output. The target is the review stage, and the evidence says the fix is capacity and process rather than another autocomplete seat: assistants did nothing to review time, agents made it worse, and AI reviewers so far touch one PR in nine.

The HN thread is split on what that process should be. One camp argues senior engineers cannot give up line-by-line review and that the culture shift will come from startups that never had human review in the first place. The other camp points out that it has been hearing about those startups for over a year and keeps asking where their products are. The paper cannot settle this. What it can tell you is that the 30% coding speedup is real, that it buys you nothing at the firm level until the drain is widened, and that so far almost nobody has widened it.

For a one-person shop the lesson inverts. You are the whole pipeline, so an agent's output lands directly in your own review queue. The paper's numbers suggest budgeting roughly as much time to read agent code as you saved writing it, and treating "changes requested" doubling as the default rather than a bad day.

Key Takeaways

• Chen and Stratton's Harvard working paper uses Jellyfish telemetry on 300 million work events from 725,938 workers at 718 firms, January 2021 to March 2026, with a staggered difference-in-differences design.
• AI coding agents raise lines of code 30%, commits 20% and pull requests 23% per worker-month. Assistants add 12%, 9% and 5%, with only commits significant.
• Resolved Jira issues and epics show no statistically significant change; the agent estimate rules out any output gain above 12% of baseline. Predicted issue size did not change.
• Review absorbs the gain: PR-to-merge time rises 49% (7.03 to about 10.5 days), PRs with changes requested nearly double from 13% to 25%, comments per PR rise 35%, and the share of workers reviewing rises 14%.
• Employment does not move. Agents rule out a total headcount decline greater than 2.9% and an engineering decline greater than 13.8%, in the short run and within adopting firms.
• Nearly 80% of firms had used AI code review by March 2026, yet AI wrote 23.3% of review comments and touched 10.8% of PRs.

Sources: Chen and Stratton, "Artificial Intelligence in the Firm: Bottlenecks in Software Production" (PDF), Fiona Chen, Harvard, Jellyfish, Ars Technica, Hacker News discussion