PayPal Got 139 Mentions From Coding Agents. And Zero Installs.

TL;DR

On September 3, Armature published a study of 16,893 coding-agent sessions in which Claude Code, Codex and Cursor were asked to choose a third-party service for a real need in a real repository and then actually implement it. 5,292 sessions across 51 codebases and 18 categories passed validation and are now public, full traces and code diffs included. Two numbers matter. All three agents land on the same tool in only 42% of the cells. And being named by an agent is close to worthless: PayPal was cited 139 times in payment sessions and installed exactly zero times, while Stripe took 124 of those same 139.

The decision you quietly stopped making

Picking which database, email API or object store to wire into a codebase used to be a human decision with a human sales cycle attached. It is now increasingly a prompt. Whether you are a vibe coder who does not know the category name or a senior engineer who does, the actual npm install is being chosen by something that read a pricing page five seconds ago.

This is not hypothetical volume. Vercel reported on April 9 that over 30% of deployments on its platform were initiated by coding agents, up 1000% in six months, with Claude Code alone accounting for 75% of that agent traffic. The buyer changed. Nobody sent a memo.

How the study was actually run

Armature built 75 repositories in 10 languages, with fake company names, fake git histories, fake API keys and real lockfiles checked against package registries, then produced variants with specific third-party integrations surgically removed so an agent had to choose one. Each session ran in an ephemeral sandbox, rotated across E2B, Blaxel and Daytona to rule out provider effects.

The clever part is the simulated human. A Gemini 3.7 Flash orchestrator plays the user across multiple turns, asking the agent to analyze the codebase and recommend before approving an implementation. Armature found that skipping this step biased agents toward building everything in-house, since they had no one to ask for permission to add a dependency. A second Gemini instance acted as judge, throwing out sessions where the repository had effectively pre-chosen the answer.

1,163 prompt variations covered four personas: vibe coder, junior engineer, senior engineer, and engineer at a large enterprise. Roughly a fifth of prompts added constraints like cost or usage volume.

The three agents do not agree with each other

Give the same category to Claude Code, Codex and Cursor and they pick the same tool in 42% of cells. In voice agents, Claude Code chose Twilio, Codex chose the OpenAI Realtime API, and Cursor chose Vapi. Three answers, one question.

The reason shows up in how they research. Codex ran a web search in 94% of sessions, and in nine queries out of ten it used operators like site: to pin itself to a vendor's own domain. Cursor leaned on the web in about two thirds of sessions. Claude Code searched in roughly 30%, preferring its own priors, though it browsed three times as many pages as Codex when it did go looking. In newer categories where its priors are thin, such as sandboxes, Claude Code's search rate jumped to about 80%.

Think of it as three contractors quoting the same job. One phones every supplier in the book, one skims the trade directory, and one just uses the guy he has always used until you ask for something he has never fitted before.

Claude Code also builds in-house nearly twice as often as the other two, 19% of sessions against 10%. If your product competes with forty lines of hand-rolled code, that is your real competitor in one session out of five.

Getting mentioned is not getting installed

The most useful finding for anyone selling a developer tool is the gap between being considered and being chosen. Agents name the incumbents constantly and then wire up someone else.

LangChain was the single most-cited framework in the corpus at 194 mentions, and agents installed it four times. Netlify was named 152 times and deployed six. Adyen got 175 mentions and three installs. Supabase was the most-mentioned database at 242 and still lost heavily to Neon, which took 66% of database sessions.

Where markets are decided, they are very decided. Stripe won nine payment sessions out of ten, losing mainly in EU-regulated scenarios to specialists like Paddle and Mollie. Amazon S3 took 45% of file storage with Azure and GCP on 20% each. Email is the closest race in the set, with Resend at 35.6% install rate and Postmark at 27.4%.

Your repo's language picks your vendor

Ask the identical question in four repositories written in four languages and you get four different winners. This is the finding that should unsettle anyone who assumed agents apply a stable ranking.

Deployment behaves the same way. Vercel won on TypeScript repositories and took 100% of runs where Next.js was present, and was never recommended on Python repositories, where Render dominated. Category leadership is not a single number any more. It is a matrix indexed by language, framework and persona.

Your pricing page is now agent-facing copy

Some of the most consequential losses in the study came down to a line of text on a marketing page. Mailgun repeatedly lost to Postmark when agents read "1-day retention" on its free plan. Supabase kept losing database sessions because its bundled pricing surfaced auth, storage and realtime features to an agent that had been asked for a database and nothing else.

Across 5,292 sessions, 388 raised platform management overhead and 195 raised cost as a reason. Armature's read is that in a meaningful share of those, the disqualifier was how the information was presented rather than the underlying product. Your pricing table is being parsed by something with no patience for a bundle and no sales rep to call.

Who is telling you this, and why it matters

Armature discloses it in the first line of the post: it sells growth services to dev tools, and this study is part of its work on getting products picked by coding agents. A co-founder repeated the disclosure at the top of the Hacker News thread, where the reception ranged from professional interest to one commenter calling the entire business model scum for importing SEO tactics into the one channel that had not been strip-mined yet.

Both reactions are correct. The dataset is genuinely the largest public measurement of this behavior, the raw traces and diffs are published so you can check the conclusions, and the company publishing it stands to profit from every vendor who reads it and panics. Treat the numbers as real and the framing as a sales deck, which is the normal posture for vendor research.

What to do with this on Monday

• If you build with agents: the tool your agent installs is a function of your repo's language and your prompt's phrasing, not of what is best. Pin your stack choices in a project instruction file rather than rediscovering them per session.
• If you sell a dev tool: track install rate, not mention rate. LangChain proves you can be the most-discussed name in your category and still lose 190 out of 194 decisions.
• If you are writing docs: write for a reader that will not scroll, will not ask a follow-up, and will disqualify you over one retention limit in a free-tier table.
• If you are worried about paid placement: so is everyone else in that thread. Nothing in the current agent stack distinguishes a recommendation from an ad, and the agent acts on it before you see it.

Key Takeaways

• Armature ran 16,893 coding-agent sessions and published 5,292 validated ones across 51 codebases and 18 categories, with full traces and code diffs.
• Claude Code, Codex and Cursor pick the same tool in only 42% of cells, because they research completely differently: 30%, 94% and roughly 67% web-search rates respectively.
• Mentions do not convert. PayPal was named 139 times and installed zero times; LangChain was named 194 times and installed four.
• Winners flip by repository language: the same email prompt produced Resend on TypeScript, SendGrid on Python, Postmark on Go and Azure ACS on Java.
• Claude Code builds the feature in-house in 19% of sessions, roughly double Codex and Cursor, so your competitor is often no vendor at all.
• The study is published by a company that sells agent-optimization services to the vendors it measures. The data is open; the incentive is not hidden, but it is there.

Sources: Armature research: Which tools do Claude Code, Codex and Cursor choose?, Armature publications, Vercel: Agentic Infrastructure, Hacker News discussion