0.1% of ClaudeBot Traffic Isn't ClaudeBot. It Wants Your Keys.

TL;DR

Known Agents, which runs an index of AI agent traffic across more than 5,000 websites, is flagging an active campaign that impersonates AI crawlers in order to scan for vulnerabilities. Its wording is blunt: "We are observing a widespread campaign impersonating AI bots to scan websites for vulnerabilities. The attacker appears to be targeting credential and configuration paths used by AI coding tools." The sample path list includes /.config/anthropic/credentials/default.json, /.claude/settings.json, /.codex/config.toml, /.continue/config.json, and /.aider.conf.yml. Measured spoof rates sit at 0.5% for Googlebot and 0.1% each for ChatGPT-User, OAI-SearchBot, GPTBot, PerplexityBot, and ClaudeBot. The Hacker News thread hit the front page overnight.

The interesting part is not the spoofing

People have been forging User-Agent headers since roughly the moment the header existed. That is not news, and several operators in the thread said so directly. Junk traffic is junk traffic.

The news is the wordlist. Commodity scanners have hammered /.env, /.git/config, and /.aws/credentials for a decade. Those are still in the list. What is new is that somebody sat down and added a section for AI developer tooling, and it is a well-researched section. Somewhere there is a wordlist repo with a merged pull request titled something like "add agent configs," and it was the highest-ROI patch that project shipped this year.

Why those files specifically? Because an agent config is not one secret, it is a keyring. A .env gets you one app's database. An aider config or a Continue config can hold provider keys for several model vendors at once, because that is the entire point of a tool that lets you switch models mid-session. One hit, a fistful of API keys, and a billing account that nobody watches as closely as they watch AWS.

The rest of the published list rounds out the greatest hits: /service-account.json, /firebase-adminsdk.json, /terraform.tfstate, /.docker/config.json, /rclone.conf, and six variations on .env including /.env.backup and /.env.old. Whoever built this understands that the file people forget to delete is worth more than the file they are careful about.

What a "spoof rate" actually measures

Known Agents defines spoofed traffic as requests that claim "the identity of a known agent but fail a supported authentication method," meaning verified IP ranges or HTTP message signatures. The rate is computed per website per day, then averaged across qualifying sites. That is a meaningfully different number from "0.1% of all crawler traffic on the internet," and you should read it that way.

Googlebot leading at 0.5% is the tell that this is economics, not fandom. The identity worth stealing is the one that gets the most doors opened, and for twenty years that has been Googlebot. The AI crawlers sitting at 0.1% are early in the same curve.

Why the costume works at all

A user-agent string is a claim, not a credential. It is the conference name badge you filled in yourself with a Sharpie: it tells the room what you would like to be called, and it proves precisely nothing. Verification is the guest list at the door, and most sites never check it.

That matters more now than it did in 2019, because a lot of operators have deliberately built exceptions for AI crawlers. Some allowlist them for AI-visibility reasons. Some block everything and carve out the ones they want indexing them. Either way, a header the attacker controls for free is now sometimes worth a softer rate limit, a cache-warmed response, or a bypassed challenge. The cost of trying is zero, so it gets tried.

How you actually check

Two mechanisms exist, and they are at different stages of maturity.

IP range verification

The old way. OpenAI publishes GPTBot's ranges as JSON, currently 21 IPv4 prefixes, mostly /24s and /25s. You fetch the file, you check the source address, you drop what does not match. It works, and it is what most operators can deploy today.

Web Bot Auth

The new way, and the one Known Agents counts as a "supported authentication method." Cloudflare's Web Bot Auth has the bot sign each request using RFC 9421 HTTP Message Signatures with an Ed25519 key, and attach a Signature-Agent header pointing at where its public keys live. The architecture draft is in IETF process and the reference implementation is on GitHub.

Cloudflare's argument for moving off IPs is not theoretical: addresses get shared between services, infrastructure churns, and privacy proxies put unrelated traffic behind the same egress. A signature travels with the request and does not care whose datacenter it left from.

What the operators are reporting

Treat this part as anecdote, because it is. Site admins in the Hacker News thread describe a surge over the past week, with several pointing at Google Cloud IP space and one claiming sustained traffic in the tens of thousands of requests per minute. Multiple people put the escalation around August 6. Several noted that volume changed simultaneously across many unrelated IPs, which reads as centralized control rather than a hobbyist.

None of that is independently verified, and none of it needs to be for the takeaway to hold. Whether it is one actor or five, the request pattern is public and the defense is the same.

The caveats, straight

• Known Agents sells traffic analytics. The data is from its own network of 5,000+ sites, not an independent audit, and the page ends with a sales pitch.
• Rounded percentages at 0.1% granularity hide a lot. A rate that is really 0.14% and one that is really 0.05% both display as 0.1%.
• Scanning for /.claude.json over HTTP only works if your web server is serving a directory that contains one. That is a deployment mistake, not an agent vulnerability. Nothing here says Claude Code, Codex, or aider is broken.
• These are per-site daily averages, not a measurement of global crawler traffic composition.

What to do about it

Nothing exotic, which is the good news.

• Stop treating the user-agent as an identity. If you have an AI-crawler allowlist, gate it on verified IP ranges or a Web Bot Auth signature, or delete the allowlist.
• Grep your access logs for the paths above. A 404 is fine. A 200 is your morning.
• Make sure your document root cannot serve dotfiles at all, rather than blocking this week's specific list.
• Keep agent config out of anything a web server can reach. Home directory, correct permissions, not next to your app.

The uncomfortable framing: your Claude Code config, your Codex config, and your aider config are now standard entries on a commodity scan list, right next to wp-config.php. That is what mainstream adoption looks like from the attacker's side.

Key Takeaways

• Known Agents reports a widespread campaign impersonating AI crawlers to scan for vulnerabilities, targeting "credential and configuration paths used by AI coding tools."
• The published sample list includes /.config/anthropic/credentials/default.json, /.claude/settings.json, /.codex/config.toml, /.continue/config.json, and /.aider.conf.yml, alongside 30 more classic secret paths.
• Spoof rates across 5,000+ sites: 0.5% for Googlebot, 0.1% each for ChatGPT-User, OAI-SearchBot, GPTBot, PerplexityBot, and ClaudeBot, measured as requests failing IP or signature verification.
• Agent config files are attractive because one file can carry keys for several model providers, unlike a single-app .env.
• Defenses are unglamorous and available now: verify crawler IPs against published ranges, adopt Web Bot Auth signatures, and stop serving dotfiles.
• This is a deployment-hygiene problem, not a flaw in any coding agent. The scanners just updated their wordlist to match where developers now keep their keys.

Sources: Known Agents Agentic Web Index, Hacker News discussion, Cloudflare on Web Bot Auth, Cloudflare Web Bot Auth docs, RFC 9421, draft-meunier-web-bot-auth-architecture, OpenAI GPTBot IP ranges