Chrome's Modern Web Guidance 1.0 Lifts Coding-Agent Pass Rates From 55% to 92%
On this page 7 sections
TL;DR
Modern Web Guidance 1.0 shipped on Nov 10: an open-source agent skill and CLI from the Chrome team that hands coding agents curated, current web platform know-how, so they stop defaulting to 2019-era JavaScript. In Chrome's own evals, about 200 tasks graded by roughly 1,700 Playwright and AST assertions, agents went from 55% pass unguided to 92% with the skill loaded. Firefox, Edge, and MDN now co-steward the project, and it installs into Claude Code, Codex, Cursor, Copilot, and most other harnesses in one command.
What shipped
Paul Irish announced the 1.0 on Hacker News on Nov 10, where it picked up 252 points and 72 comments by the next morning. The repo has existed since March; 1.0 is the finished form, a dormant skill plus a retrieval CLI, after what Irish describes as nine months of work on getting that shape right.
Install is one command from a project directory: npx modern-web-guidance@latest install. The same skill ships through the plugin marketplaces of Claude Code, Codex, Cursor, Antigravity, Copilot, WebStorm, Kimi Code, and Grok Build. You can drive it without installing anything: npx modern-web-guidance@latest search "animate a dialog modal backdrop" returns ranked guides, and legacy-scan walks a codebase for dated patterns. I ran the search command from a clean shell and got ten guide hits, from animate-to-from-top-layer to declarative-dialog-popover-control, in under five seconds.
Two skill packs exist: the core web guidance and one for Chrome extensions covering Manifest V3 and Chrome Web Store publishing. Code is Apache-2.0, guide content is CC-BY 4.0, and the repo passed 3,200 stars the morning after the announcement.
Why your agent writes 2019 JavaScript
Training corpora hold vastly more legacy code than modern code, so agents keep hand-rolling the old pattern even when a native API has existed for years. Your agent has read about <dialog>. It will still build a div overlay with a focus-trap loop, because that is what most of its training corpus did.
Chrome's evals name the same gaps from the other side. Agents fumble keyboard focus and inert management, reduced-motion support, accessible form validation, swipe and touch gestures, dialog and popover transitions, light and dark theming, main-thread task scheduling, and safe HTML sanitization with strict CSP. These are exactly the features a browser vendor can document better than anyone.
The fix is the packaging. The always-loaded part costs about 205 tokens, a card catalog rather than the library. When a frontend task shows up, the CLI fetches the one or two guides that match, and the agent works from those. Squeezing that stack small enough to feel instant took real engineering: what began as a 500 MB native search stack is now a roughly 7 MB pure-JS package that answers in about 10 ms on a CPU.
The receipts
Chrome built a closed-loop eval harness around the guides and published the dashboard. Prompts describe developer wants ("make my images load faster") without naming APIs, so the eval measures whether the agent finds the right platform feature on its own. Playwright graders check source patterns, computed styles, runtime DOM behavior, and accessibility states, and they are calibrated before use: golden solutions written by multiple agents must pass 100%, and the unmodified base app must fail 100%. Roughly 200 tasks, about 1,700 assertions, three harnesses.
The aggregate result, from Irish's post: 55% unguided, 92% guided. Per agent, from the Nov 8 runs recorded in the repo:
The harness also prunes. Where models already pass without help, the guide gets cut, and Irish admits some assumptions about what AI did not know turned out to be wrong. Guidance kept only where it measurably lifts is the part worth copying.
Early production numbers, vendor-published
Chrome's blog post carries two case studies. One covers Note, which ran a 180-run evaluation on its production codebase: pass rates on modern web checks went from 61% to 94%, and proper LCP image prioritization rose 28% across 431 merged PRs. Pensionbazaar pointed its agents at about 30 high-traffic landing pages for performance audits and cut manual auditing effort by 92%. Both numbers come from Chrome's own post, so treat them as vendor-published until someone replicates.
The catches
- Every number above is Chrome's own. The calibration discipline (golden must pass, base must fail) is the right hygiene, but the graders, the guides, and the conclusion come from one team with a stake in the outcome.
- The 92% is an aggregate. On Nov 8, Codex on GPT-6.1 landed at 82% guided, 28 points up from unguided. The jump is large, and the spread between harnesses is wide; expect different numbers on your own setup.
- The governance objection came up fast on Hacker News: one browser vendor now curates what every LLM learns about web development. Firefox, Edge, and MDN co-stewarding, open licenses, and building on Baseline soften that, and the guides are forkable. Chrome still holds the pen.
- Telemetry is on by default: anonymous install counts, the calling agent's name, guide retrieval IDs, and CLI search queries. Raw prompts are not collected. The opt-out is an environment variable:
DISABLE_TELEMETRY=1.
Key Takeaways
- Chrome shipped Modern Web Guidance 1.0 on Nov 10, an Apache-2.0 skill plus a CC-BY-4.0 guide corpus that steers coding agents toward native web platform features, co-stewarded by Firefox, Edge, and MDN.
- Chrome's evals, about 200 tasks and roughly 1,700 assertions, report agents going from 55% pass unguided to 92% guided; Nov 8 per-agent runs show +28 to +36 points across Gemini 4, Opus 5.5, and GPT-6.1 harnesses.
- The reusable design is the packaging: a roughly 205-token dormant skill plus a 7 MB, 10 ms retrieval CLI, so guidance enters context only when a task needs it.
- legacy-scan, trained on 30,000 GitHub repos at 92% precision on held-out repos, flags dated patterns your agent can then modernize.
- Vendor-published case studies: note went from 61% to 94% on modern web checks, with 28% better LCP image prioritization across 431 merged PRs; Pensionbazaar cut manual audit effort 92% across about 30 landing pages.
- All evidence is Chrome's own harness plus vendor case studies. No independent replication exists yet.
Sources: Chrome for Developers blog, Modern Web Guidance 1.0, Modern Web Guidance on GitHub, source repo and eval dashboard, official docs, Paul Irish's Show HN announcement