DEV Community

Jason Miller
Jason Miller

Posted on Originally published at axeploit.com

Google's AI Found 500 XSS Bugs, and the Model Is the Least Interesting Part

#dast #aiagents

Google turned an AI agent loose on its own web apps and it came back with more than 500 verified XSS vulnerabilities. None of them will ever pay a bounty. That second sentence is the real story.

The agent is PageBreak, built by Google's Product Security team. Pilot in November 2025, full project by January 2026, running mostly on Gemini 3.1 Pro and 3.5 Flash. Coverage has fixated on the 500. The 500 is a marketing number. The architecture is the news.

The validator is the product

Every AI security demo follows the same script: point a model at code, get a plausible vulnerability narrative, declare victory. PageBreak works because Google's team refused that script early. The model generates hypotheses. Then a suite of validators, written by humans, not generated by the model, executes real payloads against running environments. A finding only exists if the exploit actually fires.

The validators are boring, and that is the point:

  • XSS: inject a JS payload, confirm execution in a rendering harness
  • SQLi: check output, timing, or both
  • Path traversal: write a file somewhere world-readable, then read it back
  • SSRF: watch for outbound requests to internal services

The gate is roughly:

if validator.confirms(hypothesis):
    report(hypothesis)      # a human sees this
else:
    quarantine(hypothesis)  # kept as a seed, never reported
Enter fullscreen mode Exit fullscreen mode

That quarantine is the discipline. Unverified candidates never reach product teams. Compare that with tools that dump every hypothesis on a dashboard and call it coverage. If your agent cannot execute its own claims, you are paying triage engineers to read someone else's inference output.

The trade Google actually admits

Deterministic-only validation guarantees false negatives. Bug classes without validators, or bugs needing multi-step state the harness can't reproduce, get missed. Google says this out loud, which is more than most vendors manage.

Proof the trade is real: against apps built on Google's high-assurance web frameworks, PageBreak found exactly 2 XSS across hundreds of apps. Secure-by-default frameworks nearly zero out the class, and whatever sits outside validator coverage (WebSocket-heavy surfaces, for one) stays human territory.

Precision is a trust problem. Recall is a roadmap problem you fix by writing more validators. Google picked correctly: once product teams learn your queue is noise, they stop reading it permanently.

What this does to bounty economics

Here is the asymmetry nobody covered. Internal agents like PageBreak file only verified bugs. External hunters with off-the-shelf LLM tooling will file more unverified ones, because their incentive is volume and the tool's incentive is plausibility. Your external noise rises at exactly the moment mature companies stop generating their own.

And look at what Google chose to show off: a cache-poisoning bug where the tainted path segment was excluded from the cache key, a signature bypass chained across two endpoints on admin.google.com, a universal XSS in an extension's message passing. None of those is a single-step reflected XSS a 2009-era scanner catches. That is the human niche that survives: chained, context-heavy bugs.

If the owner of a surface can enumerate single-step XSS at near-zero marginal cost, the market price of finding one more collapses. Google's 500 internal bugs are 500 reports that will never be purchased from the outside market.

Takeaways you can use this week:

  • Running a bounty program? Gate triage on execution evidence: a payload that fires, a trace, a video, or a run in a sandbox you provide. Everything else goes to automated reproduction before a human reads it.
  • Reprice your payout table now. Lower the base for single-step web bugs, add multipliers for chained impact, and publish which classes your internal automation already covers.
  • Evaluating AI appsec tooling? Ask one question: show me the validator. If findings are not confirmed by execution, you are looking at a demo.
  • Hunting for a living? Stop farming single-step XSS on big surfaces. That window is closing.

For the comments: does "near-zero false positives" convince you when it is self-reported, or does the mechanism (no execution, no report) make the claim falsifiable enough to trust?

Longer writeup if you want the full argument: https://axeploit.com/blog/500-xss-bugs-zero-bounty-payouts-what-google-s-pagebreak-agent-does-to-the

Top comments (0)