It started on a phone screen, during a routine check of the product. The challenge was called Leaked UUID. The player gets an invoice, inv-9af3, and the goal is to read a different customer's invoice. The brief said the other reference "had appeared in a shared-link response". There was no shared-link response in the scene. There was nothing. The player was supposed to just… know it.
The challenge had been live for weeks. Nobody had flagged it — because a challenge that can only be solved by guessing looks exactly like a hard challenge.
Three out of eighty-one
That weekend every one of Breachloom's 81 challenges went through the same audit. An AI assistant did the tedious pass — every brief, every hint, every simulated server — answering one question per challenge: where does the player get this value? If the answer was "from the brief" or "by guessing", the challenge was broken.
Three were.
Leaked UUID. The invoice the player may see is inv-9af3. The one they are meant to reach is inv-2b7c. Nothing in the scene ever mentioned inv-2b7c. Fix: the API response for the player's own invoice now carries a sharedFrom: "inv-2b7c" field — the kind of over-sharing a real API produces when the database model is reused as the response model. The player has to notice it, then ask the question that matters: does the server check ownership once the reference is known?
Hidden field. A GraphQL API with a passwordHash field the UI never asks for. The player had to request it by name, but introspection was blocked, so there was no way to learn the name. A lock with the key left outside the room. Fix: the server now does what real GraphQL servers do — answer an unknown field with Did you mean "passwordHash"?. Field suggestions leak schema even with introspection off; a whole tool, Clairvoyance, is built on exactly that.
The CEO's project. Stage two of a multi-step chain needed a project ID that stage one never showed. Fix: the viewer query in stage one now returns the list of projects the account can see — and the list leaks prj-ceo while the detail endpoint still guards it. The oldest access-control bug there is: the list is generous, the detail is strict, and nobody compares them.
Three out of eighty-one is not a scandal. It is what should be expected from any team that writes challenges without an adversarial pass. The uncomfortable part is how long they stayed live, and why: a guessing game and a hard puzzle produce the same telemetry.
The rule that came out of it
Every value the player needs must be discoverable inside the scene, by a method that works on real systems.
Not "in the brief". Not "in the hint". In the scene — a response header, a leaky list endpoint, an error message that says too much, a notification email. If the discovery step does not exist in the wild, it does not go into the challenge.
That rule sounds obvious. It is not how most security training works.
Where the flag stops
Most labs end when the payload works. Paste ' OR '1'='1, the page shows admin, confetti. The reading material often has a "how to prevent this" section. The exercise never asks the learner to write it.
That is not malice; it is lineage. Capture-the-flag was born as offensive training. Gasiba and colleagues, who study security training in industry, cite a survey of more than 4,000 developers in which fewer than half could spot a security hole — and their answer was a new kind of challenge, built around entering code with an automated coach checking it, rather than around breaking in (Gasiba et al., 2021). The "Build It, Break It, Fix It" contest made the same point a decade ago by adding a phase in which teams fix what others broke in their code (Ruef et al., 2016).
Meanwhile the people who ship the software mostly never get taught. The Linux Foundation and OpenSSF surveyed about 400 development professionals in 2024: nearly one in three said they were unfamiliar with secure development practices; 69% learn it on the job, and the report estimates it takes around five years of that to reach a minimum level of familiarity (OpenSSF, 2024).
So there are two pipelines. One produces people who can make a payload work and cannot say which line to change. The other produces developers who were never shown the payload. The word for the first group used to be script kiddie. It was an insult. It is more accurately a description of a training design.
The same failure, at scale
What is happening to bug bounty programs has the same shape.
curl shut its bounty at the end of January 2026. Daniel Stenberg's stated reason: "remove the incentive for people to submit crap and non-well researched reports to us. AI generated or not." In one sixteen-hour stretch that month the project received seven reports; none was a vulnerability (BleepingComputer, The Register). Elastic's program received more than 1,390 reports in the first half of 2026 — more than 2024 and 2025 combined — and roughly 70% were closed at analysis; the triage team's summary was that LLMs "made it trivially cheap to generate vulnerability reports" (Elastic Security Labs). HackerOne's own numbers: submissions up 76% year over year to a record 46,947 in March 2026, about a quarter confirmed exploitable, and the backlog of validated but unfixed bugs up more than twenty-fold, because — in HackerOne's words — discovery used to be the bottleneck and now remediation is (HackerOne).
A report that says "this payload does something" without "here is the decision that is missing and here is the line that makes it" is a script-kiddie report. The industry has now automated writing them. And the one skill that would make the pipeline useful again — understanding the bug well enough to fix it — is the skill almost nobody trains for, because the exercise ends at the flag.
What changed at Breachloom
Two things.
Every challenge now has to pass the discoverability rule above. Seventy-eight honest challenges beat eighty-one with three guessing games in a security costume.
And the part that matters more: after the exploit there is a second half. Eighteen missions hand the learner the vulnerable code; they make the fix, and the server runs its checks against their version. Not "explain the fix" — ship it, and have it verified. The IDOR mission uses the invoice endpoint from the story above and asks for the decision that was missing. It is harder than the exploit. It is also the only half that would have got any of those 1,390 reports past triage.
The three fixes are live:
- The leaked-UUID invoice, now with a real leak: https://breachloom.com/ctf/idor-uuid/
- The hidden GraphQL field, now discoverable the way it is in the wild: https://breachloom.com/ctf/gql-fields/
- The chain where the list leaks what the detail protects: https://breachloom.com/chains/graphql-tenant/
- The fix-the-code half: https://breachloom.com/codeops/owner-bind/
Everything runs in the browser against fictional apps. No VM. And anyone who finds a fourth challenge that can only be solved by guessing is invited to report it — that is the kind of bug report worth paying for.
Sources: Gasiba, Lechner, Pinto-Albuquerque, Zouitni — Design of Secure Coding Challenges for Cybersecurity Education in the Industry (2021) · Ruef et al. — Build It, Break It, Fix It (CCS 2016) · OpenSSF / Linux Foundation — Secure Software Development Education 2024 Survey · BleepingComputer — curl ending bug bounty program · The Register — Curl shutters bug bounty program · Elastic Security Labs — AI vulnerability triage · HackerOne — Finding Fast, Fixing Slow · Clairvoyance
Top comments (0)