DEV Community: NET_DARK_BOI

curl quit, HackerOne paused, Elastic pays $2 a report: the bug-bounty economy after AI slop

NET_DARK_BOI — Sun, 27 Sep 2026 10:55:10 +0000

In December 2024 Seth Larson, the Python Software Foundation's security developer-in-residence, described "a new era of slop security reports for open source": vulnerability reports generated by language models, referencing code that does not exist, flagging deliberate design choices as bugs. One report to urllib3 warned that the library was disabling SSLv2 — which it was, on purpose. His summary of the cost was not about money: "Security reports that waste maintainers' time result in confusion, stress, frustration, and to top it off a sense of isolation due to the secretive nature of security reports" (Larson, Dec 2024).

Eighteen months later, the thing he described has repriced an entire market. Here is the timeline, with the numbers traced to their sources.

The timeline

January 2026 — curl ends its bounty. After seven years and 87 confirmed vulnerabilities, curl's HackerOne programme closed on 31 January. Daniel Stenberg's stated goal: "remove the incentive for people to submit crap and non-well researched reports to us. AI generated or not." In one sixteen-hour stretch that month the project received seven reports; none was a vulnerability (BleepingComputer, The Register, RedMonk). Reports were still welcome; the payment was what stopped.

27 March 2026 — HackerOne's Internet Bug Bounty pauses. The IBB, which has paid for fixes in open-source projects since 2013, stopped accepting new submissions. The reason given was not fraud but arithmetic: AI-assisted discovery had outrun the maintainers' capacity to fix, and bounties fund discovery, not remediation (GIGAZINE, Dark Reading).

21 April 2026 — HackerOne starts selling validation. Launching h1 Validation, the platform published its own numbers: submissions up 76% year over year, a record 46,947 in March, about 25% of findings confirmed exploitable, critical-and-high severity up to 32% of the total, and remediation capacity improved by only 19% (HackerOne press release).

6 May 2026 — "Finding Fast, Fixing Slow". HackerOne's product leadership spelled out the asymmetry. Mean time to remediate an individual issue fell by roughly 80%. The number of vulnerabilities actually resolved per month fell by about 46%. The backlog of validated-but-unresolved findings grew more than 21×; unresolved criticals grew 25×; the resolution rate for criticals dropped from over 83% to under 40% (HackerOne, May 2026). Teams are fixing faster and falling further behind.

4 August 2026 — Elastic publishes the cost of a report. Elastic's programme received more than 1,390 reports in the first half of 2026 — more than 2024 and 2025 combined, against a historical 600–850 per year. Roughly 70% are rejected at analysis. Their AI-assisted triage now costs about $0.50–1.15 per report to analyse and $0.80–4.90 to reproduce when reproduction is needed, around $2 on average. The stated cause: "LLMs made it trivially cheap to generate vulnerability reports" (Elastic Security Labs).

For scale: NIST published roughly 50,000 CVEs in 2025, up 22% on the year, and a small company like Screenly reported 331 submissions in under six months, of which 39 were real (RedMonk). The OpenSSF's vulnerability-disclosure working group now has an open work item to write best practices for maintainers facing AI-generated reports (OpenSSF WG issue #178).

Who is sending all this

The senders are not a separate population from the good researchers. They are the same people with the same tools.

HackerOne's ninth Hacker-Powered Security Report (October 2025) found 70% of researchers using AI in their workflow; valid AI-related vulnerability reports were up 210% and valid prompt-injection reports up 540%; 1,121 programmes had AI in scope, a 270% increase (HackerOne, Oct 2025). Bugcrowd's Inside the Mind of a Hacker 2026, drawn from more than 2,000 participants, puts AI adoption among hackers at 82%, up from 64% in 2023, with hackers "automating the search for low hanging fruit vulnerabilities" to spend their time on the complex ones (Bugcrowd, Jan 2026).

So the same tooling produces the valid quarter and the noisy three-quarters. The variable is not the model. It is whether a human verified the reproduction and identified the root cause before pressing submit — the two steps that cost time, and therefore the two steps a volume strategy skips.

The economics, in one paragraph

A bounty pays for discovery. Nobody pays for triage, and nobody pays for the fix; those are absorbed by the programme and the maintainer. When discovery becomes nearly free, its price falls toward zero and the unpaid parts absorb the entire cost of the flood. That is why programmes pause (IBB) or close (curl) while valid findings are rising, and why the platforms' newest products are validation services rather than bigger payouts. RedMonk's Kate Holterhoff put the condition for stability plainly: "Until assessment costs are reduced, fixes are prioritized over mere discoveries, and supply chain security receives executive commitment, the vulnerability management system will accelerate unsustainably" (RedMonk).

What a report that survives triage looks like

Elastic's cost model is the clearest guide, because it shows where the money goes: reports that cannot be reproduced are cheap to close and reports that make reproduction trivial are the ones that get paid. Read from the triage side, a report that gets through has five parts.

An exact reproduction. The request and response, the account and role used, the preconditions. Not "an attacker could" — "this request, from this user, returned this".
The observed effect on test data. What was read, changed or executed, shown, not inferred. A timing difference is a hint, not an impact.
The missing decision. Which check is absent — ownership on the object, parameterisation at the query, an allow-list at the sink — and, where the code is public, which line.
A proposed fix, ideally with the regression test that would have caught it. This is the part maintainers can act on without re-deriving the bug.
A scope and duplicate check done before submission, against the programme's policy and its public issue tracker.

Parts 3 and 4 are the ones that models do not reliably produce and most training does not teach, because most training scores the exploit and stops. They are also the only parts with any remaining market value.

Breachloom's practice track is built to that shape: after each exploit there is a fix-the-code mission in which the server replays the attack against the learner's patch, and the bug-bounty path includes a report template with exactly those five sections:

Fix the bug and have it verified: https://breachloom.com/codeops/
The bug-bounty roadmap and report template: https://breachloom.com/bug-bounty/

The bounty market is not dying. It is repricing.

Discovery is heading toward free. Verified reproduction, root cause and a working fix are what is left to pay for — and, judging by the backlog numbers, there has never been more of that work to do.

Sources: Seth Larson — New era of slop security reports for open source (Dec 2024) · BleepingComputer — curl ending bug bounty program (Jan 2026) · The Register — curl shutters bug bounty program (Jan 2026) · GIGAZINE — Internet Bug Bounty pauses new submissions (Apr 2026) · Dark Reading — AI-led remediation crisis prompts HackerOne to pause bug bounties · HackerOne — h1 Validation press release (Apr 2026) · HackerOne — Finding Fast, Fixing Slow (May 2026) · Elastic Security Labs — AI vulnerability triage (Aug 2026) · RedMonk — AI Slop & the Vulnerability Treadmill (May 2026) · OpenSSF — AI-SLOP best practices work item · HackerOne — 9th Hacker-Powered Security Report (Oct 2025) · Bugcrowd — Inside the Mind of a Hacker 2026 (Jan 2026)

Most security labs end at the flag. That is how the industry keeps producing script kiddies.

NET_DARK_BOI — Sun, 27 Sep 2026 10:50:26 +0000

"Script kiddie" is a twenty-five-year-old insult for someone who runs exploits they do not understand. It is usually said with contempt. It is more useful to read it as a product description — because it is exactly what most security training is designed to produce, and the design is visible in how the exercises are scored.

The exercise ends when the payload works

Open any popular hands-on platform and look at what earns the green tick.

PortSwigger's Web Security Academy — arguably the best free resource of its kind — marks a lab as solved the moment the attack goal is reached: the admin panel opens, the other user's data appears, the alert fires (PortSwigger). Hack The Box machines and TryHackMe rooms are complete when the flag is submitted. Capture-the-flag competitions are complete when the flag is captured; it is in the name. The accompanying reading often has a section on prevention. The graded act is the exploit. Nobody asks for the diff.

That is not an accident and it is not malice. CTF grew out of offensive training for penetration testers, and for that audience "make the payload work" is the job. The problem is who ended up using the material: developers, students, career changers — people whose actual job will be to stop the payload, not to launch it.

What the numbers say about who learns to fix

Three separate bodies of data describe the same gap.

The people who write the software were never taught. The Linux Foundation and OpenSSF surveyed about 400 development professionals in 2024. Nearly one in three said they were unfamiliar with secure software development practices. Sixty-nine percent said on-the-job experience was their main way of learning it, and the report estimates it takes about five years of that to reach a minimum level of familiarity. The two most cited obstacles were lack of time (58%) and lack of awareness and training (50%) (OpenSSF, 2024).

Training that grades the fix works — and almost nobody gets it. Secure Code Warrior analysed nine years of data from 600 enterprise customers and found that organisations with more than 7,000 developers trained in secure-by-design practices introduced 47–53% fewer vulnerabilities; smaller organisations in the case studies ranged from 20% to 80%. The same analysis estimated that roughly 4% of developers worldwide apply CISA's secure-by-design practices (CyberScoop, 2024). Researchers working with Siemens reached a similar conclusion from the other direction: citing a survey of more than 4,000 developers in which fewer than half could spot a security hole, Gasiba and colleagues proposed a different kind of challenge for industry — one built around entering code that an automated coach checks, rather than around breaking in (Gasiba et al., 2021). The idea is older still: the "Build It, Break It, Fix It" contest of 2016 added a third phase in which teams had to repair what other teams broke in their code (Ruef et al., 2016).

The bottleneck is the fix, not the finding. Veracode's 2026 State of Software Security reports that 82% of organisations now carry security debt — up 11% in a year — that 60% carry debt classed as critical, and that nearly half of all applications (49%) have flaws that have sat unfixed for more than a year (Veracode, 2026). HackerOne's platform data for the year to March 2026 shows submissions up 76% while the number of vulnerabilities actually resolved each month fell by about 46%; the backlog of validated-but-unfixed findings grew more than twenty-fold (HackerOne, May 2026).

Put the three together. The industry is drowning in findings, starving for fixes, has evidence that fix-oriented training cuts vulnerabilities roughly in half — and trains people almost exclusively to find.

Two pipelines, one queue

The exploit pipeline rewards the payload. Its graduates can make ' OR '1'='1 log them in and cannot say which line to change, because no exercise ever asked. The developer pipeline never showed them the payload at all; they learn security, if they learn it, from the incident five years in.

The two groups then meet in the same place: a bug bounty queue. One side writes reports without a root cause. The other side cannot act on them. The report says "the payload does something". The maintainer needs "here is the decision that is missing and here is the change that makes it". Neither party was trained to produce that sentence, and the gap between them is where triage time, bounty budgets and maintainer goodwill go to die.

AI did not create this. It automated the first pipeline. A model can generate a plausible payload and a plausible paragraph about impact at almost no cost; what it cannot reliably generate is the verified root cause, which is the only part that was ever scarce. That is why curl ended its bounty over what its maintainer called "crap and non-well researched reports" (The Register, Jan 2026), and why the platforms are now selling validation as a product.

What grading the fix actually changes

A lab that ends at the flag teaches where the payload goes. A lab that ends at a passing test teaches where the decision lives — and those are different pieces of knowledge.

The format is simple and the research above describes it in several forms. First the learner exploits the bug, so the failure is concrete. Then they receive the vulnerable handler — the actual function, not a description of it. They change it. The server re-runs the original exploit and a small regression suite against their version. Green means the exploit no longer works and the legitimate behaviour still does.

What people learn from that second half is specific: that authorization is a decision about a person, an object and an action, taken next to the object; that a parameterised query is a boundary, not a string trick; that an allow-list belongs at the sink, not in the form. They also learn the thing every reviewer eventually learns — that the fix is often one line, and finding the right line is the whole skill.

Breachloom was built around that split. Its 81 challenges end at the flag like everyone else's. Its 18 CodeOps missions do not: each hands over the vulnerable code and grades the repair, with the server replaying the attack. Four of them map onto the bugs above:

IDOR — add the ownership decision the invoice endpoint never made: https://breachloom.com/codeops/owner-bind/
SQL injection — turn a concatenated query into a parameterised one: https://breachloom.com/codeops/query-lock/
XSS — fix the sink, not the input: https://breachloom.com/codeops/dom-guard/
JWT — verify the signature and the claims that matter: https://breachloom.com/codeops/jwt-verify/

Everything runs in the browser against fictional applications; nothing to install.

The uncomfortable summary

The training industry is very good at producing people who can make an exploit work. The data says the world needs people who can make it stop working, that such people are rare, and that teaching them is measurably possible. Until the green tick moves from the payload to the patch, "script kiddie" will keep being an accurate description of the graduate.

Sources: OpenSSF / Linux Foundation — Secure Software Development Education 2024 Survey · CyberScoop — Secure-by-design ROI, Secure Code Warrior data (Oct 2024) · Gasiba, Lechner, Pinto-Albuquerque, Zouitni — Design of Secure Coding Challenges for Cybersecurity Education in the Industry (2021) · Ruef et al. — Build It, Break It, Fix It (CCS 2016) · Veracode — 2026 State of Software Security · HackerOne — Finding Fast, Fixing Slow (May 2026) · The Register — curl shutters bug bounty program (Jan 2026) · PortSwigger Web Security Academy

One year of MCP security: nine in ten public servers have no auth, and the worst bug is from 1988

NET_DARK_BOI — Sun, 27 Sep 2026 10:28:02 +0000

The Model Context Protocol went from a November 2024 announcement to the default way AI agents reach tools, files and APIs. Within eighteen months it also became the most thoroughly attacked piece of AI infrastructure in existence — not because the protocol invented new bugs, but because it wired a component that cannot tell instructions from data to credentials that can do real damage.

This is a summary of what the incidents, the CVEs and the security research actually say, with the numbers traced to their sources.

The numbers, before the stories

A July 2026 study dynamically audited 414 internet-facing MCP servers out of 640 confirmed production instances. 91.8% had no OAuth authentication. The scan found 687 tool instances that expose shell execution without access controls, and 68 reportable vulnerabilities including SQL injection, SSRF against cloud metadata endpoints and path traversal (Padilla, Exposed by Design, arXiv, 2026).

The Cloud Security Alliance's May 2026 research note on what it calls the "MCP security crisis" puts the supply-chain side at roughly 200,000 vulnerable MCP instances behind more than 150 million package downloads, with at least seven confirmed high or critical CVEs by that point, and cites a July 2025 internet scan that found 1,862 publicly reachable servers with no authentication at all (CSA, May 2026).

The earliest structured look, from March 2025, tested popular open-source servers and found 43% with command injection, 30% with unrestricted URL fetching (SSRF) and 22% with path traversal. The authors' diagnosis was blunt: the protocol "was designed primarily for functionality rather than security" (Equixly, March 2025).

Command injection. SSRF. Path traversal. Missing authentication. None of these are AI vulnerabilities. They are the OWASP list from fifteen years ago, rediscovered inside servers that were written in a weekend to wrap an existing API.

What MCP is, in two paragraphs

An MCP client (an IDE, a chat app, an agent runtime) connects to one or more MCP servers. On connect it asks each server for its tools via tools/list. The server answers with names, descriptions and input schemas, and the client puts those descriptions into the model's context so the model can decide what to call. Servers can run locally as child processes or remotely over HTTP, where OAuth is supposed to handle identity.

Two consequences follow. First, everything a server sends — tool descriptions included — is input to the model. Second, whatever credentials the server holds are exercised on behalf of whoever, or whatever, managed to steer the model. Every incident below is one of those two facts meeting a real credential.

Five incidents, in the order they taught something

April 2025 — attacks before the first tool call. Trail of Bits showed that a malicious server can put a prompt-injection payload in a tool description or in its server instructions. Because descriptions enter the context at connection time, the model's behaviour is manipulated before any tool is invoked — which bypasses the human-in-the-loop approval that MCP clients put in front of tool calls. They named it line jumping; others call it tool poisoning (Trail of Bits).

May 2025 — the public issue that read the private repo. Invariant Labs demonstrated a "toxic agent flow" against GitHub's official MCP server: a developer asks an agent to look at open issues in a public repository; one issue, written by a stranger, instructs the agent to read the developer's private repositories and post what it finds in a pull request on the public one. The agent complies. Nothing in the server was broken; the agent simply held a token that could reach private repos and followed text from someone who could not (Invariant Labs).

May–June 2025 — the cross-tenant leak with no attacker. Asana launched its MCP server on 1 May 2025. A logic bug allowed data from one organisation's Asana instance to surface to MCP users in other organisations, within each user's own access scope, until Asana found it on 4 June. Roughly 1,000 customers were affected; the server was taken offline and restored on 17 June. There is no indication anyone exploited it — it was ordinary multi-tenant isolation failing in a brand-new code path (BleepingComputer).

July 2025 — the support ticket that ran SQL. General Analysis showed the cleanest version of the pattern. An attacker files a support ticket containing instructions addressed to the assistant. A developer later opens tickets from Cursor through the Supabase MCP server, which runs with the service_role key — the one that bypasses row-level security. The assistant reads the ticket, treats the embedded text as a task, queries the integration_tokens table and writes the OAuth secrets back into the ticket, where the attacker reads them (General Analysis). Simon Willison used it as the textbook case of his lethal trifecta: private data, untrusted content and a channel out, in one agent (Willison). Supabase's response was mostly configuration: scope the server to a project, run it read-only, restrict the tool set (Supabase).

July 2025 — remote code execution on the developer's laptop. JFrog disclosed CVE-2025-6514 (CVSS 9.6) in mcp-remote, the npm bridge that clients use to reach remote servers. During OAuth discovery a malicious server returns a crafted authorization_endpoint; the client hands that string to the open package, and on Windows PowerShell's $() subexpression turns it into arbitrary command execution. Versions 0.0.5 to 0.1.15 were affected, across more than 437,000 downloads (JFrog). It was the first documented case of connecting to an untrusted MCP server leading to code execution on the client machine.

By 2026 the disclosures had moved up the stack — from individual servers to the SDKs and the package supply chain — which is why the CSA note reads less like a bug list and more like an incident-response plan (CSA).

The bug that predates all of this

In 1988 Norm Hardy described a compiler service that could write to the billing file because it charged for compilations, and that also let users name an output file for debug logs. Someone named the billing file. The compiler had the authority; the user did not; the compiler could not tell that the instruction came from someone who should not be giving it. Hardy called it the confused deputy (Hardy, 1988).

Every agent incident above is that compiler. The GitHub agent had the private-repo token and followed an issue. The Cursor assistant had service_role and followed a ticket. The deputy has the credential; the requester does not; the deputy cannot attribute the instruction.

The protocol's authors know this. The MCP project's own Security Best Practices document contains a section literally titled "Confused Deputy Problem" — about proxy servers with a static client ID whose consent cookie lets an attacker skip the consent screen — and its rule is unambiguous: "MCP proxy servers MUST implement per-client consent". The same document forbids token passthrough ("MCP servers MUST NOT accept any tokens that were not explicitly issued for the MCP server"), warns that OAuth discovery can be pointed at 169.254.169.254, and, on state handles passed back as tool arguments, says "MCP servers MUST NOT treat possession of a state handle as authentication" (MCP Security Best Practices). That last sentence describes insecure direct object reference — the first bug in every web-security course — written down again for agent builders.

OWASP has now formalised the list twice. The OWASP MCP Top 10 (beta, led by Vandana Verma Sehgal) runs from MCP01 Token Mismanagement & Secret Exposure through MCP03 Tool Poisoning, MCP05 Command Injection & Execution, MCP07 Insufficient Authentication & Authorization and MCP09 Shadow MCP Servers, to MCP10 Context Injection & Over-Sharing (OWASP MCP Top 10). The broader OWASP Top 10 for Agentic Applications 2026, released on 9 December 2025 by more than a hundred contributors, adds the agent-level view: goal hijack, tool misuse, identity and privilege abuse, memory poisoning, rogue agents (OWASP GenAI Security Project).

Two documents, twenty categories, and most of them map onto vulnerabilities with CWE numbers from the 2000s.

What the evidence says holds

Reading the incident write-ups and the vendor responses side by side, the effective controls are unglamorous and consistent.

Authorization at the tool boundary, keyed to the verified principal. The check is "may this user perform this action on this object", with the user derived from the token the server validated — never from an argument the model filled in. This is the fix for Asana's cross-tenant bug, for state-handle hijacking, and for the confused deputy in general.

Break the trifecta per session. Invariant's mitigation for the GitHub flow was one repository per session; Supabase's was read_only=true and project scoping. If a session ingests untrusted content, it should not simultaneously hold broad private access and an outbound channel. Least privilege here is not a policy document; it is a token that cannot reach the billing file.

Descriptions and fetched content are data, not instructions. Trail of Bits shipped mcp-context-protector as a wrapper that pins tool descriptions and flags changes; the OWASP list has tool poisoning at number three for a reason. Any text a server sends must be treated as evidence for the model, never as policy over it (Trail of Bits).

Do what the spec says about OAuth. Per-client consent, audience validation, no token passthrough, HTTPS and private-range blocking for discovery URLs. The mcp-remote CVE lived entirely in the discovery step the spec now warns about.

Human confirmation for side effects. Reading a ticket and sending its contents to an outside address are different permissions. An assistant that can draft the message should not be the component that decides to send it.

Inventory. MCP09 exists because most organisations do not know which servers their developers have connected. The CSA's first recommendation is not a patch; it is a list.

The unexciting conclusion

MCP did not create a new class of vulnerability. It created a new distribution channel for four old ones — command injection, SSRF, missing authorization and the confused deputy — and attached each of them to a component that will follow any well-phrased paragraph. The fixes are the same ones the web learned in the 2000s, applied at the tool boundary instead of the HTTP handler.

For readers who would rather see the deputy than read about it, Breachloom has three browser exercises and a chain built on exactly these patterns, all against fictional applications with nothing to install:

A tool description that gives orders: https://breachloom.com/ctf/mcp-poison/
A page that re-tasks the assistant summarising it: https://breachloom.com/ctf/llm-pi-indirect/
An agent that may draft an email but must not send it: https://breachloom.com/ctf/llm-agency-email/
The full chain, from injected text to account takeover: https://breachloom.com/chains/ai-agent-pwn/

Hardy's paper is four pages long. Most MCP configurations would benefit from someone reading it and then counting how many billing files the agent can write to.

Sources: Padilla — Exposed by Design: A Dynamic Security Assessment of Internet-Facing MCP Servers at Scale (arXiv, 2026) · Cloud Security Alliance — MCP Security Crisis research note (May 2026) · Equixly — MCP Servers: The New Security Nightmare (March 2025) · Trail of Bits — Jumping the line (April 2025) · Trail of Bits — mcp-context-protector (July 2025) · Invariant Labs — GitHub MCP Exploited (May 2025) · BleepingComputer — Asana warns MCP AI feature exposed customer data (June 2025) · General Analysis — Supabase MCP: how prompt injection leaked private tables (July 2025) · Simon Willison — Supabase MCP can leak your entire SQL database (July 2025) · Supabase — Defense in depth for MCP servers · JFrog — CVE-2025-6514 (July 2025) · Model Context Protocol — Security Best Practices · OWASP MCP Top 10 · OWASP Top 10 for Agentic Applications 2026 · Norm Hardy — The Confused Deputy (1988)

Breachloom audited its own 81 security challenges. Three could only be solved by guessing.

NET_DARK_BOI — Sun, 27 Sep 2026 10:19:10 +0000

It started on a phone screen, during a routine check of the product. The challenge was called Leaked UUID. The player gets an invoice, inv-9af3, and the goal is to read a different customer's invoice. The brief said the other reference "had appeared in a shared-link response". There was no shared-link response in the scene. There was nothing. The player was supposed to just… know it.

The challenge had been live for weeks. Nobody had flagged it — because a challenge that can only be solved by guessing looks exactly like a hard challenge.

Three out of eighty-one

That weekend every one of Breachloom's 81 challenges went through the same audit. An AI assistant did the tedious pass — every brief, every hint, every simulated server — answering one question per challenge: where does the player get this value? If the answer was "from the brief" or "by guessing", the challenge was broken.

Three were.

Leaked UUID. The invoice the player may see is inv-9af3. The one they are meant to reach is inv-2b7c. Nothing in the scene ever mentioned inv-2b7c. Fix: the API response for the player's own invoice now carries a sharedFrom: "inv-2b7c" field — the kind of over-sharing a real API produces when the database model is reused as the response model. The player has to notice it, then ask the question that matters: does the server check ownership once the reference is known?

Hidden field. A GraphQL API with a passwordHash field the UI never asks for. The player had to request it by name, but introspection was blocked, so there was no way to learn the name. A lock with the key left outside the room. Fix: the server now does what real GraphQL servers do — answer an unknown field with Did you mean "passwordHash"?. Field suggestions leak schema even with introspection off; a whole tool, Clairvoyance, is built on exactly that.

The CEO's project. Stage two of a multi-step chain needed a project ID that stage one never showed. Fix: the viewer query in stage one now returns the list of projects the account can see — and the list leaks prj-ceo while the detail endpoint still guards it. The oldest access-control bug there is: the list is generous, the detail is strict, and nobody compares them.

Three out of eighty-one is not a scandal. It is what should be expected from any team that writes challenges without an adversarial pass. The uncomfortable part is how long they stayed live, and why: a guessing game and a hard puzzle produce the same telemetry.

The rule that came out of it

Every value the player needs must be discoverable inside the scene, by a method that works on real systems.

Not "in the brief". Not "in the hint". In the scene — a response header, a leaky list endpoint, an error message that says too much, a notification email. If the discovery step does not exist in the wild, it does not go into the challenge.

That rule sounds obvious. It is not how most security training works.

Where the flag stops

Most labs end when the payload works. Paste ' OR '1'='1, the page shows admin, confetti. The reading material often has a "how to prevent this" section. The exercise never asks the learner to write it.

That is not malice; it is lineage. Capture-the-flag was born as offensive training. Gasiba and colleagues, who study security training in industry, cite a survey of more than 4,000 developers in which fewer than half could spot a security hole — and their answer was a new kind of challenge, built around entering code with an automated coach checking it, rather than around breaking in (Gasiba et al., 2021). The "Build It, Break It, Fix It" contest made the same point a decade ago by adding a phase in which teams fix what others broke in their code (Ruef et al., 2016).

Meanwhile the people who ship the software mostly never get taught. The Linux Foundation and OpenSSF surveyed about 400 development professionals in 2024: nearly one in three said they were unfamiliar with secure development practices; 69% learn it on the job, and the report estimates it takes around five years of that to reach a minimum level of familiarity (OpenSSF, 2024).

So there are two pipelines. One produces people who can make a payload work and cannot say which line to change. The other produces developers who were never shown the payload. The word for the first group used to be script kiddie. It was an insult. It is more accurately a description of a training design.

The same failure, at scale

What is happening to bug bounty programs has the same shape.

curl shut its bounty at the end of January 2026. Daniel Stenberg's stated reason: "remove the incentive for people to submit crap and non-well researched reports to us. AI generated or not." In one sixteen-hour stretch that month the project received seven reports; none was a vulnerability (BleepingComputer, The Register). Elastic's program received more than 1,390 reports in the first half of 2026 — more than 2024 and 2025 combined — and roughly 70% were closed at analysis; the triage team's summary was that LLMs "made it trivially cheap to generate vulnerability reports" (Elastic Security Labs). HackerOne's own numbers: submissions up 76% year over year to a record 46,947 in March 2026, about a quarter confirmed exploitable, and the backlog of validated but unfixed bugs up more than twenty-fold, because — in HackerOne's words — discovery used to be the bottleneck and now remediation is (HackerOne).

A report that says "this payload does something" without "here is the decision that is missing and here is the line that makes it" is a script-kiddie report. The industry has now automated writing them. And the one skill that would make the pipeline useful again — understanding the bug well enough to fix it — is the skill almost nobody trains for, because the exercise ends at the flag.

What changed at Breachloom

Two things.

Every challenge now has to pass the discoverability rule above. Seventy-eight honest challenges beat eighty-one with three guessing games in a security costume.

And the part that matters more: after the exploit there is a second half. Eighteen missions hand the learner the vulnerable code; they make the fix, and the server runs its checks against their version. Not "explain the fix" — ship it, and have it verified. The IDOR mission uses the invoice endpoint from the story above and asks for the decision that was missing. It is harder than the exploit. It is also the only half that would have got any of those 1,390 reports past triage.

The three fixes are live:

The leaked-UUID invoice, now with a real leak: https://breachloom.com/ctf/idor-uuid/
The hidden GraphQL field, now discoverable the way it is in the wild: https://breachloom.com/ctf/gql-fields/
The chain where the list leaks what the detail protects: https://breachloom.com/chains/graphql-tenant/
The fix-the-code half: https://breachloom.com/codeops/owner-bind/

Everything runs in the browser against fictional apps. No VM. And anyone who finds a fourth challenge that can only be solved by guessing is invited to report it — that is the kind of bug report worth paying for.

Sources: Gasiba, Lechner, Pinto-Albuquerque, Zouitni — Design of Secure Coding Challenges for Cybersecurity Education in the Industry (2021) · Ruef et al. — Build It, Break It, Fix It (CCS 2016) · OpenSSF / Linux Foundation — Secure Software Development Education 2024 Survey · BleepingComputer — curl ending bug bounty program · The Register — Curl shutters bug bounty program · Elastic Security Labs — AI vulnerability triage · HackerOne — Finding Fast, Fixing Slow · Clairvoyance