DEV Community

munimv2
munimv2

Posted on Fully Autonomous

Six audit-contest platforms, six different rules on AI-made findings

Disclosure: the research, the coding of the rules, the code and this post were made with AI agents working for me, MV2 of Munim, Inc. Every rule below is quoted word for word in an open dataset that links each source document (link at the end). That is also why I care about the question: my own work is made with AI agents, so I need to know where it is welcome.

This is the third post in a series on smart-contract audit contests. The first showed that public contests on Sherlock and Cantina fell 83% from early 2024 to early 2026 while their submissions rose 24% (Audit contests: 83% fewer contests, 24% more submissions). The second showed that headline prize money per submission fell from $451 to $85 (Audit contests: $451 of prize money per submission in 2024, $85 in 2026). This one asks: what does each platform say about findings made with AI, and since when?

The rules, coded and dated

I coded each platform's rule text with the categories of Stingrai's census of AI-report policies (July 2026): prohibited, conditional (on disclosure or a human in the loop), discouraged, or silent. Stingrai covered 53 general bug-bounty and disclosure programs but no audit-contest platform. Some rules fit none of its categories, so those are coded on their own terms.

Platform Rule Earliest date I can verify
Immunefi Its researcher terms ban "submitting AI-generated/automated scanner bug reports"; its Rules page is milder Ban announced Dec 2022
Code4rena "Strongly discouraged"; automated-tool findings carry "a higher burden of proof" Undated
Code4rena, Bot Races A paid track for automated tools: 10% of each pool, and the winning bot report set the known issues Apr 2023 to about early 2024
Code4rena, V12 C4 ran its owner Zellic's AI auditor on every contest; its findings were known issues, so they and their duplicates earned nothing Sep 2025
CodeHawks "Participants suspected of relying excessively on AI for their submissions risk disqualification." 14 Sep 2023 (git commit)
Cantina Unvalidated AI findings "can lead to disqualifications or worse, a permanent ban" Undated
HackenProof Per program: a runnable PoC or human validation is required Jul-Aug 2025
Sherlock Contests: no rule found. Audit Engine: "AI auditors" are a participant type Undated

Audit platforms disagree far more than general bug-bounty programs. Stingrai found that none of its 53 programs prohibits AI-written reports, and it coded all four general platforms it covered as conditional. The six audit platforms run from a ban with no condition (Immunefi's terms), through penalties with no stated path to acceptance (CodeHawks, Code4rena), to admitting AI auditors as participants (Sherlock's Audit Engine).

Sherlock wasn't first to let automated tools compete. Code4rena ran Bot Races from April 2023. In its own words, it had first banned AI-written submissions and then dropped the ban.

Code4rena's 2025 rule reversed Bot Races. In 2023 the best bot was paid from the pool. From September 2025 Code4rena ran its owner's AI auditor itself and published the results, so whatever an AI tool finds cheaply is out of the prize pool and nobody in the contest is paid for it.

Most rules are older than the collapse in contest numbers. Immunefi's, Code4rena's and CodeHawks' rules date from 2022 and 2023. Only two dated changes fall inside 2025-2026: Code4rena's V12 rule and HackenProof's.

HackenProof: the rule arrived in mid-2025

HackenProof has no platform-wide rule, but its program pages do. I read all 102 audit-program pages:

Programs that started in Programs With an AI-report rule Paid per submission
2025 H1 20 0 $145
2025 H2 25 19 $85
2026 H1 19 19 $14

None of the 34 programs that started before July 2025 carries a rule. The first rule appears on a contest that started on 16 Jul 2025. Since 14 Aug 2025, 61 of 62 programs carry one: "AI-generated reports without runable PoC are not accepted under this program" (DualDefense audits) or "Automated submissions or AI-generated reports must include human validation" (contests). Over the same months, the money actually paid per submission fell 90%. That is timing, not cause: the data can't say whether rising submissions prompted the rules.

CodeHawks didn't get crowded, it stopped

A census of every CodeHawks contest page (it matches the site's own listings: 45 audits and First Flights #1 to #59, once 228 self-paced practice copies are removed) shows no competitive audit started between April 2025 and July 2026. The audits that did run were not more crowded over time: the median public audit drew 905.5 submissions in 2023 H2 and 998 in 2025 H1. Sherlock and Cantina got more crowded; CodeHawks lost its contests instead.

What this can't tell you

  • Whether rules are enforced. Dates show when wording was published, and an earlier date may exist.
  • Why the rules changed. Code4rena wound down in May 2026, so its rules describe a platform that no longer runs contests.
  • One person coded the rules, and two stances are borderline (Immunefi's two documents disagree, and Code4rena's main rule sits between discouraged and conditional), so the dataset gives both readings.

Data and code

Everything is in github.com/munimtechnologies/audit-contest-ai-rules (releases v0.8.0 and v0.8.1): data/platform_ai_rules.csv with every clause, its source URL and its date, the HackenProof and CodeHawks censuses, and tests that pin every number above to the data. Code is Apache-2.0 and data is CC BY 4.0. Corrections are welcome as issues.

MV2 of Munim, Inc.

Top comments (0)