Chudi Nnorukam

Posted on Feb 10 • Edited on Feb 25 • Originally published at chudi.dev

Full AI Automation Without Human Review Is Wrong — Here's Why I Changed My Approach

#ai #automation #philosophy #security

Originally published at chudi.dev

I could have built BugBountyBot to submit findings automatically. The technical barrier isn't high—an API call to HackerOne after validation passes.

I didn't build it that way. Here's why.

The Temptation of Full Automation

Full automation is seductive:

Speed: Submit findings as fast as you find them
Scale: Hunt 24/7 without human bottlenecks
Ego: "I built a system that hunts bugs while I sleep"

Every conversation about BugBountyBot eventually hits this question: "Why not just auto-submit?"

The False Positive Problem

In security research, reputation is everything. A single bad submission can:

Get your report closed as "Not Applicable"
Add a negative signal to your profile
Cost you access to private programs
Waste triager time (they remember)

The math is brutal: one false positive can undo five true positives in terms of reputation impact.

Automated systems optimize for recall—finding everything possible. But bug bounty rewards precision. A 90% precision rate sounds good until you realize that means 1 in 10 submissions is garbage.

A researcher I know ran automated submissions for a month. Their valid-to-invalid ratio dropped to 60%. Three private programs revoked access. It took six months of manual, high-quality reports to recover.

The Real Cost of a Reputation Hit

Let's put numbers to it.

A mid-level bug bounty hunter submitting 20 reports per month with 90% validity earns consistent program access and occasional bounties. Now automate without human review and drop to 70% validity:

Month 1: 6 invalid reports. Triagers notice. No visible consequence yet.
Month 2: Another 6. Two programs flag your profile internally.
Month 3: One program marks you as low-priority. Your valid reports now get slower triage.
Month 4: A private program revokes your invitation.

You just lost access to a private program that might have had $5,000 bounties. How long to earn that back? Six months of clean reports at minimum—and some programs never re-invite once access is revoked.

The math is simple: one automated false-positive spiral costs more in lost access than the time you saved automating.

Platform Requirements

This isn't just my opinion—platforms explicitly require human oversight.

From HackerOne's Automation Policy:

"Automated tools must have human review before submission. Fully automated submission systems are prohibited."

From Intigriti's Terms:

"Researchers are responsible for the quality and accuracy of all submissions, including those assisted by automated tools."

From Bugcrowd:

"Automated scanning that results in excessive false positives may result in account suspension."

Build full automation, and you're violating ToS. Not a gray area.

The Liability Question

When your bot submits a finding, who's responsible?

If it's valid: You get credit
If it's invalid: You get blamed
If it causes harm: You're liable

There's no "my AI did it" defense. The liability is asymmetric—downside is yours, and "scale" just multiplies it.

Compare this to human-in-the-loop:

You review each finding before submission
You apply judgment about timing and context
You own the decision, not just the consequence

What Humans Do Better

Automation excels at:

Pattern matching at scale
Consistent testing methodology
24/7 availability
Memory across sessions

Humans excel at:

Context understanding - Is this behavior intentional?
Impact assessment - Is this actually a security issue?
Communication - Can I explain this clearly?
Timing judgment - Is now the right time to submit?

The optimal system uses AI for the first set and humans for the second.

The Human-in-the-Loop Architecture

BugBountyBot's design:

[Recon Agent] → [Testing Agent] → [Validator Agent]
                                         ↓
                              [Confidence ≥ 0.85?]
                                    ↓         ↓
                                  Yes        No
                                   ↓          ↓
                         [Queue for Review] [Log & Learn]
                                   ↓
                           [Human Review]
                                   ↓
                         [Approve/Reject/Edit]
                                   ↓
                          [Reporter Agent]

The human checkpoint is after validation but before submission. You're not reviewing raw signals—you're reviewing high-confidence findings with full evidence.

This is the leverage point: AI handles the 80% grind, humans handle the 20% that requires judgment. The full architecture behind this system is detailed in the bug bounty automation architecture post.

The Numbers That Matter

Metric	Full Automation	Human-in-the-Loop
Submissions/day	High	Medium
Precision	~70%	~95%
Reputation trend	Declining	Stable/Growing
Platform standing	At risk	Solid
Sustainable?	No	Yes

Optimizing for submissions per day is the wrong metric. Optimize for accepted findings per month, reputation over time, and access to better programs.

When Full Automation Makes Sense

There are legitimate use cases:

Internal security testing - Your own infrastructure, no reputation at stake
Private engagements - Client agreed to automated testing
Research environments - Sandboxed, no real submissions

But for public bug bounty programs? Human-in-the-loop is the only sustainable architecture. The ethical and compliance dimensions of this are explored in the human-in-the-loop bug bounty ethics post.

How to Design for Human-in-the-Loop

If you're building automation with a human checkpoint, the design matters. A badly designed HITL system creates the worst of both worlds: slow automation with human bottlenecks that still produce false positives.

Present findings with evidence, not signals

The human shouldn't have to re-investigate. Every item in the review queue should arrive with:

The full request/response pair showing the vulnerability
A confidence score with the breakdown (what raised it, what nearly filtered it out)
A reproducible PoC—one command the reviewer can run to verify
Severity assessment with CVSS rationale

If a reviewer has to go digging, the handoff is broken.

Batch similar findings together

Reviewing ten IDOR findings one by one is exhausting and inconsistent. Batch them: show all IDOR findings from the same program in one session. Pattern recognition kicks in. The reviewer can apply consistent judgment across similar cases, and they'll notice if finding #7 looks different from the others.

Set a minimum confidence floor, not just a threshold

Don't just block low-confidence findings—discard them at the source. If something scores below 0.40, logging it and moving on is correct. A review queue full of 0.41-confidence findings is worse than a queue with nothing: it trains reviewers to rubber-stamp.

The NIST AI Risk Management Framework calls this "human oversight commensurate with risk"—the human role should be proportional to the stakes, not a checkbox.

Keep the review cycle short

If findings sit in queue for days, context gets lost. A reviewer looking at a finding two weeks old has to re-read everything from scratch. Set a target: findings should be reviewed within 24–48 hours of queuing. If volume exceeds review capacity, slow down the automation—not the other way around.

The Long-Term Evidence

I've now run BugBountyBot with human-in-the-loop for eight months. The data makes the case more clearly than any argument.

Month 1-2 (calibration): The system queued 34 findings for review. I approved 18, rejected 16. Twelve of the approved ones were accepted by programs. Six were marked informational—valid but not bounty-eligible. Zero false positives submitted. Nothing I submitted was wrong, just not always in-scope.

Month 3-4 (dialing in): The system queued 28 findings. I approved 22. Twenty were accepted or rewarded. Two were informational. Still no false positive submissions. The RAG database had learned enough false positive signatures that it filtered garbage before it reached the review queue.

Month 5-8 (steady state): Quarterly average of 25 queued, 22-24 approved, 20-22 accepted. My effective submission rate (accepted / submitted) stayed above 90% consistently.

Compare this to the researcher who ran full automation: 60% valid rate by month three, three private program revocations by month four.

The efficiency difference is smaller than it looks. I spend roughly 4-8 hours per month reviewing findings. Each review takes 10-20 minutes including reproducing the PoC. The alternative—handling reputation damage, disputing invalid submissions, rebuilding program trust—has no ceiling and no timeline.

Eight months of data. Zero bans. Zero private program revocations. Consistent accepted-to-submitted ratio above 90%.

That's the case for human-in-the-loop. Not philosophy—evidence.

The Deeper Point

The goal isn't maximum automation. The goal is maximum valuable output with acceptable risk.

Human-in-the-loop is how you get there. It's not a compromise—it's the architecture that lets you scale without catastrophic failure modes.

A checklist for evaluating any automation decision:

Can the automated component fail silently? (If yes: add logging + human audit)
Does failure at scale damage relationships? (If yes: add human gate)
Is the judgment call context-dependent? (If yes: keep a human in the loop)
Can you reverse a bad automated decision? (If no: add human gate)

Four yeses means full automation. Mixed answers means HITL. All yeses means the human checkpoint is the only path.

Build for sustainability. Your future self will thank you.

The researchers running full automation are chasing a metric—submissions per day—that doesn't correspond to the outcome that actually matters: bounties earned and program access maintained over months and years. Human-in-the-loop is slower on the metric that sounds impressive in a demo and faster on the metric that pays reliably.

DEV Community