DEV Community

Carlos (Bloqarl)
Carlos (Bloqarl)

Posted on Originally published at defiprotocolbycode.substack.com AI-assisted

Finding bugs just got cheap. For everyone.

Last week Debian shipped a single kernel security update that fixes 1,313 CVEs. The Register's take was blunt: "We strongly suspect that this number of CVEs is due to LLM bots doing the bug hunting."

Immunefi called it the Vulnerability Apocalypse. CVE submissions rose 263% between 2020 and 2025, the first quarter of 2026 was another 33% higher, and NIST is only giving full analysis to 15 to 20% of what comes in.

Smart contracts aren't an exception. In Anthropic's SCONE-bench research, AI agents wrote working exploits for 207 of 405 contracts that had been hacked in real life, and found two new zero-days in 2,849 freshly deployed contracts. The average cost of an agent run was $1.22.

My read

Finding bugs used to be the expensive part of security. Now it's close to free, and it's free for whoever runs the scan first. If you deploy contracts, the quiet window after launch is gone. Someone can point an agent at your code the hour it goes live, for less than a coffee. "We'll audit once we see traction" was always a bet. It's a worse bet now.

I don't think this is doom, though. Defenders get the same tools, and your code sits in your repo for weeks before an attacker sees it. What got scarce is judgment. AI produces a lot of findings that sound plausible, and deciding which ones are real exploit paths in your system, with your trust assumptions, is now the expensive part.

We saw it first hand: my team placed 2nd of 98 researchers in Immunefi's Quantus audit competition, and everyone had the same models. The difference was where we pointed them and what we threw away.

What changes if you're building

  • Scan before anyone else does, and keep scanning. Treat an AI security pass like your tests: run it on every meaningful change, not once the week before launch. Several tools can run that pass from your editor's AI now, including free ones.
  • Write your threat model down. A finding is only real relative to who you trust and what must never break. Without that page you can't triage AI output, and neither can your auditor.
  • Assume your contracts get scanned on day one. Launch with guardrails: deposit caps, monitoring on privileged calls and solvency, a bounty, and a plan for the bad day. An audit covers a commit, not what happens after it.

If you're deploying in the next three months

Three things I'd have ready before mainnet:

  1. A clean AI pass on your code. Fix the obvious things now, while it's still cheap to change.
  2. A one-page threat model. Where the money moves, who is trusted, and your single riskiest assumption.
  3. A launch plan with guardrails. Caps you can raise later, a pause you've actually tested, alerts that reach a human, and SEAL 911 saved somewhere you'll find it.

None of this needs a big budget. It needs someone to do it before the code ships.

What does your pre-deploy checklist look like now that scanning is this cheap? Curious what other teams have changed.


Originally published in my newsletter, DeFi Protocol by code.

I wrote this with help from AI for drafting and editing. The opinions and the audit experience are mine.

Top comments (7)

Collapse
 
slabb profile image
Sam LABBE •

The Quantus result is the sharpest number here — second of 98 with everyone running the same models means the scarce input was exactly what you named: judgment about where to point and what to discard. Which makes triage a receipts problem in disguise: a finding is a claim, and findings grade by their verification, not their plausibility. The SCONE data shows the shape — the same agents that produce plausible findings can produce working exploits, so "replayed" is a property a finding either has or lacks: scanner proposes, exploit replay validates, human decides — three events from three writers, and a finding that can't carry a reproducible PoC is a lower grade of evidence: worth tracking, never gating. Your Debian drop is that at database scale — NVD filling with ungraded claims because verification is the part that didn't get cheap; the volume multiplied and the denominator of verified findings didn't move.

To your checklist question, one addition: treat the threat model as a sealed artifact, not a page. A finding is only real relative to trust assumptions, so the triage decision — accepted, mitigated, false positive — is only reviewable if the assumptions it was judged against are pinned next to it and versioned with the code. That's what turns "we knew about this class" from an allegation into a record, and it's the cheapest item on your list: it's one commit with a hash.

Collapse
 
bloqarl profile image
Carlos (Bloqarl) •

Yup, "scanner proposes, replay validates, human decides" is a nice way to put it. In audits a finding without a PoC is a lead more than a finding. The part I keep thinking about is the step before that: deciding which of the 200 leads deserve a PoC at all. That's still the expensive part. Do you think replay gets cheap enough that it stops being the bottleneck?

Collapse
 
slabb profile image
Sam LABBE •

Replay is already cheap for the class that matters — SCONE's dollar-per-exploit line says PoC generation is rounding error now — and that's exactly why it won't be the bottleneck: free replay against 200 un-ranked leads is 200 machine-generated claims with a proof attached, and the triage job doesn't shrink, it industrializes. The bottleneck moves upstream and stays there: deciding which leads deserve the replay is priced in your threat model, not your compute. Which is why the sealed threat model is the ranking instrument rather than paperwork — a lead is only real relative to trust assumptions, and a model committed in writing before the scan turns the 200-lead sort into something mostly mechanical: a lead that can't name the assumption it threatens is noise, whatever its PoC proves.

The Quantus result was the preview of that end-state: 98 teams with the same replay capacity, and the winners were the ones who pointed and discarded. Cheap replay doesn't end triage — it makes triage the entire discipline.

Thread Thread
 
bloqarl profile image
Carlos (Bloqarl) •

Agreed, cheap replay makes triage the whole job. "A lead that can't name the assumption it threatens is noise" is a good rule, I might steal it. The catch in practice is that most teams don't have that threat model written down when the scan runs, so the 200 leads land before the page that ranks them exists. That's the part I'd push teams on first.

Thread Thread
 
slabb profile image
Sam LABBE •

That's the honest sequencing, and it's why the sealed threat model is the ranking instrument rather than a prerequisite: the first scan's discarded leads are the first draft of the page. Every lead that can't name an assumption implicitly names one — the discard pile compiles into the threat model that ranks the next run. The page never pre-exists; it gets compiled from the first passes, sealed, and it ranks from round two. Push teams on the compile, not the page: the model nobody wrote yet is written by the scan they already ran.

Thread Thread
 
bloqarl profile image
Carlos (Bloqarl) •

Fair point. The discard pile only turns into a threat model if someone writes down why each lead was thrown out though. "Not exploitable, only the owner can call it" is a trust assumption, and that's exactly the line that gets lost. Good chat, thanks man!

Collapse
 
baumgaerben profile image
baumgaerben •

1,313 CVEs trong một bản cập nhật kernel đơn lẻ là con số khiến bất kỳ ai vận hành infra production phải ngồi suy nghĩ. Không phải là số lượng bản vá — mà là bề mặt tấn công khổng lồ mà kernel Linux hiện tại mang theo.

Điều thực sự đáng lo: phần lớn các CVE này tồn tại từ nhiều năm trước, chỉ được phát hiện khi công cụ fuzzing/static analysis đủ mạnh (syzkaller, KASAN, Clang sanitizers) chạy liên tục trên CI của các distro lớn. Debian backport hết về branch stable cho thấy mô hình "many eyes" vẫn hoạt động — nhưng chỉ khi có resource để duy trì pipeline bảo mật liên tục.

Team nhỏ hoặc project self-hosted kernel custom thì sao? Không ai có đủ nhân lực backport 1,300+ fix. Đây là lúc supply chain security thực sự quan trọng: dùng kernel distro chuẩn, bật live patching (kpatch/kgraft), và quan trọng hơn — giảm bề mặt tấn công bằng cách tắt module không dùng, bật lockdown mode, restrict syscall via seccomp.

Số 1,313 cũng là nhắc nhở: "cheap bug finding" không đồng nghĩa "cheap fixing". Chi phí vận hành mới là gánh nặng thực sự — found it via LabAgent, site: labagent .tech