There's a moment that happens in almost every security team's Slack channel these days.
Someone pastes a screenshot. An LLM just flagged a SQL injection in thirty seconds flat, in a codebase that took the team a full sprint to onboard onto. Someone reacts with a 🔥 emoji. Someone else says, only half-joking, "guess we're all out of a job."
Then, a few days later, the same team spends six hours manually tracing a race condition across three microservices — the kind of bug that never shows up in a scanner, that no model flagged, that only got caught because someone asked a strange question during a threat-modeling session: "what happens if these two requests land in the same millisecond?"
Both of those moments are true. Both of them are happening in security teams right now. And most of the "AI vs humans" debate completely ignores the second one.
The Debate Everyone's Having Wrong
Scroll through any security forum and you'll find two camps shouting past each other.
Camp one: AI is going to replace vulnerability researchers within a few years. Look how fast it reads code. Look how much it already knows.
Camp two: AI is a party trick. It hallucinates half its findings and can't reason its way out of a paper bag.
Neither camp is really looking at what's happening. The honest answer is boring compared to either headline: AI has become an extraordinary pattern-matcher, and vulnerability research was never just pattern-matching — even though a lot of it used to look that way from the outside.
That distinction is the whole story.
Where the Machine Genuinely Wins
Let's give credit where it's due, because a lot of security professionals are reflexively dismissive in a way that doesn't match reality.
Speed at scale is not a small thing. Ask a human to review a 400,000-line codebase they've never seen and you're talking about days, maybe weeks, before they even feel oriented. Ask an LLM to summarize the same codebase, map out the auth flow, and point at the five riskiest files, and you get an answer before your coffee gets cold. That's not a gimmick. That's hours of orientation work collapsed into minutes.
Repetition is where humans quietly fail, and nobody likes to admit it. If the same insecure JWT-verification snippet got copy-pasted into 80 microservices by 80 different engineers over three years, a tired human reviewer is going to catch a handful of them and miss the rest — not because they're bad at their job, but because humans lose focus around instance number twelve. A model doesn't get bored. It'll find all 80, consistently, every time you run it.
It's a translator for unfamiliar territory. Every researcher eventually lands on a stack they don't know — Elixir, Rust, some in-house framework nobody documented properly. Instead of burning half a day reading docs, you can just ask what the routing layer assumes about authentication, and get a usable answer immediately. That alone has quietly reshaped how fast people ramp up on new targets.
If a vulnerability looks like something that's already been written about a thousand times — and an enormous share of real-world bugs do — today's models are genuinely good at spotting it.
Where It Falls Apart
Here's the part that doesn't make for a good screenshot, because it's not a single dramatic failure — it's a quiet, structural gap.
Take a banking app. Every endpoint authenticates properly. Authorization checks are airtight. Input validation is clean. A static analyzer runs across the whole thing and comes back with nothing.
And yet — a user can trigger a transfer and cancel it through a separate endpoint at almost the same instant, and a race condition lets the account get credited twice.
Nothing about that is a coding mistake. The code does exactly what it was written to do. The process is wrong. Recognizing that requires understanding how money is supposed to move through a system, not how a function is supposed to execute. That's not a pattern in a training set. That's judgment about intent.
Real-world compromises rarely come from one clean vulnerability either. They come from chains: a low-severity stored XSS leads to session theft, which leads to a dashboard nobody thought to lock down properly, which leads to a privilege escalation nobody planned for. Individually, each step might get scored as "low" or "informational." Strung together, it's a full account takeover. Seeing that chain requires holding the whole attack path in your head at once and asking "then what?" after every single step — and that's still something models do inconsistently unless a human is explicitly steering them through it.
And then there's the hardest category to even describe: the unknown unknowns. HTTP request smuggling. Prototype pollution. Dependency confusion. Prompt injection itself. Every one of those attack classes existed for exactly zero seconds in any training dataset — right up until the moment a curious human noticed a weird interaction nobody had documented yet and decided to poke at it. A model trained on yesterday's knowledge is structurally incapable of inventing tomorrow's category. It can only get fast at recognizing it after someone else names it first.
The Uncomfortable Truth About Curiosity
If you spend enough time around good researchers, you notice they all share a slightly annoying personality trait: they can't leave a "that's probably fine" alone.
What happens if these two requests arrive out of order? What if a user deletes their account mid-session — does anything downstream still trust the old session token? What if support staff have a debug panel that quietly bypasses the checks everyone assumes are universal?
None of those questions come from reading code. They come from imagining how the system will actually get used, misused, and abused by real people cutting real corners under real pressure. That's not a technical skill you can distill into a prompt template. It's closer to professional paranoia, and it's earned through actually breaking things and watching how organizations really operate — not through reading about how they're supposed to operate.
So What Does Good Actually Look Like Right Now?
The researchers getting the most out of this moment aren't the ones treating AI as either a replacement or a toy. They're the ones who've figured out a division of labor that plays to both sides' strengths.
Something like: let the model chew through the codebase and hand you an orientation map. Let it flag the obvious pattern-matches so you're not wasting time on the boring stuff. Then step in yourself for the part that actually requires thinking like an attacker — building the chain, questioning the assumptions, asking the annoying "what if" questions the model wasn't prompted to ask.
That's not a consolation prize for humans. It's genuinely the more efficient workflow. The alternative — trusting a model's output uncritically — has its own failure mode that doesn't get talked about enough: false confidence. A scanner or model that reports "no issues found" doesn't mean no issues exist. It means nothing matched a pattern it recognized. Treating silence as a clean bill of health is how the business-logic bug slips straight into production.
The Comparison Nobody Likes, But It's Right
People keep reaching for the calculator analogy, and it's a little tired, but it's tired because it's accurate. Calculators didn't put mathematicians out of work. They killed manual arithmetic and freed people up to think about harder problems. Autocomplete didn't replace developers. It killed the boring keystrokes and left the architecture decisions right where they always were — with a person.
Vulnerability research is going through the same shift, just later than most fields, because security bugs are stubbornly resistant to shortcuts in a way that autocomplete-able code isn't. The repetitive 80% of the job — the orientation, the pattern-matching, the "have I seen this exact mistake before" work — is getting automated fast. The 20% that actually decides whether a breach happens or doesn't — the business logic, the chained reasoning, the invented attack class nobody's named yet — is stubbornly, currently, still human territory.
Attackers have never really broken code. They break assumptions. Right now, understanding which assumptions are worth questioning is still something only a person actually knows how to do.
That won't be permanently true. But it's true today, and pretending otherwise — in either direction — is how you end up either panicking for no reason or getting complacent for a very bad one.
Top comments (1)
Where do you land on this? Has AI changed your actual day-to-day workflow, or is it still mostly noise you have to double-check? Curious to hear how other people are actually using it in practice not the theory, the reality.