Your feed is full of it right now. An agent one-shots a $10,000 website. Sixty sub-agents swarm a repo overnight. A founder ships a whole product before lunch. The speed is real and it is genuinely impressive. What those clips almost never mention is the second number, the one that decides whether the thing you just shipped becomes a liability.
Here it is, from the people who scan this stuff for a living.
The gap between "runs" and "safe"
Veracode's Spring 2026 GenAI Code Security update ran 80 coding tasks across four languages against more than 150 large language models. Two of its findings sit right next to each other, and together they tell the story. Over 95% of the generated code was syntactically correct. It compiles, it runs, it looks finished. Only 55% of it was actually secure. The other 45% shipped a known vulnerability (Veracode).
That security number has barely moved in two years, even as the models got dramatically better at everything else. Bigger models barely helped, and Veracode notes the small advantage they once had "has largely disappeared with more recent releases." The one bright spot, reasoning models at around 70 to 72%, still leaves roughly one task in three with a hole in it.
The breakdown by weakness is where it starts to sting. Cross-site scripting passed 15% of the time. Log injection, 13%. Java overall, 29%. None of these are exotic bugs. They are the OWASP classics we have had linters for since the early 2010s, and the agents reintroduce them at scale, confidently, inside code that sails through every "does it work" check you would normally trust.
Velocity makes it worse, not better
The natural answer is "fine, we will review it." But the same wave of tools changed how much there is to review. GitClear's analysis found that the average developer now commits far more code than a few years ago, while the share of carefully reworked code fell off a cliff. For the first time on record, copy-paste overtook refactor (Dark Reading).
So the volume went up, the care per line went down, and the review budget stayed flat. That is not a combination that catches the 45%.
The bug nobody typed
There is also a whole class of problem you did not write and cannot diff, because it lives in what the agent decides to fetch. When a model does not know a package, it tends to invent one, and it invents the same fake names over and over. Attackers noticed. The Cloud Security Alliance has a name for the result: slopsquatting. You register the hallucinated package, then wait for an agent to install it. One malicious package impersonating a common ESLint plugin was still pulling around 233 downloads a week months after npm flagged it, and researchers found 127 package names, 109 on PyPI and 18 on npm, that every major frontier model reliably hallucinates. Run the same prompt ten times and 43% of the fake names come back every single time (Cloud Security Alliance).
Predictable hallucination is a gift to an attacker. They do not even have to guess where to set the trap. The models tell them.
"It fixed the bug" and "the bug is gone" are different claims
Even when you point an agent straight at a security problem, "looks fixed" and "is fixed" are not the same thing. We put agents on real, disclosed CVEs with the internet cut off (our benchmark). The flagship models wrote patches that read clean and often passed an AI reviewer, but on the hard set only 21 to 27% were clean, mergeable, and actually closed the hole. The fast AI grader we used for triage was wrong about one fix in seven, in both directions, while sounding completely certain. A patch can compile, read well, and survive review, and still leave the vulnerability wide open.
That is the Veracode number again, seen from the other side. The cheap signal, that it compiles and an AI said it was fine, is exactly the signal most likely to flatter you.
What actually holds
This does not mean AI coding agents are a bad idea. It means the signals we lean on because they are cheap, that it compiles and it looks right, are the ones we should trust least. A few things genuinely help, and none of them are exciting.
Run the tests instead of reading the diff. The only verdict that counts is the exploit failing against a real execution, broken before, working after. A reader, human or AI, will happily wave through code that does not even compile.
Put a gate on what the agent installs. Pin your lockfiles, verify hashes in CI, and do not let an agent pull a new package without an allowlist or a person in the loop. That closes the slopsquatting door at no cost.
Keep a real human on the security-critical change. Not a second agent grading the first one. Someone who understands the diff and has the standing to reject it. Nearly every failure above gets caught there, and mostly nowhere else.
The number your feed shows you is the 95%. The number that wakes you up at 2am is the 55%. Ship at agent speed if you want, just measure the thing the demo never does.
Sources: Veracode Spring 2026 GenAI Code Security, Cloud Security Alliance slopsquatting research note, GitClear via Dark Reading, Help Net Security on AI coding agents repeating old mistakes, Mobb, We asked AI agents to fix real security bugs
Top comments (0)