Security Risks of AI-Generated Code: Research and a Review Checklist

Originally published on shipfastdev.com.

AI-generated code usually works, but it is often not secure. In Veracode's 2026 tests, models produced code that compiled almost 100% of the time, yet failed security checks on nearly 44% of tasks, a pass rate that has not improved since 2025. The fix is not to ban AI tools; it is to treat their output like code from a contractor you have never worked with: review it, scan it, and test it before it reaches production.

Below is a summary of the research, the failure patterns that come up again and again, and a review checklist you can hand to your developer or agency.

What the research found

Veracode: about half of security-relevant tasks fail, and it is not improving

In its 2025 GenAI Code Security Report, Veracode gave 80 curated coding tasks to more than 100 large language models. When a secure and an insecure approach were both possible, models chose the insecure one 45% of the time. Java was the riskiest language, with a security failure rate above 70%; Python, C# and JavaScript failed between 38% and 45% of the time. Models failed to defend against cross-site scripting in 86% of cases and log injection in 88%. Larger models did not perform significantly better than smaller ones.

The 2026 edition found the average security pass rate "sits at 56 percent — virtually unchanged since last year's report." The best model on its summer 2026 leaderboard reached 68%, which still means failing nearly one security task in three. Python code passed 63% of the time, Java only 30%. Coding-specialised models (51%) did no better than general-purpose ones (52%), and model size made no clear difference, although reasoning models kept an edge (56% against 51%).

Academic studies: vulnerable suggestions and overconfident users

• An early study of GitHub Copilot, "Asleep at the Keyboard?" (IEEE Symposium on Security and Privacy 2022), generated 1,689 programs across 89 security-relevant scenarios and found approximately 40% of them vulnerable.
• A user study published at ACM CCS 2023 found that participants with an AI assistant wrote significantly less secure code than those without, and were more likely to believe their code was secure.
• A USENIX Security 2025 paper analysed 576,000 code samples from 16 models and found that at least 5.2% of packages suggested by commercial models and 21.7% by open-source models did not exist, with 205,474 unique invented package names. An attacker can publish a malicious package under a name that models tend to invent, and anyone who installs it without checking is compromised.

Team level: more changes, less stability

Google's 2025 DORA report found that AI adoption is linked to higher delivery throughput but still has a negative relationship with delivery stability. Its explanation applies directly to security: without strong automated testing and fast feedback loops, more change volume leads to more incidents.

Why AI-generated code fails security checks

Security usually depends on context the model does not have: which input comes from an untrusted user, who is allowed to see which record, which secrets exist in your environment. Cross-site scripting, which depends on knowing where a piece of data will be displayed, had one of the worst results in Veracode's 2025 tests (86% failure). The model writes code that does what you asked; whether it is safe depends on what you did not say.

The failure patterns to look for

• Untrusted input reaching output or queries: cross-site scripting, SQL injection, log injection.
• Missing authorisation checks: an endpoint that checks the user is logged in, but not that the record belongs to them.
• Secrets in code: API keys pasted into source files or front-end bundles.
• Invented or outdated dependencies: packages that do not exist, or old versions with known vulnerabilities.
• Insecure defaults: permissive CORS, debug mode on, verbose error messages in production.
• Weak cryptography or home-made auth instead of the framework's built-in tools.

A security review checklist for AI-generated code

You don't need to read code to use this list. Ask your developer to confirm each point for every feature that touches users, payments or data. Items 1 to 4 apply before any change is merged, 5 to 7 cover dependencies and secrets, and 8 to 12 cover testing and process.

1. A human who understands the code has reviewed every AI-generated change, line by line.
2. Every endpoint checks authorisation (who can access this record), not just authentication.
3. All user input is validated on the server, and output is escaped by the framework (no raw HTML rendering of user content).
4. Database access uses the framework's query builder or parameterised queries, never string-built SQL.
5. Every new package was checked: it exists, is maintained, and is the one the developer meant to install.
6. A dependency audit runs automatically (for example npm audit, composer audit or GitHub Dependabot).
7. No secret is committed to the repository or shipped to the browser; secret scanning is enabled.
8. Automated tests cover login, permissions and payment flows, and run on every change. End-to-end tests such as the Playwright setup we use catch broken access rules early.
9. A static analysis (SAST) tool runs in CI on every pull request.
10. Production runs with debug off, restrictive CORS and generic error pages.
11. AI agents cannot touch production data or credentials directly.
12. Someone owns security decisions, and you know who.

Questions to ask your developer or agency

• Which AI tools do you use, and what is your review rule for their output?
• What runs automatically on every pull request: tests, dependency audit, static analysis, secret scanning?
• How are authorisation rules tested?
• Who would notice if a new package did not exist yesterday?

These questions belong in any evaluation of a development partner. If your app was built mainly with an AI app builder, read where vibe-coded MVPs break as well: missing data-access rules are the most common problem there.

Frequently asked questions

Is AI-generated code secure?

Not by default. Veracode's 2026 tests found an average security pass rate of 56%, unchanged from 2025, even though almost all generated code compiled. It needs the same review and testing as any other code, and arguably more.

Which programming language is riskiest for AI-generated code?

In Veracode's tests, Java: a 30% security pass rate in 2026, against 63% for Python. Language matters less than review, though: every language failed a significant share of tasks.

Can an AI tool review its own code for security?

It can help spot issues, but the same models chose insecure options in a large share of Veracode's tasks. Use AI review as one layer, alongside static analysis, dependency audits, tests and a human reviewer.

Do I need a security audit for my MVP?

If it handles personal data, payments or logins, at least run through the checklist above before launch. A full external audit makes sense once you have real users or sensitive data at scale.

Sources

1. LLMs Are Getting Smarter but Not Safer: Veracode 2026 GenAI Code Security Report, Veracode, 2026-07
2. AI-Generated Code Poses Major Security Risks in Nearly Half of All Development Tasks, Veracode Research Reveals, Veracode, 2025-07
3. Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions (IEEE S&P 2022), Pearce, Ahmad, Tan, Dolan-Gavitt, Karri (arXiv), 2021-12
4. Do Users Write More Insecure Code with AI Assistants? (ACM CCS 2023), Perry, Srivastava, Kumar, Boneh (arXiv), 2023-12
5. We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs (USENIX Security 2025), Spracklen et al. (arXiv), 2025-03
6. Announcing the 2025 DORA Report, Google Cloud, 2025-09