TL;DR
- AI editors did not learn security from security documentation. They learned from tutorials, and tutorials strip security controls on purpose to stay readable.
- The model is not ignorant. Paste its own output back and it flags the bug correctly. Generating and evaluating run on different paths.
- Prompting for security helps but does not override the prior. Check the output with something that does not share it.
Last week I asked Cursor to add a search endpoint to a small Express API. It gave me three lines with the query parameter interpolated straight into the SQL string.
I pasted that exact block back into the same chat and asked whether it was safe. It told me the query was vulnerable to SQL injection, explained the mechanism, and offered a parameterized rewrite.
Same model. Same session. Ninety seconds apart.
That gap bothered me more than the bug did. A model that cannot recognize SQL injection is a training problem you fix with more data. A model that recognizes it perfectly and writes it anyway is a different thing entirely.
What the Model Actually Learned From
AI editors learned to write code mostly from tutorials, quickstarts, and Stack Overflow answers, not from production codebases or security documentation. That single fact sets the default for everything they generate.
Think about which code is actually public and duplicated at scale. A popular "build a REST API in Node" tutorial gets copied into thousands of repos, reposted across a dozen blogs, and quoted in hundreds of Stack Overflow answers. Production code that survived a real security review usually sits in a private repo, and on the rare occasion it is public, it exists once.
The training corpus is not a random sample of code. It is heavily weighted toward code written to teach.
Teaching Code Is Optimized Against Security
Tutorials cut whatever does not change what the reader sees on screen, and every security control is invisible on success. That is not carelessness. It is the editing rule that makes a tutorial readable.
Run down the list. A parameterized query produces identical output to an interpolated one for well-behaved input. A rate limiter does nothing until somebody abuses the endpoint. An ownership check passes silently for the user who owns the record. A constant-time comparison returns the same boolean as a plain equality check.
Every one of them costs lines and changes nothing the reader can observe. If your editing rule is "cut anything that does not move the demo forward," you cut the security controls first, every time, without ever making a decision about security.
The bias in the corpus is not random noise. It points in one direction.
The Warning Did Not Survive the Copy
Tutorial authors usually do flag the gap, but they flag it in prose next to the code block, and the code block is what propagated.
Tutorials are full of lines like "for simplicity we are skipping validation here" and "do not do this in production." Those sentences sit in a paragraph above or below the snippet. What got copied into a repo was the snippet. What got quoted in the next blog post was the snippet. Even the inline comment version, the // in production, load this from an environment variable line, tends to get deleted by the second person who pastes it, because it reads like clutter once the code is in a real file.
So the pattern propagated at full strength and the caveat decayed at every hop. By the time all of it reached training, the code was common and the warning was rare.
Knowing and Generating Are Different Paths
The model explains the vulnerability correctly because explanation draws on security writing in the corpus. Generation draws on code, and the code distribution is tutorial-shaped. Those are two different distributions and they disagree with each other.
Here is the reproducible version. Ask for a search endpoint:
// Prompt: "add a product search endpoint"
app.get('/api/search', async (req, res) => {
const q = req.query.q;
const rows = await db.query(
`SELECT * FROM products WHERE name LIKE '%${q}%'` // CWE-89
);
res.json(rows);
});
Paste that back and ask "is this safe?" and you get an accurate answer about CWE-89, plus the fix:
app.get('/api/search', async (req, res) => {
const rows = await db.query(
'SELECT * FROM products WHERE name LIKE $1',
[`%${req.query.q}%`]
);
res.json(rows);
});
The knowledge was there the whole time. Nothing about the first generation required it.
That reframes the problem. You are not filling a knowledge gap. You are competing with a prior.
Why Prompting Does Not Fully Fix It
Telling the model to write secure code shifts the headline pattern but not the details, and the prior reasserts itself as the session gets longer.
I have watched this happen enough times to stop trusting it. Ask for "a secure login endpoint" and you reliably get bcrypt, because "secure login" is strongly associated with bcrypt across the corpus. You do not reliably get a rate limiter, because the word "secure" was not specific enough to summon one. The instruction moved the thing it was pointed at and left everything else on the default.
Twenty files into a session, that instruction is far back in context and the local pattern pressure is immediate. The default comes back. Not because the model forgot, but because nothing in the current generation is asking it to be anything other than typical.
What Actually Works
Check the output with something that does not share the model's prior. In practice that means a scanner or a hook, and optionally a second pass whose only job is review.
The asymmetry is the useful part here. The model is good at evaluating code and biased when producing it, so putting it in evaluation mode is not wasted. A separate review pass over generated code does catch a real share of what generation introduced, precisely because it runs on the path that knows things.
But a review pass by the same model still inherits the same blind spots. Deterministic tooling does not. semgrep has no opinion about what a search endpoint usually looks like. It has rules, and the rules fire the same way on the thousandth file as on the first.
That is the whole trick. The bias is systematic, so the check has to be systematic too.
FAQ
Q: Do newer AI models write more secure code?
A: Somewhat, but the underlying corpus does not change. New tutorials are written the same way and scraped the same way, and a growing share of new tutorials are themselves AI-generated from the older distribution. Better alignment reduces the effect. It does not remove the cause.
Q: Can I just tell Cursor to write secure code?
A: It helps and you should do it, but do not treat it as coverage. The instruction reliably affects the specific pattern you named and leaves the surrounding gaps filled by the default. Verify the output with a scanner.
Q: Why can the model spot the bug but not avoid writing it?
A: Explaining a vulnerability draws on security writing in the training data. Generating code draws on the code distribution, which is dominated by teaching code that omits security controls on purpose. Both were learned correctly. They just disagree.
I have been running SafeWeave for this. It hooks into Cursor and Claude Code as an MCP server and checks generated code before I move on, on rules rather than on a model's judgment about its own output. Even a basic pre-commit hook with semgrep and gitleaks will catch most of what a tutorial-shaped default produces. The point is to verify with something that does not share the bias, whatever tool you use.
Read the full original on the SafeWeave blog: https://safeweave.dev/blog/cursor-learned-to-code-from-tutorials-that-skip-security
Top comments (0)