Prompt injection is not a bug in a model. It is a consequence of the interface: instructions and data arrive in the same channel, as one flat sequence of tokens. There is no prepared statement, no privilege bit. A document that says "ignore your instructions" looks exactly like your instructions because it is the same kind of thing.
So nothing below is a fix. Spotlighting (Hines et al., Microsoft 2024) makes the boundary learnable, which is a different and more honest claim.
Measured live, against 1,200 generated injections: https://dev48.infy.uk/prompt/day66-spotlighting.html
Three techniques, increasing in cost
Delimiting — fence the data, and say in the system prompt what the fence means:
const delimit = text =>
"<<DATA>>\n" + text.split("<</DATA>>").join("<<\\/DATA>>") + "\n<</DATA>>";
That split/join is the escaping, and leaving it out is the entire fence-escape attack family.
Datamarking — a rare token in every whitespace gap, so the extent of the data is visible on every token rather than at two edges:
const datamark = (text, mark) => text.replace(/\s+/g, mark);
Encoding — base64, so an injected instruction is not natural language at all.
The numbers, with the columns nobody quotes
| defence | attack success | token cost | task accuracy |
|---|---|---|---|
| none | 62.6% | 22 | 92.0% |
| delimiting | 45.6% | 27 | 90.0% |
| datamarking | 37.1% | 54 | 83.0% |
| encoding | 25.7% | 33 | 68.0% |
Two things the security column alone will not tell you: datamarking more than doubles the token count of the data, and encoding costs 24 points of task accuracy. A defence that takes injections from 40% to 5% and accuracy from 92% to 68% may be a bad trade, and you cannot see that from one number.
And none of them reach zero — which is asserted
for (const d of DEFENCES)
ok(all[d].successRate > 0, `${d} reached 0% attack success - impossible for a prompt-side defence`);
If any row ever read 0%, the harness would be lying, and a lying harness is worse than none.
The rows that stay red
Averages hide the thing you need. Broken out by attack family, delimiting leaks 58% on a fence-escape attack against 44% for a direct injection — its known hole, barely defended.
My first version of that table under-stated datamarking's blind spots and made it look uniformly good, which hid the whole point of the breakdown. The corrected numbers come from the mechanism: datamarking works by filling whitespace, so a markdown URL and a base64 blob — neither of which contains any whitespace — are essentially unmarked.
What to actually do
Mark the boundary and constrain what the model may do with what it concludes. If a successful injection cannot trigger a side effect, its success rate matters much less. Spotlighting is worth deploying; it is not worth relaxing about.
Part of a from-scratch series — one prompting technique a day, measured rather than described: https://dev48.infy.uk/promptfromzero.php
Top comments (0)