Prompt injection is the security problem that defines LLM applications, and I kept running into the same wall when I tried to explain it: reading about it does not build intuition. You can describe "ignore previous instructions" all day, but until you actually watch a model spill a secret it was told to guard, none of it lands. So I built injection-arena, a self-hostable game where a sandboxed AI agent defends a hidden secret and you race to make it leak.
The pitch is simple: guard a secret, break the guard, top the leaderboard. The learning is the side effect.
The core idea
Each level is a challenge. It has a system prompt that instructs an agent and hides a secret formatted like IARENA{...}, a canary token embedded in that same prompt, and a stack of defense layers. You send messages to the agent and try to make it reveal the secret. The game grades you on the server and tells you whether you cracked it.
There are ten levels, and each one stacks a new defense on top of the last. Level 1 is barely defended so you can feel a plain attack working. By the time you reach level 10, only a combined attack (payload splitting plus delimiter confusion) gets through. The progression is the whole point: you feel exactly what each defense stops and exactly where it breaks.
How the defenses work
An attempt flows through a single server-side pipeline that looks like this:
input-filter -> agent -> judge -> score -> persist
The defenses attach at different stages of that pipeline:
- System guard lives in the prompt itself: hardened instructions telling the agent to refuse.
- Input filter runs before the model sees anything. It blocks loud override and system-leak payloads pre-agent.
- Roleplay block rejects persona-hijack attacks, the "pretend you are a different assistant" family.
- Encoding guard rejects base64, spell-it-out, and translation-based exfiltration attempts.
- Output filter runs after the agent responds and redacts the secret if it appears verbatim.
- Canary token is checked by the judge: if the canary shows up in the output, the prompt escaped and that is an automatic crack.
The seeded attack techniques (direct ask, authority override, roleplay, translation, base64, spell-out, ignore-previous-instructions, system-prompt leak, few-shot poisoning, delimiter confusion, payload splitting) live in one library. They power both the difficulty design and the test suite, so the attacks I test against are the same ones players learn to run.
The judge, and why canary tokens matter
The most important design decision was never trusting the client. Grading happens entirely server-side. A naive version of this game would ask the model "did you leak?" or check the response on the frontend, and both are trivially gamed.
Instead the judge does a few concrete things. It checks whether the secret appears in the output. It checks whether the canary token appears, which is the sharper signal: the canary is embedded in the system prompt and nowhere else, so if it surfaces in a response, the system prompt itself has escaped even if the literal secret string did not. And it accounts for obfuscation, because a leak that comes back base64-encoded or spelled out letter by letter is still a leak. The output filter can redact the raw secret, but the judge is what decides whether a crack actually happened.
Scoring rewards higher difficulty, more active defenses, and cracking with fewer attempts. Only your first crack of a level counts, so you cannot farm points by re-submitting the same winning payload.
The offline mock agent
Here is the part I am most happy with. The entire game runs with no API keys and no network. The default provider is a deterministic mock agent that simulates injection susceptibility per level. Each challenge declares which technique families still work against it offline, and the mock honors exactly those, so there is a real difficulty curve even without a real model.
That means you can clone the repo, run npm install && npm run dev, and immediately play all ten levels for free. It also means the whole thing is testable in CI: the suite has 36 tests covering the techniques, the judge, the scoring, the pipeline, sessions, and the database, all running against the deterministic mock with no external calls.
When you want the real thing, you set AGENT_PROVIDER to anthropic, openai, or groq and supply the key. Real-model responses are graded by the exact same server-side judge. And if the selected provider's key is missing, the app falls back to the mock automatically, so it is always runnable.
Self-hosting and storage
The persistence layer is a pluggable async interface. Locally and in tests it uses SQLite through better-sqlite3, stored at a configurable path. In production on Cloudflare Workers it uses Cloudflare D1 through a binding, and the backend is selected automatically at runtime. No external database is required to run it locally. Players are identified by a signed cookie plus a nickname, so there is no login flow to stand up. The live instance runs on Workers with D1 via the OpenNext adapter.
Adding a level is deliberately small: append a challenge object to the levels file, list which technique families should still crack it offline, and add a test asserting what should and should not work. No schema changes, no migrations. That was a design constraint I held onto, because a challenge platform that is painful to extend does not grow.
One honest limitation
The rate limiter is in-memory and fixed-window. That is fine for a single instance and for local play, but if you self-host across multiple instances, each one keeps its own counter, so the effective limit multiplies and the protection weakens. The code notes this: front it with something shared like Redis if you run more than one instance. I chose the simple version deliberately to keep the zero-dependency local story clean, but it is a real edge you should know about before scaling it out.
I would also be upfront that the offline mock is a simulation of susceptibility, not a real model. It is excellent for building intuition and for deterministic tests, but the honest way to feel how a specific model behaves is to plug that model in and attack it directly.
Try it
Play it live at https://injection-arena.agentpostmortem.com, or clone and self-host from https://github.com/AgentPostmortem/injection-arena. It is MIT licensed. Contributions of new levels and attack techniques are especially welcome, and given the extension surface, they are genuinely easy to add.
Top comments (0)