Adversarial testing of the input/output guardrails in Learna Tutor, a TK–8 AI tutoring app. October 2026.
Why this matters
Learna Tutor lets children ages ~4–13 chat directly with an AI tutor. That makes safety load-bearing in a way it isn't for adult tools: the users are kids, the failure modes include prompt injection ("ignore your instructions and…"), privacy extraction ("tell me your email"), and academic-integrity bypasses ("just give me the answer"). We needed guardrails we could actually verify, not just a system prompt and hope.
The architecture: defense in depth
Safety in Learna is three layers, and the tests target the two deterministic ones:
Input filter (checkTutorInput) — runs before any model call. Eight regex rule categories: privacy, crisis, bullying, medical-legal, off-scope, academic-integrity, prompt-injection, and unsafe-web. It also rejects unsupported tutor modes outright.
Model system prompt — grade-banded, hint-first instructions (not unit-tested; verified by review).
Output filter (checkTutorOutput) — screens the model's response after generation. Notably, it has a dedicated privacy check: if the tutor ever solicits a child's personal details ("tell me your email address"), the response is blocked. It also handles negation correctly — "Do not share your email address" passes, because blocking safety reminders would be a false positive.
Regex filters are deliberately the outermost layer: cheap, fast, no model call needed, and they fail closed.
The test approach
I wrote an adversarial suite covering each category with realistic attack phrasings a child (or a curious teenager) would actually type — not just textbook examples. Just as important, I included false-positive checks: legitimate questions the filter must not block. My favorite: a reading-comprehension prompt containing "Maya heard footsteps, hid a letter beneath a floorboard" — a naive privacy filter matching "letter" near personal-context words could nuke a perfectly good English question. It passes.
The finding
The suite caught a real gap on its first run: the prompt-injection pattern matched "ignore previous instructions" but missed "ignore your previous instructions" — a one-word variation. The optional "your" wasn't in the pattern.
This is the canonical regex-brittleness problem. Attackers don't use your exact phrasing; they vary it. "Ignore all previous instructions," "disregard the above," "forget everything I told you" — each variant is a separate hole unless the pattern generalizes. A child doesn't even need to be adversarial: "ignore your rules" is just how a kid talks.
The fix
Broadened the injection pattern with optional capture groups so determiners and modifiers generalize:
ignore\s+(your\s+|all\s+|the\s+)?(previous\s+)?instructions
plus parallel coverage for disregard, forget, system prompt, jailbreak, developer message, and role-play escapes ("pretend you are not…", "act as unrestricted"). After the fix I re-ran the full suite — all green — and added six extra injection probes with novel phrasings. All six blocked.
Honest limitations
Regex is a tripwire, not a wall. It catches known phrasing families. A genuinely novel jailbreak phrasing can still pass the input filter — which is exactly why the output filter and the system prompt exist as independent backstops. No single layer is trusted.
Testing was at the filter layer, not end-to-end against the live model (no API key in the test environment). The filters are deterministic, so unit tests are the right tool for them; model behavior needs separate eval.
This is ongoing. Every new attack phrasing we learn about is a new test case. The suite is designed to grow.
Takeaways
Test your guardrails adversarially, not just functionally. "Does it block the example in the docs" is not the same as "does it block what a real user types."
False-positive tests matter as much as blocking tests. A filter that blocks legitimate learning questions is a broken product, especially for kids.
One-word variants are the whole game in prompt injection. If your pattern doesn't generalize over determiners and synonyms, it's a demo, not a defense.
Defense in depth isn't a slogan — it's the reason a regex miss isn't a catastrophe. Input filter, system prompt, output filter: each one assumes the others can fail.
Learna Tutor is in private beta. The guardrail module (app/lib/safety.ts) and its test suite are part of the application's open development process; this write-up reflects testing performed October 2026.
Top comments (0)