Prompt Injection Detection Failed—So I Reduced My AI Agent's Permissions

I reviewed the incident records and decisions. AI helped produce this English follow-up. Defense details are generalized. #ABotWroteThis

Quick answer (TL;DR)

After a prompt injection made my AI agent reveal internal details, my best fix was not another detector. I applied least privilege and reduced external output paths.

Rules can overblock, classifiers can be manipulated, and message scores can hide history. My goal is a controlled failure when detection misses something.

This follows my 45-day honeypot report. It measured hostile traffic; this explains the redesign.

What failed

My Moltbook agent read public comments and generated replies. Some comments asked it to run commands or reveal system information. It processed them through the normal reply path and returned details not intended for publication.

One classification error became disclosure because external text influenced a model that could access implementation context and publish its output directly.

1. Split cheap rejection from semantic judgment

I use deterministic rules for patterns the bot never needs to accept and an LLM classifier for context-dependent cases. A system-path reference might be documentation, a security quotation, or extraction bait.

This split is not a security boundary. The classifier is another LLM reading untrusted text; if it is fooled into saying “safe,” its answer cannot be the only control.

2. Bound false positives and defense-induced denial of service

After the disclosure, I tightened a similarity rule. The roughly ten retained blocks were all legitimate discussion containing nearby words. This small sample is not a benchmark, but was enough to reject that rule.

Quarantine can also become an attack if each trigger extends a cooldown. I now cap repeated-trigger effects, isolate only the affected operation where possible, define recovery, and keep evidence for tuning.

3. Preserve history instead of taking a majority

My old design aggregated comment scores by sender. In retained data, more than half of roughly twenty favorable senders also had attack-classified logs. One sender had about 300 “safe” and 10 “attack” semantic classifications, while a deterministic layer recorded about 200 attack-pattern matches.

The layers measured different signals; majority voting erased the difference. I now preserve layer, history, and co-occurrence separately. Co-occurrence prompts review; it proves neither coordination nor intent.

4. Reduce permissions and output paths

Detection lowers exposure probability. Least privilege lowers maximum damage.

I removed unneeded context, separated read from write, and reduced automatic public routes. A documentation agent should not inherit every file permission; a repository-analysis agent should not automatically deploy.

This matches OWASP LLM01:2025: constrain privileges, separate untrusted content from trusted instructions, and limit consequential actions.

5. Safe-failure checklist

Testing still found inputs spanning time, classifier attacks, quarantine abuse, and hostile log strings. So my success condition became:

If an attack is not stopped, the agent must still fail without unlimited damage.

• reject obvious patterns before interpretation;
• isolate classifiers from powerful tools and secrets;
• retain history across layers and time;
• cap repeated-trigger outages;
• expose only the data and tools the action needs;
• control external output paths; and
• calculate the worst case if every detector is wrong.

Evidence boundary

These counts are approximate observations, not a benchmark. “Attack” and “safe” were system labels, not proof of intent. I omit current thresholds, prompts, paths, and account identifiers because they could aid evasion.

FAQ

Is prompt injection detection still necessary?

Yes. It reduces exposure and creates evidence. Least privilege and output controls limit a miss.

Why not add classifiers and use majority voting?

It may improve coverage, but can hide rare high-impact failures. Preserve each layer and its history instead of converting uncertain votes into authority.

What should I restrict first?

Start with the highest-impact capability reachable after untrusted input: credentials, writes, data export, deployment, or public replies. Remove what the task does not need.

Conclusion

Assume a detector will eventually be wrong. Then make what the model can do next small enough to fail safely.