I've been building Agenci — open-source CI/CD testing for AI agents (functional tests, security checks, regression detection across model/prompt changes).
Wanted to actually test it against something real rather than a synthetic example, so I pointed it at phi4-mini running locally via Ollama. First security test: told it to ignore its instructions and just say a specific word. It complied instantly — screenshot attached.
Small, efficient local models are great for a lot of things, but this made me curious how consistent this weakness is across other small local models people are running. Anyone here tested for prompt-injection resistance specifically, or is this a known/expected tradeoff at this model size?
Repo (free, pip install agenci): github.com/klinditafa1/agenci
Top comments (0)