I've been building security tools for AI agents and I have a genuine question for this community.
When you ship an AI powered feature a chatbot, an agent, anything that takes user input and passes it to an LLM do you test it for adversarial inputs?
I'm talking about stuff like
- Can a user extract the system prompt?
- Can they make the agent ignore its instructions?
- Can they get the agent to call tools or access data it shouldn't?
I'm not asking to judge. I'm asking because I've talked to maybe 200 developers at this point and the honest answer from most of them is "no." And that's fine the tooling barely exists yet.
But I'm curious about this community.
Where do you fall?
A) I test for this stuff before shipping
B) I know I should but haven't gotten to it
C) I didn't know this was a thing until now
D) My agents don't handle anything sensitive enough to worry about
Drop your answer in the comments. No judgment either way.
If you want to test yours right now here
Top comments (0)