A developer's map of the tools, grouped by the agent attack surfaces they cover.
The failures that hurt with an AI agent are actions: a refund issued, a record deleted, a file sent to the wrong person. A chatbot that leaks its system prompt is embarrassing. An agent with a payments tool that obeys a hidden instruction in a support ticket is an incident. That difference changes how you test, and it changes which tools are useful.
This is a condensed version of our full guide, written for developers who need to pick tools quickly.
Why agents need different testing
Three properties change the job. Agents act in several steps, so a small manipulation early in a plan can compound through later tool calls. They read untrusted content such as emails, web pages, tickets, and tool outputs, and a model cannot reliably separate that content from instructions. And they hold real permissions, so the damage is limited by what the agent is allowed to do, not by what the model is willing to say.
The Cloud Security Alliance reached the same conclusion from the tooling side. Its evaluation of PyRIT for agentic red teaming found that PyRIT can test what a model says about using tools but cannot observe actual tool invocations.
What to test
The OWASP Top 10 for Agentic Applications names ten places an agent can fail. If you only have time for three, start with these, since every agent that reads outside content, calls a tool, and acts for a user has them:
| Risk | What to try | Example failure |
|---|---|---|
| Goal hijack (ASI01) | Plant instructions in emails, pages, tickets, and documents the agent reads | The agent abandons its task and follows a hidden command |
| Tool misuse (ASI02) | Push unsafe arguments, odd tool chains, and runaway loops through legitimate tools | The agent bulk-deletes records or burns an API budget |
| Identity and privilege abuse (ASI03) | Ask for other users' data or functions the agent should refuse | The agent acts on another customer's account |
The rest cover supply chain, code execution, memory poisoning, inter-agent messages, cascading failures, human trust, and rogue agents.
Four questions for any tool
- Does the attack arrive through a channel the agent actually reads? A tool that only types into a chat box misses hostile content in tool outputs, retrieved documents, memory, and MCP tool descriptions.
- Is success judged by what the agent did or what it wrote? The AgentDojo authors score outcomes by inspecting environment state, and warn that simulating tool results with a language model is a weak setup for injection tests because the simulator can be fooled too.
- Can you replay the failure after a fix? A finding you cannot rerun is a report, not a safeguard.
- What does a full run cost? Attacker and judge models use tokens in every tool here, open source included.
The tools
Promptfoo is the best open-source choice for agent attacks in CI. Its agent red teaming guide lists plugins for object-level and function-level authorization, RAG poisoning, memory poisoning, and excessive agency, and an MCP plugin tests function discovery and tool metadata injection. A model grades responses by default, but it can use OpenTelemetry traces to see real tool calls if your agent emits them. Promptfoo agreed to be acquired by OpenAI and committed to stay open source.
DeepTeam suits Python teams that want a code-first framework. Confident AI's documentation describes single-turn and multi-turn attacks against agents, RAG pipelines, and chatbots, including systems with memory and tool use. On its own it has no dashboard, and Confident AI sells the platform on top, so treat its comparisons as a vendor's view.
AgentDojo is the best way to measure how easily tool outputs hijack an agent. Per the paper, it has 97 user tasks and 27 injection tasks that combine into 629 security test cases, checked against environment state. It benchmarks generic agents, not yours.
Snyk Agent Scan checks the agent supply chain. It started as MCP-Scan from Invariant Labs and is maintained by Snyk. It scans MCP servers and skills for prompt injection, tool poisoning, and malicious skills. It needs a Snyk token, sends component details to Snyk's API, and can start the MCP servers it scans, so run it in a sandbox for anything you do not trust.
PyRIT is a flexible framework for multi-turn campaigns, but as noted above it does not run real agents. Garak is the quickest broad scan of the model underneath your agent, and a good baseline before you test behavior.
BotGauge, which we build, tests agent behavior end to end and turns each finding into a check that runs on every release. It is built for agent teams, not as a network-level AI firewall.
What covers what
| Tool | Runs against a live agent | Tool and permission abuse | MCP and supply chain | Success judged by |
|---|---|---|---|---|
| Promptfoo | Yes | Yes | Yes | Model grader, plus traces when instrumented |
| DeepTeam | Yes | Yes | Not documented | Language model judge |
| AgentDojo | No, own environments | Partial | No | Environment state |
| Snyk Agent Scan | No, scans components | No | Yes | Rule-based analysis |
| PyRIT | Partial | Partial | Not documented | Scorers |
| Garak | No, targets models | Not documented | No | Detectors |
The table reflects each project's public documentation, so confirm details in a trial before you standardize.
What to run first
A prototype with no write access needs a Garak baseline and a scan of any MCP servers. Once the agent calls tools for real users, add Promptfoo or DeepTeam campaigns in CI. In production, add an agent-level platform and use the scan as a deploy gate.
In AgentDojo's own tests, limiting a GPT-4o agent to the tools its task required cut targeted attack success from roughly 58 percent to under 8 percent, although it failed when the task's own tools were enough to carry out the attack. Tighten permissions before you rely on prompt wording.
The full guide has the complete comparison, the coverage matrix by attack surface, and a one-week plan for a first agent red team: AI red teaming tools for autonomous agents.
Top comments (0)