Your Red Team Found a Jailbreak. Now What?

Most AI red teaming tools can break your chatbot. The harder question is what happens to the finding after the scan finishes.

Run any decent adversarial testing tool against an LLM app and you will get findings. Prompt injection lands. The system prompt leaks. A multi-turn escalation gets the agent to call a tool it should never touch. That part of the category is basically solved.

The part that is not solved is everything after the report renders. Someone reads the PDF, files a ticket, edits a system prompt, and hopes. Three weeks later a model version bumps and nobody knows if the fix held.

That gap is the actual thing to evaluate when you pick a tool, and it is the lens behind NeuralTrust's rundown of the ten platforms worth knowing in 2026. This is the short version for people who have to wire one of these into a pipeline.

Why your pentest playbook does not port over

Traditional pentesting assumes a deterministic target. Same input, same code path, same result. You find an injection point, you patch it, it stays patched.

LLM applications violate all three assumptions. The exploit is natural language, not a malformed packet. The same prompt can pass on Monday and fail on Tuesday because temperature, retrieved context, or a provider-side model update changed underneath you. And the attack surface reopens on every prompt edit, every new tool binding, every RAG index refresh.

This is not a fringe opinion. OWASP has kept prompt injection at the top of its Top 10 for LLM Applications for two editions running, and the 2025 document is blunt about why: because of how generative models work, it is unclear whether any foolproof prevention exists. Their recommended mitigation list is defense in depth plus adversarial testing and attack simulation. Not a patch. A control plus continuous verification.

So a quarterly manual audit is theater. Adversarial testing has to run like a test suite, on every change.

The category consolidated fast

Worth knowing before you sign anything, because it changes who owns the roadmap:

• Zscaler acquired SPLX in November 2025 and folded it into the Zero Trust Exchange.
• Check Point acquired Lakera in 2025, making it the base of its AI security practice.
• OpenAI announced its acquisition of Promptfoo on March 9, 2026, integrating it into OpenAI Frontier. The open source project stays open under its current license.

None of this is inherently bad. If your SOC already lives in Zscaler or Check Point, consolidation is a feature. But it does mean the red teaming roadmap now follows a much larger platform's priorities. And in the Promptfoo case there is a structural question worth asking out loud: a testing tool owned by a model lab evaluates models in a context shaped by that lab.

Four things to actually check

1. Does a finding become a control? This is the one most teams underweight. If your testing vendor and your runtime vendor are different companies, you own the translation layer between them forever. A closed loop looks like: attack lands, finding maps to an enforcement policy, re-run confirms the policy blocks it. NeuralTrust pairs TrustTest with TrustGuard, its own inline detection engine, specifically so that handoff is not yours to build.

2. Is it stateful across turns? Single-turn scanning is the easy 60 percent. Crescendo, role-play drift, and context hijacking work precisely because each individual message looks fine. If the detection layer evaluates messages in isolation, it will miss the attacks that actually succeed in production. Same question applies to the offensive side: does the tool escalate across a conversation, or just replay a jailbreak corpus?

3. Does the output survive an audit? OWASP, MITRE ATLAS, ISO/IEC 42001, and the EU AI Act are the checkpoints enterprise procurement asks about. A raw JSON dump means someone on your team spends a week translating results into something a risk owner can read.

4. Does it test agents, not just chat? Tool misuse, indirect injection through tool output, memory manipulation, MCP surfaces. The threat model for a tool-calling agent has little to do with the threat model for a chatbot. The teams cataloguing this properly, like the threat library over at agentsecurity.com, treat agentic failure modes as their own class, and your testing should too.

Make it a build step

The practical bar is that a failed attack blocks a merge. Here is roughly what that looks like with TrustTest, which is a Python framework rather than a portal, so it version-controls alongside your app code. Adapted from the official docs:

import os
import trusttest
from trusttest.catalog.red_team import run_red_teaming
from trusttest.targets.http import HttpTarget, PayloadConfig

target = HttpTarget(
    url="https://your-api.com/chat",
    headers={"Content-Type": "application/json"},
    payload_config=PayloadConfig(format={"message": "{{ test }}"}),
    concatenate_field="response",
)

client = trusttest.client(
    type="neuraltrust",
    token=os.getenv("TARGET_TOKEN"),
    target_id=os.getenv("TARGET_ID"),
)

for language in ["English", "Spanish"]:
    run_red_teaming(target, language=language, client=client)

Two things to notice. The target is an HTTP endpoint, which means this is blackbox. You are testing the deployed application, including its system prompt, retrieval layer, and tool bindings, not an isolated model behind an API. That distinction matters because most real failures live in the glue, not the weights.

The other is the language loop. Multilingual testing is not a nice-to-have. Safety alignment is unevenly distributed across languages, and a refusal that holds in English frequently does not hold in a lower-resource language. If your app serves non-English users, monolingual testing is measuring the wrong thing. NeuralTrust's own red teaming product page leans on this, and it is one of the more common blind spots in homegrown eval suites.

The short version

Pick on the loop, not the attack count. Every vendor will quote you a big number of adversarial scenarios and every vendor's demo will find something. What separates them is whether a finding turns into an enforced runtime policy and whether a re-run proves the fix held.

If those are two different vendors, that integration is now your problem. Ask the question during the POC, not after the model version bumps.