Most teams shipping LLM features treat red-teaming and observability as two separate concerns. Red-teaming happens before launch, a stress test to find where the model breaks. Observability happens after launch, dashboards and logs watching for things going wrong in production.
That split is starting to break down, and for good reason. The failure modes red-teamers look for before launch are increasingly the same failure modes observability systems need to catch in production, just showing up later and in messier form.
The pre-launch mindset doesn't survive contact with real users
A red team session usually looks like this: a small group tries to jailbreak the model, extract system prompts, get it to produce harmful content, or manipulate it into ignoring its instructions. They document what worked, the team patches the obvious holes, and the feature ships.
The problem is that real users are more creative and more numerous than any red team, and they're not trying to break things on a schedule. They're doing it constantly, at scale, in ways that don't match the specific attack patterns anyone tested for. A prompt injection technique that didn't occur to your red team in a two-week sprint will occur to someone on the internet eventually, often within days of launch.
So the question isn't "did we catch everything before shipping." It's "can we catch it after shipping, fast enough to matter."
What this means for observability
Traditional observability was built for deterministic systems: track latency, error rates, resource usage. You know what "wrong" looks like because the system either returns the expected type of response or it throws an error.
LLM systems don't fail that way. A jailbroken model doesn't throw an exception, it happily returns a 200 with content nobody wanted generated. A prompt injection doesn't crash anything, it just quietly changes what the model does next. The failure is semantic, not structural, which means you need observability tooling that actually looks at what the model said and did, not just whether the request completed.
This is pushing teams toward a few concrete practices:
Continuous, automated red-teaming instead of point-in-time testing. Rather than a pre-launch sprint, teams are running adversarial prompt suites against production models on a schedule, treating it more like continuous security scanning than a one-time audit.
Logging model reasoning and intermediate steps, not just final output. If an agent takes five tool calls to get to an answer, the interesting failure might be in step three, not the final response. Observability needs visibility into the chain, not just the endpoint.
Classifiers watching classifiers. Using a smaller, cheaper model to continuously score production outputs against categories like "policy violation," "prompt injection detected," or "off-task behavior," then alerting on drift rather than waiting for a user to report it.
Treating red-team findings as monitoring rules, not just patches. When a red team finds a jailbreak pattern, the fix isn't just patching that specific case, it's turning that pattern into something your observability layer watches for going forward, since variants of it will keep appearing.
The uncomfortable overlap
Here's the part that makes teams uneasy: if red-teaming and observability are converging, it means security testing isn't a phase you complete. It's a running process, indistinguishable in practice from the monitoring you're already supposed to be doing in production.
That's a different budget conversation than "we did a red team exercise before launch, we're good." It means the tooling and the headcount for adversarial testing need to live inside the same team, and often the same pipeline, as the tooling watching production behavior. Separating them into a pre-launch checkbox and a post-launch dashboard is exactly the structure that lets both efforts miss the same class of failure.
Where this is heading
The teams doing this well aren't the ones with the most red-teamers. They're the ones who've made red-teaming an ongoing input into their observability system rather than a gate before deployment. Every new jailbreak pattern discovered becomes a new thing the production system watches for automatically, and every anomaly the production system flags becomes a new test case fed back into the next red-team pass.
That loop, not the individual practices, is what's actually new here. Anyone building on top of LLMs at any real scale is going to end up building some version of it, whether they call it red-teaming, observability, or just "the thing that keeps us from finding out about problems from Twitter."
Top comments (1)
Turning every production anomaly into a durable adversarial test closes the loop nicely. One caution is to keep detection separate from enforcement: a semantic monitor can raise or route a case, but the actual disclosure and side-effect blocks should live in deterministic policy. Otherwise the safety classifier becomes another model whose confident miss still returns 200.