DEV Community: kinga bhat

Why AI Red-Teaming and Observability Are Becoming the Same Discipline

kinga bhat — Tue, 15 Sep 2026 06:34:41 +0000

Most teams shipping LLM features treat red-teaming and observability as two separate concerns. Red-teaming happens before launch, a stress test to find where the model breaks. Observability happens after launch, dashboards and logs watching for things going wrong in production.

That split is starting to break down, and for good reason. The failure modes red-teamers look for before launch are increasingly the same failure modes observability systems need to catch in production, just showing up later and in messier form.

The pre-launch mindset doesn't survive contact with real users

A red team session usually looks like this: a small group tries to jailbreak the model, extract system prompts, get it to produce harmful content, or manipulate it into ignoring its instructions. They document what worked, the team patches the obvious holes, and the feature ships.

The problem is that real users are more creative and more numerous than any red team, and they're not trying to break things on a schedule. They're doing it constantly, at scale, in ways that don't match the specific attack patterns anyone tested for. A prompt injection technique that didn't occur to your red team in a two-week sprint will occur to someone on the internet eventually, often within days of launch.

So the question isn't "did we catch everything before shipping." It's "can we catch it after shipping, fast enough to matter."

What this means for observability

Traditional observability was built for deterministic systems: track latency, error rates, resource usage. You know what "wrong" looks like because the system either returns the expected type of response or it throws an error.

LLM systems don't fail that way. A jailbroken model doesn't throw an exception, it happily returns a 200 with content nobody wanted generated. A prompt injection doesn't crash anything, it just quietly changes what the model does next. The failure is semantic, not structural, which means you need observability tooling that actually looks at what the model said and did, not just whether the request completed.

This is pushing teams toward a few concrete practices:

Continuous, automated red-teaming instead of point-in-time testing. Rather than a pre-launch sprint, teams are running adversarial prompt suites against production models on a schedule, treating it more like continuous security scanning than a one-time audit.

Logging model reasoning and intermediate steps, not just final output. If an agent takes five tool calls to get to an answer, the interesting failure might be in step three, not the final response. Observability needs visibility into the chain, not just the endpoint.

Classifiers watching classifiers. Using a smaller, cheaper model to continuously score production outputs against categories like "policy violation," "prompt injection detected," or "off-task behavior," then alerting on drift rather than waiting for a user to report it.

Treating red-team findings as monitoring rules, not just patches. When a red team finds a jailbreak pattern, the fix isn't just patching that specific case, it's turning that pattern into something your observability layer watches for going forward, since variants of it will keep appearing.

The uncomfortable overlap

Here's the part that makes teams uneasy: if red-teaming and observability are converging, it means security testing isn't a phase you complete. It's a running process, indistinguishable in practice from the monitoring you're already supposed to be doing in production.

That's a different budget conversation than "we did a red team exercise before launch, we're good." It means the tooling and the headcount for adversarial testing need to live inside the same team, and often the same pipeline, as the tooling watching production behavior. Separating them into a pre-launch checkbox and a post-launch dashboard is exactly the structure that lets both efforts miss the same class of failure.

Where this is heading

The teams doing this well aren't the ones with the most red-teamers. They're the ones who've made red-teaming an ongoing input into their observability system rather than a gate before deployment. Every new jailbreak pattern discovered becomes a new thing the production system watches for automatically, and every anomaly the production system flags becomes a new test case fed back into the next red-team pass.

That loop, not the individual practices, is what's actually new here. Anyone building on top of LLMs at any real scale is going to end up building some version of it, whether they call it red-teaming, observability, or just "the thing that keeps us from finding out about problems from Twitter."

The Prompt Is the Bottleneck, Not the Model

kinga bhat — Tue, 15 Sep 2026 06:28:31 +0000

Every few months there's a new debate about whether LLMs have plateaued. Better benchmarks, bigger context windows, cheaper inference. And yet most people using ChatGPT, Claude, or Gemini day to day are still getting mediocre output.

The uncomfortable truth: for the vast majority of real-world use cases, the model stopped being the bottleneck a while ago. The prompt is.

The variance is bigger than you think

I've run the same task across GPT-5.x, Claude, and a couple of open-weight models, and the gap between a lazy one-line prompt and a properly structured one is often larger than the gap between the models themselves.

Type "write me a marketing email" and you get something forgettable, regardless of which model answers. Give it a role, real context, a defined objective, and explicit constraints, and the exact same model performs like it was actually briefed.

This isn't news to anyone who's spent real time with these tools. But it's still not reflected in how most people actually use them.

Why structure gets skipped

Frameworks like CO-STAR (Context, Objective, Style, Tone, Audience, Response) and RISEN (Role, Instructions, Steps, End goal, Narrowing) exist because a good prompt has real, repeatable parts. The problem is holding all of that in your head every single time you open a chat window.

So people default to the path of least resistance: a single sentence, sent cold. Then the output is underwhelming, and the conclusion becomes "AI isn't that good," when the actual issue was never given enough to work with.

A concrete example

Here's the same task, unstructured vs structured:

Unstructured:

Fix this code.

Structured:

Act as a Senior [LANGUAGE] Engineer reviewing a pull request.
Context: This function is supposed to [EXPECTED_BEHAVIOR] but instead [ACTUAL_BEHAVIOR].
Objective: Identify the root cause and propose a fix.
Format: Explanation first, then the corrected code block.
Constraints: Do not rewrite unrelated code. Flag any edge cases you notice.

Same model. Same underlying capability. Completely different quality of output, because the second version actually tells the model what "good" looks like.

The real fix isn't "learn to prompt better"

Telling people to just get better at prompting doesn't scale, the same way telling people to "just write better emails" doesn't fix inbox chaos. What actually moves the needle is making structure the default instead of something you have to remember and reconstruct every time.

That's the gap most prompt tooling is trying to close right now: not smarter models, but better scaffolding for how people talk to the models they already have. A saved template, a builder that walks through the missing pieces, a shared library so a team isn't rebuilding the same prompt from scratch every week.

If you're curious what that looks like in practice, I put together a structured prompt writer and template library built around CO-STAR, RISEN, and CRAFT, covering 29 categories from coding to marketing to HR, all reusable across ChatGPT, Claude, and Gemini.

Where this is heading

As more real work runs through these models, the ability to brief them clearly is going to matter the same way clear technical writing already does. It won't be a nice-to-have skill for power users. It'll be assumed, the same way basic Git literacy is assumed on most teams today.

The models will keep getting better regardless of what any of us do. The more interesting problem, and the one actually worth solving right now, is closing the gap between what these models can do and what most people are actually getting out of them.

What's your experience been? Curious if others have noticed the same variance between prompt quality and model choice.