An AI feature that talks to customers is not a purely technical system. It carries brand risk, regulatory risk, and a category of reputational risk that is harder to measure than either. A customer-support chatbot that answers rudely, a recommendation engine that produces discriminatory output, a summarization tool that quietly omits a compliance notice — each of these is an incident that can outlive the feature itself in the public record.
The practice that separates teams who ship these features safely from teams who ship them and then deal with the consequences is a pre-release safety review. Not a ceremony, not a long document, not a compliance box. A structured review that catches the obvious problems before customers do.
This post describes what that review looks like in practice.
What a safety review is actually checking
A useful AI safety review answers four questions before a feature goes live.
What can this feature be made to say or do that would be unacceptable?
Which of those unacceptable outputs are plausible in normal use, and which require adversarial effort?
What guards are in place for each category, and how were they tested?
What is the response plan if a customer reports an incident, and how quickly can the feature be disabled?
A feature that cannot answer these four questions, specifically, is not ready for a customer-facing launch.
The red-team session
The most useful part of a safety review is a focused session where two or three people actively try to make the system produce bad outputs. This is not penetration testing. It is a structured adversarial check.
A good red-team session has a checklist covering the common failure modes: refusal failures (the system helps with something it should not), persona drift (the system adopts a tone or identity it should not), confidentiality leaks (the system reveals internal information, system prompts, or other users’ data), factual errors with consequence (wrong pricing, wrong medical guidance, wrong legal advice), and bias (different quality of response based on demographic cues in the input).
Each finding becomes a test case added to the evaluation suite. The fix is not just a prompt change; it is a regression test that proves the fix works today and will still work after the next model update.
The escalation path
Not every bad output is a model failure. Some are prompts asking for something the model should not produce; others are the model producing something reasonable that the business should not have shown. Knowing which is which matters because the fixes are different.
A robust customer-facing system has a layered response: the model itself refuses things it should refuse; a separate safety classifier catches things the model did not refuse; a human-in-the-loop path handles things the classifier was uncertain about; and an audit trail ensures that a human can always review why a given response was produced. Each layer has its own cost, so most teams pick the combination that fits the stakes of the workload.
The kill switch
Every customer-facing AI feature needs an obvious, tested path to disable it within minutes. Not “we could deploy a change to disable it”; an actual feature flag or configuration value that the on-call engineer can flip without rebuilding anything. Test the kill switch before launch. Test it quarterly.
Teams that learn this the hard way do so because a live incident is unfolding and the path to stop it is not ready. The cost of building a kill switch in calm times is an hour. The cost of improvising one during an incident is measured in customer impact and overtime.
Who signs off
A safety review that only involves engineering misses problems that other functions would have caught. A useful review has named sign-offs from engineering (did we test the known failure modes?), product (is the feature doing what the business expects?), legal or compliance (are there regulatory implications?), and an operational owner (is there someone on-call for this feature?). Each signs off after reviewing findings from their angle.
This sounds heavy. In practice it is half an hour of asynchronous review once the engineering work is complete, because each participant is looking at a short document, not reproducing the entire engineering analysis. The weight is in the document quality, not in the meeting.
Post-launch monitoring
A safety review is a point-in-time artifact. Production is not. The commitments that outlast the review are the continuous monitoring signals that tell the team whether the feature is still behaving within expectations: the refusal rate, the rate of outputs flagged by the safety classifier, the rate of customer complaints tagged with AI-related categories, and the results of a weekly rerun of the red-team checklist against the live system.
If any of these drift materially, the feature goes back into review. This is the discipline that keeps a launched feature safe over the months and model updates after launch.
A reasonable cadence
Safety reviews are not expensive if the process is efficient. For a typical customer-facing feature, expect about a week of total elapsed time: a couple of hours to write the risk analysis, a focused red-team session, a day for the fixes and regression tests, and a day for the async sign-offs. The goal is not to make launches slower; it is to move the incidents that would have happened after launch to a controlled place before it.
Teams that institutionalize this launch more AI features, not fewer, because the confidence to launch comes from knowing the failure modes, not from hoping none exist.
Top comments (0)