Teams often file every AI risk under one word, usually "security" or "safety", and then argue about whether a guardrail has dealt with it. The four concerns are related, but they ask different questions, need different evidence, and usually belong to different people.
- Security asks whether an actor can violate confidentiality, integrity, availability or authorized control.
- Safety asks whether operating the system can cause unacceptable harm, including harm with no attacker involved.
- Robustness asks whether behavior stays acceptable under noise, distribution shift, ambiguity or stress.
- Privacy asks whether personal or sensitive information is collected, inferred, used, retained or disclosed appropriately.
One summarizer, three different failures
Take a model that summarizes clinical documents.
A malicious document instructs it to reveal another patient's record. That is a security attack, with privacy impact. A faithful summary leaves out a critical allergy because the scan was poor. That is a robustness failure, with safety impact. And a correct diagnosis, inferred where nobody authorized the inference, can be a privacy and governance failure with no attacker anywhere.
No single guardrail can prove all three cases are controlled. Treating the four words as synonyms produces controls too vague to test.
Different questions need different evidence
| Question | Discipline | Representative evidence |
|---|---|---|
| Can an untrusted document cause an unauthorized tool action? | Security | Taint-aware attack tests, policy decisions, egress and tool traces |
| Can an allowed answer create unacceptable physical or social harm? | Safety | Hazard analysis, scenario tests, review and escalation outcomes |
| Does quality hold under blur, accent, long context or missing data? | Robustness | Slice metrics, perturbation tests, calibrated abstention |
| Can training or inference expose a person's data? | Privacy | Data inventory, minimization, access logs, deletion and privacy attack tests |
Engineering Insight
Record the overlap, but keep accountability separate. Security may own exploitability and access controls, product safety the hazardous outcomes, privacy the lawful processing and minimization, and ML engineering the robustness. A shared risk register shows where they intersect; each claim in it still needs an owner, a test, a threshold and a response.
An architecture review that lists one "AI risk" control and calls the job done has usually tested one of these four. Asking which discipline a control serves, and what evidence would show it failing, is the quickest way to find the other three.
Originally published at engineertoai.com. The full lesson is free, with no sign-up: Separate Security, Safety, Robustness, and Privacy.
Top comments (0)