Finance, healthcare, and government have different regulators and the same enforcement point. Here is where it actually lives in your stack.
Most teams try to solve AI compliance in the application layer. A PII scrubber in the chatbot service. A logging wrapper around one SDK call. A policy doc that says "do not send patient data to external models."
That approach fails for a structural reason. Every regulator that cares about AI asks the same question during an audit: show me this specific decision, the exact input that produced it, and the model version that made it. If your controls live inside individual applications, you have as many partial answers as you have integrations, and no authoritative one.
The rules differ by sector. The enforcement point does not.
What actually differs
Finance. DORA (Regulation (EU) 2022/2554) has applied since 17 January 2025. If an AI system supports a critical or important function, it is regulated ICT infrastructure, with testing, incident reporting and continuity obligations attached. Article 28 pulls your cloud provider into scope too, so their resilience posture and contract terms become your compliance problem.
MiFID II adds retention. Article 16(6) and 16(7) require records of client orders and communications to be kept for five years, extendable to seven at a competent authority's request. If a model influenced a trade, that decision has to be reconstructable years later.
Healthcare. HIPAA has no AI-specific category and does not need one. Touch protected health information and the Security Rule technical safeguards at 45 CFR 164.312 apply: access control, audit controls, integrity, authentication, transmission security.
The gap people miss is jurisdictional. A Business Associate Agreement with your model provider covers the contractual relationship. It does not remove the provider's exposure to foreign legal process. Data residency in an EU or US region is not the same as legal control over the infrastructure running inference.
Government. FedRAMP is the entry ticket for cloud services sold to US federal agencies, assessed against NIST SP 800-53 Rev 5 and its 20 control families. SI-19 covers de-identification and the PT family covers PII processing and transparency. Worth knowing if your last research on this is a year old: under the Consolidated Rules for 2026, FedRAMP retired the "Authorized" label in favour of "FedRAMP Certified", and the 20x path has shifted the program toward continuous, machine-readable validation rather than static document review.
Classified workloads are a different tier entirely. Air-gapped infrastructure and cleared personnel, regardless of any certification status.
One correction worth making, because it gets repeated a lot. The EU AI Act (Regulation (EU) 2024/1689) does classify credit scoring as high-risk under Annex III, point 5(b), but it explicitly carves out systems used for detecting financial fraud. A standalone fraud detection model is not high-risk under that provision. A model that scores creditworthiness and flags fraud in the same pass still is, for the scoring half.
The requirement all three share
Strip the sector language away and you get one technical requirement: a decision-level record, produced at the inference boundary, that ties an input to an output to a model version to an identity to a timestamp.
Application logs do not satisfy this. They record that a request happened. They rarely record what the model saw after your RAG layer assembled the context, which endpoint served it, or what classification decision routed it there.
A record that survives an audit looks closer to this:
{
"event_id": "01J9Z2Q7K3XN4B8V",
"ts": "2026-09-01T09:14:22.481Z",
"actor": {"user_id": "u_8812", "role": "underwriter", "tenant": "eu-west"},
"classification": {"labels": ["pii", "financial"], "jurisdiction": "EU"},
"routing": {"policy": "eu-resident-only", "endpoint": "onprem-vllm-01", "reason": "pii_detected"},
"model": {"name": "llama-3.1-70b-instruct", "version": "2026-04-11", "params": {"temperature": 0.2}},
"input": {"raw_sha256": "9f2c...", "redacted": "Applicant [PERSON_1], DTI [NUMBER_1]..."},
"output": {"text": "...", "sha256": "4ab1..."},
"integrity": {"prev_hash": "7d10...", "sig": "ed25519:..."}
}
Three details do the heavy lifting. Hashing the raw input lets you prove integrity without storing regulated data twice. Recording the routing decision and its reason is what demonstrates a control was enforced rather than merely documented. Chaining each record to the previous hash is what makes the log tamper-evident instead of just append-only.
Pin the model version explicitly. "Latest" is not a version, and under the FDA's guidance on AI/ML-based Software as a Medical Device, a clinical model that adapts over time needs a Predetermined Change Control Plan describing what may change and how it gets validated first.
Where to put it
The enforcement point has to sit between your applications and every model endpoint, because that is the only place where you see all the traffic and can still act before data leaves the perimeter. Classification, redaction, routing and logging happen in one pass, and the routing decision is made before dispatch rather than reconstructed afterwards.
policies:
- match: {labels: ["phi"]}
route: onprem-clinical
on_missing_endpoint: block
- match: {labels: ["pii"], jurisdiction: "EU"}
route: eu-vpc-endpoint
redact: [PERSON, NATIONAL_ID, IBAN]
- default:
route: public-api
The on_missing_endpoint: block line matters more than it looks. A policy that silently falls back to a public endpoint when the compliant one is unavailable is not a control. NeuralTrust's writeup on how AI gateways enforce data sovereignty goes deeper on the routing and inspection mechanics, and the Agent Gateway product page covers how this is implemented in practice.
Agents raise the difficulty. A single user request can fan out into many tool calls and model invocations, each a potential egress point, and your audit trail needs to reconstruct the whole chain rather than the first hop. Agent Security maintains a useful public library of agent threat models and runtime controls if that is where your architecture is heading.
The short version
Three regulatory regimes. One place to enforce them. Build the inference-layer record first, make routing a hard gate rather than a default, and pin versions on everything. The sector-specific paperwork gets much easier once the underlying trail exists.
For the full regulatory breakdown by sector, the original piece is here: Sovereign AI for Regulated Industries.
Top comments (0)