The demo went well. The model answered correctly, the stakeholders nodded, someone said "let's get this into production." That was six months ago. The PoC still runs, the roadmap still lists it, and it is no closer to shipping than it was on demo day.
If you build AI systems in a bank, an insurer, or any regulated company, you have probably watched this happen. The uncomfortable part: the PoC was never blocked by a bug. It was blocked by a line nobody drew.
The PoC was built on the wrong side of a line that did not exist yet
A typical proof of concept calls a hosted frontier model API. That is the right choice for a PoC: best capability, zero infrastructure, an afternoon to integrate. With synthetic or scrubbed data, nobody objects.
Production data is a different animal. The first review meeting asks questions the architecture cannot answer:
- Which data classes does this workload send out, and who approved that list?
- Where does inference happen? A query routed to a model hosted abroad has left the jurisdiction even when the database stayed in Frankfurt.
- Where is the audit evidence, and can the vendor's log retention satisfy our supervisor?
- Whose name is on this system when the auditor asks? Under Article 26 of the EU AI Act, operator duties stay with your company no matter what the vendor contract says.
None of these are model-quality questions. The model is fine. The architecture has no place to put the answers.
The retrofit trap
The instinct is to patch: add a redaction layer, move logs, amend the vendor contract, write a policy document. Each patch triggers the next review. Contracts get re-papered. Data flows get re-architected. Sign-offs get reopened. Every new use case renegotiates its own boundary from scratch, and your backlog queues behind one compliance team.
No project gets rejected in this loop. It just never clears. That is why the PoC is still "in review" six months later.
What the teams that ship do differently
The regulated teams I see reaching production all made the same move: they drew the compliance boundary before they committed to an architecture.
The pattern has three parts:
- Classify data, not use cases. A short list of data classes that may cross to an external service, and a list that never does. Customer PII and scoring features stay in; public and cleared data may go out.
- One gateway, one routing rule. Every model call goes through a single point that routes by data class. Regulated data goes to models running inside your jurisdiction on infrastructure you control; open-weight models cover most narrow workloads (classification, extraction, summarization) well. Everything cleared goes to the frontier API and gets the best capability per euro.
- Evidence inside the boundary. Audit logs, routing decisions, and the data-flow diagram live on your side of the line, signed by one named owner.
With the boundary in place, "can we ship this?" becomes a routing question instead of a six-month negotiation. The auditor gets one line to test instead of forty integrations to untangle.
What to do with the PoC you have
You do not have to throw it away. Three steps:
- Write the data-class list for the workload the PoC actually serves, and get compliance to sign it. This is days, not months, because it is one page.
- Put a gateway in front of the model call, even a thin one. Route by data class from day one.
- Move only the regulated portion inside the boundary. Keep the frontier API for everything cleared. Most PoCs split cleanly once the classification exists.
The PoC that reaches production is rarely the most impressive one. It is the one whose data flows survive the first review meeting.
I help teams in regulated industries take AI prototypes into governed production workflows. The compliance readiness checklist I run before any architecture review is at renezander.com/gdpr-checklist.
Top comments (0)