By Micky Irons, founder and CEO of Mickai.
An organisation should run AI on-premise, not on public cloud AI, when any of three conditions holds: its data is legally bound to a jurisdiction it does not control in the cloud, its decisions must be independently auditable, or the data cannot lawfully leave its perimeter at all. The reason is singular. When you send a prompt to a public cloud service, the prompt itself is a disclosure, and once it has left your perimeter you cannot prove where it went, who read it, or whether it was retained. On-premise removes the disclosure rather than mitigating it.
This matters more in 2026 than it did two years ago. Public cloud AI has become the default way most organisations reach a capable model, yet the legal ground beneath that default has shifted. DORA has been in force across EU financial entities since January 2025. NIS2 has widened the set of essential and important entities that must govern their supply chains. The EU AI Act high-risk Annex III obligations, once due on 2 August 2026, were deferred by the Digital Omnibus to 2 December 2027, with embedded Annex I high-risk moving to 2 August 2028. We read that deferral as a build window, not a reprieve. The organisations that use it well will be the ones that decided where their AI runs before they were told to.
What is the test that tells me the answer?
Apply one question to your own workload: if a regulator, an auditor, or a court asked you to prove that a given input was never disclosed to a third party, could you? If the honest answer is no because the input passed through a public cloud AI service, that workload belongs on-premise. This is a criterion you can self-assess without a consultant. It does not depend on your industry label. It depends on whether the act of running the model is itself a controlled disclosure.
Public cloud AI services are capable and well engineered. That is not the issue. The issue is architectural. Their value proposition requires your data to reach their infrastructure. For most organisations that is an acceptable trade. For the organisations below, the trade is unlawful, unprovable, or both.
Which rule makes on-premise necessary for finance?
DORA makes it necessary. It holds financial entities accountable for the operational resilience of their information and communication technology, including third-party dependencies, and it requires that critical functions remain governable under stress. A model you cannot inspect, hosted by a provider you cannot audit, sitting outside your jurisdiction, is a dependency you cannot fully govern. Add the US CLOUD Act, which can compel a US-headquartered provider to produce data held anywhere, and a European financial entity using US public cloud AI has a residency exposure it cannot close by contract alone. On-premise closes it structurally.
Which rule makes on-premise necessary for healthcare and government?
For healthcare it is GDPR read against the special category of health data, where lawful basis, purpose limitation and cross-border transfer are all in scope the moment a patient record enters a prompt. For government and defence it is classification and national sovereignty: certain material may not leave national infrastructure, full stop, and no data processing agreement changes that. In both cases the constraint is not a preference for privacy. It is a legal boundary on where the bytes may physically be, and public cloud AI cannot honour a boundary it is designed to cross.
The decision is not which model is best. It is whether the act of asking the model is itself a disclosure you are permitted to make.
What can an auditor actually check on-premise?
An auditor can check things that are structurally uncheckable in a shared cloud. On operator-owned hardware with the right architecture, four properties become verifiable. First, offline verifiability: the system runs without an outbound connection, so there is no channel on which data could leave. Second, a zero-egress inbound perimeter: requests come in, answers go out, and nothing else crosses. Third, hardware-attested identity bound to the audit chain, so every action is tied to a specific machine and operator, not a shared tenant. Fourth, a post-quantum signed audit ledger, so the record of what was decided cannot be altered after the fact and remains verifiable against future cryptographic standards. An auditor can inspect all four without trusting a vendor's word.
Does on-premise mean weaker models or a walled garden?
No. This is the assumption worth retiring. On-premise once meant a smaller, older model and a compliance tax on capability. That gap has closed. Sovereign models can run entirely on operator-owned hardware and, with a cross-model consensus design, multiple models can check one another's output inside the perimeter, which raises reliability rather than lowering it. Mickai is a Sovereign Intelligence Operating System, a SIOS, built for exactly this: it runs offline on the operator's own hardware, with every action cryptographically sealed to the audit chain. It is not a cloud service pointed inward. It is designed from the perimeter out. Its architecture is backed by 104 filed UK patent applications, approximately 2,340 claims, owned by Mickai LTD, none granted or patented.
How do we decide across a mixed estate?
Few organisations are all-in or all-out. The practical method is to sort workloads, not organisations. Run the test above on each workload. Marketing copy drafted from public information can sit on public cloud AI. A model reasoning over patient records, trade positions, classified material, legal privilege, or personal data of EU residents cannot. Standards give you the language for this: ISO/IEC 42001 for the AI management system, and FIPS 204 and 203 for the post-quantum signature and key-encapsulation schemes that keep an audit ledger verifiable for its full retention life. Sort by disclosure risk, then place each workload where its legal boundary allows.
Frequently asked questions
Is on-premise AI required by law, or just recommended?
No single law names on-premise as mandatory, but several make it the only compliant option for specific data. DORA, GDPR for health and personal data, NIS2 supply-chain duties, and national classification rules each constrain where regulated data may be processed. Where public cloud AI cannot satisfy that constraint, on-premise stops being a recommendation and becomes the only lawful route.
Can a data processing agreement with a cloud provider solve the residency problem?
Not fully. A contract binds the provider but does not override statute. The US CLOUD Act can compel a US-headquartered provider to disclose data regardless of where it is stored and regardless of your agreement. A contract cannot make data unreachable to a lawful order in the provider's home jurisdiction. On-premise removes the reach rather than papering over it.
Did the EU AI Act high-risk deadline pass on 2 August 2026?
No. The high-risk Annex III obligations that were due on 2 August 2026 were deferred by the Digital Omnibus to 2 December 2027, and embedded Annex I high-risk obligations move to 2 August 2028. Article 50 transparency duties are largely unchanged. We treat the extra time as a window to get architecture right, not as grounds to defer the decision.
Does running AI on-premise mean accepting a less capable model?
Not any longer. Sovereign models run entirely on operator-owned hardware, and a cross-model consensus design can have several models verify each other inside the perimeter. The result can be more reliable than a single cloud model, because disagreement is caught before an answer is trusted. The capability gap that once justified sending data to the cloud has narrowed to the point where it rarely settles the decision.
How do I start without moving every workload at once?
Sort workloads by disclosure risk rather than migrating wholesale. Keep low-sensitivity tasks on public cloud AI if that suits you, and move any workload that touches regulated, classified, privileged or personal data behind an on-premise perimeter first. Use ISO/IEC 42001 to frame the management system and let the disclosure test decide each case. The estate shifts workload by workload, not overnight.
Written by Micky Irons. Originally published at https://mickai.co.uk/articles/which-organisations-should-run-ai-on-premise-instead-of-public-cloud. More from Micky Irons and Mickai at mickai.co.uk.
Top comments (0)