By Micky Irons, founder of Mickai.
Every audit you have ever trusted rested on one quiet assumption: that the thing being audited would still be there afterwards. A bridge does not silently rebuild its own load tolerances overnight. A ledger does not rewrite last quarter while the auditor sleeps. So the discipline grew up around a sensible rhythm. Inspect, sign, file, return next year. Point-in-time assurance worked because the world it inspected held still between visits.
Machine learning broke that assumption, and almost nobody changed the rhythm. A model is not a bridge. It is a moving surface. It is retrained on fresh data, nudged by a fine-tune, swapped for a cheaper checkpoint, patched against a jailbreak, quietly re-weighted because last month's behaviour embarrassed someone. The system that earned the certificate in March is often not the system running in June. The certificate is real. The thing it describes has left the building.
The annual stamp certifies a ghost
Consider what an annual attestation actually measures. An assessor arrives, samples a frozen version of the model, runs an evaluation battery, reviews the documentation, and issues a finding tied to a date. That finding is honest on the day. It is also obsolete the instant anything changes, and in a live system something changes almost at once. Weights shift. The retrieval index is refreshed. The prompt scaffolding is edited by an engineer who was never in the room when the audit happened.
This is the heart of the continuous audit problem, and drift is not an edge case you caveat away in an appendix. It is the normal operating condition of any deployed model. The moment you accept that models move, you have to accept what a yearly sign-off really certifies: a snapshot of a system that no longer exists, presented as if it were the system in production. A stamp on a ghost.
Chronos turns the great gear, and every tooth leaves a sealed imprint on the scroll of record.
Drift is silent, and the silence is the danger
The unsettling thing about model drift is not that it happens. It is that it happens without announcing itself. A model can degrade on a protected class, develop a new failure mode under a particular phrasing, or start leaking from its retrieval store, and none of it trips a klaxon. The outputs still read fluently. The dashboards still glow green. The drift stays invisible precisely because the system was built to sound confident, not to be candid about its own changes.
So the gap between two annual visits is not a gap in paperwork. It is a gap in knowledge. For roughly three hundred and sixty four days a year, the people relying on the certificate have no cryptographic idea what the system actually did. They hold a document describing a past state and a hope that nothing material moved. Hope is not a control.
A model is not a building you inspect once and certify. It is a river. Auditing a river by photographing it in March tells you nothing about what flowed through it in November.
What moves between the visits
It is worth being concrete about what can change while the audit file sits closed in a drawer. None of this needs malice. Most of it is routine engineering, done by competent people for good reasons, and every item can invalidate the last attestation without anybody noticing.
- The base model is retrained or replaced with a newer checkpoint to cut cost or latency.
- A fine-tune is applied to fix one behaviour and silently shifts a dozen others.
- The retrieval corpus is refreshed, so the same question now draws on different source documents.
- A system prompt or guardrail is edited in production, days after the assessor signed off.
- An upstream dependency, an embedding model or a moderation filter, is upgraded by a vendor.
- Quantisation or distillation is applied to fit new hardware, changing outputs at the margins.
- A rollback restores an older, previously rejected version because the new one broke something else.
Each of these is a perfectly ordinary Tuesday. Together they mean the audited system has a half-life measured in weeks, while the audit cycle is measured in years. You cannot close that gap with a longer report. You can only close it by changing where assurance lives.
Move the evidence to the moment of execution
Here is the shift I am arguing for. Stop trying to certify the system as an object. Start certifying its behaviour as a stream. Assurance should not be something an assessor reconstructs months later from logs the operator could have edited. It should be produced at the moment of execution, by the system itself, as an unavoidable byproduct of acting.
That means every consequential action carries its own evidence. What the model was, what it was asked, what it decided, under whose authority, captured the instant it happened and sealed so it cannot be quietly rewritten. Audit stops being a periodic photograph and becomes a continuous cryptographic recording. The auditor's question changes from what was this system like last spring, to what has this system done since we last looked, and prove it.
Assurance produced at the moment of execution: each action stamped as it happens, not reconstructed afterwards.
Logs are not evidence, and tamper-evidence is the difference
The obvious objection is that systems already keep logs. They do, and logs are useful operationally, but a log is not evidence in any meaningful sense. A log is a file the operator controls. It can be edited, truncated, back-dated, or selectively produced. When the thing you are auditing also owns the record of what it did, that record is only as trustworthy as the party with the most to lose. Which is precisely backwards.
Evidence has to be tamper-evident by construction, not by policy. The record must be sealed at the point of creation with cryptography the operator cannot forge and cannot silently alter, and it must be anchored somewhere the operator does not unilaterally control. The difference between a log and an audit record is the difference between a diary and a notarised deposition. One is a claim. The other is a claim you can stake something on.
Why the cryptography has to outlast the model
There is a further trap. An audit trail that leans on cryptography we already expect to break is not assurance, it is a deferred liability. Records sealed today may need to be defensible in a decade, well inside the horizon where a sufficiently capable quantum computer threatens classical signatures. Continuous assurance only means something if the seals it produces are still meaningful when somebody finally comes to verify them.
How Mickai makes this an engineering fact
This is the part I will speak to directly, because it is what we built. Mickai is a Sovereign Intelligence Operating System, fifty specialised brains running on the operator's own hardware, fully offline-capable. Inside that system, every consequential action is sealed into an Open Audit Record the moment it occurs. The seal is a post-quantum signature under FIPS 204 ML-DSA-65, so the evidence is built to outlive the model that produced it and the assessor who first read it.
Those records do not sit in a log file the operator could quietly groom. They are anchored to Pantheon, our sovereign Layer 1, which itself settles to Bitcoin. The chain of custody runs from the individual action, through a post-quantum seal, to an anchor the operator does not unilaterally control. Retrain the model tomorrow and the new version starts sealing under its own identity, in the same unbroken stream. The drift does not hide. It is recorded, action by action, with a timestamp nobody can move.
Each Open Audit Record sealed under FIPS 204 ML-DSA-65 and anchored through Pantheon to Bitcoin, beyond the operator's reach.
Continuous assurance as a property, not a promise
Notice what this changes about the conversation. Continuous audit is usually pitched as a governance aspiration, a thing organisations promise to do better, a maturity level on a slide. Promises drift as easily as models do. The point of sealing at execution is that the assurance is no longer something a team has to remember to perform. It is a property of how the system runs. The evidence exists because the action happened. You cannot act without leaving the record, and you cannot alter the record after the fact.
That is the line I care about. The difference between aspiration and engineering is whether the property survives a bad day, a rushed deadline, a quiet retrain, a change of staff. A policy that says we will audit continuously survives none of those. A system that physically cannot take a consequential action without sealing a post-quantum record survives all of them, because the assurance is welded to the execution, not bolted onto the process.
What a regulator should actually ask for
If I were writing the rule, I would stop asking for the annual certificate and start asking for the stream. Show me the continuous record. Show me that every consequential action since the last review carries a seal I can verify independently. Show me that the seals are anchored beyond your reach and built to outlast the threat model. An annual attestation tells a regulator what a system was like on one convenient day. A continuous, anchored, post-quantum record tells them what it has actually done, every day, whether or not the model was retrained between visits.
The annual attestation is not merely weak. It is structurally dishonest about its own subject, because it presents a static photograph of something that was never static. We can keep performing that ritual and calling it assurance, or we can move the evidence to where the behaviour actually happens and seal it so it holds.
Sand falling upward into light: time runs forward, and every action it carries is already sealed.
Continuous, or nothing
So I will put it plainly. In a world where models move, point-in-time audit is not a lighter form of assurance. It is the absence of assurance wearing the clothes of the real thing. Either the audit is a continuous cryptographic stream, anchored at execution and sealed to outlast the model, or it is a stamp on a ghost. There is no honest middle. Continuous audit, or no audit at all.
Written by Micky Irons. Originally published at https://mickai.co.uk/articles/continuous-audit-or-no-audit-at-all. More from Micky Irons and Mickai at mickai.co.uk.
Top comments (0)