Part 4 of "Trust the Machine" —> a series on building AI infrastructure that is secure, compliant, and governable by design.
The thread that ties the series together
The preceding posts addressed three engineering problems: seeing the AI systems in an environment, containing autonomous agents, and governing the data beneath them. This final post addresses the discipline that ties them together and, increasingly, compels them: regulatory compliance.
Three frameworks now dominate the conversation: the European Union's AI Act, the U.S. National Institute of Standards and Technology's AI Risk Management Framework (NIST AI RMF), and the international standard ISO/IEC 42001. They differ in force and detail, but they converge on a single expectation: AI systems must be documented, tested, and auditable. The organizations that struggle with this expectation are those that treat compliance as a documentation exercise performed after the fact. The organizations that meet it are those that make compliance a property of the pipeline, an outcome of how systems are built and operated, rather than a description assembled afterward.
The three frameworks, in brief
The frameworks are complementary rather than competing, and an organization will typically engage with more than one.
The EU AI Act
Binding law with extraterritorial reach. It applies to organizations that place AI systems on the EU market or whose systems affect people in the EU, regardless of where the organization is based. It classifies systems by risk (prohibited, high-risk, limited-risk, and minimal-risk) and imposes obligations proportional to that classification. High-risk systems carry the most substantial requirements: risk management, data governance, technical documentation, logging, human oversight, transparency, and accuracy and robustness. These obligations are phasing in through 2026 and 2027, which makes present preparation a practical necessity rather than a future concern.
The NIST AI RMF
A voluntary framework rather than a regulation. It organizes AI risk management into four functions (Govern, Map, Measure, and Manage) and provides a structured, widely adopted vocabulary for identifying and mitigating AI risk. It is frequently used as the operational backbone that gives structure to compliance efforts under other regimes.
ISO/IEC 42001
A certifiable international standard for an AI management system, analogous to ISO 27001 for information security. It specifies how an organization should govern AI across its lifecycle and provides an auditable basis for demonstrating that governance to external parties.
How they fit together
The practical relationship among them is straightforward: ISO/IEC 42001 and the NIST AI RMF supply the management system and risk vocabulary that help an organization satisfy the specific legal obligations of the EU AI Act. Implementing the former substantially advances readiness for the latter.
The shift: from documentation to engineering property
The common failure mode is to read a framework, produce a set of policy documents, and file them. This approach is expensive to maintain, quickly diverges from reality, and provides little genuine assurance. The alternative is to recognize that most framework obligations describe outcomes that the controls from Posts 1 through 3 already produce. Compliance then becomes a matter of connecting existing engineering controls to the requirements they satisfy, and generating evidence as a byproduct of normal operation.
The following mapping illustrates the point.
| Framework obligation | Engineering control (from this series) |
|---|---|
| System inventory and record-keeping | The continuous AI inventory and AI-BOM (Post 1) |
| Data governance and lawful data use | Provenance and lineage (Post 3) |
| Human oversight of AI systems | Human-in-the-loop approval for consequential actions (Post 2) |
| Risk management proportional to impact | Risk-tiering of systems (Post 1); autonomy bounds (Post 2) |
| Logging and traceability | Per-action audit logging (Post 2); lineage (Post 3) |
| Accuracy, robustness, and testing | Continuous evaluation as a pipeline gate (below) |
| Transparency and technical documentation | The AI-BOM plus evaluation results (below) |
Read this way, compliance is not additional work layered on top of good engineering. It is largely the documentation and evidence that good engineering already generates.
Evaluations as compliance gates
Frameworks require that AI systems be tested for accuracy, robustness, and (for higher-risk systems) safety and security properties. In conventional software, the equivalent requirement is met by tests running in continuous integration. The same mechanism applies to AI, with evaluations in the role of tests.
- Run evaluations in the pipeline. Accuracy, safety, robustness, and security evaluations should execute automatically on model and prompt changes, and should block deployment when results regress below defined thresholds. This converts a compliance requirement into an enforced engineering gate.
- Include adversarial and red-team evaluations. For systems exposed to untrusted input or capable of consequential action, evaluations should include tests for prompt injection, jailbreaks, and misuse, the failure modes described in Post 2.
- Retain results as evidence. Every evaluation run is a dated, versioned record that a system was tested and met its thresholds. Retained systematically, these records constitute much of the evidence a framework requires.
Treating evaluation as a deployment gate satisfies the testing obligation and simultaneously improves the system, aligning the compliance incentive with the engineering one.
Audit logging and evidence
Auditability is a recurring requirement across all three frameworks, and it is served directly by controls already established. The per-action logging that contains agents (Post 2) and the lineage that governs data (Post 3) together produce the audit trail that compliance demands. The essential practices are to ensure logs are comprehensive enough to reconstruct consequential decisions, tamper-resistant, and retained for the required period.
The AI-BOM, introduced in Post 1, becomes the central evidence artifact. Combined with evaluation results and audit logs, it constitutes the technical documentation the frameworks call for, assembled continuously and automatically rather than reconstructed under deadline.
Continuous compliance
AI systems drift. Models are updated, prompts are revised, data sources change, and capabilities expand. A point-in-time certification captures a system as it was, not as it is. Compliance must therefore be continuous rather than periodic.
- Trigger re-evaluation on change. A model update, a prompt revision, or a new capability should automatically re-run the relevant evaluations and refresh the associated evidence. This is an application of the drift monitoring introduced in Post 1.
- Keep the inventory and AI-BOMs current. Because the inventory is the foundation of the compliance picture, its accuracy directly determines the accuracy of every downstream claim.
- Re-assess risk tier on material change. A change that alters a system's data sensitivity, autonomy, or decision impact may alter its regulatory classification and the obligations that follow.
The AI trust control plane
Across four posts, a single structure has emerged. Visibility, agent containment, data governance, and compliance are not four separate initiatives but four layers of one capability, an AI trust control plane:
- See it — a continuous inventory and AI-BOM establish what exists (Post 1).
- Secure it — identity, least privilege, and brokered, bounded, logged actions contain what acts (Post 2).
- Govern the data — provenance and lineage account for what systems learn from (Post 3).
- Prove it — evaluations, logging, and evidence demonstrate that the whole meets its obligations (Post 4).
Each layer serves security, compliance, and governance at once, because these were never three separate problems. They are three views of one capability: knowing and controlling what AI systems do with data and actions. An organization that builds that capability into its infrastructure does not have to choose between moving quickly and meeting its obligations. The infrastructure meets them by construction.
The compliance checklist
- The applicable frameworks (EU AI Act, NIST AI RMF, ISO/IEC 42001) and their obligations are identified for each system.
- Each obligation is mapped to a concrete engineering control, not a standalone policy document.
- Evaluations run as pipeline gates, blocking deployment on regression, and include adversarial tests for exposed systems.
- Audit logs are comprehensive, tamper-resistant, and retained for the required period.
- The AI-BOM, evaluation results, and logs together form the technical-documentation evidence base.
- Compliance is continuous: changes trigger re-evaluation, evidence refresh, and risk-tier reassessment.
- Evidence is generated automatically as a byproduct of operation, not reconstructed on demand.
Closing the series
"Trust the Machine" began with a simple assertion: security, compliance, and governance for AI are not three backlogs but three views of a single capability, knowing and controlling what AI systems do with data and actions. Each post built one layer of that capability, and each layer paid off in all three disciplines at once.
The organizations that will operate AI confidently in the years ahead are not those with the strictest policies or the largest compliance teams. They are those that have built trust into the infrastructure itself, so that seeing, securing, governing, and proving are not tasks performed alongside the work, but properties of how the work is done.
Top comments (0)