Part 3 of "Trust the Machine" -> a series on building AI infrastructure that is secure, compliant, and governable by design.
Why data is the bottleneck, not the fuel
The performance of an AI system draws most of the attention: the model, the architecture, the benchmark scores. The trustworthiness of that system, however, is determined largely by something less visible: the data behind it. A model is a compressed representation of its training and retrieval data. Whatever that data contains, lacks, or was not permitted to include propagates directly into the model's behavior and into the organization's risk.
This has become a practical constraint on adoption, not merely a theoretical concern. Industry reporting in 2026 has indicated that a substantial majority of enterprises (on the order of 81%) have delayed, scaled back, or abandoned AI initiatives because of data-permission and governance problems. Data is no longer the fuel that accelerates AI; for many organizations it has become the bottleneck that stalls it.
This post examines why data governance sits at the root of AI trust, and how lineage and provenance provide a single control that satisfies security, compliance, and governance requirements at once.
Three ways data undermines trust
Data-related failures in AI systems fall into three categories, each with distinct consequences.
Unauthorized use
The organization trains on or retrieves from data it did not have the right to use: personal data without a lawful basis, licensed content beyond its terms, or information collected for an unrelated purpose. Once such data is absorbed into a model's weights, it cannot be surgically removed; remediation may require retraining. This is the failure mode behind much of the stalled-initiative statistic above.
Leakage
Sensitive information present in training or retrieval data resurfaces in outputs. A model may reproduce personal data, credentials, or confidential material either through ordinary generation or in response to deliberate extraction attempts. The exposure is a direct function of what entered the system.
Poisoning
An adversary who can influence training or retrieval data can shape model behavior, introducing backdoors, biases, or targeted failures. Because the data pipeline is often less scrutinized than application code, it can be an attractive and under-defended target.
Each of these failures originates in the data layer, which is why controls applied only at inference are insufficient. Trust must be established upstream.
Lineage and provenance, defined
Two related concepts underpin data trust.
Provenance is the origin and history of a piece of data: where it came from, how it was collected, under what terms, and how it has been transformed. Provenance answers the question, "are we permitted to use this, and for what?"
Lineage is the end-to-end trace of data as it flows through a system: from source, through processing and transformation, into training sets or retrieval indices, and ultimately into a model or an output. Lineage answers the question, "if this source is compromised or must be removed, what is affected?"
Together, provenance and lineage make the data layer legible. Without them, an organization cannot demonstrate what its models learned from, cannot scope the impact of a compromised source, and cannot respond to a deletion or consent-withdrawal request. With them, each of these becomes a query rather than an investigation.
Establishing provenance for training and retrieval data
Provenance must be captured at the point of ingestion, when context is still available; reconstructing it later is unreliable and often impossible.
- Record source and rights at ingestion. Every dataset should carry its origin, collection method, license or consent basis, and permitted purposes as structured metadata. This connects directly to the dataset fields of the AI-BOM introduced in Post 1.
- Distinguish data categories. Separate personal data, confidential material, and freely usable content, and track each accordingly. Categories determine which downstream controls apply.
- Preserve consent and purpose limitations. Where data is used under consent or for a specified purpose, that constraint must travel with the data so it can be honored throughout the pipeline.
- Extend provenance to retrieval sources. Retrieval-augmented systems draw on data at inference time; the sources feeding a retrieval index require the same provenance discipline as training data, because they influence output just as directly.
Handling personal and sensitive data
Minimizing sensitive data in the pipeline reduces both leakage risk and compliance burden.
- Minimize by default. Include personal and sensitive data only where it is genuinely necessary for the system's purpose. The most reliable protection against leakage is the absence of the data.
- Redact, mask, or synthesize. Where sensitive fields are not essential, remove or transform them before training or indexing. Synthetic or anonymized substitutes can preserve utility while reducing exposure.
- Filter at output as a secondary control. Output-side detection and redaction reduce residual leakage, but should complement upstream minimization rather than substitute for it.
- Support deletion and withdrawal. Lineage makes it possible to identify what a given individual's data touched, which is a precondition for honoring deletion and consent-withdrawal requests, an increasingly enforced regulatory expectation.
Defending the data pipeline against poisoning
The data pipeline warrants the same security rigor as production code.
- Validate and vet sources. Apply provenance checks and integrity validation to ingested data, with particular scrutiny for external or third-party sources.
- Detect anomalies. Monitor for statistical irregularities, unexpected distributions, and content that deviates from expected patterns, which can indicate tampering.
- Control write access. Restrict who and what can contribute to training sets and retrieval indices, and log all modifications. This is the least-privilege principle from Post 2 applied to the data layer.
- Version and checkpoint datasets. Immutable, versioned datasets allow a compromised state to be identified and rolled back, and support reproducibility.
Lineage as a control across three disciplines
The practices above may appear to serve compliance alone. In fact, a complete lineage and provenance capability functions simultaneously as a security, compliance, and governance control:
- As a security control, lineage enables impact analysis. When a source is found to be compromised or poisoned, lineage identifies precisely which datasets, models, and outputs are affected, scoping the response.
- As a compliance control, provenance provides the evidence of lawful data use that regulators increasingly require, and lineage supports deletion and consent-withdrawal obligations.
- As a governance control, the combination gives the organization a defensible account of what its models are built from, the foundation for responsible decisions about deployment and use.
This is the series thesis applied to the data layer: a single capability, built once, satisfies three demands. An organization that can trace its data end to end has simultaneously strengthened its security posture, its compliance position, and its governance maturity.
The data governance checklist
- Provenance is captured at ingestion: source, collection method, license or consent basis, and permitted purposes, as structured metadata.
- Data categories (personal, confidential, freely usable) are distinguished and tracked.
- Consent and purpose limitations travel with the data throughout the pipeline.
- Retrieval sources are held to the same provenance standard as training data.
- Sensitive data is minimized by default, with redaction, masking, or synthesis where it is not essential.
- End-to-end lineage links sources to datasets, models, and outputs.
- The pipeline defends against poisoning through source vetting, anomaly detection, access control, and dataset versioning.
- Lineage supports deletion and consent-withdrawal requests.
What's next
Post 4: From Policy to Pipeline: Making Compliance an Engineering Property. The final post brings the series together, mapping the requirements of the EU AI Act, the NIST AI Risk Management Framework, and ISO/IEC 42001 onto the concrete controls established in Posts 1 through 3, and showing how compliance can become a property of the pipeline rather than a document maintained beside it.
The takeaway
Data is the true source of a model's trustworthiness, and lineage and provenance are the controls that make the data layer accountable. An organization that cannot describe what its models learned from cannot claim to secure, govern, or certify them.
Top comments (6)
Lineage is where AI governance becomes practical. If you cannot trace where the data came from and how it changed, model behavior is almost impossible to explain later. Provenance is the audit trail behind trust.
Spot on. Provenance explains why you can trust it, lineage tells you what's affected when you can't. Together they become both the governance mechanism and the compliance evidence, which is exactly where the next post (4) picks up.
Exactly. Provenance gives the trust story, but lineage gives the blast-radius story. For AI systems that distinction matters because the hard question is usually not only 'can we trust this output?' but 'what else changes if this source is wrong?'
Lineage is where the governance story stops being a slide and starts being debuggable. I’d add one operational rule: every model-facing dataset should carry a change receipt, not just a catalog entry. Source, transform, approval, expiry, and downstream consumers all need to be queryable when something goes wrong.
How are you thinking about retrieval data here? It drifts faster than training data, but it can create the same audit problem.
Lineage and provenance are the right control for two of your three failure modes and weaker than claimed for the third, and the split is worth drawing because it changes what you actually have to build. For unauthorized use and leakage the failures are negligent, nobody is trying to defeat the record, so a faithful trace is a real control: it catches the thing because the thing was not hiding. Poisoning is different. The actor who can poison the data can usually author its provenance in the same motion, because both are written by the ingestion pipeline they already compromised. Poisoned data carrying a clean lineage record is the expected output of a successful poisoning, not an edge case.
So a provenance record written by the pipeline that handled the data is a self-report exactly where it matters most. The fix is the same shape as any honest-artifact problem: the trace has to be authored somewhere the poisoner cannot reach. Content-address the source at ingestion, hash it, sign the record with a key the pipeline does not hold, freeze it at a boundary upstream of the transformation steps. Then a mismatch between the signed origin and what the model actually trained on is detectable, and lineage stops being a story the pipeline tells about itself. Without that, lineage is a strong compliance and accountability instrument, which is real value, but it is not yet a security control against the one failure mode you named that has an adversary, because the adversary is holding the pen on the evidence.
The lineage problem compounds in agentic pipelines because the agent can retrieve, transform, and re-store data in a single run, creating derived artifacts whose provenance depends on tracing the full chain of retrieval steps rather than just the original source. In knowledge graph construction over enterprise data, we found that tracking which document contributed which triple (and with what permission context) was the hardest engineering problem in the whole system — harder than the entity extraction itself. The 81% figure is striking but also points to something the governance conversation often misses: the bottleneck is usually not policy definition but policy operationalization, the gap between what the data governance team says is permitted and what the retrieval pipeline actually enforces at query time. Without a permission context that travels with the retrieved chunk all the way to the model context window, governance becomes a pre-flight checklist rather than a running control.