Himanshu Agarwal

Posted on Jul 15

Regulation as a Test Oracle

#ai #testing

Engineering verifiable evidence for the EU AI Act and GDPR when the model is the defendant

By Himanshu Agarwal
Last verified: 15 July 2026

Before you read on — the two resources this article is built from

AI Compliance Testing: GDPR, EU AI Act & Beyond — the practitioner's decoder that turns terrifying legal text into a clean checklist of things you can actually test and prove. Risk-tier mapping worksheet, GDPR-to-test-case translation guide, audit-ready evidence matrix. 34 pages, zero legalese.
→ https://himanshuai.gumroad.com/l/AI-Compliance-Testing — currently 50% off (auto-applied purchasing power parity for India: ₹1,636.81)

Complete AI Testing & GenAI Engineering Master Bundle (18 Books) — the full stack: LLM evaluation, RAG testing, agent reliability, red-teaming, MLOps quality gates, and compliance.
→ https://himanshuai.gumroad.com/l/GenAI-Testing-Master-Bundle-18-Books
→ Use code SPECIAL70 for flat 70% off

Questions about the bundle, the learning roadmap, or whether it fits your experience level? DM me before you buy. I'd rather point you to the right path than sell you the wrong book.

0. Read this before you touch your 2026 roadmap

If your AI Act compliance plan says "high-risk obligations bite on 2 August 2026," your plan is out of date and you are about to burn engineering quarters against a deadline that no longer exists.

Here is the actual state of play as of today.

The Digital Omnibus on AI — the amending regulation the Commission tabled on 19 November 2025 — survived a near-collapse in trilogue on 28 April, reached political agreement on 6–7 May, was endorsed by the European Parliament on 16 June, and received final Council green light on 29 June 2026. It enters into force on the third day after publication in the Official Journal. The operative timeline it leaves behind:

Date	What applies
2 Feb 2025	Article 5 prohibitions; Article 4 AI literacy. Live.
2 Aug 2025	GPAI model obligations (Art. 51–55), governance architecture, penalties. Live.
2 Aug 2026	Article 50 transparency obligations — provider disclosure, deployer disclosure, deepfake labelling. Except Art. 50(2) machine-readable marking for systems already on the market. Live in 18 days.
2 Dec 2026	Art. 50(2) for legacy systems. New Art. 5 prohibition: AI systems generating NCII / CSAM ("nudifiers"), with a safe-harbour for demonstrably effective preventive safeguards.
2 Aug 2027	Member States must have at least one national regulatory sandbox. Commission delegated acts for Annex I sectoral interplay.
2 Dec 2027	Chapter III obligations for stand-alone Annex III high-risk systems. Deferred 16 months.
2 Aug 2028	Chapter III obligations for Annex I embedded high-risk systems. Deferred 12 months.

Three consequences most teams have not internalised:

One. The deferral is a deferral, not a repeal. The risk-based architecture, the Chapter III requirements, the conformity assessment machinery, the Article 99 penalty schedule — €35M or 7% of global turnover for prohibited practices, €15M or 3% for Chapter III breaches — all survive untouched. You did not get a pardon. You got 16 months.

Two. The deferral happened because the standards weren't ready. CEN-CENELEC JTC 21 did not deliver harmonised standards in time; national competent authorities were not designated; the conformity infrastructure did not materialise. That means the technical content of "compliance" is still being written while you build against it. If you wait for the standard to freeze before you start testing, you will have roughly eight weeks of runway in late 2027 instead of eighteen months now. The hard part of AI Act compliance was never the documentation template. It is finding every AI system in your estate, classifying each one, and getting product and engineering to keep the inventory alive as new systems ship weekly. None of that depends on a standard being final.

Three — and this is the one that gets missed — the GDPR half of the Digital Omnibus did not pass. The general Digital Omnibus (the one amending GDPR, ePrivacy, NIS2, the Data Act) is on a separate, slower track. The Council's compromise text of 20 February 2026 stripped out the proposed Article 4(1) redefinition of personal data entirely, along with the Article 22 restructuring and the scientific-research expansion. The EDPB and EDPS filed Joint Opinion 2/2026 on 11 February 2026 opposing the core of it — the EDPS chief publicly urged co-legislators not to touch the personal data definition. Parliament had not formally opened negotiations on the GDPR portion as of late spring. Realistic adoption estimates have slipped to mid-2027.

So: GDPR applies to your AI systems today, exactly as written in 2016, with no AI carve-outs. The proposed Article 88c legitimate-interest basis for AI training is not law. The pseudonymisation escape hatch is not law. Anyone building a 2026 architecture on the assumption that pseudonymised training data falls out of GDPR scope is building on a proposal that the EU's own data protection authorities are actively trying to kill.

The AI Act gave you time. The GDPR did not. That asymmetry should drive your entire sequencing decision, and it is the reason this article spends as much oxygen on Article 17 as on Article 15.

1. The oracle problem

Every test needs an oracle: a mechanism that decides whether observed behaviour is correct. In functional testing the oracle is trivial — you asserted expected == actual and the compiler helped you. In ML testing the oracle degrades into a statistical claim about a distribution. In compliance testing the oracle is a legal text written by people who have never seen your architecture, in a language deliberately drafted to be technology-neutral, which is a polite way of saying "unimplementable as written."

Consider Article 15(1). High-risk systems shall be designed to achieve "an appropriate level of accuracy, robustness and cybersecurity, and perform consistently in those respects throughout their lifecycle." That is not a test. There is no number in it. There is no threshold, no dataset, no metric. A test engineer reads this and correctly concludes that it cannot be automated.

The trap is to conclude from this that compliance is therefore a legal exercise and not an engineering one. That conclusion is wrong, and it is the single most expensive mistake in this domain. Here is why.

The regulation is not the oracle. The regulation is a specification for building an oracle. Article 15 does not tell you the threshold; it tells you that you must have a threshold, that the threshold must be appropriate to the intended purpose, that you must declare it in the instructions for use per Article 13(3)(b)(ii), and that you must demonstrate consistency against it across the lifecycle. The regulator has deliberately delegated oracle construction to you — and then reserved the right to audit your construction.

This reframing changes everything about how you approach the work:

You are not being asked "is your model accurate?" You are being asked "can you show me the accuracy target you set, the reasoning that justifies it against your intended purpose, the test that measures it, the evidence that you ran that test on this exact artefact, and the monitoring that proves it hasn't drifted since?"
The auditor is not checking your model. The auditor is checking your test suite. The model is downstream evidence.
Therefore compliance failures are almost never model quality failures. They are test infrastructure failures, lineage failures, and evidence chain failures.

This is why compliance is a QA problem, not a legal one. Lawyers can tell you what property must hold. Only a test engineer can tell you whether it holds, how you'd know if it stopped holding, and whether you can prove either statement to a hostile third party eighteen months from now.

There is a second structural property worth naming early. Nearly every obligation in Chapter III and in the GDPR is a property-level, not example-level, requirement. "The model must not discriminate" is a universally quantified statement over an unbounded input space. You cannot prove it by enumeration. You can only:

Bound it statistically (sample from a defined distribution, report a confidence interval),
Bound it adversarially (search for a counterexample with a defined budget, report the search failed),
Bound it structurally (prove by construction that the property cannot be violated — the only genuinely strong option, and almost never available).

Every compliance test you write will fall into one of those three buckets. Knowing which bucket you're in — and saying so in your evidence — is the difference between an evidence package that survives scrutiny and one that reads as overclaiming. An auditor who catches you presenting a statistical bound as a structural guarantee will discount your entire package. Calibrated claims are load-bearing.

2. Classification is a test artefact, not a legal memo

Everything downstream is gated on classification. Get it wrong and you either build €2M of controls you didn't need, or you ship an unclassified high-risk system and discover it during an enforcement action.

Most organisations treat classification as a one-time legal exercise: a lawyer reads Annex III, writes a memo, the memo goes in SharePoint, everyone moves on. This fails within a quarter, for a boring reason: the classification is a function of the system's intended purpose, and the intended purpose changes every time product ships a feature.

Your résumé-parsing tool extracts skills. Not high-risk — arguably Article 6(3)(a), narrow procedural task. Then a PM adds a "candidate strength score" to the hiring dashboard. You are now squarely in Annex III point 4(a): AI intended to be used for recruitment or selection, in particular to filter applications or evaluate candidates. Nobody filed a ticket called "reclassify system as high-risk." The reclassification happened in a Figma file.

The only stable answer is to make classification executable and re-evaluated on every change.

2.1 The decision procedure

Encode this as a real decision tree in code, not prose:

Q1. Is it an "AI system" per Art. 3(1)?
    Machine-based, designed to operate with varying levels of autonomy,
    may exhibit adaptiveness, infers from input how to generate outputs.
    → The Feb 2025 Commission guidelines matter here. A pure
      deterministic rules engine with no inference is out. A logistic
      regression fitted on data is in. Do not let anyone tell you
      "it's just regression, not AI."

Q2. Is it prohibited under Art. 5?
    Subliminal/manipulative techniques causing significant harm;
    exploitation of vulnerabilities; social scoring; predictive policing
    on personality traits alone; untargeted facial scraping for FR
    databases; emotion inference in workplace/education (with narrow
    safety/medical carve-outs); biometric categorisation inferring
    protected attributes; real-time remote biometric ID in public for
    law enforcement.
    → NEW, from 2 Dec 2026: generation/manipulation of NCII or CSAM.
    → If YES: stop shipping. No control fixes a prohibition.

Q3. Is it a safety component of, or itself, a product under Annex I?
    → Chapter III via Art. 6(1). Deadline 2 Aug 2028.
    → POST-OMNIBUS: "safety component" is narrowed. AI used solely for
      user assistance, performance optimisation, efficiency, automation,
      convenience, or quality control is NOT a safety component unless
      failure could endanger health or safety. Machinery Regulation moved
      from Annex I Section A to Section B — the full Chapter III suite no
      longer applies directly; the Commission legislates AI safety
      requirements into the Machinery Regulation via delegated acts by
      2 Aug 2028. If you build industrial AI, re-run your classification.
      Several systems that were high-risk in your Q4 2025 memo are not now.

Q4. Does it fall in an Annex III use case?
    1. Biometrics (remote ID, categorisation, emotion recognition)
    2. Critical infrastructure safety components
    3. Education and vocational training (admission, evaluation,
       proctoring)
    4. Employment (recruitment, filtering, evaluation, promotion,
       termination, task allocation, monitoring)
    5. Access to essential services (credit scoring, insurance risk
       pricing for life/health, emergency triage, public benefits)
    6. Law enforcement
    7. Migration, asylum, border control
    8. Administration of justice and democratic processes
    → If YES → Q5.

Q5. Does an Art. 6(3) derogation apply?
    (a) narrow procedural task;
    (b) improve the result of a previously completed human activity;
    (c) detect decision patterns/deviations from prior patterns, NOT
        replacing or influencing the previously completed human
        assessment without proper human review;
    (d) preparatory task to an Annex III assessment.
    HARD BLOCK: if the system performs PROFILING of natural persons,
    it is high-risk regardless. Art. 6(3) final subparagraph.
    → If you claim a derogation, you MUST document the assessment
      (Art. 6(4)) AND — post-Omnibus, reversing earlier drafts — you
      STILL register in the EU database, albeit with reduced
      information. The carve-out from registration did not survive.

Q6. Art. 50 transparency triggers (orthogonal to risk tier — a
    minimal-risk chatbot still owes these):
    - Direct interaction with humans → disclose it's AI (unless obvious)
    - Synthetic audio/image/video/text output → machine-readable marking
      (Art. 50(2))
    - Emotion recognition / biometric categorisation → inform the person
    - Deepfakes → disclose artificial generation
    - AI-generated text published to inform the public on matters of
      public interest → disclose

2.2 Making it hold

Wire the classifier into your SDLC so it cannot rot:

Classification lives in the service manifest, versioned in the repo next to the code, not in a document store. ai-classification.yaml with intended_purpose, annex_iii_category, art_6_3_derogation, derogation_rationale, art_50_triggers, gpai_dependency, last_reviewed, reviewer.
A CI check fails the build if intended_purpose changes without a re-classification signature. This is a diff check. Twenty lines of Python.
A drift detector on the model card: if the declared output schema gains a field named like a score, a rank, a probability, or a recommendation, flag for human review.
Classification is a test. It has inputs (the manifest), an expected output (the tier), and a regression suite: a corpus of previously-classified systems with known-correct tiers that you re-run whenever you change the decision tree. When Commission guidance lands and you update the tree, you re-run the corpus and see which of your live systems just changed tier. That is a regression report, and it is the single highest-leverage artefact in the whole programme.

I have never seen an organisation do the last bullet. Everyone should.

3. Chapter III, decomposed into things you can actually run

Articles 9 through 15 are the substantive core. Let's take them one at a time and convert each into test cases, because that translation is where nearly all the value is and where nearly all the published guidance stops.

3.1 Article 9 — Risk management as a control loop, not a document

Article 9 demands a "continuous iterative process planned and run throughout the entire lifecycle, requiring regular systematic review and updating." Read that as a spec for a control loop: identify → estimate → evaluate residual → adopt measures → test → monitor → feed back.

The word doing the work is continuous. A risk register updated annually by a governance function is non-compliant on its face, and an auditor will spot it in ninety seconds by looking at the modification timestamps.

What makes this testable:

Every identified risk must map to at least one executing test. Enforce with a CI check: parse the risk register, parse the test suite's tags, fail if any risk_id has no test_id, or if any linked test hasn't executed in N days. Orphaned risks are the most common finding in every internal audit I've been part of.
Article 9(5) requires that residual risk be judged acceptable. "Acceptable" is your oracle to define. Define it numerically, per risk, in advance, and version the definition. residual_risk: {metric: fpr_on_protected_subgroup, threshold: 0.03, justification_ref: DPIA-2026-014}.
Article 9(6) requires testing against "prior defined metrics and probabilistic thresholds appropriate to the intended purpose." This is the Act explicitly telling you to write the oracle first. It is, quite literally, a legal mandate for test-driven development. Use that sentence in your next planning meeting.
Article 9(8) — testing in real-world conditions where appropriate, per Article 60. If you do this, the sandbox/real-world testing plan is itself an audited artefact.

The output artefact is a risk-to-test traceability matrix. Not a PDF. A generated view over live data:

risk_id	description	severity×likelihood	mitigation	test_ids	last_run	status	residual	accepted_by
R-014	Under-prediction of default risk for thin-file applicants	H×M	Reweighting + reject-inference	T-221, T-222, T-401	2026-07-14	PASS	0.021	J.Doe, 2026-06-02

Generate it from your CI database at audit time. If you can generate it on demand, you have a risk management system. If you maintain it by hand, you have a spreadsheet with a compliance-shaped name.

3.2 Article 10 — Data governance, or: testing the dataset as a first-class artefact

Article 10 is where ML engineering and legal obligation collide hardest, and where the most under-appreciated work sits.

Article 10(3): training, validation and testing datasets shall be "relevant, sufficiently representative, and to the best extent possible, free of errors and complete in view of the intended purpose."

Notice the escape hatch — to the best extent possible. That is not a licence to skip the work; it is a licence to document why perfection was unattainable. The evidence you owe is the search, not the result.

Concretely testable properties, and how:

Representativeness (10(3), 10(4)) — 10(4) requires consideration of characteristics particular to the geographical, contextual, behavioural or functional setting.

Compute population-vs-sample divergence per attribute: Population Stability Index, KL divergence, Jensen-Shannon, Wasserstein-1 for continuous features. Set thresholds in advance. PSI > 0.25 on any covariate against your declared deployment population is a finding; PSI 0.1–0.25 is a warning.
The hard part isn't computing the divergence. It's that you need a reference distribution for the deployment population — and most teams have never defined one. Defining it forces you to state your intended purpose precisely, which is exactly why the Act wants you to do it.
Multivariate, not just marginal. A dataset can match every marginal distribution and still contain zero examples of the intersection that matters. Test the joint on the cells you care about.

Error rates and completeness (10(3))

Label noise estimation: confident learning (Northcutt et al.'s cleanlab approach) gives you an estimated noise rate per class without ground truth. Report it. A model trained on labels with 12% estimated noise in the minority class is a documented risk, not a hidden one.
Missingness mechanism: MCAR vs MAR vs MNAR is not academic pedantry here. If missingness correlates with a protected attribute (MNAR), your imputation strategy is a discrimination vector and Article 10(2)(f) requires you to have examined it.
Duplicate and near-duplicate detection across the train/test boundary. MinHash-LSH at the document level, embedding-space cosine at the semantic level. Leakage inflates your reported accuracy, and your reported accuracy is what you declare under Article 13(3)(b)(ii). Overstated accuracy in the instructions for use is a false declaration, which is an Article 99 problem, not a modelling problem.

Bias examination (10(2)(f) and 10(2)(g)) — note the exact verbs: 10(2)(f) requires examination in view of possible biases likely to affect health, safety, fundamental rights, or lead to prohibited discrimination; 10(2)(g) requires appropriate measures to detect, prevent and mitigate those biases. Examination is mandatory. Elimination is not — mitigation is. Document the examination even where you concluded no action was needed. The examination is the deliverable.

The special-category paradox — and what the Omnibus did to it. To test whether your model discriminates by ethnicity, you need ethnicity labels. To hold ethnicity labels, you need an Article 9 GDPR condition. Article 10(5) of the AI Act provides a derogation for bias detection and correction, subject to conditions: strict necessity, technical limitations on reuse, state-of-the-art security, pseudonymisation, no transmission to third parties, deletion once the bias is corrected or the retention period expires.

The Omnibus broadened this — the derogation now extends beyond high-risk systems to AI systems and models generally — but Parliament and Council explicitly returned it to a strict necessity test, reversing the Commission's more permissive draft. You must be able to demonstrate no less intrusive means exist. The GDPR side (proposed new Art. 9(5), residual processing of special category data in AI contexts) is not law and may never be.

Practical consequence: build the derogation as an engineered, auditable pipeline, not a policy exception.

Proxy-first. Attempt the examination with proxies (Bayesian Improved Surname Geocoding and its variants; geographic inference) and document the attempt and its measured inadequacy before you collect the real attribute. Your "less intrusive means" argument is only as good as your evidence that the less intrusive means failed. Measure the proxy's error rate against a held-out labelled sample and report it.
Physically isolate. The bias-testing dataset lives in a separate store, separate keys, separate IAM boundary, no network path to the training environment. The engineering control is what makes the legal argument credible.
Access is logged, purpose-bound, and time-bound. Automatic expiry with a documented retention basis.
Write the deletion test. Not the deletion policy — the test that proves the deletion executed, run on a schedule, emitting evidence.

3.3 Article 11 + Annex IV — Technical documentation is a build artefact

Article 11 says technical documentation shall be drawn up before the system is placed on the market and kept up to date. Annex IV enumerates the contents across nine sections, and it is longer and more specific than most people realise: system description and intended purpose, development process and design choices, system architecture, computational resources used, data requirements and provenance, human oversight measures, validation and testing procedures with metrics and results including on accuracy and discriminatory impact, cybersecurity measures, the risk management system, lifecycle change log, list of harmonised standards applied, EU declaration of conformity, post-market monitoring plan.

Two failure modes, both fatal, both preventable:

Failure mode 1: the documentation is written by a human at the end. It will be wrong the moment it's finished, because the artefact it describes has already changed. And Annex IV requires "computational resources used to develop, train, test and validate" — nobody reconstructs that six months later from memory.

Failure mode 2: the documentation is not bound to a specific artefact. An auditor asks: does this document describe this deployed model? If your answer involves the word "should," you have failed.

The fix is mechanical and cheap. Generate Annex IV from your pipeline, at build time, and bind it cryptographically.

# annexIV.lock — emitted by the training job, not written by a human
system_id: credit-scoring-v4
annex_iii_ref: "5(b) — creditworthiness evaluation"
model_artifact_sha256: 9f2c...a13
training_dataset_sha256: 3ab8...ee1     # Merkle root over shards
eval_dataset_sha256: 71cd...004
code_commit: 4a9e2f1
container_digest: sha256:cc41...9de
base_model:
  name: "vendor/foundation-7b"
  version: "2026.04"
  provider_art_53_doc_ref: "VD-2026-118"   # what the GPAI provider gave you
compute:
  accelerator: "8x H100-80GB"
  wall_clock_hours: 41.2
  estimated_energy_kwh: 3180
  region: "eu-central-1"
hyperparameters: {...}
metrics:
  auc_roc: 0.871
  ece: 0.019
  fpr_by_declared_group: {...}
  robustness_suite_pass_rate: 0.97
declared_accuracy_art_13_3_b_ii:
  metric: "AUC-ROC on holdout H-2026-Q2"
  value: 0.871
  ci_95: [0.863, 0.879]
  n: 48211
risk_register_snapshot: R-2026-07-14T09:12Z
harmonised_standards_applied: []          # honest: none final yet
signature: <cosign signature over the above>

Sign it. Store it immutably. Now "does this document describe this model?" is answerable with a hash comparison instead of a meeting. And the honest empty harmonised_standards_applied list is itself good evidence — it shows you know the standards aren't final and you're not pretending otherwise.

The gap you must staff: provider_art_53_doc_ref. If you build on a third-party foundation model, Article 53(1)(b) obliges that provider to give downstream integrators the information they need to comply. Getting it is a procurement problem, and procurement will not solve it unless you write the clause. Add to every model vendor contract: Annex XII-equivalent documentation, training data content summary per the Article 53(1)(d) template, copyright policy attestation per 53(1)(c), and — the one everyone forgets — notification of model updates with a defined lead time. A silent weight update on a hosted endpoint invalidates every downstream evidence artefact you have generated. If your vendor can swap the model under you without telling you, your Annex IV lock file is a lie the moment they do it.

3.4 Article 12 + Article 19 — Logging as testable traceability

Article 12: automatic recording of events over the lifetime, enabling identification of situations that may result in the system presenting a risk under Article 79(1) or a substantial modification, facilitating post-market monitoring per Article 72. Article 12(3) sets minimum content: period of use with start/end timestamps, the reference database against which input data has been checked, the input data for which the search led to a match, identification of the natural persons involved in verifying results per Article 14(5). Article 19: providers keep logs for at least six months, subject to Union or national law.

The obligation is trivial. The engineering is not, because the logs must be sufficient to reconstruct a decision, and reconstruction is a much stronger property than recording.

The reconstruction test — run it as an actual automated test, not a thought experiment:

Given: a decision D emitted at time T
When: I fetch D's log record
Then: I can recover the exact model version, the exact feature vector,
      the exact preprocessing code path, the exact thresholds,
      the exact reference data version, and the human reviewer identity
And:  re-executing the pipeline on the recovered inputs reproduces D
      bit-for-bit (deterministic) or within a declared tolerance
      (stochastic — and the tolerance is declared, with the seed and
      sampling parameters logged)

Automate this. Sample N decisions per day from production, attempt reconstruction, alert on failure rate above threshold. This single test catches more real compliance defects than any bias metric, because it fails the moment someone deploys a hotfix that changes preprocessing without a version bump, and that is a thing that happens constantly.

The places reconstruction dies in practice, in rough order of frequency:

Feature store point-in-time correctness. You logged feature_x = 0.42. You did not log that feature_x was computed from a materialised view refreshed at 03:00, which has since been backfilled. Re-execution now yields 0.51. You need point-in-time-correct joins and immutable feature snapshots, not "the current value of the feature."
Non-determinism you didn't declare. GPU non-determinism, atomicAdd ordering, cuDNN algorithm selection, unseeded sampling, temperature > 0. Either force determinism (and eat the throughput cost) or declare the tolerance and log the seed. Silently non-deterministic is the worst of both.
Upstream model drift. Your embedding service silently upgraded. Same input text, different vector, different retrieval, different answer. See the vendor-notification clause above.
PII in logs. You need the input data to reconstruct. The input data is personal data. GDPR Article 5(1)(c) says minimise; Article 5(1)(e) says don't keep it longer than necessary. The AI Act says keep it six months. These are not actually in conflict — Article 12 logging is a legal obligation under Article 6(1)(c) GDPR, and Article 5(1)(e) permits retention where required by law — but you must document the reconciliation in your ROPA and your DPIA. It is not a conflict; it is a documented conflict resolution. Those are different things and only one of them survives an audit.

Log volume is the practical killer. A high-throughput system generating feature-level logs will produce terabytes. Design for it: structured, columnar, partitioned by decision date, hot for 30 days and cold for the rest of your retention window, with a tested restore path. Test the restore. An untested cold-storage restore is not evidence; it is a hope with an S3 bucket.

3.5 Article 13 — Transparency, and the declaration that becomes a liability

Article 13 requires the system to be sufficiently transparent to enable deployers to interpret output and use it appropriately, and mandates instructions for use containing — per 13(3)(b) — the intended purpose, the level of accuracy including its metrics, robustness and cybersecurity levels, known circumstances that may lead to risks, performance regarding specific persons or groups, input data specifications, and human oversight measures.

Here is the part practitioners consistently miss: the accuracy you declare in the instructions for use is a commitment you can be measured against.

Declare AUC 0.94 measured on a dataset that doesn't resemble deployment, and you have created a documented gap between declared and actual performance. A market surveillance authority under Article 74 can request that evidence. A claimant in a civil action can request it. Under Article 99(5), supplying incorrect, incomplete, or misleading information to authorities carries up to €7.5M or 1% of turnover.

So the discipline is:

Declare the metric, the dataset, the sample size, and the confidence interval. Not a point estimate. AUC-ROC 0.871 (95% CI [0.863, 0.879]), n=48,211, holdout H-2026-Q2 sampled from EU-27 applications Jan–Mar 2026.
Declare performance disaggregated by group where 13(3)(b)(iv) applies. An aggregate number that hides a 14-point gap on a subgroup is the definition of misleading.
Declare the operating envelope. 13(3)(b)(iii): known or foreseeable circumstances that may lead to risks. This is where you say the model degrades below n=50 transaction history, or on documents over 40 pages, or for languages outside the training locale set. Every honest limitation you declare is a liability you have transferred to the deployer via Article 26(1) — they must use the system in accordance with the instructions. Every limitation you know about and don't declare stays with you.

Write the instructions for use as a generated artefact from your eval suite, with the numbers injected from the test run. Human prose around machine-generated numbers. Never hand-typed metrics. The number of production incidents caused by a stale metric in a PDF is not zero.

3.6 Article 14 — Human oversight: testing the human, not the model

Article 14 is the requirement that most engineering teams treat as somebody else's problem and that will most reliably fail an audit.

It requires that high-risk systems be designed so they can be effectively overseen by natural persons, with measures enabling the overseer to: understand capacities and limitations and monitor for anomalies (14(4)(a)); remain aware of automation bias (14(4)(b)); correctly interpret output (14(4)(c)); decide not to use the output or disregard/override/reverse it (14(4)(d)); and intervene or halt (14(4)(e)).

14(4)(b) is a design requirement about a human cognitive failure mode, written into a product regulation. And 14(5) — for Annex III point 1(a) biometric systems — requires that no action be taken unless verified and confirmed by at least two natural persons with the necessary competence, training and authority.

You cannot satisfy this with a UI that shows a score and an "approve" button. That UI causes automation bias; it does not counter it. And "there's a human in the loop" is a claim, not a control. The regulator's question is: is the oversight effective? That's an empirical question about human behaviour, and it is measurable.

Testable oversight properties — this is the section where you build things nobody else builds:

Override rate. What fraction of recommendations do reviewers override? If it's 0.2%, either your model is superhuman or your human is a rubber stamp. Base-rate reasoning tells you which. Instrument it, chart it, alert on it. An override rate that trends to zero over a reviewer's tenure is automation bias with a timestamp — and it's a beautiful piece of evidence that you caught it.
Time-on-task. Median review time of 3.1 seconds on a decision with 40 features is not review. Log it.
Planted-error injection. This is the strongest control in the entire oversight arsenal, and almost nobody does it. Periodically inject known-incorrect recommendations into the review queue — decisions you have constructed to be wrong in a detectable way. Measure the catch rate. This gives you a direct empirical measurement of oversight effectiveness, expressed as a number, over time, per reviewer cohort. It's a sensitivity/specificity measurement on your human layer. Present that to an auditor and watch the tenor of the conversation change. (Handle the ethics and the works-council consultation properly: consent, no punitive use of individual scores, cohort-level reporting, documented in the DPIA. Get this wrong and you have created an Annex III point 4 worker-monitoring problem while trying to solve an Article 14 problem. Yes, really.)
Override friction symmetry. If accepting is one click and overriding requires a free-text justification and a supervisor sign-off, you have engineered the outcome. Measure the friction differential. Make it symmetric, or document why asymmetry is justified.
Competence. 14(4) says overseers must have the necessary competence, training and authority. That means a training completion record, a competence assessment, and a documented authority to override that the person's actual manager will honour. Test the last one by asking reviewers, anonymously, whether they believe they'd face consequences for overriding. If they say yes, your Article 14 control is decorative.
Two-person rule enforcement (14(5)). For biometric ID: enforce in code, not policy. Separate identities, separate sessions, no self-approval, and — the one people miss — the second reviewer must not see the first's decision before forming their own. Otherwise you've built anchoring, not verification. Test it: attempt to self-approve, attempt sequential-visible approval; both must be rejected by the system.

3.7 Article 15 — Accuracy, robustness, cybersecurity

Three requirements bundled into one article, each of which is a discipline.

Accuracy (15(1), 15(3)). Covered above; the operative move is that you declare the metric and the level, and 15(3) requires those levels and metrics to appear in the instructions for use. The subtle part is Article 15(1)'s "perform consistently in those respects throughout their lifecycle." That word converts a point-in-time measurement into a monitoring obligation. See §8.

Robustness (15(4)). "Resilient against errors, faults or inconsistencies that may occur within the system or the environment," with technical redundancy where appropriate. And the sentence that should be tattooed on every MLOps engineer: high-risk systems that continue to learn after deployment shall be developed to eliminate or reduce as far as possible the risk of possibly biased outputs influencing input for future operations (feedback loops), and to ensure such loops are duly addressed with appropriate mitigation.

The Act names feedback loops explicitly. Test for them explicitly:

Does your training data for version N+1 contain outcomes that were caused by version N's decisions? In credit, you only observe repayment for applicants you approved. Your next model trains on a censored sample. That's selection bias, it compounds monotonically across retrains, and it is precisely the loop Article 15(4) is about. Reject inference isn't optional here; it's the mitigation you document.
Run a retrain simulation: take your pipeline, simulate 10 generations of retrain-on-own-outputs, measure metric drift and subgroup gap divergence across generations. If the subgroup gap widens monotonically, you have a documented feedback loop and a documented need for mitigation. If it doesn't, you have documented evidence that it doesn't. Either output is valuable. Neither takes more than a sprint to build.

Robustness testing proper:

Metamorphic testing. The workhorse for ML systems with no oracle. Define relations that must hold: paraphrasing a résumé must not change the score by more than δ; swapping a name from Aleksandra to Amara must not change it at all; adding an irrelevant sentence must not flip a classification; monotonic features must behave monotonically. Each relation is an executable test over generated input pairs. This is the single highest-value robustness technique available and it costs a week to stand up.
Perturbation batteries. Typos at realistic rates, unicode confusables and homoglyphs, OCR-style noise for document pipelines, encoding variants, whitespace and case, truncation at token boundaries. Measure prediction flip rate. Set a threshold in advance.
Distribution shift. Covariate shift (P(X) moves), label shift (P(Y) moves), concept shift (P(Y|X) moves). Test all three deliberately: hold out by time, by geography, by channel. Report degradation per shift type. The temporal holdout is non-negotiable — random splits on temporally-ordered data are the most common source of overstated accuracy in production ML, and overstated accuracy is now a regulatory declaration.
Redundancy and graceful degradation. What happens when the model service times out? Fail-open (approve everything) and fail-closed (reject everything) are both discriminatory in different directions. Test the fallback path. The fallback is part of the high-risk system.

Cybersecurity (15(5)). Article 15(5) names the attacks: data poisoning, model poisoning, adversarial examples ("evasion"), model evasion, confidentiality attacks, and model flaws. The regulator has handed you a test plan. Use it as one.

Art. 15(5) named threat	Test technique	Evidence artefact
Data poisoning	Provenance verification on every training shard; hash-pinned datasets; anomaly detection on new data batches; simulated label-flip at 0.5/1/2/5% and measure metric impact to establish your poisoning sensitivity curve	Poisoning sensitivity report; dataset Merkle root in Annex IV lock
Model poisoning	Signed model artefacts (cosign/sigstore); registry with immutable tags; supply-chain attestation (SLSA); verify signature at load time, fail closed	Signature verification log; deployment gate
Adversarial examples	Gradient-based (PGD, C&W) where white-box; transfer and query-based (Square Attack, HopSkipJump) where black-box; certified radius via randomised smoothing where the risk justifies the cost	Robust accuracy at declared ε; certified radius where applicable
Model evasion	Domain-specific attacker simulation. For content moderation, that's obfuscation, leetspeak, adversarial suffixes. For fraud, it's an economically-rational attacker probing your decision boundary — model the query budget explicitly	Evasion rate at attacker budget B
Confidentiality attacks	Membership inference (LiRA / shadow-model), model inversion, extraction. See §5 — this is where the AI Act and the GDPR fuse	MIA AUC; extraction rate; canary exposure
Model flaws	Fuzzing the serving path; malformed input; pickle deserialization (use safetensors, always); tokenizer edge cases; prompt injection for anything LLM-backed	Fuzzing coverage; CVE scan of the serving stack

If you build anything with an LLM in the loop, prompt injection belongs here. Not because Article 15(5) names it, but because it is the mechanism by which "model flaws" become an Article 79 risk in agentic systems. The relevant test is not "can I jailbreak the model" — it's "can untrusted content reaching the model cause an action with a real-world effect that the human overseer did not authorise?" Enumerate every tool your agent can call, classify each by blast radius, and test whether injected content can reach it. The answer for most production agents in 2026 is yes, and almost nobody has written that finding down.

4. GDPR: the layer that is already live and has no grace period

The AI Act gave you until December 2027. The GDPR gave you until May 2018. The DPAs are not waiting for the AI Act and never were.

The evidence: Italy's Garante fined OpenAI €15M in December 2024, on GDPR grounds alone — lawful basis for training, transparency, age verification — with no AI Act in sight. The EDPB's Opinion 28/2024 (December 2024) on AI models and personal data settled the question everyone had been avoiding: a trained model is not automatically anonymous, and whether it contains personal data is a case-by-case assessment that must consider the means reasonably likely to be used for extraction, including by the controller itself. The Hamburg DPA's discussion paper took a different view. That disagreement is unresolved and you must engineer for the stricter reading.

And, per §0: the Digital Omnibus GDPR amendments — Article 88c legitimate interest for AI training, the entity-relative personal data definition, the pseudonymisation carve-out, the Article 22 restructuring — are not law and several have already been stripped from the Council's compromise text. Build for the GDPR that exists.

4.1 The rights that don't survive contact with a model

Article 17 — erasure, and the unlearning problem.

A data subject requests erasure. You delete their row from the training set. Congratulations: the model still contains the information. Their contribution is baked into the weights via every gradient step their examples participated in.

The engineering reality, stated honestly because the honest statement is the valuable one:

Exact unlearning means the resulting model is indistinguishable from one retrained from scratch without that data. The only general method is retraining from scratch. For a 7B-parameter model that's weeks and six figures. SISA (Sharded, Isolated, Sliced, Aggregated) makes it tractable by sharding training so you only retrain the affected shard — but it costs you accuracy from the loss of cross-shard learning, and it is an architecture decision you must make before training, not a remediation you can bolt on. If your model is already trained monolithically and you get an erasure request that bites, your options are retrain or argue.
Approximate unlearning — gradient ascent on the forget set, influence-function-based weight surgery, Fisher-information scrubbing, task arithmetic — is faster and not verifiable by construction. You cannot prove the information is gone. You can only fail to find it.
Therefore: the verification is an attack. Run membership inference against the forgotten records. If a LiRA-style attack, given the pre-unlearning and post-unlearning models and shadow models, can still distinguish members from non-members at meaningfully better than chance, your unlearning did not work. Report the AUC. An unlearning claim with no MIA evidence behind it is an unsupported assertion, and a competent DPA — or a competent claimant's expert — will treat it as one.
The strongest position is architectural. Design so that erasure is structurally satisfiable: personal data lives in a retrieval layer, not in the weights. Delete from the vector store and the record is gone from the system's knowledge, verifiably, in seconds. This is not a compliance hack. It is a good architecture that happens to also be defensible. If you are making a build decision today for a system that will handle personal data at inference, this is the decision that will matter most in three years.

The tests you need:

T-ERASE-01  Deletion propagation: subject S deleted → assert absence
            across primary DB, replicas, backups (with documented
            restore-time re-deletion procedure), feature store,
            embedding index, cache, log store (subject to Art. 12
            retention reconciliation), analytics warehouse, and every
            downstream partner under Art. 19 GDPR notification.
T-ERASE-02  Regeneration guard: does the next nightly ETL resurrect S
            from an upstream source? Run the pipeline post-deletion and
            re-assert absence. This catches the single most common
            erasure defect in production.
T-ERASE-03  Model memorisation: MIA on S's records against the deployed
            model. Report AUC, CI, attack budget, shadow model count.
T-ERASE-04  Extraction: targeted prompting / nearest-neighbour probing
            of the embedding index for S's attributes.
T-ERASE-05  SLA: elapsed time from request to verified completion,
            against Art. 12(3)'s one month (extendable by two).

T-ERASE-02 is the one that catches real bugs. Every organisation I've worked with has had a resurrection path they didn't know about.

Article 15 — access, against a RAG system.

The subject asks: what personal data do you hold about me? Your RAG system holds documents, chunks, embeddings, and a cache. Which of those are "their data"?

The source document mentioning them: yes, obviously.
The chunk containing the mention: yes.
The embedding of that chunk: yes. Embedding inversion research (Morris et al., 2023) demonstrated that dense text embeddings can be inverted to recover a large fraction of the original text. An embedding of personal data is personal data. Any argument that "it's just a vector of floats, it's anonymous" collapses on contact with the literature, and it will collapse in front of a regulator who has read it.
The model's parameters: contested. EDPB 28/2024 says case-by-case. Engineer for the stricter reading.

So a genuine Article 15 response over a RAG system requires: full-text search across the source corpus, mapping from source to derived chunks, mapping from chunks to vector IDs, and a way to identify which cached responses were generated from those chunks. Most vector databases do not have this mapping by default. If you don't design the reverse index at build time, you cannot build it later without reprocessing the corpus. Build the subject_id → {doc_ids, chunk_ids, vector_ids, cache_keys} index on ingest. It costs almost nothing on day one and is prohibitively expensive on day four hundred.

Article 15(1)(h) additionally requires, for automated decision-making within Article 22's scope, "meaningful information about the logic involved, as well as the significance and the envisaged consequences." See §6.

Article 22 — automated decision-making, post-SCHUFA.

CJEU C-634/21 (SCHUFA, December 2023) held that generating a probability score constitutes an Article 22(1) decision where a third party draws strongly on that score to determine the outcome. Read that again if you build scoring models and tell yourself Article 22 doesn't apply because a human makes the final call.

The consequence: the "human in the loop" defence is only as strong as the human's actual influence. This is the same measurement problem as AI Act Article 14, and the same instrumentation answers both. Your override rate, your time-on-task, your planted-error catch rate — these are simultaneously your Article 14 effectiveness evidence and your Article 22 "not solely automated" evidence. One instrumentation investment, two regulatory regimes. That's the highest-ROI test infrastructure in this entire article.

The threshold to defend: at what override rate does a decision stop being "solely automated"? There is no case law giving you a number. Which means you set the threshold, document the reasoning, and monitor against it. Oracle construction, again.

Note that the Omnibus proposed lowering the Article 22 bar in contractual contexts (permitting automation where necessary for entering or performing a contract, regardless of whether a human could do it) — and the Council's February compromise deleted the Article 22 restructuring entirely. Do not build on it.

Article 5(1)(b) — purpose limitation as a lineage test.

You collected data to provide a service. You now want to train a model. Compatible? Article 6(4) gives the factors: link between purposes, context of collection, nature of the data, consequences, safeguards.

This is a data lineage question, and it is testable if — and only if — you have lineage. See §5.

Article 25 — data protection by design, as a CI gate.

Article 25 is the one that converts everything above into engineering. "Appropriate technical and organisational measures, at the time of the determination of the means of processing and at the time of the processing itself." Both times. Design-time and run-time. A design-time control with no run-time verification does not satisfy Article 25, and a run-time control with no design-time record does not either.

The DPIA (Article 35) is a mandatory trigger for most AI systems — systematic and extensive evaluation based on automated processing including profiling, on which decisions producing legal or similarly significant effects are based, is an explicit 35(3)(a) trigger. Which means: your DPIA and your AI Act Article 9 risk register describe overlapping risks and must not contradict each other. They will contradict each other, because different teams own them and neither reads the other's. An auditor will read both. Generate them from one source of truth or reconcile them on a schedule with a diff report.

5. Lineage: the load-bearing wall

Almost every obligation above reduces to one question: where did this bit come from, and what is it allowed to be used for?

Purpose limitation (GDPR 5(1)(b)): can this record be used for this purpose?
Erasure (17): which derived artefacts contain this record's influence?
Data governance (AI Act 10(2)(b)-(c)): what is the provenance of your training data?
Access (15): what do we hold about this person, including derived?
Copyright (AI Act 53(1)(c)): was this content subject to a TDM reservation under Art. 4(3) of the DSM Directive?
Special category derogation (AI Act 10(5)): is this attribute in the isolated bias-testing store or did it leak into training?

If lineage is missing, none of these are answerable and all of your controls are theatre. And lineage is where every real-world programme dies, for a specific and predictable reason:

Consent and purpose flags do not survive transformation.

Trace a single field:

1. User submits form. consent.marketing = true, consent.model_training = false
   → row in Postgres, flags intact ✓
2. Nightly ETL → data warehouse. The DE writes:
   SELECT user_id, age, income, region FROM users
   → flags dropped. Not maliciously. They weren't in the SELECT. ✗
3. Feature engineering → feature store. income → income_percentile_by_region
   → derived feature, no flag, no link to source row ✗
4. Training set assembly: SELECT * FROM feature_store WHERE created > X
   → consent.model_training = false records are now in your training set ✗
5. Model trains. Weights encode data the user explicitly refused. ✗
6. Erasure request arrives. You delete step 1. Steps 2-5 are unaffected. ✗
7. Vector store built from the same warehouse. Embeddings contain it. ✗

Step 2 is the fatal one and it takes four seconds to commit. It happens in every organisation. The person who wrote it did nothing wrong by any standard their team holds them to.

The controls that actually work:

Flags are columns, not metadata. Consent and purpose flags travel in the row, physically, through every hop. Not in a separate consent service you're supposed to join against — in the row. Denormalisation is a feature here, and if a data architect objects on normalisation grounds, the counter-argument is that a join you can forget to write is not a control.
Purpose-bound views, enforced at the storage layer. Nobody queries the raw table. They query v_training_eligible, which is WHERE consent.model_training = true AND deletion_requested_at IS NULL AND retention_expiry > now(). Revoke direct table grants. Now the default path is the compliant path, and the non-compliant path requires a privilege escalation that generates an audit event. Make the compliant path the lazy path. Every control that depends on engineers remembering something will fail; every control that makes non-compliance require extra effort will hold.
Column-level lineage, automatically extracted. OpenLineage, or your platform's equivalent, parsing the actual SQL and Spark plans — not a wiki page. You need income_percentile_by_region ← income ← users.income, derived from execution, not from documentation. Documented lineage is a description of what someone believed in March.
The lineage test. This is the test almost nobody writes and everybody needs:

T-LINEAGE-01
  Given: a training dataset D used to produce model M
  When:  I resolve every row of D back through the lineage graph to
         source records
  Then:  every source record carries consent.model_training = true
  And:   no source record has deletion_requested_at set
  And:   no source record's retention window has expired
  And:   every column of D resolves to a declared purpose in the ROPA
  And:   the count of unresolvable rows is ZERO
  Failure mode to watch: "unresolvable" is the interesting bucket.
  Rows you cannot trace are rows you cannot defend.

Run it as a hard gate in the training pipeline. Not a report. A gate. The job does not start if the check fails.

Yes, this will break your pipelines the first time you turn it on. That is the point. Every break is a compliance defect you were shipping.

The canary. Inject synthetic records marked consent.model_training = false into your source systems. If any canary reaches a training set, your lineage controls have a hole, and you know exactly which hop leaked it. This is chaos engineering for consent, it takes a day to build, and it is the only technique that finds the leaks your lineage graph doesn't know about. (Handle the canaries carefully: they're synthetic, they must be marked, and they must never reach a production decision path.)

6. Memorisation, extraction, and the tests that fuse both regimes

AI Act 15(5) names confidentiality attacks. GDPR Articles 5(1)(f), 17, 25 and 32 demand you prevent unauthorised disclosure. Same tests. Run them once, cite them twice.

Canary insertion (Carlini's "Secret Sharer" method). Before training, insert synthetic sequences of known format at known frequencies — "The access code for account {random} is {random}" at 1, 4, 16, 64 repetitions. After training, measure exposure: the log-rank of the true canary among all candidates of the same format, under the model's likelihood.

This gives you a quantified memorisation curve as a function of duplication rate — which tells you the deduplication threshold your pipeline must enforce. That is a concrete, defensible, engineering-grade answer to "what did you do about memorisation," and it is far better evidence than any policy statement. Nearly nobody does this and it is a week of work.

Membership inference. LiRA (Carlini et al., 2022) is the current serious baseline: train shadow models with and without a target record, fit Gaussians to the loss distributions, compute a likelihood ratio. Report the attack's TPR at low FPR (e.g. TPR@0.1%FPR), not just AUC — AUC hides the tail, and the tail is where the privacy harm lives. A model with MIA AUC of 0.52 but TPR of 40% at 0.1% FPR is leaking badly on a small set of vulnerable records, and the AUC will tell you everything is fine.

MIA is your primary verification instrument for:

Article 17 erasure claims (§4.1)
Whether a model is "anonymous" under EDPB 28/2024 (it gives you the means-reasonably-likely evidence)
Whether your DP guarantees hold empirically, if you trained with DP-SGD

Extraction. Carlini et al. demonstrated verbatim training-data extraction from GPT-2. The 2023 divergence attack — repeat a token indefinitely — extracted training data from production ChatGPT. Your test: a prompt battery designed to elicit training data (prefix continuation, divergence, format-priming, PII templates), a detector that matches outputs against known training records with fuzzy matching, and a reported extraction rate at a defined query budget. Declare the budget. An extraction rate of 0.02% at 10,000 queries is meaningful evidence; an extraction rate of 0% with no stated budget is meaningless.

Embedding inversion. For any RAG system: train an inversion model against your embedding space, measure recovery. If your vector store is more accessible than your document store — and it usually is, because vector stores are newer and their access controls are worse — this is your real exposure surface, not the documents.

RAG access control. The failure that will bite you: the retriever is not permission-aware. Alice asks a question, the retriever returns a chunk from a document Alice cannot open, the LLM summarises it, Alice reads it. You have built a permission-laundering machine.

T-RAG-ACL-01
  For each (user U, document D) where U lacks read access to D:
    craft queries designed to retrieve D's chunks
    assert D's chunks never enter U's context window
    assert D's content never appears in U's response
  Enforce filtering at retrieval, not in the prompt. "Do not reveal
  documents the user cannot access" is an instruction, and instructions
  are not access controls.

Pre-filter by ACL at the vector query, not post-filter after retrieval. Post-filtering means the data already entered the context, and a sufficiently clever prompt gets it out.

Differential privacy, honestly. DP-SGD gives you a mathematical guarantee, which is the only structural bucket available in §1's taxonomy. The costs are real: accuracy loss, hyperparameter fragility, and an ε that is hard to explain to anyone. But note what DP buys you legally: it converts a statistical claim into a bounded one, and it gives you a principled argument for anonymity under EDPB 28/2024's means-reasonably-likely test. If you are training on genuinely sensitive data and you can absorb the utility hit, DP is the strongest position available. If you can't, say so, document why, and compensate with the empirical tests above. Do not claim DP-adjacent protection from techniques that don't provide it — "we added noise" is not DP, and the difference is the entire point.

7. Bias testing that survives an auditor

Everyone runs fairness metrics. Almost nobody produces fairness evidence. The gap is methodological, and it's where a competent adversarial expert will take your programme apart.

Pick the metric before you see the results, and justify it against the intended purpose. Demographic parity, equalised odds, equal opportunity, predictive parity, and calibration are mutually incompatible — Kleinberg et al. and Chouldechova independently proved you cannot satisfy calibration and balance-for-both-classes simultaneously when base rates differ, except in degenerate cases. This is arithmetic, not policy. So you must choose, and the choice is a normative judgment tied to the harm you're preventing.

For a credit model: predictive parity says the score means the same thing for everyone; demographic parity says approval rates match. These conflict when default base rates differ, and they will. Your Article 9(5) residual-risk acceptance is exactly where you record which you chose and why. If you compute all of them and report the flattering ones, you are metric-shopping, and the timestamps in your notebooks will show it.

Report uncertainty, always. A 3-point gap on a subgroup with n=40 is noise. Bootstrap the CI. If the CI straddles zero, say so. If your subgroup is too small to measure — and it usually is, which is the actual finding — report that you lack statistical power rather than reporting a meaningless point estimate. "We could not measure this to useful precision at n=40; here is our plan to reach adequate power" is credible. "Gap = 0.031" with no CI is not.

Intersectionality, with honest multiple-comparison handling. Marginal parity by gender and by ethnicity does not imply parity for their intersection — and the intersection is where the harm concentrates. But testing 200 cells guarantees false positives. Pre-register the cells that matter based on your risk analysis, and apply Benjamini–Hochberg to the rest. Report both the pre-registered results and the exploratory ones, separately labelled. Blending them is how you get caught.

Test the pipeline, not the model. The model is one component. Bias enters at candidate sourcing, at feature availability (thin-file applicants have fewer features — that's a data-availability gap that becomes a score gap), at threshold selection, at the human review layer, at the appeal process. A perfectly fair model behind a biased threshold is a biased system, and Article 10(2)(f) is about the system.

Proxies. Redundant encoding means removing a protected attribute doesn't remove it. Postcode encodes ethnicity. Employment gaps encode disability, caregiving, and gender. Device type encodes income. Test for it: train an adversarial probe to predict the protected attribute from your feature vector. If it succeeds at high AUC, your feature set contains the attribute, and "we don't use ethnicity" is not a defence anyone will accept. Report the probe AUC. It's a strong, honest, easily-understood number.

8. Explaining the explanation: Article 86 AI Act, Articles 13–15 and 22 GDPR

The AI Act's Article 86 gives affected persons the right to obtain from the deployer clear and meaningful explanations of the role of a high-risk AI system in the decision procedure and the main elements of the decision taken, for Annex III decisions with legal or similarly significant effects.

The GDPR requires "meaningful information about the logic involved" (13(2)(f), 14(2)(g), 15(1)(h)). CJEU C-203/22 (Dun & Bradstreet, February 2025) clarified this decisively: the controller must describe the procedure and principles actually applied in a way the data subject can understand — not dump the algorithm, and not hide behind trade secrets as a blanket refusal. Trade secrets are a reason to disclose to a supervisory authority or court rather than a reason to disclose nothing.

The engineering problem: your explanation must be faithful, and faithfulness is testable.

An unfaithful explanation is worse than none. It is a false statement about your processing, made to a data subject, in response to a statutory right. Test yours:

Faithfulness / deletion-insertion. Remove the features the explanation claims are important, in order of claimed importance. Prediction should degrade fast. Remove random features. It should degrade slowly. The AUC gap between those curves is your faithfulness score. If they're similar, your explanation is decoration.
Comprehensiveness and sufficiency (from the ERASER framework): does removing the cited rationale change the prediction (comprehensiveness), and does the rationale alone reproduce it (sufficiency)? Both, as numbers.
Stability. Perturb the input imperceptibly, re-explain. If the top-5 features reorder, your explanation is unstable and you cannot claim it describes the decision. KernelSHAP with insufficient samples is notoriously unstable — measure the variance across seeds and report it. Most teams run SHAP once and treat the output as ground truth.
The Rashomon problem. Multiple explanation methods (SHAP, LIME, integrated gradients, counterfactuals) will disagree on the same decision. Pick one, justify it, use it consistently. Reporting whichever one looks best per case is indefensible and will be found by anyone who samples your explanation logs.
Counterfactual actionability. Article 86 asks about "the main elements of the decision." A counterfactual — "had your debt-to-income been below 0.38, the outcome would have differed" — is more meaningful to a human than a SHAP bar chart, and it is testable: verify the counterfactual by executing it. Feed the counterfactual back through the model and assert the outcome actually flips. An unverified counterfactual is a guess. Also constrain to mutable features: telling someone the decision would change if they were younger is both useless and evidence of a discrimination problem.

Log every explanation you generate, bound to the decision ID and the model hash. When someone asks in 2029 why they were rejected in 2026, you must reproduce the explanation for the model that made the decision, not the model you run now.

9. GPAI and Article 50: the obligations that are already live

GPAI (Art. 51–55) has applied since 2 August 2025. The Omnibus did not touch it. If you provide a general-purpose model:

Art. 53(1)(a)+(b): technical documentation, and information for downstream providers. Your Annex XII pack is a product, with an SLA, that your customers' Annex IV depends on.
Art. 53(1)(c): a copyright policy, including identifying and respecting TDM reservations under Art. 4(3) of Directive (EU) 2019/790. This is testable: crawl-time robots.txt and TDM-reservation compliance, per-domain, with logs. Evidence = a crawl audit showing reservations were honoured, with counts.
Art. 53(1)(d): a sufficiently detailed summary of training content, per the AI Office's published template. Generate it from lineage (§5). If you cannot generate it, you do not have lineage.
Art. 51(2): systemic risk is presumed above 10²⁵ FLOP cumulative training compute. Log your FLOP. Not estimate — log. It's an integral over your training run and you should be emitting it into the Annex IV lock file anyway.
Art. 55: if systemic-risk: model evaluation including adversarial testing, systemic risk assessment and mitigation, serious incident tracking and reporting to the AI Office, adequate cybersecurity for the model and physical infrastructure. The GPAI Code of Practice (published July 2025) is the presumption-of-conformity route.

If you consume a GPAI model — which is nearly everyone — your obligation is contractual and architectural: get the Art. 53(1)(b) pack, get update notification with lead time, and pin. See §3.3.

Article 50 is your nearest live deadline, and almost nobody is ready.

From 2 August 2026 — eighteen days from now — provider and deployer transparency obligations apply. From 2 December 2026, Article 50(2) machine-readable marking applies to systems already on the market, and the new NCII/CSAM prohibition takes effect.

Article 50(2) requires providers of systems generating synthetic audio, image, video or text to ensure outputs are marked in a machine-readable format and detectable as artificially generated or manipulated, with solutions that are effective, interoperable, robust and reliable as far as technically feasible.

The engineering, and the tests:

Requirement	Implementation	Test
Machine-readable marking	C2PA Content Credentials manifest, signed; SynthID-style or Kirchenbauer-style statistical watermark for text/pixels	Assert manifest present and signature valid on 100% of outputs; assert watermark detectable
Robust	Survives re-encoding, resize, crop, screenshot, format conversion, paraphrase (text)	Adversarial removal suite: apply each transform, measure detection rate. Report the survival curve.
Reliable	Detector FPR budget	Measure FPR on a large corpus of known-human content. At scale, a 0.1% FPR means thousands of false accusations. Set the budget first.
Interoperable	Open standard, not proprietary	Cross-validate: does another C2PA-conformant validator read your manifest?
Deepfake disclosure (50(4))	Visible label, not just metadata	UI test; assert label survives the export path
AI interaction disclosure (50(1))	Disclosed at first interaction unless obvious	UI test

Text watermarking is the honest weak spot. Statistical green-list watermarks degrade badly under paraphrase, and paraphrase is one API call away. Article 50(2) says "as far as technically feasible" — which means your evidence is the measured survival curve plus a documented statement of the technical limits. That is a defensible position. "We watermark" with no robustness data is not.

The NCII/CSAM prohibition from 2 December 2026 has a safe harbour for effective preventive safeguards judged against the state of the art. If you provide image generation, the safeguard design must be in your risk management documentation now. There is no retrofit path for a prohibition.

10. Compliance as a pipeline, and the evidence package

Everything above is worthless if it lives in a notebook. Here's the shape that works:

commit
  └─ [gate] classification manifest unchanged OR re-signed
  └─ [gate] lineage: 0 unresolvable rows, 0 consent violations,
            0 deleted-subject rows, 0 expired-retention rows
  └─ [gate] dataset: PSI < 0.25 all covariates vs declared population;
            train/test leakage = 0; label noise estimated & reported
  └─ train
       └─ emit annexIV.lock (hashes, compute, FLOP, metrics) + sign
  └─ [gate] performance: AUC >= declared floor, per-group AND aggregate
  └─ [gate] fairness: pre-registered metric within pre-registered bound;
            CIs reported; adversarial proxy probe AUC reported
  └─ [gate] robustness: metamorphic relations hold; flip rate < ε;
            temporal-holdout degradation < declared
  └─ [gate] security: adversarial robust-accuracy @ declared ε;
            MIA TPR@0.1%FPR < threshold; canary exposure < threshold;
            artefact signature verified; SBOM clean
  └─ [gate] explainability: faithfulness AUC-gap > threshold;
            explanation stability variance < threshold
  └─ [gate] oversight: override path present; friction symmetry check;
            planted-error harness wired
  └─ [gate] Art. 50: watermark present + survival curve regenerated
  └─ [gate] traceability: risk register has 0 orphan risks
  └─ generate: Annex IV pack, instructions for use (metrics injected),
               DPIA delta, declaration of conformity draft
  └─ deploy
  └─ monitor: drift, override rate, time-on-task, incident clocks

Three design principles that determine whether this survives:

Gates, not reports. A report is a thing someone ignores under deadline. A gate is a thing that stops the deploy. Every gate you soften to "warning" is a control you have deleted.
Evidence is emitted, not assembled. If the evidence package is a thing a human builds before an audit, it will be inconsistent with reality and the inconsistency is what the auditor will find. Emit at build time, sign, store immutably.
Everything binds to a hash. Model, dataset, code, container, config. "Which model made this decision" must be a lookup, not an investigation.

The evidence package, as an auditor opens it:

classification.yaml + the derogation assessment (Art. 6(4)) if you claimed one
annexIV.lock, signed, one per released model version
Risk-to-test traceability matrix, generated live (Art. 9)
Data governance report: provenance, representativeness, bias examination, Art. 10(5) derogation controls (Art. 10)
Test execution records with timestamps, artefact hashes, and pass/fail — not summaries; the raw records
Instructions for use, with the exact declared metrics that match annexIV.lock (Art. 13)
Oversight effectiveness report: override rate, time-on-task, planted-error catch rate (Art. 14)
Security testing report against each Art. 15(5) named threat
Logging design + reconstruction test results (Art. 12)
Post-market monitoring plan + live drift dashboards (Art. 72)
Incident log + reporting timelines (Art. 73)
DPIA + ROPA + lineage attestations + LIA, cross-referenced to (3) and (4)
Declaration of conformity (Art. 47), CE marking evidence (Art. 48), EU database registration (Art. 49/71)

The first question a serious auditor asks is not about any of these. It is: "show me a decision from last March, and prove this document describes the model that made it." If the answer is a hash lookup, the audit is short. If it's a meeting, the audit is long, and the finding is written before you leave the room.

Post-market monitoring (Art. 72) and incidents (Art. 73). Article 15(1)'s "throughout their lifecycle" plus Article 72's monitoring plan means your gates must run continuously, not once. Monitor: covariate drift (PSI on live traffic), performance drift where you have delayed labels (and model the delay — credit outcomes arrive over 24 months, so your drift signal lags reality by two years and you need leading indicators), subgroup gap drift, override rate drift, and calibration drift (ECE on live data).

Article 73's clocks are short and they start at the moment you establish a causal link, or a reasonable likelihood of one: 15 days generally; 2 days for widespread infringement or serious disruption to critical infrastructure; 10 days for death. Two days is not a quarterly governance forum. It is a pager rotation with a pre-drafted template and a named decision-maker. Run the drill. Time it. A team that has never rehearsed the two-day path will miss it.

11. Anti-patterns, ranked by how much they cost

Compliance owned outside engineering. The governance function writes a policy; engineering never reads it; the artefacts describe a system nobody built. Every finding traces back here.
Point-in-time compliance. Certified once, drifted forever. Article 15(1) says lifecycle. Article 72 says continuous. A pass from March is not evidence about July.
Evidence assembled at audit time. Guarantees inconsistency; inconsistency is what gets found.
Metric shopping. Computing every fairness metric and reporting the flattering one. The notebooks have timestamps.
Human-in-the-loop as an incantation. A human with 3-second review time and a 0.2% override rate is not oversight. SCHUFA closed this door in 2023 and Article 14 nailed it shut.
Testing the model, not the system. The bias is in the threshold, the sourcing, the fallback, and the appeal path.
Ignoring the GPAI supply chain. Your compliance is a function of your vendor's, and you have no clause requiring them to tell you when they change the model.
Treating the Omnibus deferral as relief. Sixteen months is not "later," it is "now with less panic." The teams that will fail in December 2027 are the ones who stood down in July 2026.
Building on unadopted law. Article 88c, the pseudonymisation carve-out, the Article 22 restructuring: not law, and several already stripped from the Council text.
The unverified deletion. A deletion policy is not a deletion test, and nightly ETL resurrects the dead.

12. What to do in the next eighteen days, and the next eighteen months

By 2 August 2026 (Article 50 goes live):

Inventory every system with a human-facing conversational surface → Art. 50(1) disclosure shipped.
Inventory every generative output surface → Art. 50(4) deepfake labelling shipped.
Deployer-side transparency obligations mapped and shipped.

By 2 December 2026 (Art. 50(2) legacy + NCII/CSAM prohibition):

Watermarking pipeline live: C2PA manifests signed, detector deployed, survival curve measured and documented, FPR budget set. This is ~4 months of engineering and it is the closest hard deadline you have.
NCII/CSAM safe-harbour design documented in the risk management system if you ship image generation.

Now through Q4 2026 (the foundation, all of which is GDPR-driven and therefore already late):

Executable classification + regression corpus (§2).
Lineage: flags-in-row, purpose-bound views, column-level lineage, T-LINEAGE-01 as a hard gate, canaries live (§5).
Erasure architecture decision: retrieval-layer or SISA-sharded. Make it before you train the next model, because you cannot make it after.
annexIV.lock emission and signing (§3.3).
Reconstruction test in production (§3.4).
Vendor clauses: Art. 53(1)(b) pack + update notification with lead time.

2027 (the AI Act build-out):

Article 9 loop with live traceability matrix.
Article 14 oversight instrumentation and planted-error harness — the single highest-ROI item, because it serves Art. 14 and Art. 22 GDPR.
Full Art. 15(5) security battery.
Notified body dialogue if you're in Annex III point 1 without harmonised standards — queues will form; December 2027 is not far.

By 2 December 2027: Annex III conformity complete, registered, CE-marked.

13. The one-paragraph version

The regulation does not tell you what "correct" means. It tells you that you must define correct, justify the definition against your intended purpose, test against it, prove the test ran on this exact artefact, and monitor that it keeps holding. That is oracle construction, and it is the job of a test engineer, not a lawyer. The AI Act just handed you sixteen extra months to build it; the GDPR handed you nothing and never did. The teams that will pass in 2027 are the ones who spent 2026 building gates, lineage, and signed evidence — and the teams that will fail are the ones who read the word "postponed" and closed the tab.

If this was useful, the long-form versions go considerably deeper.

This article is a compressed map. The books are the terrain.

📕 AI Compliance Testing: GDPR, EU AI Act & Beyond — 34 pages, practitioner-first, zero legal background required. The EU AI Act risk-tier mapping worksheet (classify every feature in minutes). The GDPR-to-test-case translation guide covering data, consent, and explainability. The audit-ready compliance evidence matrix regulators can follow line by line. Plus coverage of emerging global frameworks so you're ahead, not catching up.
Regulation is now a QA problem, and ignorance is now a fineable offense.
→ https://himanshuai.gumroad.com/l/AI-Compliance-Testing — 50% off (India PPP auto-applied: ₹1,636.81)

📚 Complete AI Testing & GenAI Engineering Master Bundle — 18 Books — everything upstream of compliance: LLM evaluation harnesses, RAG testing, agent reliability, red-teaming, prompt security, MLOps quality gates, and the compliance layer above.
→ https://himanshuai.gumroad.com/l/GenAI-Testing-Master-Bundle-18-Books
→ Code SPECIAL70 → flat 70% OFF

Not sure which is right for you? DM me. Tell me your years of experience and what you're building, and I'll tell you honestly which path fits — or whether you need either. I'd rather you buy the right one than the expensive one.

— Himanshu Agarwal

Regulatory position verified 15 July 2026. The Digital Omnibus on AI received final Council approval on 29 June 2026 (Annex III high-risk → 2 Dec 2027; Annex I → 2 Aug 2028; Art. 50(2) legacy + NCII/CSAM prohibition → 2 Dec 2026). The general Digital Omnibus amending the GDPR remains a proposal; the Council's compromise text removed the Article 4(1) personal-data redefinition and the Article 22 restructuring. Treat the GDPR as fully applicable, unamended. This is engineering guidance, not legal advice — your DPO and counsel own the legal position; you own the evidence.

DEV Community