Structuring a DPIA for a Generative AI Feature

#ai #machinelearning #tutorial #llm

A DPIA is not a form. Article 35(7) specifies four things the assessment must contain, and every template you can download is an expansion of those four. Knowing which of your paragraphs is discharging which subparagraph is what stops the document turning into a system description with a risk table stapled to the end.

What Article 35(7) requires

Article 35(7) of Regulation (EU) 2016/679 requires the assessment to contain at least:

(a) a systematic description of the envisaged processing operations and the purposes of the processing, including where applicable the legitimate interest pursued by the controller;
(b) an assessment of the necessity and proportionality of the processing operations in relation to the purposes;
(c) an assessment of the risks to the rights and freedoms of data subjects;
(d) the measures envisaged to address the risks, including safeguards, security measures and mechanisms to ensure the protection of personal data and to demonstrate compliance with the Regulation, taking into account the rights and legitimate interests of data subjects and other persons concerned.

Three further provisions shape the document. Article 35(2): seek the DPO’s advice. Article 35(9): where appropriate, seek the views of data subjects or their representatives. Article 35(11): review where the risk changes. And Article 36: consult the supervisory authority if the residual risk is still high. Whether you must do a DPIA at all is a separate question, worked through on DPIA triggers for AI.

Not legal advice. The structure below follows the text of Article 35(7), but the adequacy of any particular assessment is a judgement on your own facts, and several supervisory authorities publish their own preferred template that you should check against.

Section 1: the systematic description

The worked example throughout is a customer-facing assistant on a SaaS product: the user types a question, the application retrieves matching passages from that customer’s own data, sends them with the question to a hosted model, and streams the answer back. Conversations are retained for quality review.

“Systematic” is the load-bearing word. This section has to be specific enough that somebody who has never seen the system can follow a single piece of personal data through it. In practice that means naming, for each processing operation:

The categories of data subject and data. Not “user data”. The end user’s account identifiers and prompt text; third parties named inside the prompt, who are data subjects who never interacted with you; whatever personal data is in the retrieved passages.
Each purpose separately, with its lawful basis. Answering the question, retaining transcripts for quality review, and any use of transcripts to improve a model are three purposes with three bases, not one. Conflating them is the single most common defect in these documents.
Recipients, and their role. The model provider is almost always a processor under Article 28. Say which provider, in which capacity, under which contract.
Transfers. Where the inference actually happens, the transfer mechanism, and the reference to your transfer impact assessment — see transfer impact assessments for AI providers.
Retention, per purpose. Prompt logs, retrieved context, model outputs and any evaluation dataset built from them each need their own period.

A flow list is usually clearer than prose here:

1  user prompt          -> app          (EEA, controller)
2  prompt + retrieved   -> model API    (processor, Art 28 DPA, US, SCCs)
3  model output         -> app          -> user
4  transcript           -> log store    (EEA, 30 days, quality review)
5  redacted transcript  -> eval set     (EEA, 12 months, separate purpose)

Step 2 is the row that changes most often and is hardest to keep accurate, because swapping the model behind a feature usually swaps the processor, the processing location and the contract at the same time. If you route through a gateway, that row names one contracting party and the DPIA needs a sub-processor list underneath it — which is how Multigrid describes itself in customers’ assessments, and it is worth checking that any gateway you use publishes a change notice process for that list, because Article 28(2) gives you an objection right that is worthless if you find out after the fact.

Section 2: necessity and proportionality

This section is where most DPIAs are thinnest and where a supervisory authority reads most carefully, because it is the section that can conclude the processing should not happen in the form proposed.

Necessity asks whether each processing operation is actually required for the stated purpose, and it is answered by elimination. For the assistant: is the account identifier necessary in the prompt, or only a pseudonymous session id? Is thirty days of full transcript retention necessary for quality review, or would a sampled subset do? Is the retrieved passage necessary in full, or can the personal data in it be masked before it leaves your infrastructure? Every answer of “no” here is a design change, which is why this document has to be written before the build.

Proportionality is the balance between the intrusion and the benefit, and it is where you address the specific complaints a reasonable data subject would make. If the assistant is optional and its benefit is convenience, that weighs differently from a system that gates access to a service. If the lawful basis for any operation is legitimate interests, the balancing test belongs here or is cross-referenced from here — the format is set out on writing a legitimate interest assessment.

Section 3: risks to rights and freedoms

The risk being assessed is risk to the data subject, not to the company. A regulatory fine is not a risk for the purposes of Article 35(7)(c) and listing it is a reliable sign the section was written by somebody assessing the wrong thing.

Risks that are specific to a generative feature, rather than generic:

Confabulated personal data. The model states something false about a named individual and the user acts on it. This engages the accuracy principle in Article 5(1)(d) and the rectification right; see rectifying AI-hallucinated personal data.
Cross-tenant or cross-user leakage through retrieval. A permissions bug in the retrieval layer is a confidentiality breach with a very large blast radius, because the model will summarise whatever it is given.
Prompt injection turning the assistant into an exfiltration channel. Content the user did not write instructing the model to include other data in its output.
Unexpected personal data in prompts. Users paste things. Whatever your terms say, the health data, the third party names and the case details will arrive, which is a reason to plan for Article 9 data rather than to prohibit it and assume compliance.
Function creep into training. Transcripts retained for quality review being used to fine-tune is a new purpose, and it is the change most likely to happen without anyone reopening this document.

Score each risk for likelihood and severity before mitigation, and again after — the residual score is what Article 36 turns on.

Section 4: measures addressing the risks

Each measure should point at a numbered risk. Measures that are real and checkable, rather than aspirational, look like: enforcing retrieval permissions at the query layer rather than filtering after the fact; masking direct identifiers before the prompt leaves your boundary; contractually disabling provider-side training and evidencing that in the Article 28 agreement; separating the quality-review store from the evaluation store so that the second cannot silently grow; a user-visible statement that outputs may be inaccurate about people; a documented route for a data subject to contest a statement the system made. Article 25’s data-protection-by-design duty is discharged largely through this section — see Article 25 applied to an AI feature.

Assembling and signing it off

Draw the data flow first, at the level of the five-row list above. Everything else in the document is derived from it, and errors here propagate.
Split the flow into purposes and assign a lawful basis to each. If two purposes share a row, split the row.
Write section 1 against the flow, naming categories, recipients, transfers and retention per purpose.
Run the necessity elimination on every field and every retention period. Record what you removed — the deletions are the evidence that the test was actually applied.
List risks to data subjects, score likelihood and severity, and deliberately include the three that are specific to generative systems (confabulation, retrieval leakage, injection).
Attach measures to numbered risks and re-score. Anything still high is an Article 36 prior consultation question, not a paragraph to soften.
Send it to the DPO under Article 35(2) and record the advice given, including where you did not follow it.
Decide under Article 35(9) whether to seek data subjects’ views, and record the decision either way.
Set a review trigger under Article 35(11) in the place engineers will actually see it — the pull request template for the model configuration is better than a calendar reminder.