“Send me all the personal data you hold about me” arrives at a company that trained a model on a corpus containing that person. Most of the request is ordinary and answerable. One part of it is a genuinely open question, and the honest answer to that part is longer than a refusal and shorter than an invention.
Two different requests wearing one label
Article 15 of Regulation (EU) 2016/679 contains two distinct entitlements that a single sentence in an email triggers at once. Article 15(1) is a right to information: the purposes, the categories of personal data, the recipients, the envisaged retention period, the existence of the other rights, the source where the data was not collected from the subject, and under 15(1)(h) the existence of automated decision-making with meaningful information about the logic involved. Article 15(3) is a right to a copy of the personal data undergoing processing.
Separating them is the first move, because the copy limb is the one that collides with the model and the information limb is almost entirely answerable. A response that treats the whole request as impossible because one part of it is hard is a response that fails on the easy parts, and supervisory authorities enforce on the easy parts.
Not legal advice, and this is one of the areas where the legal position is unsettled rather than merely complicated. If you are drafting an actual response, particularly one that declines to provide something, take advice on it.
What you certainly hold
Before reaching the interesting question, work through the boring one, because in most AI companies the boring one accounts for the great majority of the personal data:
- The training corpus, if you retained it. If the documents that went into training are still on disk, they are personal data in an ordinary retrievable form and Article 15(3) reaches them like any other file store. Whether you retained the corpus is a question with its own consequences — see retention limits for training data.
- Prompts and outputs. Conversation logs are records. If a user’s prompts mention them or a third party, that is personal data sitting in a database with a primary key.
- Evaluation and annotation sets derived from real traffic, which are frequently forgotten because they live outside the production data model.
- Feedback, ratings and moderation flags attached to individual conversations.
- Account, billing and telemetry data — ordinary SaaS data that has nothing to do with the model and everything to do with the response being complete.
Article 15(1)(g) also requires, where the data was not collected from the data subject, any available information as to its source. For a scraped corpus that obligation is uncomfortable and it is not excused by being uncomfortable; the extent to which “available” softens it depends on what provenance you actually recorded, which is an argument for recording it.
Whether the weights contain personal data
Now the open question. Personal data under Article 4(1) is information relating to an identified or identifiable natural person. A set of floating-point parameters is not a record about anyone, and yet a model can emit accurate personal information about a person on request, and research on training-data extraction shows that memorised sequences can sometimes be recovered verbatim.
There are at least two serious positions and they conflict. In July 2024 the Hamburg Commissioner for Data Protection and Freedom of Information published a discussion paper arguing that the storage of a large language model does not itself constitute processing of personal data within Article 4(2), so that the model as such does not hold personal data even where its training did — with the consequence that data subject rights bite on the training and the outputs but not on the weights. The paper is a position of that authority, not a decision, and other authorities have not adopted it.
The European Data Protection Board took a different and more careful route in Opinion 28/2024, adopted on 17 December 2024, on data protection aspects of processing in the context of AI models. Its position is that a model trained on personal data cannot be assumed to be anonymous; anonymity has to be demonstrated case by case, and the test it sets is that the likelihood of directly extracting personal data from the model, and the likelihood of obtaining such data from queries, should both be insignificant taking account of the means reasonably likely to be used. That framing does not say weights are personal data; it says you do not get to assume they are not, and the burden of showing otherwise is yours.
So the honest state of the law as at this page’s date: unresolved. What would settle it is a ruling from the Court of Justice on whether a set of model parameters is personal data relating to an identifiable person, or a formal EDPB position going beyond the anonymity framing. A controller writing an access response should not assert either extreme.
Article 15(1)(h) and the logic involved
Where you carry out automated decision-making within Article 22(1) and (4), Article 15(1)(h) requires meaningful information about the logic involved and about the significance and envisaged consequences of the processing for the data subject. In Case C-203/22, Dun & Bradstreet Austria, decided on 27 February 2025 and available from the Court’s case register, the Court held that this requires an explanation of the procedure and principles actually applied, in a concise and intelligible form that allows the data subject to understand which of their personal data was used and how — and that disclosure of the algorithm itself is not what is required. It also held that an asserted trade secret does not permit a blanket refusal; the allegedly protected information should be provided to the supervisory authority or court so that the competing interests can be balanced.
Note the precondition. If your model is not making Article 22 decisions, 15(1)(h) is not engaged, and a great many access responses over-answer here. If it is engaged, the ruling makes clear that “it is a neural network and therefore inexplicable” is not an answer.
What an honest response looks like
Return everything in the retrievable stores, in full, on time — Article 12(3) gives one month, extendable by two further months for complexity with notice within the first month. Answer the 15(1) information items directly rather than by reference to a privacy notice. Then, on the model itself, say what you do and do not know: that the person’s data may have formed part of a training corpus, that the model does not store records and there is no operation that retrieves an individual’s data from it, that whether the parameters constitute personal data is a question on which regulators have not reached a common position, and what measures you apply — output filtering, memorisation testing, deduplication — to reduce the chance of the model reproducing their data. A response in those terms is defensible. A response asserting flatly that the model contains no personal data is taking a contested position as though it were settled.
The practical bottleneck in most of these responses is not the model at all — it is finding every place a prompt was logged, which in a multi-provider setup means one export per provider console, each with a different retention window and a different notion of a request id. A gateway that keeps one request log across providers turns that into a single query, which is worth knowing about before the first access request arrives rather than after; it is one of the reasons Multigrid keeps a unified request record, though any consolidated logging layer solves the same problem.
The limits the Regulation actually gives you
Three provisions are relevant and none is a general escape. Article 11 says that where the controller can demonstrate it is not in a position to identify the data subject, Articles 15 to 20 do not apply unless the subject provides additional identifying information — useful for a genuinely unindexed corpus, useless once you hold an account. Article 12(5) permits a refusal or a reasonable fee where a request is manifestly unfounded or excessive, in particular because of repetitive character, and puts the burden of showing that on you; volume alone is not excessiveness. Recital 63 records that the right of access should not adversely affect the rights of others, including trade secrets, and that where a controller processes a large quantity of information it should be able to ask the data subject to specify what the request relates to — a recital, therefore an aid to interpretation rather than an operative provision, and not a basis for withholding by itself. The related question of what happens when the request concerns a model fine-tuned on one customer’s data is taken up on data subject requests against a fine-tuned model.
Top comments (0)