Deployer Due Diligence Before Buying a High-Risk AI System

#ai #machinelearning #tutorial #llm

Article 26 of the EU AI Act puts a set of duties on the deployer of a high-risk AI system that can only be discharged with artefacts and capabilities the provider controls. Every one of those is easier to obtain before the contract is signed than after an incident, and this is the list.

Why this is a pre-purchase exercise

The asymmetry is structural. Your obligations as a deployer are owed to a market surveillance authority and to the people the system is used on. The provider’s obligations are owed to you under Article 13, and to the authority under Articles 16 and 47. Nothing in the Regulation makes the provider hand you evidence on demand after the fact at the level of detail an investigation wants; what it does is require the system to be accompanied by instructions for use containing specified content. If the instructions are thin, your remedy is contractual, and you have no leverage to negotiate a contract you have already signed.

This page is a procurement aid, not legal advice, and it assumes the system is high-risk under Article 6. Whether it is — and whether the Article 6(3) derogation applies — is a legal question about your specific use, and getting it wrong in the permissive direction is the expensive error. Primary text: Regulation (EU) 2024/1689 on EUR-Lex.

Timing: the Chapter III obligations for Annex III high-risk systems apply from 2 August 2026, with the Article 6(1) product-embedded cases following on 2 August 2027. A procurement running now is buying something that will be inside the regime for most of its service life.

What you will owe as a deployer

Article 26, condensed to the duties that generate document requirements:

Use the system in accordance with the instructions for use, with appropriate technical and organisational measures. You cannot do this if the instructions do not describe the operating envelope.
Assign human oversight to natural persons with the necessary competence, training and authority, and with support. The oversight measures the system is built to support are specified by the provider under Article 14 and described under Article 13.
Ensure input data is relevant and sufficiently representative for the intended purpose, to the extent you control the input. You need the provider’s input data specification to know what “relevant” means here.
Monitor operation and, where you identify a risk within the meaning of Article 79(1) or a serious incident, inform the provider or distributor and the relevant market surveillance authority and suspend use. This needs a named contact and an agreed channel, not a support portal.
Keep the automatically generated logs to the extent they are under your control, for a period appropriate to the intended purpose and at least six months unless another rule says otherwise. Logs you cannot export are logs that are not under your control.
Where you are an employer, inform workers’ representatives and affected workers before putting the system into service in the workplace.
Where the system is used to make or assist decisions about natural persons in Annex III cases, inform those persons.
Use the Article 13 information to carry out a data protection impact assessment under Article 35 GDPR where one is required, and a fundamental rights impact assessment under Article 27 where you are within that provision’s scope.

The documents to demand

Article 13(3) sets out what the instructions for use must contain. Ask for them by that structure and a gap becomes visible immediately rather than in an audit:

Article 13(3) instructions for use — request as a numbered response
 (a) provider identity and contact details
 (b) i.   intended purpose (verbatim, as assessed)
     ii.  level of accuracy + the accuracy metrics used
          robustness and cybersecurity levels tested against
     iii. known or foreseeable circumstances that may lead to
          risks to health, safety or fundamental rights
     iv.  technical capabilities to explain the output
     v.   performance in respect of specific persons or groups
     vi.  input data specifications / relevant training,
          validation and testing data information
 (c) predetermined changes to the system and its performance
 (d) human oversight measures per Article 14, incl. the technical
     measures to help deployers interpret output
 (e) computational and hardware resources, expected lifetime,
     maintenance and care measures incl. software update frequency
 (f) description of the log collection, storage and interpretation
     mechanisms (Article 12)

Alongside those, ask for the EU declaration of conformity under Article 47, evidence of CE marking under Article 48, and the system’s entry in the EU database under Article 49 — which you can verify independently rather than take on trust. See the Article 13 page for what “concise, complete, correct and clear” has been taken to mean.

The checklist

Fix the intended purpose in writing first. Write down what you will use the system for, in one paragraph, before you read the provider’s materials. Then compare it to the intended purpose in the instructions. Any daylight between the two is the most important finding of the whole exercise, because it is the trigger for Article 25 and for you becoming a provider.
Request the Article 13(3) response in the structure above. Not a brochure, not a model card. If the provider offers a model card instead, accept it as supporting material and keep asking — see what a model card is and is not evidence of.
Verify accuracy claims are metric-bearing. A declared level of accuracy without a named metric and a stated test population is not an Article 13(3)(b)(ii) disclosure. Ask what population it was measured on and whether it resembles yours.
Test the human oversight story against a real operator. Article 14 requires the system to be designed so oversight is possible; Article 26(2) requires you to staff it. Have the person who will actually do it look at the interface and say whether they could overrule it, and what they would see if they should.
Prove log export before signing. Ask for a sample export in the format you would receive, confirm the retention period available to you, and confirm you can hold it for at least six months independently of your subscription. A logging capability that only exists inside the vendor’s console fails Article 26(6) the day you terminate.
Agree the incident channel and the clock. Name the recipient, agree a notification period that lets you meet Article 73 timelines where the reporting duty runs through you, and record it in the contract. See serious incident reporting.
Secure the assistance obligation contractually. Even where Article 25(4) applies upstream, write the cooperation duty into your own contract: information, technical access and assistance on request, surviving termination for the retention period.
Run the DPIA and, where in scope, the FRIA now. Both are inputs to the buying decision, not paperwork after it. The questions to put to the vendor for the DPIA are in the DPIA vendor question list.
File the evidence pack as it arrives. Dated, with the version of the system it describes. Reconstructing which document described which release is the failure mode that makes an otherwise compliant deployment undefendable.

Three traps

The rebrand trap. Putting your own name or trade mark on the system makes you the provider under Article 25(1)(a), with the whole of Article 16 attaching. White-labelling a high-risk system is a decision with a regulatory price, and procurement teams make it for brand reasons without pricing it. See the Article 25 integrator page.

The configuration trap. Deployers routinely tune thresholds, add rules on top of an output, or feed the system a different population than it was assessed for. Depending on the change this can be a substantial modification, and the line is not bright. Record configuration changes with reasons; the record is what lets you argue the change was foreseen in the provider’s assessment.

The pilot trap. A pilot with real people and real decisions is a deployment. The obligations do not wait for the rollout, and running a “trial” without the human oversight staffing or the affected-person notice is the most common way an organisation finds itself in scope without having decided to be.