TL;DR: ship a runbook, a client-owned eval set, pinned versions with a rollback, a named alert owner and a readable decision log.
An AI agent works on delivery day. What decides whether it still works six months later is who owns it afterwards, and whether that person has what they need. Handover is part of the build, not a meeting at the end.
What changes when nobody touches it
Agents decay without anyone editing them. The model provider retires a version, and the alias you relied on now points at something else. An upstream API renames or drops a field. The documents the agent reads drift away from the ones it was tested on: new templates, new product codes, a different way of writing the same thing. None of this shows up in a code review, because no code changed. It shows up as a mistake in front of a customer, usually noticed first by someone who does not know why it happened.
Five things that leave with the system
The first is a runbook. It says what the agent is allowed to do, what it must never do, which systems it can write to, and what to do when it behaves strangely. It is written for the person on call, not for the developer who built it.
The second is the evaluation set, owned by the client. If the questions and the expected answers live only on the vendor's laptop, the client cannot check anything after a change. With the set in their hands, they can re-run it after a model update, a prompt edit or a new batch of documents, and compare with the last result.
The third is pinned versions with a written rollback. Model, prompt and index are recorded as exact versions, and the way back to the previous combination is a documented step, not a memory.
The fourth is a named recipient for alerts, and it is not the person who built the system. An alert that goes to a mailbox nobody reads is the same as no alert.
The fifth is the decision log, with a way to read it that does not depend on the supplier. When someone asks why the agent did something in March, the answer should not require a phone call to the vendor.
Who gets the first message
Consider the first model deprecation notice. If the eval set and the pinned versions are in the client's hands, it is a calendar item: schedule a test, compare, switch. If they are not, it is an emergency that lands on whoever built the system, at a time they did not choose. That is also a cost problem for the supplier, because unlimited informal support is rarely in the price.
Maintenance is a design constraint
A support contract added after delivery cannot fix a system that was built to be understood only by its authors. The constraint has to be there from the first sprint: what will the client be able to run, check and change without us? Designing for that makes the build slightly slower and the following years considerably calmer.
When your agent breaks in a year, who gets the first message?
Full text: the full handover checklist
Articolo originale: https://www.omarbaruzzo.it/en/blog/agente-ai-dopo-la-consegna
Top comments (1)
I’d make the client demonstrate the handover once with the supplier unavailable: notice a failed run, pause new actions, inspect what already happened, and restore a supported version using their own access. That tests whether the runbook is usable. Rolling back the model or prompt won’t undo an email or payment already sent, so reconciliation belongs in the drill alongside the rollback.