Originally published at Trida AI.
The most expensive sentence in our line of work is "we've done this before." Said loosely, it means somebody on the team remembers a similar project. Said properly, it should mean something you can point to: a shelf of artifacts left over from the last deployment, which the next one starts from. The distance between those two meanings is where forward deployed engineering either becomes a business or stays a run of heroic one-offs.
A first deployment is discovery at full price. The team learns what the client's data actually looks like, which approvals really matter, where the workflow breaks on a bad Tuesday. That learning is the most valuable thing the engagement produces, and the most perishable. If it lives in the heads of the engineers who did the work, it depreciates on a schedule nobody controls: people rotate, context fades, someone leaves. The second deployment starts close to zero, and the client pays again for lessons already bought once.
Our position is simple. Judge a deployment team by what the first deployment leaves behind. The residue is the product. But residue does not curate itself, and most of what a project leaves behind is not reusable. Knowing which parts are, and doing the unglamorous work of making them so, is the discipline this post is about.
What carries over
Failure catalogues carry furthest. The ways an AI system fails in production repeat across clients because they come from the machinery, not from the business: a source replaced after indexing, quoted as if current; an ambiguous instruction resolved silently instead of asked about; a tool result trusted after the thing it described had changed; an approval requested after the action, not before. A catalogue entry names the shape, says how the system detects it, and says what happens instead. The detection and the response transfer almost intact. The examples inside the entry do not — they get redrawn from the new client's own data, because the old client's data is not ours to carry, and because a failure seen only in someone else's examples is one you will not recognise in yours.
Eval sets transfer as skeletons. The question classes, the scoring rubric, the taxonomy of hard cases: those carry. The gold answers do not — what counts as correct is a business judgment, and it belongs to the business being judged. Starting from a skeleton still changes the second deployment's shape: a regression gate exists from the first week, in form, waiting for the new client's labels, instead of being invented under deadline pressure near the end, which is when eval sets usually get built, and why they so often get built badly.
Connector patterns transfer; mappings do not. How a class of systems behaves — its authentication flow, its pagination, how it signals rate limits, whether its writes are idempotent — is knowledge that survives the move to the next client running the same class of system. What a field means inside one company's estate does not survive, and pretending otherwise is how a second deployment imports the first client's data model by accident.
Review and tracing shapes transfer for a duller reason: the accountability questions are the same everywhere. What did the system do, on what evidence, who approved it, how do we undo it. The records that answer those questions can be rebuilt from a pattern, at a fraction of the cost of designing them from a blank page.
What the first deployment hands over. Artifacts move through a curated library; judgments stay in the room.
One catalogue entry, carried across. The check and the hold behaviour moved unchanged; the stamp and the staleness window were rewritten. Example content.
What has to stay bespoke
Data semantics stay. What "active customer" means, when a ticket counts as closed, which of three conflicting records wins: no library knows this, because it is not knowledge about systems. It is knowledge about one business, and it has to be learned in the room, every time.
Thresholds stay. How much uncertainty routes a case to a human, how stale a source may be before an answer is held: these are risk decisions, shaped by regulation, by the cost of an error, by how much review capacity the client actually has. Copying the first client's thresholds into the second deployment imports somebody else's risk posture and calls it experience.
Ownership stays. Who may approve, who gets paged when a catalogue entry fires, who owns the fix: a handoff designed around one organisation's shape lands wrong in another's, and the failure shows up at the worst moment, mid-incident, when the named owner turns out to be a team that does not exist here.
The working rule we use: transfer the shape of the work, re-decide every judgment inside it. If an artifact contains a mechanism, reuse it. If it contains a decision, it gets made again, in the new room, by the people who will live with it.
The work between deployments
None of this happens by accident at the end of a project. Between deployments, somebody has to strip the client specifics out of each artifact, rewrite the examples in generic form, attach a note saying where it does not apply, version it, and give it an owner who can retire it. That work is invisible and easy to cut under delivery pressure. Cutting it is how a library becomes a junk drawer: old code and old prompts nobody trusts enough to build on, kept because deleting them feels wasteful.
A template without its limits note is worse than no template. The next team applies it where it does not fit, and applies it confidently, because it arrived with the authority of a previous success. The limits note is what turns a souvenir into an instrument.
Reuse also has to be measured, or the library is a museum. Did the eval skeleton actually catch a regression, or did it just feel familiar? Did the connector pattern save days, or hours? Artifacts that stop earning their keep get retired.
There is a boundary to all of this. Move to a second deployment in a different industry and the compounding shrinks: the failure catalogue still helps, because failures are machinery-level; the eval skeleton helps less; the connector patterns may not apply at all. Compounding is strongest inside a family of similar problems, and a team claiming otherwise is selling something. There is an opposite failure too, quieter than the junk drawer: a team so pleased with its library that it reaches for a template before it has understood the client in front of it. The artifacts should shorten discovery. The moment they replace it, the second deployment gets worse than a first one would have been, because at least a first deployment knows it is ignorant.
So the question to ask any deployment team, ours included, is not how many deployments it has done. It is what the last one left on the shelf, who owns it, and what it proved the next time it was picked up. Experience that cannot be handed to another engineer is a rental. Artifacts can be handed over.
Work with us
If you are planning a first AI deployment, plan it so the second one does not start from zero: evals, failure catalogues and integration patterns captured as assets from day one, with the judgments kept where they belong, in your room. Tell us about the deployment you need through Trida AI's intake and we will show you what we would build to be reused, and what we would insist on deciding fresh with you.
Top comments (0)