DEV Community: FDE at Trida AI

IBM and Microsoft Now Ship Engineers With the AI

FDE at Trida AI — Sat, 14 Nov 2026 15:40:57 +0000

Originally published at Trida AI.

Talk to enough enterprise AI programs and the same shape keeps turning up: licenses bought, pilots launched, a platform team stood up, and a production list that stays stubbornly short. The model is rarely the blocker. The blocker is the middle of the work — wiring an agent into the systems, data and controls it has to live inside, in a business where a wrong move has a regulator attached to it.

On November 10, IBM and Microsoft announced Imagine Enterprise, a joint platform for building, running and governing AI agents that execute industry-specific processes. The headline number is IBM Consulting's library of more than 6,000 pre-built AI agents and assets. The quieter line matters more: the platform comes with forward-deployed engineering support (engineers embedded with the client's own teams) from both companies, for qualifying engagements.

We do the same job for a living, so read what follows as practitioner notes on a competitor-scale move, not a neutral market survey.

What ships in the bundle

Read the announcement as a parts list. IBM Consulting brings the agent library and the industry expertise behind it, built on IBM Consulting Advantage, its delivery platform. Microsoft brings the foundation the agents run on: Foundry, GitHub Copilot and Copilot Studio to coordinate them, Fabric and Microsoft 365 for data and daily work, Entra, Purview and Defender for identity, governance and security, all of it on Azure. The first industry frameworks are financial services, insurance and retail — the first two of them regulated ground where deployment is hardest. Governance is part of the pitch rather than an afterthought: controls, observability and compliance support are built into the platform, which is what regulated buyers ask about first. IBM's own write-up adds that the engineers are co-embedded with client teams, working from co-design through to production.

The Imagine Enterprise bundle, by provider: IBM Consulting's agents and frameworks, Microsoft's foundation, and forward-deployed engineers from both companies working inside the client's estate.

Two companies that sell software at extraordinary margin have decided the software does not land on its own. Nobody bundles people into a product because people are cheap. They bundle them because deals stall without them — agents that demo well, then sit in pilot while the integration, the controls and the change management go undone. Enterprise AI is not short of models. It is short of deployment capacity: engineers who can take an agent from a slide to a governed process inside a real estate. The work itself is unglamorous and specific. Sit with the team that owns the process. Connect the agent to the systems it has to touch. Write the evals that say whether it is behaving. Stay until it runs with nobody from the vendor in the room. IBM and Microsoft have now said as much in a press release, which is a more honest signal than a keynote.

Whose engineers are they?

None of that makes the bundle a bad buy. If your estate already runs on Microsoft, a deployment team that knows the stack from the inside may be the fastest route to production you can purchase. But be clear about what a vendor's forward deployed engineer is optimizing for. Their knowledge is deep and deliberately narrow: the asset they adapt is the vendor's asset, the roadmap their work feeds is the vendor's roadmap, and when the engagement ends, the working knowledge of why your agents behave the way they do leaves with them. The question for the room is not "are the engineers good?" It is "what do we own when they go — the adapted agents, the evals, the context they built up about our business?"

An independent deployment team's incentives point the other way. It can tell you a process should not become an agent yet, or that the platform you already own is enough and the real gap is context and evals. That advice is worth paying for precisely because nobody's license revenue depends on the answer. It is also why this category keeps widening: once one seller bundles engineers, every competitor has to answer for deployment, not for features.

A deployment brief for one process, written the way a pod would scope it on site. Example content; the bank, the process and the rough edge are illustrative.

There is also a gate in the fine print. The engineering support is for qualifying engagements, and in enterprise software that means the largest deals get the scarce people first. Everyone below that line gets the platform and the library. The deployment problem does not shrink for a mid-size bank or insurer. Only the supply of bundled engineers does. Companies in that position need the same work done by their own hires, or by engineers whose only product is the deployment itself.

What the announcement does not prove

Hold a launch to the standard any launch should meet. No pricing was published. The one customer named, SEI, is described as exploring the technology with the two companies' support — exploring, not running the platform in production. And the 6,000 figure counts agents and assets together, so read it as a starting library of uneven depth, not 6,000 finished workers. For now the platform is also sold one way: through an IBM Consulting engagement, with self-management listed as an option rather than the main route. None of this argues against the direction. It is a reminder that a bundle announcement is a claim about supply, and the proof arrives later, in deployments that stay up.

The direction itself settles an argument. For two years the industry's pitch was that the model was the product. Two of the largest enterprise software sellers have now priced in the opposite conclusion: the product includes the people who put it into production. Whoever supplies your deployment engineers, budget for them as a first-class line, not as whatever is left after the licenses.

Work with us

If your program has licenses, pilots and a production list that is still short, the missing line is usually deployment engineering. Send us the process you need running in production through Trida AI's intake, and we will tell you plainly what it will take to get it there.

The Reranking Bake-Off

FDE at Trida AI — Sat, 14 Nov 2026 12:54:17 +0000

Originally published at Trida AI.

Somewhere in our own archive sits a paragraph explaining why a reduced answer has to be labelled as reduced. Ask that archive a plain question — why must a reduced-quality answer be visibly marked as reduced? — and ordinary vector search puts the paragraph nineteenth. Nineteenth might as well be nowhere. A generation step reads the top few chunks, not the top twenty, so the answer gets built without the one chunk that answers it. The paragraph exists. The index works. The retrieval still fails where it counts.

That gap, between retrieved and actually read, is what a reranker is sold to close. Vendor numbers for it come from public benchmarks, which tell you little about your corpus. So we ran the comparison on the one corpus we can publish without asking anyone's permission: our own posts.

The rig

The corpus is 36 published posts from this blog, body text only, split into 230 chunks with a median length of 165 words. The question set is 24 questions, written and frozen before any retrieval ran. Each question has exactly one chunk that answers it, pinned by a phrase that occurs in that chunk and in no other chunk of its post. Eight questions are paraphrases, deliberately reworded so they share little vocabulary with their answer. Eight are exact, carrying a distinctive term or figure from the chunk. Eight are confusable: the answer lives in one post, and a sibling post covers a neighbouring topic in similar words.

Stage one is vector search with all-MiniLM-L6-v2, a bi-encoder that maps each chunk to a 384-dimensional vector; the question is embedded the same way and ranked by cosine similarity. Stage two takes stage one's top 20 and rescores each pair with ms-marco-MiniLM-L-6-v2, a cross-encoder. The difference is mechanical, and the Retrieve & Re-Rank pattern in the Sentence Transformers documentation states it plainly: a cross-encoder receives the query and the document together and outputs a single relevance score, so attention can cross between the question's words and the chunk's words. A bi-encoder never gets that look. It commits each chunk to one vector before it knows what will be asked.

We scored both stages on whether the answer chunk reached rank 1 and the top 5 (recall), and on mean reciprocal rank at 10. Latency was measured on the small shared CPU that ran the test, no GPU involved. Treat our milliseconds as a ratio between the stages, not as a spec sheet. One thing we did not measure: generated answers. This test scores the gate, not what walks through it. If the answer chunk is in the context, a competent model has what it needs; if it is not, no prompt will save the answer.

One question, two stages. Vector search narrows 230 chunks to a top 20; a cross-encoder rescoring decides the top 5 the generation step reads. The vector-only path skips the second stage. Stage medians are from this run.

What the second stage bought

Vector search alone put the answer chunk first on 13 of 24 questions and inside the top five on 19. Adding the reranker moved those to 18 of 24 and 22 of 24. Mean reciprocal rank went from 0.65 to 0.82. On this corpus, in aggregate, the second stage is a clear win.

The bill is the other half of the result. The vector stage's median was 14 milliseconds per question, query embedding included. Rescoring 20 pairs took a median of 3.6 seconds on the same machine, roughly 250 times as much. Production hardware shrinks both numbers; it does not change the shape. One stage is a lookup against vectors computed in advance. The other is twenty pair scorings per question, every question, forever. A reranker is not a setting you turn on. It is a per-query cost you agree to pay.

The rescues

Paraphrase questions are where the money showed. The answer chunk reached the top three on 4 of 8 under vector search and on 8 of 8 after reranking. The buried paragraph from the opening went from rank 19 to rank 1. That is the cross-encoder's mechanism doing exactly what it promises: the question and the chunk share few words, but read together, they plainly match.

Exact questions improved too, which we did not expect. The chunk naming the three properties in Meta's Agents Rule of Two sat at rank 12 under vector search, because much of the corpus talks about sessions, tools and permissions in the same vocabulary. The reranker put it first. A chunk naming the benchmarks in a retrieval paper went from 8 to 1. Where a question carries a rare term, the embedding seems to spread its vote across every chunk that shares the surrounding topic; reading the pair directly concentrates it.

Where it lost ground

The confusable class went backwards. First place fell from 6 of 8 to 5 of 8, and mean reciprocal rank from 0.77 to 0.66. Three runs show the shape of the problem.

Three questions from the run sheet. The degraded-mode question was rescued, from vector rank 19 to reranked rank 1; the receipts question was demoted, from 7 to 12; the circuit-breaker question's answer chunk sat at rank 21, outside the top-20 window the reranker was shown, and was never scored.

On the receipts question (what must be attached to a claim so a reader can check it), the answer chunk fell from rank 7 to rank 12. Look at the reranked top five and little seems wrong: three of its five chunks come from the receipts post itself, and the other two from its nearest neighbours. At document level the system looks healthy. At chunk level, the one chunk that answers the question dropped out of reading range, and chunk level is the level a generator consumes. On the cancelling question, a chunk vector search had ranked first fell to fifth. A general-purpose scorer, trained on MS MARCO passage ranking, has no way to know the distinctions our neighbouring posts draw between similar machinery. Faced with five plausible chunks, it picks by surface fit, and surface fit is what made them confusable in the first place.

The third failure is structural, and no reranker can fix it. On the circuit-breaker question, the answer chunk sat at rank 21, one place below the candidate window. The reranker never saw it. A second stage reorders what the first stage fetches; it cannot rescue what was never fetched. Across the set, vector search placed 23 of 24 answer chunks inside its top 20. That missing one belongs to the first stage's budget: a wider candidate window, hybrid search, or better chunking, not a better reranker.

What we would ship

Rerank when your questions arrive as paraphrases, which is most human traffic: support queues, internal search, anyone asking in their own words rather than the document's. That is the class where the second stage paid for itself several times over here.

Score per question class, never only in aggregate. Our aggregate says reranking wins. Our confusable class says it ships a regression on exactly the questions where two documents disagree, which are usually the questions worth answering carefully. An aggregate-only eval would have hidden that completely.

Widen before you rank. Candidate depth sets the ceiling for everything downstream, and the one total miss in this run was a depth failure. If you can afford only one improvement, buy first-stage recall before you buy a larger reranker.

And keep the latency honest in the design doc. If your p95 cannot absorb a second stage, the answer is not to pretend the cost away; it is to spend the same money on the first stage and accept the ranking you get.

The limits of this test

Twenty-four questions is small. One question is worth about four points of recall, so differences of one or two questions are noise; we trust the class pattern, not any single number. The corpus is our own prose, 36 posts on neighbouring topics, which makes the confusable class denser than most enterprise collections and the vocabulary more consistent. The method travels. These specific numbers describe this corpus, not yours. Both models are general-purpose, and neither was trained or tuned on this corpus; a larger or domain-tuned reranker may well handle the confusable class better, and that is a test worth running on your own text rather than assuming. Finally, we scored strictly, against one gold chunk per question, even where a sibling chunk would partly answer. That understates both systems, equally.

Work with us

If your system keeps answering from the wrong chunks, the place to start is not the prompt or the model. It is a measurement: which chunks your questions actually need, and where those chunks rank today. Bring the corpus and the questions it fails through Trida AI's intake and we will run that measurement with you.

Compile the Knowledge at Index Time

FDE at Trida AI — Thu, 12 Nov 2026 12:12:25 +0000

Originally published at Trida AI.

Ask a document store the same question on Monday and again on Thursday, and watch what the system does. It embeds the question, pulls roughly the same chunks, and hands them to a model that has to work out, from scratch, which sentences in those chunks actually answer it. Nothing Monday's run figured out survives to help Thursday's.

That repeated work is what RAGFlow 1.0 is built to remove. InfiniFlow now positions the open-source engine as a supply layer for retrieval agents rather than an end-to-end RAG pipeline, and the mechanism behind the repositioning is knowledge compilation: move the understanding from query time to index time. Compile once, when a document arrives, and store the result as artifacts the agent can use directly, instead of re-deriving the same understanding inside every prompt.

What compilation stores

Compilation produces several artifact types from the same documents. There is a navigation tree over chapters and sections. There is an entity graph, timelines for collections that change over time, and wikis an enterprise can choose to generate. And there are atomic propositions, the piece the rest of this post is about.

A proposition is a one-sentence fact extracted from the source, paired with a verbatim quote. The quote has to be a contiguous substring of the original text, and that is checked mechanically. A proposition whose quote does not match the source word for word does not count as evidence.

The constraint sounds fussy. It is the design. A summary can drift a little each time it is regenerated. A paraphrase can quietly upgrade a "may" into a "will". A verbatim quote can do neither without failing its check, so a compiled proposition can be cited the moment it is retrieved, with no second pass back through the passage to confirm what it said. Each artifact also records its lineage, which document and which passage it came from, and when a document is deleted, its compiled contributions are removed with it, rather than lingering in the store as facts nobody can trace.

One paragraph, compiled. Each proposition carries a quote that must match the source word for word; the one that cannot is never stored. Example content.

The tree was not the point

The finding that makes this release worth reading closely is one the team published against its own product story. Organizing documents into trees, on its own, did not help. In the accompanying paper, the authors report that structural artifacts gave no detectable accuracy gain on the multi-hop FRAMES benchmark under a competent retrieval harness, and that an outline-style index degraded at scale. That should not surprise anyone who has watched a good agent work a corpus. Given search, reading, and the freedom to reformulate, it finds the right region anyway. A prettier map of that region adds little.

What moved the numbers was attaching claim-evidence pairs to the tree's leaves. When the evidence at a leaf is sufficient to answer, the agent answers from the evidence and skips reading the raw chunk entirely. The paper reports that this cuts query tokens by 91% when the evidence suffices and adds 1.9% when it does not — a 38% average reduction, at a measured sufficiency rate near 43% against a break-even of about 2%.

Read those as the team's numbers, from the team's runs: a harness built on BM25 and grep-style search with a single model, MiniMax M3, measured on BrowseComp-Plus and FRAMES. They are not independent benchmarks, and nobody should plan capacity around the decimals. The shape matters more than the figures. A compiled hit is cheap to use. A miss costs almost nothing, because the system checks the evidence before the model starts searching and, if the set is not enough, discards the whole block and searches as it would have anyway. Compilation does not have to win often to pay for itself. It has to win occasionally and lose cheaply.

The discard rule deserves its own paragraph, because it runs against instinct. Retrieved material usually gets stuffed into the prompt just in case, where every later round pays to carry it and the model pays to ignore it. Here, insufficient evidence is assigned no citation numbers and never enters the context at all. Partial evidence treated as partial credit is how confident, half-grounded answers get built. This design treats it as no evidence, which is the more honest accounting.

Where the understanding work happens. Chunk-and-embed pays it inside every query; compilation pays it once at ingestion and stores the result as evidence.

Where compilation stops paying

Compilation is a bet on reuse, and bets have losing sides. A corpus that gets asked once, or asked in ways that never repeat, pays the index-time cost for artifacts nobody retrieves. Long-tail archives with cold traffic are usually better left raw, with a strong search harness over them.

Freshness is the second cost. A compiled proposition is derived data, and derived data goes stale on the source's schedule, not yours. RAGFlow rebuilds compilation as an idempotent full recomputation — cleaner than half-updated structures, and a real compute line item on a large, fast-changing corpus. Anyone adopting this pattern owes their users a compile timestamp they can see, because "the document changed Tuesday; the evidence was compiled in October" is a failure mode that looks exactly like a correct answer until it isn't.

Third, the verbatim check proves less than it appears to. It guarantees the quoted words exist in the source. It does not guarantee the extractor picked the sentence that matters, or that a true sentence answers the question being asked. Extraction here can run on a small model, which keeps indexing cheap; it also means the quality ceiling of the whole store is set by a component nobody demoed. The sufficiency gate guards the answer side of that problem. Nothing guards the extraction side except testing the compiled store the way you'd test any other index: with real questions, before users do.

And buy it for the right reason. By the team's own measurement, compilation did not make answers more accurate under a strong harness. It made them cheaper and easier to trace. A team whose problem is accuracy has a different problem, and spending its budget on compilation will leave that problem exactly where it was.

None of this is an argument against compiling. It is an argument for compiling the corpora that earn it: heavily queried, slow to change, and expensive to read — policy libraries, product documentation, regulatory collections — and leaving the rest to search. The interesting shift in this release is not that one vendor added a feature. It is that the unit of retrieval is changing, from a passage a model must interpret under time pressure to a checked fact it can simply use. Systems built around the second unit will be designed differently from the ingestion pipeline outward, and most production RAG stacks were not.

Work with us

If your retrieval stack pays full reading costs on every question, the same documents understood from scratch, run after run, we can help you work out which of your corpora would earn compilation and which should stay raw. Start with Trida AI's intake and bring the question your users ask most often; that is where the arithmetic begins.

Good Context Is Built Before the Run Starts

FDE at Trida AI — Thu, 12 Nov 2026 03:06:57 +0000

Originally published at Trida AI.

Ask an agent in June who owns the Acme renewal and it may well say Priya. Priya handed the account to Daniel in March. Both facts were in the material the agent searched — the March handover note and the older account record, side by side, neither marked as finished. The model picked the wrong one, and nothing in the context it received said which statement had died.

Nothing failed in the model. Retrieval worked; both statements were genuinely relevant. What was missing was a record of the change itself: which fact is current, when the old one stopped being true, and where each came from. Keeping that record is a job of its own, and over the past year it has become a product category.

The category goes by context engineering. Zep, founded in 2023 and out of Y Combinator's Winter 2024 batch, calls itself the context engineering platform for AI agents and sells a unified context layer: company data in, governed context out. Mem0 works the narrower memory slice of the same problem and has distribution most startups would envy: AWS picked it as the memory provider for its agent SDK, and the company has raised $24 million. Letta, grown out of the MemGPT research at UC Berkeley, treats memory as something the agent maintains itself. And on November 6, Anthropic published The components of agent memory, its own architecture for what an agent keeps between sessions.

Different companies, different shapes. But the layer is distinct from the two things it sits between. It is not the model. It is not the vector database most teams reach for first, either. A vector store keeps chunks of documents, while this layer keeps facts: extracted, dated, and corrected as the company changes underneath them.

The work moves to background time

The older pattern assembled context during the run. A question arrived, the system searched raw material, and the model sorted signal from noise inside the prompt, under time pressure, at run-time prices. The new pattern does the understanding earlier — at ingestion, or in background passes between runs — so the run starts with context that has already been sorted.

Anthropic's name for the background pass is dreaming; the post credits the idea to earlier sleep-time compute work from Letta and UC Berkeley. Between sessions it rereads transcripts, tests notes against what actually happened, corrects the wrong ones, merges duplicates and builds a linked wiki over the material. On an internal corpus of about 350,000 files, Anthropic reports that answers drawn from the wiki alone met 13% of its grading criteria after two dreaming sessions and 48% after 128. Search and wiki together reached 61%, against 53% for search alone. Those are Anthropic's numbers, measured on Anthropic's corpus. Take them as a direction, not a benchmark. The direction is hard to argue with: the wiki kept getting better while nobody was querying it.

The discipline shows in the small print. Claude Code's auto memory loads only the first 200 lines or 25 KB of its index file into a session. The part of the layer that loads every time has to stay small enough to deserve it.

We've written before about what one agent should bother writing down. This layer is that instinct, industrialized: the note-keeping moves out of the agent and into infrastructure the whole company shares.

The compile-ahead pattern. Extraction, compilation and correction happen between runs — Anthropic calls the pass dreaming — so a run makes one retrieval and receives an assembled context block with validity dates and provenance already attached.

Zep's engine is Graphiti, an open-source framework (Apache 2.0) for temporal knowledge graphs. A fact there is a relationship between two entities, and it carries two timelines (when it was true in the world, and when the system learned it), four timestamps in all. When a new fact contradicts an old one, the old fact is invalidated rather than deleted: its end date is set, and it stays in the graph, queryable as history. Underneath, search mixes semantic embeddings, keyword matching and graph traversal, and what the agent receives is one assembled context block with each fact's validity dates attached, instead of a pile of chunks to adjudicate.

The same idea is arriving from the retrieval side. RAGFlow 1.0 calls it knowledge compilation: at ingestion, documents are translated into atomic propositions (one-sentence facts, each pinned to a verbatim quote from the source), so a question can later be answered from compiled evidence instead of a fresh excavation of the corpus. The team reports that a question answered from compiled evidence can save over 90% of its tokens. Compilation is the right metaphor. Source text goes in, a checked and cited artifact comes out, and the expensive thinking happens once, ahead of time.

One fact changes hands

Architectures are easiest to compare on an ordinary change. In March, a message in the account channel says Daniel takes over the Acme renewal from Priya. Three systems ingest that message. Here is what each one stores, and what an agent working from each one sees in June.

One fact changes hands in March: the Acme account's renewal owner moves from Priya to Daniel. Zep closes the old fact with a date and keeps it as history, Mem0 rewrites its extracted list in place, and Letta's agent edits its own memory file, with the edit kept as a versioned change.

All three agents answer "Daniel." The difference is what else survives. Zep's agent can also say who owned the account in February and when that changed, because the old fact was closed, not erased. Mem0's list holds the current truth with the least machinery around it; the record of the change sits behind the list, in the platform's own history, rather than in front of the agent. Letta's agent holds the file itself, and the March edit is a versioned commit, so the audit trail is the file's own past. Same answer in June. Three very different answers to the question a compliance team asks in September: show us what the agent believed, and when it stopped believing it.

Buy on correction mechanics, not benchmarks

The category's numbers should be handled with tongs. On LOCOMO, the benchmark this field argues about most, Zep has reported 84%. Mem0 re-ran the evaluation and reported 58.44% for Zep's system; Zep answered with a rebuttal saying the setup was misconfigured, and its own re-run put the figure at 75.14%. Nobody independent has settled it, and the gap between the claims is wider than most of the effects being argued over. Every vendor here, Anthropic included, grades its own homework.

So judge the layer the way you would judge a records system. When a fact changes, is the old one closed with a date, or does it linger? When the agent states a fact, can you walk it back to the source it came from? Who is allowed to see which context, and is that enforced or merely promised? And when the layer gets something wrong (it will), what does a correction look like, and does it reach the next run?

None of this makes the layer free. A compiled context is one more system to run, and background passes burn compute whether or not anyone asks a question. In one of Anthropic's tests, the wiki grew larger than the documents it summarized — a map the size of the territory, abstracting nothing. A wrong fact also behaves worse in a maintained layer than in a raw pile. A stale chunk might never be retrieved. A wrong note promoted into every session gets read every time.

That is where this stops being a tooling choice and becomes deployment work. In an enterprise rollout, the model is the part most likely to be swapped next year. The context layer — the client's facts, their corrections, their provenance — is what the next model inherits. Building it means deciding which systems count as sources of truth, which changes need a person's sign-off before they land, and which facts expire. Unglamorous maintenance, and it compounds: every correction makes the next run cheaper to trust.

Work with us

If your agents are answering from data that changed last quarter, the fix is a layer, not a longer prompt. Bring us the systems your agents read from — Trida AI's intake is where that conversation starts.

Circuit Breakers for Agent Tools

FDE at Trida AI — Wed, 11 Nov 2026 12:14:59 +0000

Originally published at Trida AI.

The dangerous moment in an agent run is not the first failed tool call. It is the fortieth.

A person whose lookup tool times out twice tries something else, or goes and makes tea. An agent does neither. It replans, and the new plan starts with the same lookup. It rewords the arguments and calls again, because from where it sits, each attempt looks like a fresh idea. The retries built into the tool layer add their own attempts underneath. None of these callers is misbehaving. Every one of them is doing exactly what it was told to do, which is to get the information and finish the task. Together they aim a steady stream of calls at a service that has already said, several times, that it cannot answer.

That stream is not harmless. Google's SRE book defines a cascading failure as a failure that grows through positive feedback: part of a system fails, the load shifts onto what remains, and the remainder becomes more likely to fail in turn. A failing tool under agent traffic fits the pattern neatly. The calls hold threads, connections and time. The service spends its shrinking capacity telling callers no, slowly. And the run itself burns its budget on attempts that were settled before they were sent.

Retries answer a different problem. A retry assumes the failure might be momentary, and sometimes it is. What an agent deployment needs alongside retries is a way for the route to a tool to remember that the last several calls failed, and to stop sending new ones for a while. That memory has a name in distributed systems, and it is older than agents by decades.

A breaker gives the route a memory

The Circuit Breaker pattern in Microsoft's Azure Architecture Center puts a proxy in front of an operation that might fail. The proxy counts recent failures. While the count stays below a threshold set for a time window, calls pass through — the closed state. Cross the threshold and the proxy opens: calls fail immediately, without being sent, and a timer starts. When the timer expires, the proxy turns half-open and lets a limited number of calls through as probes. If they succeed, the breaker closes and the counter resets. If one fails, it opens again and the wait restarts.

Two details in that design do most of the work for agents. The first is that an open breaker answers at once. A call that would have taken eight seconds to time out now takes milliseconds to refuse, and the refusal itself is information the agent can act on. The second is the half-open probe. Recovery is tested by a trickle, not by the full weight of every waiting run arriving in the same second, which can knock a recovering service straight back down.

Put one breaker on the path to each tool, inside the gateway or runtime the calls already pass through — never inside the model's discretion. The model should not decide whether a tool is healthy any more than it decides whether its own retries happened. Health is a counted fact about recent calls, and counting belongs in code.

One breaker per tool, on the path every call takes. Each breaker counts only its own tool's failures, so the open records lookup leaves search and email working. The agent reads the state; it does not set it.

The open state is a tool result

Here is where an agent deployment has to go further than the classic pattern. A conventional caller receives an exception and its code decides what happens next. An agent receives whatever the tool layer hands it, and reasons about it in language. So the open response has to be written for that reader.

It should say which tool is unavailable, when it is worth trying again, and what to do in the meantime. Park this step and continue with the parts of the task that do not need it. Work from the records already fetched. Or stop and report, if the missing call is the task. Given that, a competent agent does something sensible with the refusal. Given a bare timeout error repeated ten times, it will mostly produce an eleventh attempt in different words.

The retry layer has to respect the same signal. The Azure guidance makes the point directly: a retry policy invoking an operation through a breaker should stop when the breaker indicates the fault is not a passing one. If retries keep firing underneath an open breaker, the breaker is decoration.

Keying matters just as much. Give each tool its own breaker, and split further where a tool fronts genuinely separate resources — by destination, by shard, by customer environment. The Azure guidance warns against merging independent providers behind one breaker: one sick shard can block healthy ones, and the failures you do count stop describing anything real. Count the failures that implicate the tool, too. A call rejected because the agent supplied a malformed argument is the agent's fault, and letting it trip the breaker teaches the system that a bad prompt is an outage.

One outage, as the run recorded it. Example log, with example settings (a threshold of three failures in a minute, a ninety-second wait). Across the whole outage the service received six calls. The step that needed it was queued, not repeated, and the run spent the wait on work that was still possible.

Where a breaker stops helping

A breaker is a blunt instrument, and the bluntness cuts the wrong way often enough that the limits deserve stating plainly. The Amazon Builders' Library is candid about the trade. Circuit breakers are widely promoted for this problem, it says, but they introduce modal behaviour into systems: behaviour that can be difficult to test, and that can add significant time to recovery. Its own mitigation for runaway retries is plainer. Limit retries locally with a token bucket: calls retry while tokens remain, and when the bucket empties they retry only at a fixed rate. Amazon added that behaviour to the AWS SDK in 2016. For a tool that fails partially, answering most calls and dropping some, that gentler shape may serve better than a switch.

Breakers shared across many agent instances have their own trap. One fleet-wide breaker opening is at least honest information, but if every instance probes in the same instant when the wait expires, the probe becomes a small flood. Stagger the probes. And a breaker protects a tool, not a task: it cannot tell you whether the answer that finally comes back is right, only that the route to it is answering again. If the agent has no useful work to do while a breaker is open and no honest way to report the gap, all you have bought is a faster failure. The refusal, the alternative work, and the report are part of the design, not polish added later.

None of this is exotic. That is rather the point. Agents inherited the failure modes of distributed systems along with the tools, and the countermeasures transfer — as long as someone remembers that the caller, this time, will not get discouraged on its own. Persistence is the quality you bought. The breaker is how you aim it.

Work with us

If your agents keep calling tools that have already failed, the fix is rarely a cleverer prompt. It is a breaker on each tool's route, an open response the agent can reason about, and work it can do while it waits. Tell us where your runs stall through Trida AI's intake.

Compaction Is a Lossy Handoff

FDE at Trida AI — Tue, 10 Nov 2026 12:10:07 +0000

Originally published at Trida AI.

Every long agent run eventually forgets on purpose. The only real question is whether anyone decided, in advance, what it would forget.

That scheduled forgetting is compaction. Anthropic's context engineering guidance describes it as taking a conversation that is nearing the context window limit, summarizing its contents, and reinitiating a new window with the summary. In Claude Code, the summary keeps architectural decisions, unresolved bugs and implementation details while discarding redundant tool outputs, and the agent carries on with the compressed context plus the five most recently accessed files.

Read that example slowly, because the interesting part is the choosing. A summarizer looked at everything the run had seen and decided which of it the future deserved. The run that continues on the other side cannot ask what was cut. It cannot even know that something was cut. It has the summary, and from that moment the summary is the past.

That is why we treat compaction as a handoff rather than a compression setting. A handoff is a transfer to someone who was not there, and it fails in predictable ways: outcomes arrive without their reasons, exact figures get rounded into adjectives, and an open question comes back sounding settled. Compaction fails in exactly those ways, because underneath the mechanism it is the same act.

Write the keep list before you need it

You cannot brief the summarizer well at the moment the window fills. By then the context is crowded, the run is mid-task, and whatever instructions you bolt on are competing with the very material they are meant to sort. The keep list belongs in the design of the run, written while everyone is calm.

Ours is short. The goal, in the words it was given. Decisions, with the reason each one went the way it did. Open questions, still marked open. And exact values (case IDs, amounts, error strings, version numbers) copied character for character, because a summarizer paraphrasing an identifier produces a confident reference to something that does not exist.

Anthropic's tuning advice runs the same direction: start by maximizing recall, capturing everything that might be relevant, then iterate toward precision by cutting what proves superfluous. Err toward keeping too much on the first pass. You can measure and trim a summary that kept a useless paragraph. You cannot measure a fact that never made it across.

The provider machinery will not make these choices for you. Claude's server-side compaction fires when input tokens cross a trigger you set, 150,000 by default with a floor of 50,000. OpenAI's compaction works the same way server-side, and also offers a standalone endpoint that returns a compacted window you pass on as-is. Both answer the question of when. Neither knows what matters in your run.

Timing still deserves thought. Compact too late and the summarizer is working from a crowded, degraded history, summarizing the noise along with the signal. Compact too often and the losses stack, because the second compaction does not summarize your run. It summarizes a summary.

What the new window contains, and what stays outside. The full history is kept outside the window and stays fetchable; the summary carries pointers back to it, so a doubted claim can be checked against the original instead of trusted.

Keep the original within reach

A summary that replaces the only copy of the history is a single point of failure with a fluent writing style. Keep the full transcript and the run's artifacts outside the window, in a store the agent can read, and make the summary carry pointers: which tool result, which document version, which stretch of the log stands behind each kept claim. The pointer is what turns compaction from an act of faith into something auditable.

Start lighter than compaction where you can. Anthropic calls tool result clearing one of the lightest-touch forms of compaction: remove raw tool outputs once the agent has acted on them. A long test log that produced one failing test name rarely needs to survive verbatim. The failing name and a pointer to the log do. Plenty of runs can clear aggressively for a long time and compact rarely, which is the right order. Deleting what is provably dead is a safe operation. Summarizing what is still alive is a judgment call.

One summary, checked against the original

Here is the check we mean, on an example compacted summary from a contract review. The lines are the example summary's own. The notes are what a comparison against the original history found.

One summary, checked against the original. Example content. Three lines are safe to act on; two need the original opened first — the flattened line, which kept a check but lost which version it was checked against, is the dangerous kind, because it reads as complete.

Notice which line does the damage. It is not a missing line. The summary kept the payment-terms check and dropped the fact that the agreement it was checked against had been superseded, so the surviving sentence is accurate and misleading at the same time. Fluent summaries prefer closure. An unresolved dispute tends to come back as a reviewed item, a superseded source as a source. If you sample only one thing after compaction, sample for false closure: open items that returned sounding finished.

Where compaction stops being the answer

Anthropic's work on long-running agent harnesses is blunt on the limit: compaction on its own is not sufficient, because it does not always pass perfectly clear instructions across to the next session. Their answer is to leave structured artifacts outside the window (a progress file, a feature list) that the next session reads directly. When a task's state can live in artifacts like that, a clean restart reading good artifacts can beat a summary trying to hold everything in prose.

Compaction also cannot rescue a run whose problem was never length. If the agent is looping, summarizing hands it a shorter version of the same wrong history to continue from. If the underlying data is bad, the summary preserves the badness more compactly. And some stretches of work should not be summarized mid-flight at all: where the exact wording is itself the evidence, end the run at a boundary, keep the record whole, and start the next run from artifacts.

So test it the way you would test anything else that can silently change behaviour. Replay finished runs, compact them at the threshold you plan to use, and diff what the summary claims against what the original shows. A compaction setting that has never been through that check is a hope with a token count attached.

The design question was never whether to compact. Long runs leave you no choice. It is what your run is allowed to forget, who wrote that list, and whether the original is still there on the day the summary turns out to be wrong.

Work with us

If your long-running agents come back from compaction missing the one detail the task turned on, the fix is rarely a bigger window. It is a keep list written in advance, pointers back to the original, and a check on what survived. Tell us about the run through Trida AI's intake.

Agent Runs Need Deadlines

FDE at Trida AI — Mon, 09 Nov 2026 12:09:10 +0000

Originally published at Trida AI.

The quote went out at half past nine. The agent that was checking its prices finished at twenty to ten, found two prices changed, and did what it had been asked to do: it updated the customer record and drafted a correction. Nothing had gone wrong, technically. The work was simply late, and late work, acted on, became a second problem on top of the first.

Take that scene as a pattern rather than a report. It repeats wherever an agent's answer has a moment it is needed by: a summary wanted before a meeting starts, a check wanted before a payment is released, a price wanted before the quote is sent. Many runs carry budgets for steps and tokens. Fewer carry the time by which the answer stops being useful. Those are different limits, and the second one is the one the business actually feels.

A timeout is not a deadline

A timeout is a local allowance. This tool call may take 30 seconds, that model call may take a minute, and each allowance starts fresh when the call starts. Ten calls in a row, each inside its own timeout, can keep a run going for as long as the steps hold out. Nothing anywhere in that arrangement knows what time the whole thing was supposed to be done by.

A deadline is a point in time. The gRPC deadlines guide puts it plainly: a deadline is the point past which a client is no longer willing to wait for a response. It does not reset. Every step inside the run shares the same shrinking remainder, and when the remainder reaches zero, the question changes from "how much longer can this take?" to "why is it still running at all?"

Where the deadline comes from matters. It is not an infrastructure setting, because infrastructure does not know when the quote goes out. It belongs to the task, and it is set when the task is accepted: this answer is worth having until 09:30, after that it is worth much less, and after 10:00 it may be worth less than nothing, because acting on it creates clean-up work. Some tasks get hours. An overnight research run gets until morning. The point is not that the window is short. The point is that there is one, the run knows it, and it was chosen by whoever knows what the answer is for.

One run, and the clocks nested inside it. Example times. Each step and tool call gets what is left of the run's window, never a fresh allowance of its own — and the last slice is reserved for reporting back, not for starting new work.

The remaining time has to travel with the work

A deadline that sits in the orchestrator's configuration and nowhere else will be missed politely, one well-behaved step at a time. It has to travel. When a step calls a tool, it hands over the time remaining, not a new budget. When a run starts a child run, the child inherits the parent's deadline, minus the reserve the parent keeps for reading the result and reporting back. Where gRPC propagates a deadline, it handles the clock problem in the conversion: the deadline becomes a timeout with the elapsed time already deducted, so machines that disagree about the time still agree about how much is left. Agent runtimes need the same habit. Pass durations, not timestamps.

The check belongs before the work, not after it. Google's SRE book, in Addressing Cascading Failures, describes a request that sat in a queue for eleven seconds against a ten-second deadline: by the time a server picked it up, the caller had given up, and any work done on it earned no credit at all. Their rule is that a server handling a request in stages should check the time left at each stage before doing more work on it. Translated to an agent: before each step, and before each expensive tool call, compare what is left with what the step plausibly needs. If the step cannot finish inside the window, do not start it and call the outcome a timeout later. Decline it now, while declining is still cheap, and spend the remainder closing out.

The split of where time goes is worth copying too. Temporal's timeouts separate how long an activity may wait to be picked up from how long it may run once started, because those are different failures with different fixes. A run can miss its deadline in a queue, in an approval inbox, or mid-execution. If the record only says "deadline exceeded," you will fix the wrong one. Log which clock ate the time.

When the clock runs out

At zero, the run stops starting things. That sounds obvious and is widely not what happens: retries fire, a slow source gets one more chance, a final synthesis step begins with ninety seconds of context assembly still ahead of it. A deadline with no change in behaviour at zero is a comment in a config file.

The harder question is the late result. Sources do not know about your deadline. The supplier portal that was slow at 09:18 may answer at 09:41, correct and complete. The rule should be: a result that arrives after the deadline is recorded as late and is not acted on automatically. The world it was gathered for has moved. The quote went out, the meeting started, the record was written by someone else. Applying the late result without a person looking at it first turns a timing problem into a correctness problem. Logged with its arrival time, it is still useful evidence. Applied silently, it is a surprise.

What the run owes instead is a partial answer before the deadline, not a complete one after it. When the reserve slice begins, the run closes with what it has: the prices it verified, the one it could not, and the gap stated in plain words. A person can send a quote with one price flagged as unverified. Nobody can use a perfect check that arrives after the quote has gone.

Run log for a pricing check that missed one source. Example entries. The partial answer goes out inside the window with its gap flagged; the slow source's reply is kept as a late record, not applied.

Deadlines do real damage when they are picked badly. Set one from impatience instead of from the task, and you will train the system to fail expensive requests consistently — the SRE book warns about exactly that failure shape, where some more expensive requests are the ones a short deadline makes fail consistently. Set one that pauses silently whenever a human approval is involved, and you have a loophole, not a deadline: if waiting on a person should move the deadline, move it explicitly, in the record, where someone can see who moved it and by how much. And a genuinely open-ended task should not be given a pretend window to satisfy a policy. Give it the honest one, even if that is tomorrow morning.

So the test is two questions, asked of any agent run you operate. When does this answer stop being useful? And does the run itself know? If the first answer lives only in a person's head, the second is no, and the run will keep doing correct work at the wrong time — which, from the receiving end, looks exactly like being wrong.

Work with us

If your agents are doing sound work that lands after the moment it was needed, the fix is rarely a faster model. It is a deadline drawn from the task, carried through the run, and honoured in the last minutes. Tell us where your runs are landing late through Trida AI's intake.

Async Agents Move the Failure Modes

FDE at Trida AI — Sun, 08 Nov 2026 12:06:19 +0000

Originally published at Trida AI.

The most important design decision in the new async agent pattern is not the parallelism. It is who stays behind to answer the phone.

LangChain's Agents Are Going Async describes the split its Deep Agents framework now ships: an orchestrator holds the conversation with the user while async subagents run the long work in the background. In the older shape, a subagent call blocked the orchestrator's loop until it returned, so one slow task froze everything, including the conversation. Now the hand-off returns at once, the loop stays free, and the amount of work an agent can carry grows with the number of workers instead of the length of one loop.

We think the direction is right. We also think teams will misread what it fixes. Async removes a waiting problem. In exchange it creates a supervision problem, because work that used to fail in front of you now fails, waits, and finishes somewhere you are not looking. The failure modes did not leave. They moved.

What the orchestrator actually does all day

The mechanics are worth stating plainly, because the duties hide inside them. The orchestrator manages background work through five operations: start a task, check its status, send it new instructions, cancel it, and list everything in flight. Every unfinished task is listed in its system prompt, so it always carries a picture of what is outstanding.

Steering splits into two choices with different costs. An interrupt (the default) stops the worker's current run and restarts it with the new instruction — right for a correction, wasteful for an addition, because the run in progress is stopped before it finishes. An enqueue holds the follow-up until the current run finishes, which preserves that work and delays the change. Neither is a detail. Picking between them, task by task, is a judgment the orchestrator now makes on your behalf.

Results find their way back three ways. A busy orchestrator picks up finished results before its next model call. An idle one is woken by the worker reporting in. A paused one — waiting on an approval — is left alone, and the result waits with it. And when a background worker itself needs an approval, the request surfaces on that worker's own thread, where the user answers it without disturbing the main conversation.

Read that list again as a job description. Track everything outstanding. Decide what an interruption should cost. Hold results until the moment they can be received. Route approvals to the right thread. That is not routing. That is supervision, and it is the part of the system that now decides whether the whole arrangement feels reliable or haunted.

What each part is doing while the work runs. One vendor comparison, delegated once, still being steered the next morning — the conversation and the work now run on separate clocks.

Where the failures go when nobody is watching

A synchronous agent fails in the conversation. The user sees the error, asks a question, redirects. A background worker fails at 2 a.m. in a thread nobody has open. Unless the system gives that failure a destination (a record a person can inspect, with what the task was, how it failed, and what it touched before it stopped), the failure is just missing output. The user finds out when the result they were promised does not arrive, which is the worst notification design there is.

Waiting rots in the same quiet way. An approval request sitting on a background thread has no natural owner: the user may not know it exists, and the orchestrator is paused around it. A task can wait for days while every dashboard shows the system as healthy, because waiting is not an error state. It needs an age, and at some age it needs to escalate to a person.

Two more relocations are easier to miss. A worker that runs after the user has logged off runs on access granted inside a conversation that is over, so the grant's lifetime belongs to the task, not to the session. A credential that outlives the work is a standing invitation, whoever ends up holding it. And a result that lands the next morning arrives after the user's context has moved on. It has to carry its receipts with it: what it read, when it read it, and which claims stand on which sources. A bare answer, delivered late, asks to be trusted at exactly the moment checking it is hardest.

The test is the task list

Here is the evaluation we would apply to any async agent setup, our own included. Show the user a single list of everything running in their name: each task, when it started, what state it is in, what it is waiting on, and what happens if it fails. If that list exists and stays honest, the supervision duties above have somewhere to live. If it does not, and the only way to learn what is outstanding is to ask the orchestrator and take its word, then the background is a black box with good manners.

The background task list, as the user sees it. Waiting and failed appear as states with ages and owners, not as gaps between messages.

None of this makes async a bad trade. For genuinely long work (research that takes an hour, a migration that runs overnight) a blocking conversation is the worse design, and we would not go back. But there are places the split earns nothing. Short tasks pay the coordination cost and get nothing in return. Work the user needs to watch unfold, because the reasoning is the product, gets worse when it happens offstage. And tools that cannot report progress or accept a cancellation make poor background workers: they cannot be supervised, only launched.

So build the orchestrator as a supervisor, not a dispatcher. Give its tasks a ledger, its failures a destination, its approvals an age, and its results receipts. The parallelism is the easy part, and it is mostly solved. What async actually moves into your product is the oldest management problem there is: work you cannot see, done by workers you have to trust, reported back after you have stopped watching. Design for that, and the pattern holds up. Design only for the parallelism, and you will meet the failure modes at their new address.

Work with us

If your agents are starting to run work in the background (research, migrations, overnight jobs) and nobody can say exactly what is outstanding, what it is waiting on, or where its failures land, we can help you build the supervision layer before the list gets long. Tell us about the deployment through Trida AI's intake.

Cancelling an Agent Run Cleanly

FDE at Trida AI — Sat, 07 Nov 2026 12:04:49 +0000

Originally published at Trida AI.

Clicking stop is the easy part. Everything after the click is where cancelling an agent run actually happens, and in most products it doesn't. Someone watches a run head in the wrong direction, clicks stop, the spinner disappears, and the run is filed as cancelled. Meanwhile a tool call that was already sent finishes a minute later and writes anyway. The screen said stopped. The world disagrees.

That gap isn't a UI bug. It's what cancellation defaults to when nobody designed it: a display state. Making stop mean something in the systems the run touched is a design problem, and durable workflow engines have already mapped most of it.

Cancellation is a request the run has to hear

Temporal draws the line clearly in its documentation of cancellation and termination. Cancellation is cooperative: the service delivers the request and the workflow code decides what to do with it, so it can release resources, compensate for steps that finished, and, in most SDKs, pick its own closing status. Termination is the opposite. The execution closes immediately, the code never sees the request, and no cleanup runs. Anything the run was holding stays held: Temporal's own list is reserved inventory, held locks, and partial writes left as written.

Agent runtimes need both words, used honestly. If your stop path kills the process, call it termination and accept what it leaves behind. If you call it cancellation, the run has to receive the request while it can still act on it, which means the signal needs a delivery route to every part of the run: the orchestrator, any child runs it spawned, and the tool call in flight.

Delivery is the part people assume. Temporal delivers an activity cancellation when the activity heartbeats: a long-running activity that doesn't heartbeat can't receive the request at all, and runs to completion, failure, or timeout instead. Translate that to agents and the equivalent is a tool wrapper that never checks in. A forty-minute export job with no progress reporting isn't just hard to watch. It's uncancellable, and no button upstream changes that.

Model calls have their own version of the same limit. OpenAI's API reference says a response can be cancelled only when it was created with the background parameter set to true; a synchronous call has no handle to cancel by. Whether a run can be stopped is often decided at the moment the work starts, not when someone clicks.

The call in flight decides what stop means

Picture a run with four steps. Two have finished. The fourth hasn't started, and stopping that one is easy: it simply never runs. Everything hard about cancellation lives in step three, the call that is in flight when the request arrives.

One run, the moment cancel arrives. The finished steps are history, the unstarted step is easy, and the call in flight is the one that has to report what it actually did.

That call ends in one of three states, and the runtime has to learn which. It stopped before acting. It acted, and the effect landed. Or it timed out in the middle, and nobody yet knows whether the write went through. The third state is the honest problem. Treating an unknown outcome as "cancelled, nothing happened" is how a stopped run still changes a record, sends the message, or creates the draft, and the operator finds out from the other system.

So a cancelled run shouldn't close with a single status word. It closes with an account: what stopped, what landed before the signal arrived, what cleanup removed, and what is still unknown and needs a person to settle it in the target system. If the unknown list isn't empty, the run isn't cleanly cancelled yet, whatever the UI says.

Child runs widen the account. A supervisor that spawns background work needs cancellation to reach that work too, or the parent stops and the children keep going on the old instructions. LangChain's async subagents documentation makes this an explicit supervisor tool: cancelling a task calls the server's cancel on the run, marks the task cancelled, and, where earlier runs are still active, also cancels those. The propagation is a feature someone built, not a default of spawning work.

Cleanup cannot live inside the thing you cancelled

There's a trap in the obvious implementation. The run receives the signal, catches it, and tries to tidy up: delete the temporary file, release the hold, withdraw the draft. But the tidy-up is new work scheduled inside a scope that has just been cancelled, so it gets cancelled too. Temporal's scopes make the fix explicit: cleanup runs in a nonCancellable scope, protected from the very signal that triggered it.

The agent version is the same shape. The component that runs cleanup, a reaper outside the run or a protected section of the runner, must not take instructions from the run it's cleaning up after, and it needs its own small budget of time and tool calls. Cleanup that can be starved by the cancellation itself is decoration.

What the operator sees after the cancel. One item landed before the signal arrived, so it stays, flagged for a person. Nothing on the report is listed as unknown.

Two limits are worth stating plainly. First, cancellation is never instant, and UIs should stop pretending otherwise: there's always a window where the request is travelling and the in-flight call hasn't reported back, and the status during that window is "cancelling," not "cancelled." Second, some tools offer no way to check in and no way to withdraw a request once sent. For those, the honest options are waiting for the outcome before closing the run, or recording the unknown and handing it to a person. A confident green checkmark isn't on the list.

None of this argues against giving people a stop button. It argues for building the plumbing the button implies: a signal that propagates to child runs and tool calls, check-ins from anything long-running, a protected cleanup path, and a closing report that separates what landed from what didn't. A run you can stop cleanly is also, not coincidentally, a run you understand while it's going.

Work with us

If stopping one of your agents today clears the screen while the work carries on underneath, we can help you build the version where the signal reaches the tool calls, the child runs, and the cleanup. Tell us about the deployment through Trida AI's intake.

Dead Letter Queues for Agent Failures

FDE at Trida AI — Fri, 06 Nov 2026 12:18:29 +0000

Originally published at Trida AI.

A failed agent run usually ends in the worst place it can: a log line. The run retried, spent its budget, gave up, and the only evidence is a stack trace in a stream that scrolls past thousands of other lines. Nobody owns it. A week later the same failure has happened forty more times, and each one was news.

Messaging systems settled this problem years ago. When a message can't be processed, it doesn't vanish and it doesn't block the queue behind it. It goes to a dead letter queue, a holding place built for inspection. Amazon SQS describes dead-letter queues as the place to isolate messages that failed processing so you can work out why. Google Cloud Pub/Sub forwards a message to a dead-letter topic after roughly a configured number of delivery attempts (five by default), wrapping the original with attributes that identify where it came from. Azure Service Bus adds a reason and a description to every dead-lettered message, and is blunt about the other half of the contract: there is no automatic cleanup, so messages sit there until someone retrieves them.

Agent runs need the same destination, and most don't have one. The shape of the problem is familiar (retries exhausted, failure made permanent, work set aside for a human), but the unit is different. A message is a payload. A run is a piece of work that may have read three systems, written to one of them, and stopped halfway through a plan. Parking that safely takes more than a queue. It takes a record.

A log is not a queue

Logs answer "what happened?" for someone who already knows where to look. A dead letter queue answers a different question: "what work is still owed, and who owes the next move?" That difference is operational, not cosmetic. A queue has an owner, a count, and an age for its oldest entry. It can alarm when it grows. Items leave it one of two ways, replayed or resolved by a person, and both are deliberate acts.

So the first design decision is boring and important: terminal failure is a state, not an event. When a run exhausts its retries or its budget, the runner doesn't just report an error and exit. It writes a dead letter record and stops. Everything after that (inspection, the fix, the replay, the handoff) works from that record.

Where a failed run goes. The runner parks a terminal failure as a record; a person either clears it for replay from its checkpoint or takes it over with the record attached.

What the record has to hold

A dead letter record for a message can be thin: the payload, a reason code, a delivery count. A run's record has to carry enough for a stranger to make a safe decision about work they didn't watch happen.

The original task, first: the instruction and inputs the run started from, snapshotted rather than referenced, because the source may have changed since. Then the failure, described in a small fixed vocabulary (tool timeout, invalid input, permission denied, output rejected by a checker), because free-text errors can't be counted, and uncounted failures can't be fixed in batches. The attempt history belongs beside it: how many tries, at what, and where the run stopped, with a pointer to its checkpoint and its trace.

The field people skip is the side-effect register: what the run already changed in the world before it stopped. Which writes landed, under which idempotency keys, and, just as explicitly, which ones are unknown. That register is what separates a replayable failure from a dangerous one, and it's cheap to write at the moment of failure and nearly impossible to reconstruct a day later.

Two smaller fields earn their space. An owner, because an unowned queue is a landfill with good tooling. And an expiry, because a failure nobody has looked at in thirty days is rarely worth replaying. The world it belonged to has moved on.

One dead letter record. The replay decision is written down with the reason, and the unknown side effect — not the timeout — is what sends this one to a person.

Replay is the dangerous part

Getting a failed message back is a solved, almost casual operation: SQS calls it redrive, and the message returns to its source queue. Replay an agent run the same casual way and you can do real damage. The run may have already created the draft, sent the first email, or placed the hold. Starting it again from the top repeats whatever landed.

So replay has a gate, and the gate reads the side-effect register. If every landed effect is recorded and the replay resumes from the checkpoint reusing the original idempotency keys, it can go. If any effect is marked unknown (a tool call that timed out after it may have committed), the record doesn't replay at all. It goes to a person, with the unknown named in plain language, and the person settles that one question before anything resumes. One ambiguous write should cost a human two minutes, not a duplicate payment.

The queue also pays rent in aggregate. Read as a pile, dead letter records are a bug report against your own system: the same failure class forty times is a broken tool contract or a bad instruction, not forty accidents. The fix lands in the tool or the prompt, the waiting records replay against the fixed system, and the class empties. A queue that only ever gets replayed, never read, is a retry loop with extra storage.

Where this stops helping: small deployments. If your agents fail terminally twice a week, a table and a named owner give you most of this; the full apparatus is for fleets where failures arrive faster than anyone can read logs. And the record itself needs care — task snapshots can hold personal or commercial data, so redact on the way in and keep retention short. A dead letter queue that becomes a second, unguarded copy of your inputs has traded one problem for a worse one.

Work with us

If your agents' failures currently end as log lines, retried, dropped, and rediscovered weeks later, we can help you give them a destination: records a person can decide from, and replays that are safe to run. Tell us about the deployment through Trida AI's intake.

Agent Credentials Should Expire

FDE at Trida AI — Thu, 05 Nov 2026 12:26:14 +0000

Originally published at Trida AI.

Hand an agent a key that never expires and you have not really given the agent access. You have given access to whoever eventually finds that key, for as long as they care to use it. Keys leak out of ordinary systems often enough. An agent raises the odds, because handling material and passing it along is most of its job.

Our position is simple enough to fit on an index card: an agent's credential should be born with the task, scoped to that task, and dead shortly after it. If a deployment cannot say when its agents' credentials expire, the honest answer is probably that they don't, and that is the problem worth fixing first.

A key does not stay where you put it

A conventional service holds a credential in one place, a config store or an environment variable, and uses it from there. An agent moves things. The key, or something that can be exchanged for it, can surface in a prompt, ride along in a tool call, get written into a run log, be quoted in a summary the agent writes about its own work, or travel inside the context handed to a second agent. None of that requires an attacker. It is the normal plumbing of these systems, doing what it was built to do.

That changes what a leak means. With a human operator, a copied key is an incident. With an agent, copies are a by-product of ordinary operation, scattered across stores that were designed for reading, not for secrecy. You cannot reliably hunt down every copy after the fact. You can make every copy worthless on a schedule you choose in advance. That is what expiry is for.

Scope matters as much as lifetime, and the two decisions belong together. A grant that can read one collection for fifteen minutes is a different object from a grant that can write anywhere for a year, even if both are called "the API key" in a config file. Expiry forces the scope conversation, because a credential that has to be re-issued for each task gets its permissions re-asked for each task too.

The machinery already exists

None of this needs inventing. Cloud platforms settled the pattern years ago for ordinary workloads. The AWS Security Token Service issues temporary credentials that work much like long-term keys, except that they are generated on request rather than stored, are valid for a configured span from fifteen minutes up to twelve hours, and stop being recognised at all once they expire. AWS makes the quiet point that follows from that: a credential with a limited lifespan does not have to be rotated, and it does not have to be explicitly revoked when the work ends. Time does it.

Google Cloud lands in the same place from the service account side. Its documentation calls obtaining short-lived credentials the most secure way to authenticate as a service account, with access tokens expiring after one hour by default. It also blocks a neat circumvention: a short-lived credential may not be used to mint a fresh token for the same service account, because that would let a stolen token refresh itself indefinitely. Expiry that can be quietly extended by the holder is not expiry.

Agents are simply late to a settled practice. The workload changed, from a service with a stable job to a system that takes a new instruction every few minutes, and the credential should change with it: one grant per task, sized to the task, rather than one standing key behind every task the system will ever run.

Two ways to give an agent access to the same tool: one standing key that spreads into prompts and logs, and a broker that issues a short grant per task and injects it when the call is made.

Keep the credential out of the model's hands

Expiry alone still leaves the raw credential passing through a system that writes things down. The stronger shape puts a broker between the agent and the credential. The agent asks the broker for access to a named tool, for a named task. The broker checks that request against policy, obtains or mints a short grant, and the tool gateway attaches that grant to the outgoing call. The model sees that the call succeeded or failed. It never sees the thing that made the call authorised.

This is the arrangement the OWASP secrets guidance points at with dynamic secrets: credentials generated on demand rather than retrieved from a drawer, given a short lease, and revoked when they are no longer needed, which cuts down how much any one credential can be reused. Applied to agents, the payoff is concrete. There is nothing in the prompt to leak, nothing in the transcript to scrape, and nothing in a handoff packet that quietly carries write access to the next run. A stolen transcript becomes a record of work, not a set of working keys.

Renewal belongs to the broker as well, and it should be a fresh decision, not an automatic extension. When a grant runs out mid-task, the broker re-checks the task before issuing another: is this still the job that was approved, with the same scope? Most renewals will pass in milliseconds. The point is that they are checked at all, so a run that has wandered somewhere new does not drag its original permissions along with it.

One grant followed from issue to refusal: read-only export access issued for fifteen minutes, accepted once, then refused when the same token was replayed after expiry. Example content, invented for illustration.

Where this gets awkward

Plenty of the tools agents need to reach do not offer short-lived credentials at all. Plenty of SaaS products still work the older way: a single API key per account, with no expiry, no per-task scope, and no way to ask for less. There the broker earns its keep differently: it holds the one standing key, never releases it to the agent, and enforces the expiry and scope on its own side of the gateway, by refusing calls that fall outside the current task's grant. The underlying key stays long-lived. The agent's effective access does not.

Long tasks are the other friction. A research run or a migration that legitimately takes hours will cross several grant windows, and a grant that dies in the middle of a multi-step write can leave the work half-applied. The answer is renewal at step boundaries, arranged before the work starts, plus the discipline covered by idempotent tool design: a step that is retried after a renewal must be safe to run twice. What you should not do is respond to one awkward long task by issuing day-long grants to everything.

And the broker itself becomes a high-value target, which is worth saying plainly. You have concentrated the ability to mint access in one component. It needs the boring protections any such component needs, tight policy, a full record of every grant it issues, and no path by which the agent can talk it into a wider scope than the task was approved for. Short-lived credentials shrink the blast radius of almost every failure except a failure of the broker. Staff it accordingly.

None of these are reasons to keep handing agents standing keys. They are the shape of the work once you stop.

Work with us

If your agents are running on keys that never expire, bring that setup through Trida AI's intake and we will work out how to put a broker and a per-task grant between the model and the credential, starting with the tools that carry the most risk.

Agent Answers Need Receipts

FDE at Trida AI — Wed, 04 Nov 2026 12:10:34 +0000

Originally published at Trida AI.

"Where did nineteen days come from?"

Picture a routine review. An agent has summarised a supplier's documents and put a lead time at nineteen days. The number looks ordinary, sitting in a tidy paragraph next to two others nobody questions. Then someone asks which document said nineteen, and the room goes quiet. The trace shows the searches, the reads, a long generation. What it cannot show is which passage that one number rested on. Nobody kept the connection between the claim and its source.

Most agent systems keep a good record of the run and a poor record of the answer.

A trace answers a different question

A trace is an operator's record. It tells you the order of operations, what each call received and returned, where the time went. The OpenAI Agents SDK traces runs this way by default: generation spans hold the model's inputs and outputs, function spans hold a tool call's arguments and its result, and you can turn that capture off because the contents may be sensitive. Useful, and necessary. Also not the reader's question.

The reader's question is narrower and ruder: this sentence, the one I am about to act on: what is it standing on? A trace can help an engineer reconstruct an answer after the fact, the way a flight recorder helps investigators. A receipt is the thing you hand over at the time, before anyone has to reconstruct anything.

The idea is old. The W3C PROV family modelled provenance as entities, activities, and agents: a thing, the process that produced it, and whoever bears responsibility for that process. Agent answers fit that shape well. The claim is the entity. The retrieval is the activity. The system that served the answer is the agent who stands behind it.

Attach the receipt to the claim

Answer-level citations are where good intentions go soft. A paragraph ends with two links, and the reader guesses which link supports which sentence, or whether either supports the number in the middle. Receipts earn their name at claim level: one claim, one record, four fields.

The source, named exactly — the document, the record, the tool result, not "the database". The passage inside it, located so a person can land on it: a page range, a character range, a row key. The time it was read, because sources change and a receipt from March should not underwrite a claim made in November. And the claim itself, stored as written, so a later edit cannot borrow a receipt issued for different words.

Some of this machinery now ships with the models. Claude citations return structured objects rather than prose promises: the cited text, which document it came from, and a location that is a character range for plain text, a page range for a PDF, or a block range for custom content. That is the right shape. It is not, on its own, a guarantee. Anthropic's own guidance for extracting fields this way ends with an instruction to check that each field has a value and at least one citation, because the API guarantees neither. The check is yours to run. A claim that comes back without a receipt has to go somewhere you decided in advance: held for a person, rewritten to say less, or dropped. What it cannot do is ship wearing the same confidence as its supported neighbours.

One claim's path through the system: the tool result is stored as a receipt before the answer is written, and a checker holds any claim that arrives without one.

Receipts have to survive the machinery around the answer, too. Summaries compress, handoffs pass context along, and both are fine places for a source to fall off a claim. Pass receipt identifiers along with the text they support, the way you would pass a primary key, and let the checker follow them back to the stored passage. If the passage cannot be opened, the claim is unsupported again.

What the reader actually sees

One short answer, three claims: two carry receipts a reader can open, and the third, which had no receipt for its date, was rewritten to say only what its source supports. Example content, invented for illustration.

Notice what changed for the third claim. It was not deleted, and it was not left standing on a plausible tone of voice. It shrank to fit its evidence. That is most of the daily value here: receipts rarely expose a dramatic fabrication. They expose the sentence that drifted a little further than its source, the figure that was true of last quarter, the commitment a document mentioned as a possibility and the answer repeated as a plan.

Where receipts run out

They only cover claims that come from somewhere. A recommendation, a judgment, a forecast — these rest on reasoning, and dressing them in citations borrowed from nearby facts would be worse than leaving them bare. Mark them as the system's judgment and let the reader price them that way.

Coverage costs, too. Claim-level checking adds a pass to every run, the receipt store becomes one more system holding possibly sensitive passages, and a team that demands receipts for trivia will train its agents to write answers with no checkable claims in them at all. Receipts belong where a reader will act: figures, dates, commitments, statuses. The rest can travel lighter.

And a receipt proves where a claim came from, not that the source was right. A wrong document produces a perfectly valid receipt. That still helps: the argument moves from "the agent said so" to the document itself, in one click, and the document can be fixed at its root.

Keep the passage, not a pointer to it. Attach it to the claim, not the answer. Hold the claim that arrives without one. Three dull rules, and the question at the top of this post — where did this come from — gets an answer in seconds instead of a reconstruction project.

Work with us

If your agents produce answers your team cannot check without replaying the whole run, the missing piece is a receipt layer, not a longer trace. Bring the answers your readers act on through Trida AI's intake and we will work out which claims need to carry their sources with them.