DEV Community: FDE at Trida AI

Trida AI Joins Google Cloud Partner Advantage

FDE at Trida AI — Wed, 21 Oct 2026 17:13:09 +0000

Originally published at Trida AI.

Trida AI is now a member of Google Cloud Partner Advantage, confirmed on October 20, 2026. TRIDA AI LLC is registered in the program across the Google Cloud services and co-sell paths and the Google Workspace technology and services paths.

What we can do as a member

Partner Advantage is Google's program for companies that build on, sell, and deliver services around Google Cloud. Membership puts Trida inside that program, with access to Google's partner training and technical resources, partner support channels, and the co-sell structure through which Google's field teams and partners take customer opportunities forward together.

For our customers, the practical change is this: when a forward-deployed team from Trida takes an AI workflow to production on Google Cloud, it now does so as a program member — building with the platform's current training, resources, and support behind the work, not from outside the ecosystem.

What we are doing from here

Three things. First, Google Cloud delivery becomes a standard part of our forward-deployed engagements for customers whose stack runs on it. Second, our engineers certify on Google Cloud as they deliver — the program measures partners on certified staff and customer outcomes, and our next milestone, Select, is earned exactly that way. Third, we build the co-sell path: registering real customer opportunities with Google as they come, so joint work starts with the first deployment, not after it.

That is the same sequence our Academy runs on: skill first, credential after proof. The membership is the starting line, and we intend to show the progression in the open as it happens.

Work with us

Planning an AI deployment on Google Cloud? Start with our intake — tell us the workflow, the stack it has to live in, and the date it has to be live by.

Idempotency for Agent Actions

FDE at Trida AI — Wed, 21 Oct 2026 15:59:24 +0000

Originally published at Trida AI.

An agent issues a refund. The payment call times out. Nothing in that timeout tells the agent whether the money moved. The request may have died on the way out, or the refund may have completed while the response died on the way back. Both worlds look identical from where the agent sits.

So it retries. That is what it was built to do, and in the second world the customer has now been refunded twice.

This is not a reasoning failure. The model cannot reason its way to a fact the network didn't return. It's a plumbing failure, and it has a known fix that predates agents by years. The reason it keeps biting agent systems is that agents retry from more places than ordinary software does, and most tool layers are still built as if a call either happened or it didn't.

Reads forgive. Writes don't.

A retried read costs you a second lookup. A retried write can cost you a second refund, a duplicate order, a duplicate ticket, or two copies of an email a customer will read as carelessness. The dividing line in any agent is exactly there: which of its tools change something outside itself?

Agent stacks multiply the retry sources. The model reconsiders and calls the same tool again with slightly different wording. The framework retries a failed step. The SDK retries a transient error nobody configured. A queue redelivers a message, a worker restarts mid-run and resumes from a checkpoint that predates the side effect. Each layer means well. Stacked together, one intended action can leave the building three or four times, and no single layer sees the others doing it.

Ordinary services learned to name this property. An operation is idempotent when repeating it with the same input produces the same end state: the effect happens once, however many times the request arrives. Agents need that property at every tool that writes, and they need it designed in, because it cannot be prompted in. No instruction in a system prompt survives a worker restart.

One key for the whole job

The mechanism is a key. Before the first attempt, the system mints one identifier for the intended action (this refund, for this order), writes it down somewhere that survives a restart, and sends that same key with every attempt. The receiving side records the outcome against the key. A repeat with the same key gets the recorded outcome back instead of a second execution.

Stripe's API is a reference implementation most teams meet early, and it is explicit that it supports idempotency for safely retrying requests without performing the same operation twice. The details matter more than the headline. The key is client-generated, with V4 UUIDs suggested. In API v1, the result of the first request is saved once execution begins, and retries return that saved result. Reuse the same key with different parameters and the request errors rather than quietly returning the wrong response. And keys can be pruned once they are at least 24 hours old, after which a reused key starts a new request. The provider's memory sets a horizon, and any workflow that might retry beyond a day needs its own record as well.

Two design choices decide whether this works in an agent. First, where the key comes from. A fresh random key per attempt is useless; a fresh random key per run is better, but it still misses the duplicate that matters most in production, where two separate runs get launched for the same job. Deriving the key from the business identity, the order, the ticket, the invoice being acted on, means retries, restarts and duplicate runs all collide on the same key and collapse into one effect. Second, when the key is written down. It has to be persisted before the side effect, not after. A key recorded in a success handler is a key that a crash, mid-action, takes with it.

The action ledger: an intended action gets one key, minted and persisted in the ledger before any call is made; the tool executes downstream once and its result is stored against that key, so a retry carrying the same key is answered from the ledger and never reaches the downstream system a second time.

When there is no key, reconcile

Plenty of the systems agents act on offer nothing of the sort. No key header, no deduplication, just an endpoint that does the thing every time it is called. Sending again and hoping is not a strategy for those; the timeout case has to route somewhere else.

Amazon's Builders' Library walks through the same dilemma for a provisioning call: after a timeout, it may be unclear whether the workload is running, and simply retrying could create a second one, so the caller is left performing a reconciliation to find out what actually happened. Its design answer is a unique client request identifier carried with the request, so a retry can be recognised as the same request rather than a new one.

Where you cannot add that identifier to the downstream call, imitate it from your side. Attach your own reference to the action wherever the target system lets you (a client reference on the record, your key in a notes or metadata field) and make the timeout path a lookup before it is ever a resend: search by that reference, adopt the record if it exists, resend only if it provably doesn't. If the system can't be queried either, the honest options narrow to a status check by other means or a person. An ambiguous write that a human resolves in a minute is cheaper than a duplicate a team unwinds in a week.

One retry owner, or the retries multiply

Even safe retries need a budget and a single owner. When every layer of a stack retries independently, the counts don't add, they multiply: Amazon's retry guidance describes a five-deep call stack with three retries at each layer, where a failing database ends up taking 243x the load, and the same arithmetic applies to side effects when the layers sit in front of a write. Pick the layer that owns retries for each action, turn the others off for that action, and give the owner a cap, exponential backoff and jitter. The agent loop itself is usually the worst owner: it retries semantically, rephrasing as it goes, which defeats exact matching everywhere downstream. Retries belong in the tool layer, where the key and the record live.

One refund, traced

Say the action is a refund for a returned order, and the payment provider's response is lost to a timeout on the first attempt.

One refund traced end to end: the intent and its key are recorded before the call, the first attempt completes downstream while its response is lost, the retry arrives carrying the same key and returns the original refund record instead of refunding again, and the customer ends with one refund and one confirmation.

The intent is recorded first: refund this order, key derived from the order itself, status in progress. The first attempt reaches the provider and the refund is created; the response never arrives, so the agent knows only that the attempt timed out. It retries with the same key and the same parameters. The provider finds the key, returns the original refund, and creates nothing. The ledger marks the action complete with that refund's identifier, and the customer receives one confirmation, because the confirmation step keys off the completed record rather than off the attempt count.

Nothing in that trace required the model to be careful. The safety sits underneath it, which is where safety has to sit in a system whose top layer is probabilistic.

Where this stops helping

A key makes a repeat safe. It does not make the first attempt right. An agent that refunds the wrong order, confidently and exactly once, has still refunded the wrong order. Approval rules and evals carry that problem, not idempotency.

The in-progress case is genuinely hard. If a worker dies after the downstream action completes but before the result is recorded, the record sits at in progress and the next run has to decide whether that means "still happening" or "happened, unrecorded". Mature implementations expire in-progress records after a bounded window: the Powertools for AWS Lambda idempotency utility, for one, expires an in-progress execution if the function times out halfway through. Expiry is a judgment call about how long the action can legitimately take, and getting it wrong in either direction reopens the duplicate or strands the work.

And the window problem cuts both ways. Keys expire, ledgers get pruned, providers forget. A retry that arrives after the record is gone is a new action again. Set retention from your real retry horizon, including the human who re-runs a failed job on Monday, not from a default somebody else picked.

Work with us

If your agents can send, charge, book, create or delete anything, the duplicate-action question is already sitting in your retry paths, whether or not it has fired yet. Working through which actions need keys, ledgers and reconciliation, before a customer meets the same refund twice, is deployment work we do. Tell us about it through Trida AI's intake.

Model Upgrades Are Deployment Events

FDE at Trida AI — Wed, 21 Oct 2026 01:34:08 +0000

Originally published at Trida AI.

Swapping a model takes a minute. You change one string in a config file, restart the service, and a different system starts answering your users. Nothing in your code changed. A good deal of your system's behaviour just did.

That gap is where migrations go wrong. Over months, a production system quietly grows around one model's habits. Prompts are tuned against its phrasing. Parsers expect its formatting quirks. Tool calls follow its rhythm, retry logic assumes its latency, cost forecasts assume its tokenizer. The replacement may top every public benchmark and still be worse at your specific job, in specific places, for reasons no leaderboard will show you.

So we treat a model change the way we treat any other deployment: an event with a before and an after, evidence gathered on both sides, and a way back. The model is a dependency. Upgrading it deserves at least the suspicion you would give a major-version bump of a library your product cannot run without. More, actually. A library does not rewrite its own behaviour between releases.

Pinning buys time. It does not buy permanence.

You have two ways to name a model, and each fails differently. Call an alias, the short undated name, and the provider can move it to a newer snapshot underneath you. Nothing errors. Your system just drifts, and the first symptom is a slow slide in output quality that nobody can date. Pin a dated snapshot instead and behaviour stays put, but you have scheduled your own migration: snapshots retire.

The clocks are public and they are not generous. On Friday, OpenAI shuts down a batch of legacy snapshots (GPT-4, GPT-4 Turbo, the first GPT-4o snapshot, o1, o3-mini and o4-mini among them), with replacements named on its deprecations page. The named replacements are GPT-5.6 models, Sol, Terra and Luna by tier, and a second wave is already scheduled behind this one: GPT-4o's later snapshots, GPT-4o mini, GPT-4.1 and GPT-4.1 mini retire on January 14, 2027. OpenAI commits to at least six months' notice for generally available models, three months for specialised variants, and possibly as little as two weeks for previews. Six months sounds comfortable until the work lands in the quarter when everything else ships too.

For generally available models, Anthropic's floor is shorter. Its model deprecations page promises customers with active deployments at least 60 days' notice before a publicly released model retires, and requests to a retired model simply fail. Claude Sonnet 4.5 was deprecated on September 30, 2026 and retires on November 30. Anthropic names Claude Sonnet 5.5 as the replacement. The same page carries a quieter trap: on Claude 4.7 and later, setting temperature, top_p or top_k to a non-default value returns a 400 error. Code that never changed models can still break on the day it is forced to move.

And if you reach the same model through Amazon Bedrock or Google Cloud, check that platform's schedule too. The provider's clock is not always your platform's clock.

A better model can still regress your workload

The evidence that upgrades hurt real prompts is not folklore. In a case study of toxicity detection on social media text, 58.8% of prompt and model combinations lost accuracy after an API update, and among those that dropped, 70.2% dropped by more than 5%. The update was newer. The workload was worse.

Slices are the operative word. An aggregate eval score can hold steady while one segment of your traffic falls off a cliff. If your evals only report the average, you will promote a regression and call it an upgrade.

Run the migration like a release

Start with an inventory, because the model ID never lives in one place. It sits in application config, in worker env vars, in scheduled jobs, in the eval harness, in the docs nobody updated. List every caller before you touch anything.

Then freeze a baseline. Run your eval set against the current model and keep the outputs, scored by slice. The slice is the unit that matters: document type, language, request shape, whatever partitions your traffic.

Replay the same set against the candidate. Compare slice by slice rather than by the total, and compare cost and latency alongside quality. A candidate that is two points better and three times slower is not obviously a win.

The migration gate: production traffic runs on the current pinned model while the candidate receives a shadow copy of the same requests; both sets of outputs are compared slice by slice in the eval replay, and only a candidate that passes the gate earns a 5% canary share of live traffic, with rollback to the pinned model available until its retirement date.

Only then let the candidate touch real work. Shadow first: it answers live requests, its outputs are logged and scored, nobody acts on them. Canary second: a small share of traffic, watched slice by slice. Promotion is a config change, and so is rollback, flipping back to the pinned ID. That is exactly why the whole sequence has to finish while the old model is still being served. A rollback plan with an expiry date is a deadline wearing a costume.

One migration, traced

Say the workload is invoice extraction: an agent reads incoming invoices and posts structured totals to an accounting system. The old snapshot retires in six weeks. Here is the shape of a migration that respects the clock.

One migration traced end to end: an invoice-extraction agent's baseline is frozen by document-type slice, the candidate model wins overall but drops on credit notes, a prompt fix targets that slice alone, the re-run and a 5% canary pass, and the candidate is promoted with two weeks to spare before the old snapshot retires.

The baseline is frozen by document type: standard invoices, credit notes, multi-page scans, invoices in a second language. The candidate scores two points higher overall and drops sharply on credit notes, where it returns the gross figure where the old model returned the net. Nothing in the aggregate score objected. The slice did.

The repair is narrow: an explicit instruction about credit notes, tested against that slice first, then the full set to confirm nothing else moved. The candidate shadows for several days of real traffic, takes a 5% canary share, and is promoted with two weeks to spare. Every step is ordinary release engineering. The only unusual part is remembering that a model needs it.

Where the gate leaks

An eval set measures the workload you had when you built it. Traffic drifts, new document types arrive, and a gate that passed in spring can certify a migration in autumn against a world that no longer exists. Refresh the set from recent production failures, or it becomes a museum.

Outputs are also non-deterministic, even at settings that should pin them down. A single replay run will show differences that are noise wearing the costume of signal. Repeat runs and thresholds, not exact-match comparisons, are what separate a regression from a coin flip.

And the rollback really does expire. Once the retirement date passes, the old behaviour is gone. Not deprecated, gone. Any problem you find afterwards gets fixed forward, on the new model, under time pressure. That is the argument for starting at the deprecation notice rather than the shutdown reminder. It is also the argument for not jumping on day one: a replacement that has been serving other people's traffic for a few months has a track record your evals can learn from. Use the notice window. It is the one part of this the providers actually give you for free.

Work with us

If your systems are pinned to snapshots with retirement dates on them, or worse, riding aliases nobody has scheduled, we run the migration like the release it is. Start at Trida AI's intake.

The Trida AI Academy: Courses, Skill Tests, and Blueprint Builds

FDE at Trida AI — Wed, 21 Oct 2026 01:28:35 +0000

Originally published at Trida AI.

Every company hiring forward deployed engineers runs into the same problem: the role is proven in production, but it is hired on paper. A resume lists technologies. A profile lists keywords. Nothing in the standard process shows whether an engineer can take a problem, build the system, and ship it. So hiring becomes a bet, and when the bet is wrong, it surfaces inside a customer's deployment.

Trida AI Academy is the proof system we built instead. At its core are three kinds of evidence: certification courses an engineer passes to earn a credential, skill tests that badge individual skills under time pressure, and blueprint builds — complete agents, built to a specification and reviewed by a person. Each one leaves evidence. All of it lands on a single profile. And that profile is what carries an engineer into Trida Network, where recruiters and companies see verified skill instead of claims. That is the flywheel: prove you can ship, be seen for it, get matched on it.

Proof one: certification

A certification course is the foundation. Engineers work through readings and hands-on labs, material built from real deployment work: production repositories, evaluation sets, the failure modes that only appear after launch. Every module ends with a quiz. There is no proctor and no cohort schedule; if a module does not clear, the engineer goes back, learns the part that was missed, and takes it again. Clear the core modules and an elective, and the credential issues automatically, carrying a unique ID that anyone can verify.

The credential is not attendance. It is a recorded pass, and the ID means a recruiter, a customer, or we ourselves can verify it.

Proof two: skill badges

Certifications establish breadth. Skill tests pin down the individual skills a deployment turns on. Each test is timed and scored, one skill at a time. Pass, and the badge goes on the engineer's profile.

The granularity matters. An engineer can be strong on retrieval and new to orchestration. A shelf of badges says precisely where the strength is, skill by skill, each one earned under time pressure rather than typed into a profile. For anyone evaluating an engineer, it reads like a map instead of a claim.

Proof three: blueprint builds

The third proof is a finished system. In Blueprint Builder, the engineer takes a specification for a real working agent in a real business vertical, written the way production systems are actually run, with a human holding the decisions that carry consequences. They build the agent end to end, publish it on GitHub, and submit it for review. A person reviews and certifies the build; it is not auto-graded, because what makes a build good is judgment, and judgment does not reduce to a test suite.

That matters for forward deployed work, because the job is never a slice of a system. It is the whole system, shipped.

Three kinds of proof: a certification course proves the foundation, skill tests prove individual skills under time pressure, and a blueprint build proves you can ship the whole system — reviewed, public, and verifiable.

The profile is the point

None of the three proofs lives in a drawer. The credential, the badges, and the reviewed build collect on one Academy profile: a record a third party can open and inspect. Verify the credential ID. Read the build on GitHub. See which skills carry badges. An engineer moving toward forward deployed roles does not have to argue that they can ship. The profile shows the evidence, in the open, before the first conversation.

Then the Network takes it from there

A verified profile is only useful if the right people see it. Trida Network matches Academy-certified engineers with companies, ranked by verified skill, not resume keywords, and introductions are reviewed by a person before they happen. Recruiters browsing the Network are looking at proof profiles: the credential, the badges, the public build. Companies can shortlist on evidence that was produced in the open, and engineers get seen for what they demonstrably can do.

The Trida flywheel: certifications, skill badges, and blueprint builds collect on one verified profile; the Network puts that profile in front of recruiters; and the Academy's material is built from real deployment work.

Why we built it this way

Trida AI is a deployment company, and the quality of a deployment is the quality of the engineers on it. "Trust us" is not a quality system. The Academy makes the standard inspectable: the courses, the tests, and the blueprints are public, and anyone can look at what they ask of an engineer. For companies, that means an engineer's skills can be evidenced before they ever touch a production system. For the market, it means the supply side of forward deployed talent gets built on proof instead of paper — and the loop is designed to compound: engineers prove themselves in the Academy, the Network matches them on that proof, and the Academy's material is built from real deployment work, so every round of the system is grounded in the work itself.

What proof does not settle

Proof covers skill. It does not measure fit, taste, or how an engineer works inside a particular team, and no credential settles those questions. That is why the blueprint review is a human judgment and why Network introductions are reviewed by a person. Evidence removes the cheapest doubt in hiring — whether the skill is there at all. The rest still takes people.

See the proof system

Start with the courses, where the foundation is earned and certified.

Test a single skill against the clock with a skill test, and put the badge on a profile.

Or go straight to the build itself: build a blueprint, in the open, reviewed by a person.

Sandboxing Agents That Run Code

FDE at Trida AI — Wed, 21 Oct 2026 01:26:35 +0000

Originally published at Trida AI.

Give an agent a shell and you have given it your shell.

Same home directory. Same SSH keys sitting in it. Same cloud credentials in the environment, same open network connection. The agent does not need to be malicious for this to go wrong. It needs to be obedient, following an instruction that arrived inside a web page, a package README, or a tool result you asked it to read. Prompt injection stops being a content problem the moment the reader can execute commands.

The usual first answer is permission prompts, a human approving each command before it runs. That decays in practice. Approving the fortieth command of a session is not a review, it's a reflex. Anthropic reports that sandboxing safely reduced permission prompts by 84% in its internal usage with Claude Code. Fewer prompts, applied where they matter, beat more prompts applied everywhere.

A prompt asks. A boundary refuses.

A sandbox (the isolated environment an agent's code runs in, in this post) is a boundary enforced below the agent, by the operating system or a hypervisor, not by the model's good behaviour. It is a short list of allowances. These directories may be read, these may be written. These network hosts may be reached. This much CPU and memory, for this long. Inside the list, the agent works freely and nobody approves anything. Outside it, a strict configuration has nothing to approve either: the call fails the way a call to a nonexistent file fails. Looser configurations soften the refusal into a notification and a chance to run the command outside the boundary instead, and that softness is the permission prompt creeping back in through the side door.

That placement is the whole argument. A system prompt that says "never read files outside the project" is advice the model can be talked out of by the next document it reads. An operating-system boundary cannot be talked out of anything. It also covers the part teams forget: the scripts, installers and subprocesses a command spawns. A boundary drawn around the parent has to hold for all of them, and the sandboxed Bash tool in Claude Code is built that way: its limits apply to every process a command starts, too.

Two walls, not one

The boundary needs a filesystem wall and a network wall, and each one is pointless alone. Filesystem isolation without network control still lets a script send anything it can read, anywhere. Network isolation without filesystem control still lets it read and alter the machine it lives on, including the configuration of the very proxy meant to contain it.

Together they close the loop. The filesystem wall says the working directory is the only writable place, and the paths that matter cannot be read at all. The network wall says there is no direct route out: traffic passes through a proxy running outside the sandbox, and the proxy checks each destination against an allowlist of domains.

The read half of that wall is the one teams leave open by accident. Claude Code's documentation is blunt about its own default: sandboxed commands can read most of the machine, credential files included, unless those paths are explicitly denied. A sandbox switched on with its read defaults untouched still lets a manipulated script read your keys; it just cannot send them anywhere while the network wall holds. Deny the credential paths explicitly. Do not let one wall cover for the other.

Credentials get the same treatment, and this is the part worth copying even if you copy nothing else. The real secret never enters the sandbox. Cloudflare's sandbox platform does this as secure credential injection: an egress proxy adds authentication to outgoing calls at the network layer, so the agent can use a service without ever holding its credential. If the code inside is compromised, there is no key inside to steal; the proxy authenticates only specific requests, to specific hosts.

The sandbox boundary: the agent and the code it runs sit inside an OS-enforced boundary with a writable working directory and no direct network route; all traffic passes through a proxy outside the boundary, which checks hosts against an allowlist and injects credentials; the host filesystem, SSH keys and cloud credentials remain outside, unreachable under the configured boundary.

Match the boundary to the tenant

Not every deployment needs the same strength of wall. Overspending here is its own failure: a sandbox so slow or so locked down that the agent cannot install a dependency will be switched off by Friday.

For a single developer's own agent on their own machine, an OS-level sandbox is usually the right weight: Linux bubblewrap and macOS Seatbelt primitives, with no container to spin up or manage. The threat is a manipulated agent reaching beyond the project, and the OS boundary answers it.

For shared infrastructure (many sessions, many users, code you did not write), you want a kernel between the workload and the host. An application kernel such as gVisor intercepts the container's system calls and services most of them in user space, limiting what reaches the host kernel directly, which is why Google Cloud offers it as GKE Sandbox for pods that need hardening. A step beyond that, a microVM gives each workload its own kernel behind a hypervisor. Firecracker microVMs were built at AWS for this multi-tenant case, and the project's own figures put boot time under 125 milliseconds, with under 5 MiB of memory overhead per microVM. Isolation priced in milliseconds.

Whichever layer you pick, three settings matter as much as the layer: a wall-clock limit, so a runaway loop dies on its own; resource caps, so one session cannot starve the host; and an ephemeral filesystem, so anything the run created dies with it unless deliberately exported.

One run, traced

Take a typical run, configured the way this post argues for, and watch where the walls meet the work. The task: clone a repository, run its test suite, fix the one failing test.

One run traced end to end: the agent clones through the credential proxy, edits and tests freely inside the workspace, an install script's attempt to read the SSH directory and contact an unlisted host are both refused by the boundary, and the finished diff leaves through the proxy while the sandbox is destroyed.

The agent's git client authenticates to the proxy with a scoped credential; the proxy verifies the repository and branch, attaches the real token, and forwards the request. The agent then edits files and runs the test suite inside the workspace — dozens of commands, zero permission prompts, because none of them can leave the boundary.

Mid-run, a dependency's install script does two things its author should not have asked for. It tries to read the SSH directory, and it tries to send a request to a host that is not on the allowlist. Inside the sandbox the first read is refused: that path was explicitly denied when the boundary was configured, so for this run it might as well not exist. The second request reaches the proxy and is refused, and the refusal is logged with the host named. The install continues without that step; the tests pass. The diff and the logs are the only things that leave; then the sandbox is destroyed, along with everything the install script wrote.

What a sandbox does not do

It does not judge intent inside the walls. An agent allowed to write to a repository can still write a bad change to that repository, and an agent allowed to call an API can still make a wrong call the API permits. Containment answers "how far can this reach?" It says nothing about "should it?" Reviews, evals and scoped permissions still carry that question.

Its coverage also has edges you have to read, not assume. In Claude Code, the documented boundary wraps shell commands; its file tools, MCP servers and hooks run outside it, following permission rules instead. A team that enables the sandbox and declares the agent contained has checked one door of several. Whatever runs the agent, the agent process itself included, needs to sit behind a boundary if the claim is going to hold.

Set the allowlist from a specification and it will be wrong in both directions: hosts the work needs will be missing, and hosts nobody needs will sit there because someone added them "just in case". Set it from observed traffic during a supervised run, then tighten. A proxy log that has never denied anything usually means the list is too wide to mean much.

Not every agent needs one, either. An agent with no shell, no code execution and a handful of scoped, read-only API tools has a small surface, and a sandbox around it buys little. Sandbox the execution. That is where the reach is.

Work with us

If your agents run code on machines that also hold production credentials, the useful question is not whether the model behaves — it is what a bad run can reach. Drawing that boundary, and setting the allowlist from your agents' real traffic, is deployment work we do. Tell us about it through Trida AI's intake.

Too Many Tools Hurt Agents

FDE at Trida AI — Wed, 21 Oct 2026 01:19:50 +0000

Originally published at Trida AI.

Connecting one more MCP server is the easiest way to make a working agent worse.

Nobody does it on purpose. The agent handles support tickets well, so someone adds the billing server. Then the CRM. Then the deployment tools, because the agent kept asking a human for things it could have looked up itself. Each addition is reasonable. The result isn't. The agent that was reliable in June is picking the wrong tool by September, and the post-mortem blames the model.

The model is usually innocent. What changed is the size of the choice it faces at the start of every request.

Every tool an agent might call has a definition: a name, a description, a parameter schema. In the standard setup, all of those definitions are loaded into the model's context before it reads a word of the request. Anthropic's worked example of a five-server setup (GitHub, Slack, Sentry, Grafana, Splunk) comes to 58 tools consuming roughly 55K tokens before the conversation starts, and Anthropic reports seeing definitions reach 134K tokens. That's a tax paid on every request, for tools the request will mostly never touch.

The tax is the visible part. The accuracy cost hides better. Anthropic notes that the most common failures in that setup are wrong tool selection and wrong parameters, especially between similar names. Its example pair is notification-send-user and notification-send-channel, two definitions that differ by one word and opposite intent. When Anthropic's Tool Search Tool loaded only the few tools a request actually needed instead of the whole catalogue, token use fell by about 85%, and accuracy on its internal MCP evaluations rose sharply: Opus 4 from 49% to 74%, Opus 4.5 from 79.5% to 88.1%. Same models. Smaller choice.

The catalogue is not the working set

Here's the distinction we think teams miss. A catalogue is everything the agent is allowed to reach, ever. A working set is what this request needs, usually three to five tools. Loading the catalogue to serve the working set confuses storage with attention. You wouldn't hand a new engineer the manual for every system in the building before their first ticket. You'd tell them where the manuals live.

So the fix is retrieval, applied to tools the way it's already applied to documents. A search step reads the request, ranks the catalogue, and loads a shortlist. The RAG-MCP study tested exactly this against a large pooled catalogue: with every definition in context, tool-selection accuracy sat at 13.62%; with retrieval narrowing the candidates first, it reached 43.13% accuracy, while prompt tokens fell by more than half. A 43% score is nothing to celebrate in absolute terms. The direction is the finding: the model didn't get smarter, the question got smaller.

A more recent shortlist study adds a useful wrinkle. On the BFCL benchmark, an adaptive shortlist that showed about seven tools per query, chosen per request, matched the coverage of showing fifty every time, 90.3% against 90.8%. The same study found the catch on ToolBench: a fixed shortlist of five found nothing at all on the hardest queries, where the right tool ranked sixth to twentieth, while the adaptive one, allowed to look deeper when the first pass was thin, still found 16.7% of them. Shortlists have to be able to widen. A retrieval step that can't miss loudly will miss silently.

It also means the raw count was never quite the villain. Fifty well-separated tools can be an easier choice than ten that overlap. What degrades selection is neighbours: tools whose descriptions promise nearly the same thing, so the model splits its confidence between them and picks by wording accidents. Count is just the easiest way to accumulate neighbours.

Deferred loading in practice: the request is searched against the full tool registry, a shortlist of three to five tools is loaded, and the model chooses from that shortlist plus the small resident core. The rest of the catalogue never enters context.

What we'd change first

Start by splitting the catalogue. A handful of tools get used on most turns. Those stay resident, loaded every time. Everything else is deferred: present in a registry the agent can search, absent from its context until a request actually calls for it. On most stacks this is configuration, not surgery.

Then consolidate. Anthropic's guidance on writing tools for agents is blunt that more tools don't always lead to better outcomes, and its remedy is to build the tool around the workflow instead of the API: one scheduling tool that checks availability and books, rather than separate tools for listing users, listing events and creating events that the model has to chain correctly every time. Every consolidated tool deletes several near-neighbours at once.

Descriptions get the same treatment. A good tool description reads like directions to a colleague: what it does, when to reach for it, and — the part everyone skips — when not to. "Use this to message one person. For a channel, use the channel tool." Two sentences like that do more for selection accuracy than a model upgrade, because most wrong-tool calls are really two descriptions that never agreed on the boundary between them.

Last, measure selection itself. Log which tool was chosen, which were on the shortlist, and which one the completed task actually needed. Wrong-tool rate is a metric you can watch fall as you defer, consolidate and rewrite. If it doesn't fall, the problem is somewhere else and you've learned that cheaply.

One request traced end to end: posting a deploy failure to the incident channel. Search over the deferred catalogue returns three candidates, the model picks the channel tool over its one-word neighbour, and the other fifty-five definitions never enter context.

Where this bends

A six-tool agent doesn’t have this problem, and adding a search step to it buys latency and a new failure point in exchange for nothing. Anthropic’s own guidance puts the payoff at ten-plus tools, or once definitions pass roughly ten thousand tokens. Below that, load everything and spend your effort on the descriptions.

Retrieval also moves the failure rather than deleting it. If the right tool isn’t in the shortlist, the model can’t call it, and it will often complete the task with the wrong tool rather than report the gap. That’s why the widening behaviour matters, and why the resident core should be chosen from logs of what your agent actually calls, not from what the architecture diagram says it should.

And none of this rescues bad tools. Defer a confusing pair and they still confuse, just later and in a smaller room. The shortlist is a way to stop paying for the whole catalogue on every request. The catalogue still has to deserve the agent’s trust: one tool per job, boundaries written down, names that don’t collide.

Work with us

If your agent’s tool list has grown server by server and its wrong-tool calls have grown with it, the catalogue is where we’d start: what stays resident, what gets deferred, and which tools should never have been separate. Bring us the list through Trida AI’s intake.

Structured Output Is Not a Guarantee

FDE at Trida AI — Wed, 21 Oct 2026 01:18:07 +0000

Originally published at Trida AI.

The invoice parses perfectly. Every field is there, the total is a number, the currency is one of the allowed values, the date is a date. Your pipeline writes it downstream without a murmur — and the total is wrong. The model transposed two digits, the schema never noticed, and nobody was ever going to notice, because the check everyone trusted had already passed.

This is the misunderstanding we see most often in production AI work right now. Structured output, schema-constrained generation, is a real advance, and teams keep reading its guarantee as bigger than it is. The guarantee covers the shape of the answer. It was never about the truth of the values inside it.

What the constraint actually buys

The mechanism matters here, because the mechanism is what makes the guarantee strong in one place and absent in another. When you send a JSON Schema with the request, the provider compiles that schema into a grammar and constrains generation against it: tokens that would break the schema are not available to the model as it writes. OpenAI describes Structured Outputs as ensuring the model will always adhere to your supplied JSON Schema, so a required key is not omitted and an enum value is not invented. Anthropic describes its implementation the same way: a guarantee delivered through constrained decoding.

That buys you something genuine. A whole class of plumbing failures disappears: no more prose wrapped around the JSON, no missing field three lines before a crash, no retry loop whose only job is begging for the same shape again. If your system consumes model output in code (and in production, something always does), this is the difference between a format you hope for and a format you can build on.

But notice what the grammar cannot see. It knows that total must be a number. It does not know what the invoice said. It knows status must be one of four values. It does not know which one the document in front of it supports. Shape is a property of the answer alone; truth is a relationship between the answer and the world, and the grammar only ever sees the answer.

The validation stack: constrained generation produces the shape, a schema check confirms it, semantic checks test the values against the source and the rules, and only then does output reach the downstream system. Failures go to one bounded repair attempt, then to a reject queue.

Three ways a valid answer is still wrong

The first is the plain wrong value: a transposed total, a date read from the wrong line of the document, a name taken from the signature block instead of the letterhead. Every field is the right type. The extraction is simply incorrect, and schema validation is structurally incapable of catching it.

The second is internal inconsistency. Line items that sum to 4,280 and a total field that says 4,820. A start date after the end date. A headcount of twelve and a list of nine names. A schema describes fields one at a time; many production errors live in the relationships between fields, which is exactly the territory a schema does not cover.

The third is subtler, and it is created by the constraint itself. In OpenAI's strict mode, every property in the schema is required — there is no honest way to say "the document doesn't state this." So when the source is silent, the model still has to put something in the field. A required field pointed at a document that doesn't contain the answer manufactures a plausible value. The tighter you make the schema, the more important it becomes to give the model an explicit way to say "not stated" (a null where the schema allows one, or a dedicated value in the enum), and to treat that answer as a routing signal, not a failure.

The guarantee has edges, and the vendors say so

Read the provider documentation closely and the guarantee is narrower than the marketing version. Anthropic's documentation is unusually candid about this. Some constraints in your schema — numeric bounds, string lengths — are not enforced during generation at all; the SDKs rewrite them into field descriptions, and validating SDK helpers check them against the original schema afterwards. A refusal or a response cut off at the token limit can return output that doesn't match the schema. Even enum casing is not guaranteed: a value can come back differing from the schema only in capitalization, with no error raised.

None of this is a reason to avoid structured output. It is a reason to keep your own validator downstream of it, permanently, even on the days it never fires. The bad days are the ones that matter.

Validation first, repair second, rejection always available

The production pattern we argue for has three layers, and the order is the point.

First, deterministic semantic checks — the ones a schema can't express. Totals against line items. Values against the source text: does this figure actually appear in the document? Codes and identifiers against the systems that own them: does this purchase order exist? These checks are ordinary code, cheap to run, and they catch the errors that actually cost money.

Second, bounded repair. When a semantic check fails, send the output back once — with the specific error attached, not a generic "try again." "Total does not match the line-item sum of 4,280; re-read the source" is an instruction a model can act on. One attempt, occasionally two. A repair loop with no bound is not robustness; it is a system that has decided to keep guessing until something passes, and something eventually will.

Third, a reject path that is a real destination: a queue a person works, with the source, the failed output, and the failed check all attached. The reject queue is also your best design feedback. A field that lands there every week is telling you the schema, the prompt, or the source document is wrong, and that signal is worth more than another retry.

One invoice through the stack: the model's schema-valid answer carries a transposed total, the line-item check catches it, a single repair with the error attached corrects it, and the verified record is what reaches the downstream system.

Where this leaves the schema

None of this argues against schemas. It argues for treating the schema as the first layer of a validation stack rather than as the stack. Design it deliberately: field order matters, because generation follows it — put the evidence a value rests on before the value where you can, and let the model work outward from the source instead of committing early and rationalizing backwards. Keep schemas inside the providers' complexity limits, and expect the first request against a new one to pay a grammar-compilation cost that later requests don't.

The honest summary is short. Structured output ended the era of parsing apologies: the almost-JSON, the missing key, the shape that changed between calls. What it did not end, and never claimed to if you read the documentation it ships with, is the need to check whether the answer is right. Shape is guaranteed. Truth is still your job.

Work with us

If your team's extraction or agent pipeline trusts schema-valid output a little more than it should — and most do, right up until the first wrong value reaches a system of record — that gap between shape and truth is exactly the kind of problem we work on. Tell us about it through Trida AI's intake.

Multi-Agent Systems in Production

FDE at Trida AI — Wed, 21 Oct 2026 01:11:22 +0000

Originally published at Trida AI.

Every vendor demo now seems to end the same way: one agent hands work to another, a third checks it, a fourth reports back. It looks like a team. It demo-animates like a team.

Production is less kind. The honest summary of what has actually been measured so far is narrower and, we think, more useful: multiple agents help a lot on one shape of problem, and actively hurt on another. The dividing line isn't the framework you pick or the model you run. It's how tightly the pieces of the work depend on each other.

Where it earns its keep

The strongest published result is a research system, and the shape of the task explains the result. Anthropic's multi-agent research system puts a lead agent in charge of planning and synthesis, with subagents searching in parallel, each in its own context window. On Anthropic's internal research evaluation it outperformed a single agent by 90.2%, and token usage alone explained most of the performance spread the team measured. Research rewards that shape: the question breaks into independent directions, each direction is mostly reading, and a subagent that goes down a dead end doesn't corrupt the others. Its findings come back as text for the lead to weigh.

Notice what the subagents don't do. They don't build on each other's half-finished work. They don't make decisions the others have to live with. They gather, compress and report, and the one agent that decides sees all of it.

That pattern generalises beyond research. Checking a claim against many sources. Reading a large document set where each document can be judged on its own. Triaging a queue of independent items. Fan-out works when the units are separable and the cost of one unit failing is a retry, not a rebuild.

Where it breaks

Split a piece of making in half and the failure shows up immediately. Cognition's widely argued essay, Don't Build Multi-Agents, makes the point with a small example: two subagents building halves of one game, each working reasonably, producing pieces built on conflicting assumptions that no final agent can cleanly merge. Their underlying claim is the one to hold onto: every action an agent takes carries decisions it never stated, and agents that can't see each other's decisions will eventually make incompatible ones.

This isn't only an argument from one team's experience. The MAST study built a failure taxonomy from more than 1,600 annotated traces across seven multi-agent frameworks, found failure rates of 41% to 86.7% across the systems it examined, and sorted the failures into three buckets: system design issues, misalignment between agents, and failures of verification. Read that list slowly. None of the three is "the model wasn't smart enough." They're coordination failures, the same class of problem that makes human projects fail, except that agents hold the misunderstanding silently and execute it at machine speed.

There's a second, quieter cost. More agents means more tokens, and not a little more. Anthropic reports its multi-agent system using roughly 15 times the tokens of a chat interaction, against about four times for a single agent. Fifteen times the spend buys a real gain on the right task. On the wrong one it buys you a faster, more expensive route to an answer one agent would have reached, plus a synthesis step where errors from several threads get averaged into something that reads as consensus.

The test we apply

A worked example — one claim checked end to end: the brief the lead writes, what a worker returns, and the check at the boundary before anything is merged.

Before a deployment goes multi-agent, we want four answers, and they have to be yes in the work itself, not in the pitch for it.

Can the task be split into pieces that don't need to see each other working? If piece B changes its mind when it learns what piece A decided mid-flight, you don't have two tasks. You have one task and a coordination problem wearing a costume.

Is the work mostly reading, searching and judging — rather than writing into shared state? Parallel readers are cheap to reconcile. Parallel writers are not.

Can each piece's output be checked on its own, before anything is merged? A verifier that only sees the combined result arrives too late; by then you can't tell which thread introduced the error. Verification belongs at the boundary of each subagent's work, where a bad output is still small.

Is the task valuable enough to pay roughly an order of magnitude more in tokens for a faster or broader answer? Sometimes it plainly is. Often the honest answer is that a single agent, given better tools and a well-built context, gets there for a fraction of the cost.

When those answers are yes, the architecture that holds up is boring in a good way: one lead agent that plans, delegates with written briefs that state the objective, the boundaries and the output format, and synthesises; workers that return findings rather than half-built artifacts; and a check on each return before it enters the lead's picture of the world. Anthropic's team found the briefs mattered more than expected: vague delegation was behind duplicated work and gaps in its early versions. That's familiar to anyone who has managed people. It turns out to be just as true here, and less forgiving, because an agent won't come back and ask what you meant.

The architecture that holds up — one lead agent plans and synthesises, workers search in parallel in their own context windows, and every return is checked before it merges.

What still isn't working

Real-time coordination is the open problem. Agents are poor at renegotiating a plan mid-task, at noticing that a sibling's output invalidates their own, at knowing when to stop. Frameworks will keep adding protocols for agent-to-agent communication, and some of that plumbing is useful. But a message bus doesn't fix a spec problem, and most of the failures in the MAST taxonomy are spec and coordination problems, not plumbing problems.

Our position is simple, and it will probably age: default to one agent. Add a second when the task's shape demands breadth that one context can't cover (not the demo, not the org chart in the slide), and keep the deciding in one place while you do it. The teams publishing the best results are, underneath the diagrams, doing exactly that.

Work with us

Splitting a workflow across agents is easy to demo and hard to keep honest in production. If you're deciding where a single agent ends and a team of them should begin, bring the workflow to Trida AI's intake and we'll work through the shape of the task with you.

Long-Horizon Agents Need a Map, Not a Bigger Context Window

FDE at Trida AI — Wed, 21 Oct 2026 01:09:38 +0000

Originally published at Trida AI.

An agent on a long task usually loses its way well before it runs out of context. The window still has room. The transcript holds everything it has done, in order, at full length. What it no longer holds, in any usable form, is where the work stands.

That distinction matters because the fix most teams reach for is size. Give the model a larger window, feed it more history, let it carry the whole job in its head. Size helps with one problem: information that was dropped entirely. Most long-horizon failures are a different problem. The information is present. It is buried under tool output, dead ends and superseded attempts, and the model has to reconstruct its own position from that pile on every step.

The trend line makes this urgent rather than academic. METR's measurements put the length of tasks agents can finish at 50% reliability as doubling roughly every seven months, and the same work found success falling below 10% on tasks that take a person more than about four hours. Tasks are getting longer faster than raw memory is getting more reliable. So the question for a deployment is not how much an agent can hold. It is how the agent knows where it is.

A transcript is not a map

A transcript records a journey the way a dashcam does: complete, sequential, and almost impossible to navigate by. Ask it a simple question mid-task (what have we already ruled out?) and the answer has to be mined from hundreds of turns, some of which contradict later turns.

Worse, a long transcript actively degrades the reasoning done over it. Chroma's Context Rot study tested 18 production models and found performance getting less reliable as input length grew, in uneven ways, even on simple retrieval and copying tasks. Similar but irrelevant material hurt more than plain filler. A long agent transcript is exactly that material at scale: near-miss attempts, stale intermediate results, and tool output that was useful for one step and noise ever after.

A map solves a different problem than memory does. It does not try to keep everything. It keeps position: the goal, what is done, what was decided and why, what is still open, and where the underlying detail lives if it is needed again. A driver crossing a continent does not memorise every mile driven. They carry a map, a current location, and a route — and they can re-orient in seconds after any detour.

Same journey, two records. The transcript keeps everything and answers nothing quickly. The map keeps position and points to the detail.

What goes on the map

Keep it small enough to read at a glance, and structured enough that updating it is a habit rather than a chore. In our view five entries earn their place.

The goal, stated once, in the terms the work will be judged by. Then current position: the last verified state of the work, not the last thing attempted. Decisions belong here too, with the reason attached, because a decision without its reason gets relitigated three hours later by an agent that cannot tell it was settled. Open ends come next: the questions still live, the branches not yet taken, the failures already ruled out. Finally, pointers. File paths, document links, query names: lightweight references the agent can follow to reload detail on demand, instead of carrying that detail in context the whole way.

That last entry does quiet, important work. Anthropic's writing on effective context engineering describes the same pattern from the other side: keep identifiers in context, load the underlying data just in time through tools, and treat context as a finite attention budget rather than a storage bin. Their long-horizon toolkit — summarising history when it grows unwieldy, keeping structured notes outside the window, and letting focused sub-agents return short distilled summaries instead of raw workings — is, in effect, map maintenance described three ways. We agree with the toolkit. Our argument is about priority: build the map first, deliberately, as a designed artifact. The rest follows from it.

A map also changes what a fresh start costs. When a run is compacted, restarted, or handed to another agent, the newcomer does not need the saga. It needs the map and a few pointers. Anthropic's later work on harnesses for long-running agents makes a parallel point in software work: durable progress files and a structured list of what is done and what is not let a new session orient itself and make incremental progress instead of declaring victory early or redoing finished work. Orientation, in both accounts, is a file — not a memory.

Keep the map honest

A map that is written once and never updated is worse than none, because the agent trusts it. So the working rule is simple: no step counts until the map reflects it, and verification is what updates position. An agent that has attempted a fix has moved its transcript forward. It has only moved its position when a check (a test, a query result, a human confirmation) says the fix holds.

That rule also gives long runs a natural rhythm. Work in increments small enough to verify. Record the outcome, including the failures, since a ruled-out branch written on the map never gets explored twice. When the map and the territory disagree, believe the territory, fix the map, and note the disagreement. It usually marks the spot where the task was misunderstood.

The loop that keeps a long run oriented: position changes only when a result is verified.

Where a map does not help

Short tasks do not need one. If a job finishes in a handful of steps inside one context, map-keeping is ceremony, and the transcript is the record. Maps also decay. On a fast-changing task the maintenance cost is real, and a stale map misleads with confidence, which is why the verification rule above is not optional decoration.

There is a harder limit underneath. A map fixes orientation; it does not fix judgment. An agent that misreads its goal, or verifies its work against the wrong check, will maintain an excellent map of the wrong journey. And summarising a long history into map form always discards something whose importance only shows up later. Every team that compacts context learns to tune what gets kept, starting generous and tightening with experience. The map is a discipline, not a guarantee.

Still, the direction is clear. Windows will keep growing, and long tasks will keep degrading inside them for the reasons the context-rot results describe. The teams whose agents finish four-hour, eight-hour, day-long work reliably will not be the ones who waited for a window big enough to hold the whole job. They will be the ones who taught their agents to carry a map, keep it current, and check their position against the territory as they go.

Work with us

If your team is stretching agents from ten-minute demos toward work that runs for hours, the bottleneck is probably orientation rather than model size — and it is fixable with structure, verification and honest state-keeping. Tell us where your long runs lose their way through Trida AI's intake, and we will map out what it takes to keep them on course.

The FDE Shortage Is a Proof Problem

FDE at Trida AI — Wed, 21 Oct 2026 01:02:54 +0000

Originally published at Trida AI.

Earlier this month, Business Insider reported that Asa Kalavade, Amazon Web Services' vice president for forward-deployed engineering, calls qualified FDE candidates "unicorns". AWS is not short of commitment. It announced a $1 billion forward-deployed engineering organization in June and plans to staff it with thousands of engineers. Forrester estimates that Amazon, Microsoft, OpenAI and Anthropic pledged roughly $9 billion to FDE ventures between May and July alone. The money is there. The people are the constraint.

Kalavade was specific about what AWS screens for. Three things: a product customer mindset, deep software expertise, and hands-on experience building with agents. Most engineers have one of the three, maybe two. The unicorn is the one carrying all of them at once.

So where does AWS find them? Mostly, it doesn't — yet. The first hires are seasoned operators, people who have already done deployment work elsewhere. That works as a seed. It does not scale into thousands, because the pool of finished FDEs is small and every hyperscaler is fishing in it at the same time. Universities are not refilling it either; per the same reporting, colleges are still building their first FDE training programs. An industry that needs thousands of a new kind of engineer has almost no way to certify one — and no shared proof that a given engineer can do the job.

Two of the three skills never show up on a resume

Look at AWS's list again. Software expertise is the easy third: work history shows it, a technical interview can probe it. The other two are different. A product mindset means the engineer walks into a customer's operation and finds the problem worth solving, not just the ticket they were handed. Agentic experience means they have actually shipped systems where models act — with the guardrails, evals and failure handling that production demands, not a demo.

Neither appears on a CV in any form a recruiter can check. Candidates claim both; interviews guess at both. This is why FDE hiring stays slow even with unlimited budget behind it. The bottleneck is not finding applicants. It is verifying the two skills that decide whether a deployment works, and a resume is the wrong instrument for that job.

Trida AI's route: engineers earn an Academy certification against the three skills the role demands, the credential makes that skill verifiable by anyone, and the Network matches certified engineers to companies by verified skill.

How Trida AI built the certification route

Trida AI was set up around this exact gap. We deploy forward-deployed engineers into enterprise clients, and we kept hitting the same wall as everyone else: skill that could not be checked before an engagement started. So we built the other side of it — a way for engineers to prove the skill, and for companies to hire on that proof.

The Trida AI Academy does not train anybody. Its certifications are free, self-paced and always open — no cohorts, no schedule — and they are earned, not attended: readings, hands-on labs, and quizzes the engineer passes themselves, on material built from real deployment work, production repos, eval sets and failure modes. Clear the bar and the credential issues with a unique ID that anyone can verify. What gets certified is the same three dimensions AWS screens for: production-grade software, agentic systems built and evaluated by hand, and business problems worked end to end instead of toy examples.

The Trida Network is where certified engineers meet the work. It matches Academy-certified forward-deployed engineers with companies that need them, ranked by real, verified skill rather than a resume keyword search — and when a role and a candidate line up, a person reviews the match before the introduction is made. An engineer who has demonstrated the three-part package does not need to call themselves a unicorn. The evidence is attached to the profile.

That gives a company two ways in: deploy with Trida AI's own engineers, or hire certified talent directly through the Network and run the deployment yourself. Either way, the starting point is the same — skill that has been tested before the engagement begins, not discovered during it.

The hyperscalers will solve their own hiring problem eventually; $9 billion buys a lot of persistence. But most companies do not need thousands of FDEs. They need two, or five, or a pod for one hard workflow — and they cannot wait for the talent market to sort itself out. For them, the question is whether proof of FDE skill exists outside any single vendor's hiring process. It does now. Building it was the point.

Work with us

If you need forward-deployed engineers and the job market is offering you unicorns, start with talent that is already certified and verified — tell us what you are trying to deploy through Trida AI's intake.

Insurance Cash Reconciliation Agent Blueprint

FDE at Trida AI — Wed, 21 Oct 2026 01:01:09 +0000

Originally published at Trida AI.

Every morning, the cash team at a specialty insurer faces the same pile: money landed in the bank overnight, and someone has to work out which policies it pays. The wire amounts rarely match the bills. One transfer covers dozens of policies, explained only by an advice written as prose. An insured has paid late, or short, or twice. Some of the money matches nothing anyone can see.

The errors in that pile are quiet ones: cash applied to the wrong policy looks exactly like cash applied to the right one. So this open blueprint, the Insurance Cash Reconciliation Agent, takes the working-out as its job. The matching arithmetic is checked in code. A model is used only where judgment is needed: reading the unstructured paper, proposing where unexplained money belongs. And nothing posts itself: a named controller approves every posting, and a day counts as reconciled only when every cent of it is accounted for.

One note before the machinery: the carrier in the repository is Northgate Specialty Insurance, and it is a reference prototype. The architecture and the code are the deliverable, and both are public.

Why the cash never equals the bill

Most premium arrives through brokers, and brokers remit net: they deduct their contracted commission and send the rest. A wire for less than the billed premium is usually not a short payment. It is the correct payment, and recognising it means knowing the contract behind it, and which of its rates was in force the day the cash arrived. Rates change mid-book; the change reprices nothing backwards.

That history also exposes a particular failure shape. When a broker deducts at a rate that is clean and plausible but not the contracted one, the wire settles nothing exactly, and the pattern in the numbers says why. The blueprint raises it as its own exception and drafts the query note for the broker: what was expected, what arrived, the difference. Drafted, never sent: sending is a person's act.

The day's flow. Matching is arithmetic, checked in code; the model reads and proposes; a person approves the posting.

Reading is the model's job. Matching is not.

An exact reference with an exact amount settles first in the fixed order of attempts. Then the netting-adjusted figure, premium minus the commission the contract allows. A reference whose amount differs is never force-fitted to the bill: the cash applies to what it genuinely covers, and the difference becomes a partial or overpaid exception with an aging clock. Duplicates are caught before any of it runs.

The model's work sits inside that order, in three places. It reads remittance advices, the emails and documents that explain a lump sum, and turns them into per-policy allocations, each required to cite the words in the document it came from. It proposes an allocation for a lump sum nobody explained, restricted to the paying broker's own open items, summing to the wire. And it drafts those broker query notes. Every reading is validated before it is worth anything: advice lines that do not sum to the wire are refused, allocations to cancelled policies are flagged rather than posted, and an unreadable document is held whole while the day carries on.

Installments get worked one by one, oldest first; a reference naming a specific installment settles that one first, and the record shows the choice. A free-text hint pointing elsewhere is information, not instruction: the cash applies oldest-first, and the difference is flagged.

Suspense is a place, not a bin

Cash that survives every attempt unexplained goes to suspense, carrying a reason code and the date it arrived. Auditors ask about this pile first, so it is treated as work in progress, never as a bin. Each item sits in an aging view, oldest first, owned by a named member of the team. Information often lands after the cash does, and a re-match pass works the open items against the books as they now stand; recoveries still arrive as proposals, in a pack, at the gate.

The hardest items have no payer to work from at all. For those, a resolution scan reads the whole book and builds candidates under a strict discipline: a candidate exists only when the arithmetic is exact, or when the wire's text names the policy, quoted verbatim. A model may rank them, and may add one only by quoting that text. Policies the evidence names but the cash cannot settle come back flagged, considered and unchoosable. The item stays open while all this stands proposed, because a hypothesis is not a claim on the cash. A controller picks one candidate, or none.

Two partial wires from the same payer can also settle one bill, when they price out exactly, each at the rate of its own day, so a pair can straddle a rate change. A pair that lands a cent short proves nothing, and neither does an exact sum across two different payers.

Some cash is never identified. After a governed age it can be proposed for a governed end: written off to income, or moved to an unclaimed-property review list, because it may be owed to someone the desk could not find. Either way it takes two approvers, and the assembler can be neither. Review-list money stays visible, with its full chain beside it.

The footing rule. A day is reconciled only when the three figures add up to what the bank says arrived.

A day counts only when it foots

The day's report is the control everything answers to: the bank total, split into what was applied, what sits in suspense, and what has not been worked yet. If they do not add up, to the cent, the report says so in words and refuses to call the day reconciled. On a book with more than one currency, the identity holds per currency first. Cash is never converted to hunt for a match, so a wire that arrives in the wrong currency waits in suspense in its own denomination, with its reason named. Conversions recorded for reporting use the rate in force on the day the cash arrived, and keep all three figures: the cash, the rate, the converted amount.

Money flows back out too. A policy cancelled mid-term earns a return premium, computed pro-rata in code and owed back to the insured, or to the broker who collected it. Cash that arrives later for that policy meets the amount owed before it meets anything else. Payments can also be taken back by the bank; a return un-applies the cash by replaying the original arithmetic exactly, and a foreign-currency return reverses at the rate recorded when the cash first arrived, even if the rate has since moved. Approved packs are never rewritten; they gain a flag, and the return stands as its own entry.

Month end gets the same control at a larger scale: the close pack rolls suspense from opening to closing, currency by currency, and foots or says that it has not. Each broker's own monthly statement is checked against the books line by line, and every disagreement is classified: agreed, commission differs, amount differs, missing on the books, missing on the statement. The model's proposals are graded too, by what the gate actually did with them, and each broker's paperwork is scored on the record.

What the checks prove, and what they don't

The blueprint is tested against planted cases built into its months: the pair of wires one cent short, the wire in the wrong currency quoting a bill's exact amount in the other one, the broker who applied next quarter's commission rate a month early. Passing those cases means the code still behaves as designed on the traps the builders thought of. That is a regression gate, not evidence about your book or your brokers.

The habit that gives the gate its value is unglamorous: when the desk meets a new failure shape, the case goes in first, and the behaviour changes second.

Honest limits

The book in the repository is Northgate's, chosen to exercise the shapes a real desk meets. Rates, thresholds and waiting periods are the blueprint's starting set; a deployment replaces them with the carrier's own contracts and policies first.

The built-in fallback reader handles only advices written in a tidy, predictable shape, and real remittance paper is rarely tidy. Reading free-form documents well is the connected model's work, and live verification against a real model is still pending; the blueprint would rather say that than grade its own homework. The boundary at the far end is deliberate too: the system never moves money and never writes to a general ledger. Its output is an approved posting file, an audit record, and reports that foot. A person still uploads the file.

Work with us

If your cash team starts its mornings with a pile of wires that do not match the bills, and a suspense balance nobody wants to explain to an auditor, start with Trida AI's intake: a few questions about your book and your brokers, then a conversation with the founders and engineers who would build it with you.

Get the Code on GitHub: https://github.com/tridaai/blueprint-insurance-cash-reconciliation

Text-to-SQL Against a Real Schema: Where the Demo Database Stops Telling the Truth

FDE at Trida AI — Wed, 21 Oct 2026 00:54:25 +0000

Originally published at Trida AI.

Twelve questions, three production models, each asked three times. On the clean database the scores ran from 5 to 11 of 12. On the same data in legacy dress they ran from 6 down to 1. The failures were the same failures, model after model.

We built the test twice. The small dataset is a sales book: seven customers, six products, eleven orders. The larger one is a wholesale business: 47 active customers, 20 products, 161 live orders and four deleted ones besides, three companies filed under two customer records each, two credit holds. Each dataset is stored two ways. The first copy is the demo version: names that say what they mean, dates in one format, statuses stored as words. The second holds exactly the same business data under the rules real databases accumulate: abbreviated names, undocumented integer status codes, dates in three or four formats, deleted orders still in the table, product attributes in a key-value table, credit holds written only in a notes field.

Then we asked Anthropic's Claude Sonnet 4.5, Opus 4.5 and Haiku 4.5 the same twelve questions against each copy, three runs apiece: 432 answers in all. Same prompt, the schema definition as the only context, no retries. Gold answers were written first, and the gold queries return identical results on both copies of each dataset, so any gap is the schema's doing, not the data's. A question counted as correct only when the executed result matched the gold result exactly.

The test rig: one dataset, two schemas, the same twelve questions, scored against gold answers.

The clean schema failed in exactly one way

Every easy question passed on the clean schema: counts, totals, a category lookup. The six misses, three medium and three hard, share a single cause. The model had to guess the literal values stored in the status columns, and it guessed the way a person writes: 'Completed' with a capital C, 'partially paid' where the stored value is just 'partial'. The database holds 'completed', lowercase.

Those guesses returned nothing. Not wrong numbers: empty result sets and null totals, because a filter that matches no rows produces silence, and a sum over silence is null. The joins were right. The aggregations were right. Revenue by category came back with the correct shape and zero rows inside it.

There was no consistency to lean on, either. The same model wrote 'completed' correctly in one question and 'Completed' in another. A schema definition tells a model which columns exist. It says nothing about the values inside them, and in this run the values were the whole game.

The legacy schema failed in four different ways

Four misses trace to a deleted order nobody filtered out. One order in the legacy table carries a deleted flag, and it is worth 1,400. The model ignored the flag four times: a total inflated by exactly 1,400, a count of 10 completed orders instead of 9, quarterly revenue of 5,160 instead of 3,760, and a revenue-by-category table where Electronics alone absorbed the phantom 1,400. Each answer was precisely, confidently wrong.

One miss was an attribute that was not a column. Asked which products are made in Germany, the model found the attribute table and joined it, then filtered on the attribute name 'country'. The stored name is 'origin_country'. Zero rows.

One miss was a fact that lives in a sentence. Asked for the completed-order value of customers who are not on credit hold, the model returned 4,050 instead of 3,450. On the legacy schema the credit hold exists only as text in a notes field, three words, 'CREDIT HOLD', attached to one customer. No column carries it, and the model never looked at the notes. A system reading only the schema cannot know which customer is on hold, because the schema never says. That is not a prompting problem.

One miss was not SQL at all. Asked how much invoiced money is still outstanding, the model replied with a paragraph of reasoning and a half-finished code block. No executable query. One answer in twenty-four came back unusable as an answer.

Not everything broke. The model converted epoch timestamps into a 2024 date range without trouble. Faced with two customer rows for the same company on the legacy schema, it summed across both and got the duplicate question right, a question the clean copy failed on a capital letter. The reasoning was rarely the problem. The misses were facts about the data that the schema doesn't state.

Three models, two datasets, the same traps

The worked example above is Sonnet on the small dataset. The full scores (a range where the three runs disagreed): small clean schema — Sonnet 5–6, Opus 10, Haiku 9; small legacy — Sonnet 5, Opus 6, Haiku 3. On the larger dataset the clean scores held up (Sonnet 6, Opus 11, Haiku 8) and the legacy scores fell through the floor: Sonnet 2, Opus 3, Haiku 1.

The traps did not care which model walked into them. Guessing stored values sank all three; the deleted flag was the biggest killer, 84 of the 156 legacy misses across both datasets. The attribute-name guess caught Sonnet and Opus on both datasets. Haiku failed the same questions two other ways: prose on the small dataset, and on the larger one a query that went looking for Germany in the customer table. The credit-hold question beat all three models on both datasets. And 21 of the 432 answers came back as paragraphs rather than a query at all, usually on the hardest questions — all from Sonnet and Haiku; Opus always returned a query.

A bigger model helped on the clean schema. On the legacy schema it barely mattered: the best legacy score in these runs was 6, and nobody beat 3 on the larger dataset. The information the models needed was not in the prompt. It was not in the schema either.

What we would do about each one

Publish the values alongside the columns. A data dictionary carrying the actual stored values of status-type columns, or a few sample rows, belongs in the model's context next to the schema definition. Six clean-schema misses say the values are the context that matters.

Make deletion a default rather than a detail. Serve the model a view with deleted rows already excluded, or pin the rule to the schema where no query can miss it. A flag every query must remember is a flag some query will forget — we watched it happen four times in twelve questions.

Treat attribute tables as first-class. The attribute names belong in the dictionary too. Given 'country' or 'origin_country' to guess, the model will guess the obvious one and get silence.

Promote notes to data. If a fact decides money, a credit hold, a do-not-ship, it needs a column. Text fields are where facts go to be missed.

Execute before you trust. Run every generated query and inspect what comes back. An empty set or a null total, on a question that should have an answer, is a detectable smell, and it would have caught the silent clean-schema failures. The inflated totals are harder: they sail through any check that only looks for emptiness, which is why the dictionary and the defaults come first.

Then we applied the fixes

So we applied the fixes a schema and a prompt can carry, and ran everything again: dictionary in the prompt, deleted rows excluded by default, credit holds as a column. Another 432 answers. Small legacy: Sonnet 5 → 8, Opus 6 → 10, Haiku 3 → 10. Larger legacy: Sonnet 2 → 9, Opus 3 → 10, Haiku 1 → 10. Clean schemas: every model reached 11 or 12. Across all 432 fixed answers, not one miss fell into the four failure modes above.

What remained was different in kind. Duplicated customers still got double-counted on a signup question — nothing in these fixes merges two records for one company. One model added the gross and net amount columns together. A few answers still came back as prose, or with the right rows carrying an extra column. Smaller problems, and honest ones: identity and output discipline, not facts the schema hides.

The limits of this test

Two synthetic datasets, both ours, the larger one still small next to a real warehouse. Three models from one provider, three runs each, one prompt per pass, no tools. Exact-match scoring is strict: a right answer carrying an extra column counted as a miss, and prose counted as a miss even when a query was buried in it. The legacy transformations were ours, documented and deliberate, and probably kinder than plenty of real schemas, where codes go undocumented for a decade and three systems disagree about what a customer is. What transfers is the shape of the failures: they clustered where the schema stops describing the data, and production schemas stop much earlier than demo ones.

Work with us

If your team is putting natural-language questions over a database that has been accumulating decisions for a decade, the demo is the easy part. The real work is the dictionary, the defaults and the checks around the model, and it starts before the first question is asked. Tell us about the database through Trida AI's intake, and we will talk through what it takes to make its answers worth trusting.