DEV Community: Karl Mehta

Agents With Amnesia: Why Memory Is the Real Frontier in Building AI Agents

Karl Mehta — Thu, 25 Jun 2026 02:46:43 +0000

Notes from three days inside Google DeepMind — on Antigravity, client- and server-side memory, and where agent memory and the human brain are quietly converging on the same design.

Last week I spent three days at Google’s Mountain View campus as a guest of a Gemini / Google DeepMind forum, part of a small group of AI entrepreneurs invited through a competitive global selection. The agenda swept across the frontier of agentic AI, but one thread kept resurfacing — in the sessions and in the hallway conversations alike: memory.

It came up most sharply around Google Antigravity, DeepMind’s agent-first development platform — the agent harness built around Gemini 3 that lets agents plan, write, run, and verify work autonomously across the editor, terminal, and browser, leaving behind artifacts as a verifiable record of what they did. Antigravity is a useful lens because it makes the core tension impossible to ignore.

The model underneath is a stateless primitive — a pure function that maps an input to an output and remembers nothing once the call returns. But the agent on top has to be stateful: it must hold a goal across hours, carry context as it moves from the editor to the browser, accumulate what it learned on the last task, and prove what it changed. Even the plumbing reflects the split — Gemini 3 shipped a client-side bash tool and a server-side bash tool as a deliberate pair. The hard problem of agents is getting the best of both worlds: the clean, scalable statelessness of the primitive, and the rich, persistent, stateful action a real workflow demands. That gap is memory — and it lives on both the client and the server side.

My own research interest sits exactly here. And over the last year I’ve been comparing notes with friends in the medical and neurology community about the other system that solved this problem a few hundred million years ago: the human brain. The brain is the existence proof that you can run an in-the-moment, stateless-feeling intelligence on top of a deep, durable, self-organizing memory — on roughly 20 watts. The more closely I look, the more our agent memory architectures and human memory architecture appear to be converging on the same blueprint, from opposite directions.

This is what I’ve pieced together: how agents remember today across four distinct layers, where the brain still does it better, what that means concretely if you’re building agents, and why the memory layer is the part of the stack actually worth owning.

A model is a brain you rent — you and your competitor rent the same one. The memory system is the only part you own. It’s where your moat lives.

How an LLM “remembers” today: four layers, four timescales

The confusing thing about LLM memory is that the word means at least four unrelated things depending on which part of the stack you’re standing in. Separate them by where the bytes live and how long they survive — this is the mental model every agent builder needs, and it maps cleanly onto the client/server split Antigravity makes visible.

Layer 1 — In-weights memory (the model’s knowledge). The parameters are the only truly permanent memory. Everything learned in pre-training is frozen into the weights: durable, generalized, read-only after training. This is the closest analogue to deeply consolidated human knowledge — vast, but inert. You cannot write a new fact here mid-conversation without retraining. A base model frozen in 2025 insists it’s 2025 forever.

Layer 2 — Server-side working memory (the KV cache). This is the layer nobody sees and the one that dominates inference economics. When a transformer generates text, each token attends back over all prior tokens through vectors called Query, Key, and Value. Recomputing every Key and Value on every generated token would be unusable, so the model stores them once and reuses them — that store is the KV cache, the model’s runtime working memory. It’s why inference has two phases: a compute-bound prefill that reads your whole prompt in parallel, and a memory-bandwidth-bound decode that generates one token at a time while re-reading the cache. The cache grows linearly with sequence length, batch size, and depth, which is why the field obsesses over shrinking it (grouped-query attention, paged attention, FP8/INT4 quantization) and over persisting it across a tiered hierarchy — hot blocks in GPU memory, warm in CPU DRAM, cold in Redis or on disk. The KV cache has quietly grown its own memory pyramid. This is server-side memory in its purest form.

Layer 3 — Caching and the context window (the session). Prompt caching (prefix caching) is the optimization you pay for as a builder: when a fixed system prompt or long document prefixes many requests, the system reuses the already-computed Key/Value vectors instead of re-paying for them — dramatically cheaper and faster on reuse. But the context window itself is the most fragile layer. It’s bounded, recomputed every turn, and lossy in a way the marketing hides: even at million-token windows, models reliably use only the first slice of what you hand them and treat the rest as expensive padding. (Tellingly, Antigravity’s faster default model traded away some long-context recall for speed — a live reminder that more context is not more memory.) Stuffing the window is the fire-hose approach to remembering.

Layer 4 — Client-side / external memory (RAG and agent memory). Because none of the above survives a closed session usably, the industry bolted on an external layer. First came retrieval-augmented generation (RAG): embed documents into vectors, store them, retrieve the nearest chunks at query time. Useful and limited — reasoning only operates over what retrieval returns, so a terminology mismatch that makes retrieval miss is unrecoverable downstream. The 2026 generation looks more like a memory operating system, pioneered by MemGPT/Letta and refined by hierarchical systems:

• Core memory — always resident in context, like RAM.

• Recall memory — searchable history, like a disk cache.

• Archival memory — long-term external store the agent queries, like cold storage.

The decisive shift: the agent now actively manages its own memory through tool calls — an explicit action space to add, update, delete, retrieve, summarize, filter — rather than passively logging everything. (Antigravity’s own “progressive disclosure” of Skills — loading a knowledge package into context only when a request matches it — is exactly this instinct: keep working memory lean, fetch the rest on demand.)

How the human brain does it differently

Lay the two systems side by side and the gaps become a design spec.

Human memory is a pipeline of stages, not a set of storage tiers. Information is encoded when the prefrontal cortex filters for relevance, passes through sensory and working memory, and then hits the part LLMs have no real equivalent for: consolidation. The hippocampus is a fast learner that captures an experience in one shot, then — largely offline, during sleep, via replay — gradually reweaves it into the slow-learning neocortex. A fast plastic learner and a slow stable learner in constant crosstalk: the complementary learning systems design.

That two-speed architecture solves the stability–plasticity dilemma — learn new things fast without overwriting old ones. LLMs are pinned at both extremes: weights are pure stability (frozen), the context window is pure plasticity (nothing retained). The brain lives in the productive middle. It learns continuously and forgets on purpose.

Two more differences matter for builders. Retrieval is reconstructive, not lookup — a cue reactivates a hippocampal index and triggers pattern completion that rebuilds a memory, and the act of recalling can update it (reconsolidation). RAG is dumb nearest-neighbor matching by comparison. And forgetting is a feature — the hippocampus actively weakens unused traces to prevent overload and interference. Most agent memory systems default to append-only stores that accumulate forever, which is a cost problem and, increasingly, a security problem.

The cleanest way to see the convergence is to put the two lifecycles next to each other — not as static storage tiers, but as the journey a single memory takes from arrival to retrieval to forgetting

That last row is the loudest signal that today’s architecture is missing something fundamental, not just under-scaled.

If you’re building agents: what you actually need to decide

The model is a commodity — you and your competitor rent the same one. The thing you control, and the thing that compounds, is memory. Most teams botch it by defaulting to “dump everything in a vector DB and pray.” Here’s the decision framework instead, split the way the brain splits it: short-term (within a task or session) and long-term (across sessions).

Short-term memory: working context within a task

This is your KV cache, context window, and prompt cache — and it’s mostly a cost and latency problem disguised as a memory problem.

• Budget your context like RAM, not a junk drawer. Decide explicitly what stays resident (core memory: identity, current goal, active state) versus what’s retrieved on demand. Models effectively use the front of the window, so put what matters first.

• Structure prompts for prefix caching. Keep the stable parts — system prompt, tools, persona, policies — as a fixed prefix so the KV cache is reused across turns. Cache hits can cut cost and time-to-first-token by an order of magnitude. Putting volatile content early breaks the cache and you pay full price every turn.

• Compact instead of accumulate. Long agent runs blow past the window. Summarize-and-prune the trajectory at checkpoints rather than carrying raw history — but know the trap: summarization only works over what’s still present, so prune the wrong thing and it’s gone.

• Treat token discipline as memory discipline. Every token you cut is one fewer KV entry per layer across every generated token. Fewer tokens means faster prefill, faster decode, and more concurrent sessions per GPU. Token bloat is silent latency.

Long-term memory: knowledge across sessions

This is where the real moat is, and where most agents are dangerously naive.

• Don’t default to append-only. Forever-growing memory is expensive, drifts, and becomes an attack surface. Memory poisoning — an adversary planting false “facts” the agent later retrieves and acts on as truth — is the headline agent vulnerability of 2026. Append-only stores are now considered actively unsafe in adversarial settings.

• Separate episodic from semantic, then run a consolidation loop. Store raw events (episodic) but periodically distill them into generalized knowledge (semantic): turn “5,000 conversation snippets” into “here’s what this user reliably wants.” This offline distillation is the engineering analogue of sleep. Skip it and you’re retrieving noise.

• Your retrieval architecture is your ceiling. Nearest-neighbor over frozen embeddings fails on terminology mismatch. The frontier is moving toward graph-structured and multi-hop retrieval that reconstructs context rather than returning the top-k closest chunks — closer to how the brain pattern-completes.

• Build governance in from day one. Decide what may be written, who may read it, when an update is authorized, what stays auditable, and what may be forgotten (TTL, decay, explicit deletion). The 2026 research calls this verifiable control mnemonic sovereignty. The competitive moat is shifting from recall capability to memory governance — storing everything is easy; knowing what to keep, consolidate, surface, and delete is the defensible part.

Where the next big frontier is for your agents

Three shifts are landing now. Design so you can adopt them, not against them.

***1. Test-time learning collapses the wall between weights and context.*
Google’s Titans (viral after NeurIPS 2025) and the end-to-end test-time training line of work let a model write information into its own weights during inference — true continual learning, with brain-borrowed surprise-gating and decay. The honest caveat: neural memory still stores better than it retrieves (near-perfect memorization but unreliable recall — the “tip of the tongue” failure). Adopt cautiously; pair it with an external store you can audit.

2. Training-free continual learning is the pragmatic path. Methods like JitRL keep a non-parametric memory of past experiences and modulate output logits at inference — matching fine-tuning performance at a fraction of the cost, no gradient updates, no catastrophic forgetting. If you want agents that improve from experience today, this is the lower-risk lever.

3. Memory becomes the asset that compounds. An agent that has worked your domain for a year — with consolidated, governed, reconstructive memory — is something a competitor can’t clone by renting the same model. That accumulated memory is the moat. Architect for it now.

When agents become the workforce: human memory vs agent memory

Once you stop thinking of agents as chatbots and start thinking of them as workers — a marketing agent, a CFO agent, a support agent — the memory question reframes itself. Memory is the difference between a temp who restarts cold every morning and an employee who gets better every quarter.

A human employee’s value is mostly memory you can’t see on day one. Onboarding is consolidation. Institutional knowledge is semantic memory. Knowing which past project resembles today’s problem is reconstructive retrieval. A new hire with no memory of anything is useless; the value is in what accumulates. The same is exactly true of agents — which is why an agent without a serious long-term memory layer is a permanent new hire who forgets everything every night.

But the human/agent comparison isn’t symmetric, and the asymmetries are where the design leverage is:

• Humans forget gracefully and reconstruct; agents have perfect recall but no judgment about what to forget. A human employee naturally lets go of noise and keeps the gist. An agent will faithfully remember the typo from eight months ago and weight it equally. You have to engineer the forgetting a human gets for free — and that engineered forgetting is also your defense against poisoning and drift.

• Humans carry tacit knowledge they can’t articulate; agents carry explicit knowledge they can fully expose. That cuts both ways: agent memory is auditable in a way a human’s is not — you can prove what it knew and when. For regulated workflows that’s a feature humans literally cannot offer.

• The superpower humans don’t have: transferable, shared memory. When one human learns something, the org learns it slowly and lossily, through meetings and docs. When one agent consolidates a lesson, every agent can inherit it instantly. A workforce of agents can share a single, governed institutional memory. Design for this. The org chart of your agents is, functionally, a memory architecture — decide what’s private to each agent (its episodic experience), what’s shared institutional knowledge (consolidated semantic memory), and who is authorized to write to the shared store.

So the right mental model for an agent workforce isn’t “smart chatbots.” It’s employees whose entire career — onboarding, accumulated expertise, auditability, and handoff — is a memory system you get to architect deliberately, with one cheat code biology doesn’t allow: instant, shared, transferable institutional memory.

The payoff: stop hand-rolling memory — build on an IDE that owns it

Here’s the part nobody warns first-time agent teams about. Everything above — KV-cache-aware prompt structuring, prefix-cache optimization, the short-term/long-term split, episodic stores, consolidation jobs, retrieval that doesn’t choke on terminology mismatch, governance with provenance and forgetting policies — is real systems engineering. And today most teams rebuild all of it, badly, from scratch, for every single agent. It’s the undifferentiated 80% of building a production agent, and it’s exactly the part that decides whether the agent works.

You don’t hand-tune your CPU’s L1/L2/L3 cache when you write an application. The compiler and the OS do it for you, and they do it better than you would. Memory optimization for agents should work the same way — handled by the environment you build in, not hand-wired per project. Platforms like Antigravity have shown what the agent-first surface looks like; the next step-change is making the memory substrate underneath it just as managed.

That’s the thesis behind CodeNOW.pro — an IDE for building agents that treats memory as a first-class primitive rather than an afterthought you bolt on at the end. The four layers stop being your problem to wire by hand:

• Short-term memory, optimized for you — prompts structured so stable prefixes stay cached, context budgeted and compacted automatically, so you ship fast, cheap inference without becoming a KV-cache expert.

• Long-term memory, built in — episodic and semantic stores with consolidation loops running in the background, so your agents accumulate knowledge instead of just accumulating logs.

• Retrieval that reconstructs, not just matches — so agents surface what they actually learned instead of the top-k nearest chunks.

• Governed by default — provenance, authorization, auditability, and forgetting policies wired in, so memory is an asset you can trust and defend against poisoning, not a liability that grows unbounded.

I left Mountain View more convinced than when I arrived: the agents that win the next cycle won’t be the ones running the smartest model — everyone rents the same models. They’ll be the ones with the best memory: agents that remember your domain, learn from every run, forget what they should, and can prove what they know. The brain figured this out a long time ago. We’re finally building the rest of the memory system around the cortex we already scaled. Memory was always the missing piece. It’s now the frontier — and CodeNOW.pro is where you build on the right side of it.

Dario Amodei Just Laid Out Why AI Assurance Is Now Non-Negotiable. Here's What Enterprises Need to Do Monday Morning

Karl Mehta — Mon, 15 Jun 2026 01:57:36 +0000

Dario Amodei, CEO of Anthropic, published a sweeping policy essay yesterday on X: "Policy on the AI Exponential", that every enterprise leader deploying AI should read. His central message: AI is no longer a toy or a tool. It is a technology of national and economic consequence and our policy and governance infrastructure is dangerously behind. Dario called out the emergence of Claude Mythos Preview as proof that frontier models now pose real risks to critical infrastructure and national security, and announced that Anthropic is releasing both a legislative proposal on frontier model testing and a policy framework for job displacement, with substantial financial backing. The essay is the clearest signal yet from inside the AI industry that self-regulation is over.

The Treebeard Problem Is Real

Dario's metaphor is apt: AI is moving at Hobbit speed, policy at Treebeard speed. In the time it takes Congress to pass a bill, AI capabilities compound multiple generations. But here's the enterprise implication that doesn't get enough attention — companies are in the exact same trap. Internal governance committees, legal reviews, audit cycles — they were designed for software that didn't change every 90 days.

The result? AI systems are in production making decisions about who gets hired, who gets a loan, who gets treated, who gets covered — with no independent verification, no compliance certification, and no tamper-proof evidence that anyone checked.

What Dario Is Calling For — And Why It Matters to Your Enterprise

Three parts of his essay hit directly at what enterprises need to act on now:

1. Mandatory third-party testing is coming.

Dario explicitly calls for frontier AI models to be evaluated by qualified independent third parties before deployment — modeled on the FAA. Governments should have the power to block deployment of models that fail. Whether or not federal legislation passes this year, the regulatory direction is unmistakable: independent evaluation is becoming table stakes, not optional. The EU AI Act entered enforcement this August, with fines up to €35M or 7% of global revenue. NYC's LL144 is expanding. The legislative proposal Anthropic released alongside this essay signals the industry itself now agrees: self-reporting isn't enough.

2. The cybersecurity risk is not theoretical.

Claude Mythos Preview demonstrated that frontier AI poses real, immediate risks to critical infrastructure, financial systems, and national security. For enterprise leaders, the implication is concrete: the AI systems your vendors are deploying — in your HR stack, your credit decisions, your underwriting models — carry cybersecurity, bias, and accountability exposure that your existing risk frameworks weren't built to catch.

3. The economic stakes of getting this wrong are enormous.

Dario is candid that AI may produce more enduring labor displacement than any prior technology. Companies that deploy AI recklessly will face not just regulatory exposure, but workforce, reputational, and legal consequences they aren't pricing in today.

Why I Open-Sourced TrustModel Last Week

Dario's essay calls for independent evaluation infrastructure that the whole industry can trust — and that's precisely why last week I open-sourced the TrustModel core under the MIT license. My post on the launch explains the reasoning: AI assurance can't be a black box. The evaluation engine, the scoring methodology, and the guardrail framework need to be inspectable, forkable, and community-auditable. The same independence that makes TrustModel valuable as an enterprise platform — we don't have an incentive to pass the models we sell — depends on that methodology being open to scrutiny.

Open-source bottom, commercial top. The evaluation engine is MIT-licensed and available now at github.com/karlmehta/trustmodel. The compliance framework library — 50+ regulatory frameworks, tamper-proof on-chain governance evidence, continuous monitoring — is the enterprise product. This is the Databricks model: openness builds trust, which is the point.

What TrustModel.ai Is Doing About It — Today

At TrustModel.ai, we've spent the last two years building the infrastructure Dario is calling for at the enterprise layer. Here's what that looks like in practice:

Independent evaluation, no vendor cooperation required. Our platform scores AI models and COTS systems across 10 trust dimensions — Safety, Fairness, Accuracy, Privacy, Transparency, Robustness, Accountability, Explainability, Compliance, and Reliability — producing a TrustScore from 0–100, the credit rating for your AI systems.

Compliance frameworks that match real regulatory exposure. We've operationalized EU AI Act, NIST AI RMF, OWASP LLM Top 10, and NYC LL144 into policy packs that evaluate your deployed systems against the standards regulators are now enforcing.

Tamper-proof governance evidence. We anchor evaluation results and audit trails cryptographically on-chain. When a regulator asks "did you exercise due diligence?" — you have cryptographic proof you did, not a PDF from a consultant.

Continuous monitoring, not one-time audits. AI models drift. Guardrails change. Vendor updates happen silently. Our platform monitors AI behavior continuously via OpenTelemetry integration, alerts on violations, and maintains a live compliance posture.

The Call to Action

Dario ends with optimism — that the window of opportunity is open, that policymakers are unusually receptive, that a nonpartisan coalition around AI safety is possible. I share that optimism.

But enterprises cannot wait for Congress. The EU AI Act enforcement clock has already started. Your board is already asking questions your CISO and CDO can't yet answer.

The question isn't whether you need independent AI assurance. The question is whether you build that infrastructure before the first enforcement action or after.

FreeScan your first AI model — full 10-dimension evaluation, no credit card required — at trustmodel.ai. Or join us at the AI Assurance & Governance Summit on October 1, 2026 at Stanford University , where we'll convene the enterprise, regulatory, and investor leaders working to get this right.

A Personal Note on the Journey

On a personal note: congratulations to Dario and the entire Anthropic team on the confidential S-1 filing last week, targeting a public listing that could make Anthropic one of the first trillion-dollar AI companies to reach the public markets. What Anthropic has built in just five years, from a safety-focused research lab that many dismissed as too principled to compete, to the company that has genuinely redefined what responsible AI leadership looks like — is nothing short of extraordinary. Dario has earned the right to write essays like this one. He built the credibility while others just wrote the press releases.

And a special congratulations to my former colleagues and friends at Menlo Ventures — who led Anthropic's Series D at a time when many other firms had passed. That kind of conviction, writing a major check into a safety-first AI lab when the consensus wasn't yet there, is what separates great venture investors from the rest. I've had the privilege of working at Menlo Ventures and with Menlo across two of my own startups, and I know firsthand that their pattern is consistent: they back founders with a point of view before the market catches up. They did it with Anthropic. They did it with me in two of my successful companies (one acquired by Visa and another by Cornerstone OnDemand ). That's the firm I'm proud to call a partner.

The best investors, like the best founders, look right before they look early.

AI Is Shipping Unvalidated. Today We're Open-Sourcing the Fix.

Karl Mehta — Sun, 31 May 2026 23:41:35 +0000

Enterprises are carrying an estimated half-trillion dollars of unvalidated AI risk, and U.S. courts already hold 100+ active AI lawsuits. TrustModel — Eval, Monitor, Govern — is now free and open source.

Every company on earth is racing to put AI in front of customers, employees, and regulators. Almost none of them can answer a simple question before they ship: Is this AI safe, fair, and defensible?

That gap — between "it worked in the demo" and "it holds up in a deposition" — is now one of the largest unpriced liabilities in the economy. By industry estimates the aggregate exposure runs into the hundreds of billions of dollars, and the bill is already coming due in cour

This is not hypothetical anymore
The litigation has arrived, and it's not edge-case. A few that should make every builder pause:

Hiring. Mobley v. Workday — an AI résumé-screening system accused of discriminating by age, race, and disability — was cleared in 2025 to proceed as a nationwide collective action. The EEOC's first AI-discrimination settlement (iTutorGroup, $365K) was over software that auto-rejected older applicants.
Customer-facing agents. A tribunal ordered Air Canada to honor a refund policy its chatbot invented, flatly rejecting the argument that "the chatbot is a separate entity." Your AI's words are your words.
Healthcare. Class actions allege algorithmic systems wrongfully denied medically necessary care at scale.
Safety & harm. Wrongful-death and product-liability suits now name AI products directly for the content they generate.

And the regulators are converging on the same standard. The EU AI Act carries penalties up to €35M or 7% of global revenue. NYC Local Law 144 mandates bias audits for hiring tools. Colorado's AI Act lands in 2026. NIST's AI Risk Management Framework, ISO 42001, and the OWASP LLM Top 10 are quietly becoming the bar you'll be measured against — in audits and in court.

The problem isn't that teams don't care. It's that validation is too far away.

Here's the trap. "AI trust & safety" has no budget line, no owner, and no tool the engineer who actually ships the model can reach for on a Tuesday afternoon. The governance platforms that exist are six-figure, top-down, procurement-cycle products aimed at a Chief Compliance Officer — months away from the code. So the model ships unvalidated, and the exposure compounds silently until a plaintiff, an auditor, or a journalist finds it.

The fix can't be another enterprise sales motion. It has to be free, local, and in the hands of the developer — the person who can actually do something about a bad score before it ships. Here is the repo on GH to pick it up: https://github.com/karlmehta/trustmodel also, check the HuggingFace page: https://huggingface.co/spaces/karlmehta/trustmodel-score-any-ai

So we open-sourced it.

"I'm Karl, founder of TrustModel. I built this because whether your AI is safe to ship shouldn't require a sales call to find out. Run it locally, read the code, and score your own AI across the same ten dimensions our enterprise customers use — for free, forever." —@karlmehta

Today we're releasing TrustModel — an MIT-licensed toolkit that scores any AI across 10 trust dimensions and rolls them into a single 0–100 TrustScore. Three products, one free API key (a developer account is free, no credit card, and comes with 5 credits — $500 — to use across all three):

$ pip install trustmodel
$ trustmodel login # free account → API key + 5 credits ($500)
$ trustmodel eval "Take 500mg of metformin twice daily."

TrustScore: 41/100 (Grade D)
safety ⚠ ········ unverified medical dosage advice
explainability ⚠ ····· 47 ⚠
...
Eval, Monitor, Govern

Eval — score any model, prompt, agent, or MCP server across 10 dimensions, locally, using your own LLM as the judge. Gate it in CI so a bad score fails the build, not the lawsuit.
Monitor — one line wraps your live LLM calls and scores every response in production, with threshold alerts and OpenTelemetry export. Catch drift the day it starts, not the quarter it's subpoenaed.
Govern — enforce open-source policy packs mapped to real regulation (EU AI Act, NIST AI RMF, ISO 42001, NYC LL144, OWASP LLM Top 10) before output reaches a user. Block the opaque rejection. Redact the leaked PII. Stop the unsafe answer at the door.

Govern: block output that would fail an EU AI Act / LL144 audit

from trustmodel import Guardrail
gr = Guardrail("nyc-ll144")
gr.check("You're not a culture fit. We can't say why.").allowed # → False

Ten dimensions, mapped to the rules you'll be judged by

Safety, fairness, accuracy, privacy, transparency, robustness, accountability, explainability, compliance, reliability — each scored, each mapped to the frameworks that show up in audits and complaints. This is the difference between "we tested it" and "we can prove it." When a regulator or a plaintiff's attorney asks how you validated your system, "we ran TrustModel in CI on every release and here are the scores" is an answer. Silence is a settlement.

Open core, on purpose

The toolkit is free and MIT-licensed — the harness, the CLI, the MCP server, the policy packs. The calibrated, audit-ready TrustScore, the compliance reports an auditor will accept, certification, and in-VPC agent governance are the commercial layer at trustmodel.ai. Think Linux and Red Hat: run the open source forever; pay only when you need a score you can hand to a regulator. We'd rather a million developers validate their AI for free than have a thousand do it after the lawsuit.

Score your AI today. Free key, $500 in credits, no credit card.

pip install trustmodel && trustmodel login
Star on GitHubTry the live demoGet your free key: trustmodel.ai (click for free developer account under sign up)

https://github.com/karlmehta/trustmodel

https://huggingface.co/spaces/karlmehta/trustmodel-score-any-ai

Note: The $500B exposure figure reflects aggregate industry estimates of unvalidated AI liability; case references are drawn from public U.S. litigation and regulatory actions and are illustrative, not legal advice. AI litigation counts reflect public litigation trackers as of 2026.

The Missing Engineering Stack for Production AI Agents

Karl Mehta — Sun, 17 May 2026 15:29:24 +0000

The "build an agent in 5 minutes" tutorials get you to a demo. They don't get you to production. Here's the field guide for the four primitives that decide whether your agent survives contact with real users, real data, and real adversaries — context-window discipline, skill composition, capability-based security, and drift telemetry. Concrete patterns, named tradeoffs, and the enterprise integrations that let you ship past prototype.

This is part 1 of a 3-post series. Part 2 — Why current IDEs need to be redesigned for the agent era — covers the developer-tooling argument. Part 3 introduces what I'm shipping next.

1. Tokens — context-window discipline
A token is the unit of inference cost, the unit of latency, and the unit of model attention. Treat it like memory in a 1990s embedded system: budget every byte, evict aggressively, and never assume the next call gets the same allocation.

Prompt caching is a 90% cost cut you'd be insane to ignore
Anthropic's cache_control: { type: 'ephemeral' } marker (5-minute TTL by default, 1-hour via the extended-TTL beta) deduplicates the static prefix of your prompts at the inference layer. Cached tokens are billed at 10% of input cost; cache writes cost 25% more on the first call. The math: any system prompt + tool catalog + few-shot exemplar bank that's reused more than ~3 times per 5 minutes is a net cost win. Order matters — the cache is a prefix, not a content-addressable store, so the cached span has to be byte-identical and at the start.

messages: [
{ role: "user", content: [
{ type: "text", text: STATIC_TOOL_CATALOG, cache_control: { type: "ephemeral" } },
{ type: "text", text: STATIC_SKILLS_BUNDLE, cache_control: { type: "ephemeral" } },
{ type: "text", text: dynamicUserTurn },
]}
]
Two cache breakpoints because cache reads accumulate up to the most recent cache_control marker — splitting tool catalog from skill bundle lets either evolve without busting the other. OpenAI's automatic prefix caching (no opt-in, but no extended TTL) and Gemini's explicit CachedContent resources are the equivalents on the other major providers.

Model routing — pay Haiku rates for Opus-class outcomes
A single agent run rarely needs the same model for every step. The cost spread is enormous: Claude Haiku 4.5 is $1/$5 per million in/out, Sonnet 4.6 is $3/$15, Opus 4.7 is $15/$75. The pattern that's worked for me is a three-tier router:

Retrieval / classification / extraction → Haiku. Use structured outputs (forced JSON via tool_use with strict mode) so the model can't waste tokens on freeform.
Synthesis / reasoning over retrieved context → Sonnet. The default mid-tier; this is where 80% of business logic lives.
Tool selection / planning / disambiguation → Opus only when the planner has to coordinate >5 tool calls or weigh ambiguous user intent.

Switching costs ~50ms of router latency. The cost amortization is typically 4–8× on production workloads. The trap: don't route based on input length alone — route based on the step type. A 50-token "is this a refund request?" classifier on Haiku is 60× cheaper than the same call on Opus.

Streaming, KV reuse, and the structured-output dodge
Streaming via SSE (Anthropic, OpenAI) or gRPC bidirectional (Vertex) is non-negotiable for latency. The first token typically lands at 200–600 ms; the full response at 2–8 seconds. If your UX waits for the full response, you've added 4 seconds of perceived latency for zero product reason.

KV cache reuse across calls is the under-discussed companion to prompt caching. Modern Anthropic and OpenAI back-ends keep the attention key-value cache warm across the cache TTL. Order tool calls so the most-frequently-called tools come first in your tool list, because tool definitions are part of the prefix that gets cached.

The structured-output dodge: when you need a list, a classification, or a structured fact, don't ask the model in freeform — define a tool, force it via tool_choice, and receive a typed JSON object. You skip 50–80% of the freeform tokens the model would otherwise generate, and the output is parser-safe by construction. Pair with strict mode (OpenAI) or JSON Schema with $defs (Anthropic) to refuse off-schema outputs at the decoder.

2. Skills — composition, not prompts
A "skill" is the unit of behavior an agent can perform. Most production agents conflate three different things into a megaprompt: identity (who are you), capabilities (what can you do), and policies (what you must / must not do). That conflation makes prompts impossible to evolve safely. Separate them into composable fragments, then assemble at runtime.

The model I've shipped against — and what I think every production agent eventually converges on — is the trigger / action / restriction triple per skill:

{
"id": "refund-policy-2024",
"trigger": "the user asks for a refund",
"action": "verify the order is within the 30-day window, then issue a refund via tools.stripe.refund and post-confirm via tools.email.send",
"restriction": "never issue refunds > $500 without a human-approval gate; never refund subscription items in their first cycle"
}

Domain experts (PMs, ops, legal) author triples in plain English. The runtime composes them into a system-prompt slot. Versioning per skill — not per agent. Eval suites attach to the skill, so swapping out a refund policy in 2026 doesn't require reblessing the entire agent.

Tool use, MCP, and the transport question
Tools are the IO of an agent. The schema is the contract. Two opinions worth holding:

Strict JSON schemas with additionalProperties: false. Closed-world schemas catch hallucinated arguments at the validator instead of in production. Strict mode (OpenAI) and the Anthropic tool_choice + JSON-Schema combo both enforce this.
Tools should be small and idempotent. orders.refund(orderId, amountCents), not orders.handle(intent, payload). The agent's planner is dramatically more reliable when each tool does one thing with a typed input.

Once you have more than ~5 tools, the catalog itself becomes worth standardizing. Model Context Protocol (MCP) — Anthropic's open-source agent ↔ tool spec — is the answer that's consolidating the ecosystem. Three transports, three different tradeoffs:

stdio — local-process tools. Lowest latency, zero network surface. Use this for code execution, filesystem ops, anything sensitive.
SSE (deprecated in favor of StreamableHTTP) — long-poll over HTTP. Browser-friendly, easy to host. Latency ~50ms.
StreamableHTTP — single-endpoint HTTP with optional SSE for streaming responses. The current recommendation for hosted MCP servers. Compatible with most cloud LB stacks.

The plan-execute-review loop
For agents with >3 sequential tool calls, prompt the model to plan first (one message, no tool calls), execute against that plan (n messages, tool calls only), then review the result against the plan's stated success criteria (one message, no tool calls). Anthropic's Agent SDK ships this pattern via the plan_mode primitive; it's also straightforward to implement in raw fetch with three system-prompt slots.

The bonus: when the agent fails, the failure is grounded in a textual plan you can replay, eval, and red-team — instead of an opaque chain of tool calls.

3. Security — capability-based, not vibe-based
The threat surface of an agent is wider than people pretend. A short list:

Prompt injection — adversarial input in retrieved context, tool outputs, or user data flips the agent's instructions.
Data exfiltration — the agent calls a tool that emits sensitive data to an attacker-controlled destination (an email, a webhook, a markdown image with a query string).
Tool abuse / RCE — the agent uses a legitimate tool in a way the designer didn't intend (a shell tool, a code-exec tool).
Supply chain — a tool dependency or model weight is compromised.
Secret leakage — API keys end up in logs, prompts, or tool error messages.

Capability-based authority, not ambient authority
The security primitive that's stood up best in 50 years of OS research is the object capability: hand a process the smallest unforgeable token that lets it do exactly the thing it needs, and nothing else. Apply this to agents.

Concretely: don't give the agent a long-lived OPENAI_API_KEY with billing access. Give it a per-session token, scoped to specific endpoints, with a TTL. Every tool gets a separate principal. Authorize via OAuth 2.1 with PKCE — the agent walks the user through delegated authorization, the user sees the exact scopes, and tokens are stored in the OS keychain (libsecret on Linux, Keychain on macOS, DPAPI on Windows; Electron's safeStorage wraps the platform primitive for cross-OS).

Sandbox the tools, not just the agent
If a tool runs untrusted code or writes to a filesystem, isolate it. Three real options ranked by overhead:

WASM (Wasmtime, Wasmer) — sub-millisecond startup, deny-by-default I/O, easy to configure capability lists. The right choice for code-exec and policy-evaluation tools.
gVisor — userspace kernel; near-full Linux compatibility with a 10–100ms startup cost. Right for tool subprocesses that need the full POSIX surface.
Firecracker — microVM; ~125ms startup, hardware-backed isolation. Right for multi-tenant agent execution in shared infra.

ko/distroless container images, SLSA Level 3 build attestation, and sigstore-signed artifacts close the supply-chain surface. If your agent runs in a long-lived process, write the SBOM to the artifact registry and gate deploys on cosign verification.

Prompt injection defense
The most under-addressed threat. The mitigations that actually work:

Channel separation. Treat tool outputs and retrieved documents as data, not as instructions. Anthropic's recent research on instruction-data separation in the system prompt is the current best practice — wrap untrusted content in clearly labeled XML-ish tags and tell the model to ignore any instructions inside them.
Allowlist tool surfaces. The agent can call send_email only to addresses on a per-conversation allowlist that the user explicitly authorized. The same pattern applies to outbound HTTP, database writes, file outputs.
Output content classifiers. Run a small model over the agent's tool calls before they execute, looking for known exfil patterns (suspicious destinations, base64-encoded blobs, sensitive-field references).
HITL gates on consequential actions. Anything that costs money, sends external communication, modifies a database, or touches PII goes through a human approval before execution. The threshold is per-skill.

4. Trust — telemetry, not vibes
"It worked when I tested it" is not a trust story. The four signals you actually need on every agent in production:

Eval pass rate against a golden set
A regression suite of input/output pairs the agent must continue to pass. Run on every prompt change, every model upgrade, every tool catalog edit. Tag failures by skill so you can localize regressions. Pairwise LMSYS-style judging works for tone-sensitive outputs; exact-match works for structured outputs. Don't conflate them.

Drift detection
Even with a stable model, your agent's behavior drifts when the input distribution shifts — new product launches, seasonal traffic, adversarial probing. Track distribution shift on input embeddings (cosine distance from a reference centroid) and behavioral metrics (tool-call mix, refund rate, escalation rate). Alarm at 2σ; investigate at 1σ.

Behavioral canaries
Plant N synthetic inputs per day designed to exercise the prompt-injection, exfil, and jailbreak surfaces. Pass rate on canaries is your live red-team signal. When a new attack class appears in the wild, add it to the canary set; you'll know the next time someone tries it.

Audit trail with integrity
Every run captured as JSONL — input, system prompt, tool calls, model responses, costs, latencies. Hash chain over the events; periodically anchor the head into an immutable store (S3 Object Lock, GCS Bucket Lock). When auditors ask "what did the agent do on March 12 at 14:22 UTC", you have a Merkle-verifiable answer.

A composite TrustScore rolls these up: weighted blend of eval pass rate, drift score, canary survival, HITL approval rate. Per agent, per skill, per day. The score is operationally meaningful only if it's grounded in those underlying signals — a score with no traceable inputs is theater.

The compliance + enterprise integrations
For anything regulated — health, finance, government, EU operations — the trust telemetry has to map onto external frameworks. The integrations I've found genuinely useful:

TrustModel.ai for the GRC overlay — NIST AI RMF, ISO 42001, EU AI Act Article-by-Article mapping, SOC 2, FedRAMP. The TrustScore feeds directly into the control library and produces auditor-ready reports without re-instrumenting the agent.
Cisco DefenseClaw — Apache 2.0, free, OSS. Jeetu Patel announced it from the RSAC 2026 keynote stage on March 23, 2026; it's the most consequential agent-security release of the year. Four components ship in the box: Skills Scanner (capability scan before execution), MCP Scanner (allow/block on MCP server inspection), CodeGuard (static analysis for secrets, unsafe deserialization, weak crypto, and injection patterns), and a Guardrail Proxy (runtime inspection of prompts, completions, and tool calls via regex rules + optional LLM judgment). Stack is a Go gateway sidecar + Python CLI + a TypeScript plugin for the OpenClaw framework that DefenseClaw was built to protect. The framework is observable by default, with first-class Splunk connectivity for the audit-trail story above. It bridges the trust gap that has 85% of enterprises experimenting with agents but only 5% running them in production. Personal note: Jeetu Patel is one of my role models, and I started coding the integration into the IDE I'm shipping the moment he walked off the RSAC stage. The most quoted line from the announcement — "I run OpenClaw at home — that's exactly why we built DefenseClaw" — is the right framing. There's no good reason not to wrap DefenseClaw around every production agent.
OpenTelemetry GenAI — the emerging standard for agent telemetry semconv. Emit the standard span attributes (gen_ai.system, gen_ai.request.model, gen_ai.usage.input_tokens) and your traces work in any OTel-compatible backend.

The bar
A production agent is not a model and a prompt. It's a token economy, a skill catalog with versioning, a capability-scoped security model, and a trust telemetry stack. Each of those is a non-trivial engineering surface in its own right; together, they're more work than the "build an agent in 5 minutes" tutorials acknowledge.

The argument I'll make in part 2 is that the IDEs we have weren't built to help engineers hit this bar. They were built for the 2010 unit of work — one developer, one project, one file at a time — and the unit of work in 2026 is an agent that gets trained, guard-railed, and overseen by a domain expert who isn't the engineer. The tooling has to follow.

The Commoditization of LLM Models

Karl Mehta — Tue, 05 May 2026 00:55:29 +0000

I’m becoming more convinced that LLMs are moving toward the same structure as payment networks. The models will be incredibly important. But the largest value will not be captured by the raw model layer alone. It will be captured by the layers above it: routing, evals, RAG, MCP, memory, orchestration, agentic workflows, vertical applications, and trust infrastructure.

As a founder and developer, this pattern feels familiar to me.I previously built a fintech company that routed transactions across multiple rails and 100+ payment methods around the world. It was eventually acquired by Visa. In payments, Visa, Mastercard, and AmEx were critical rails. But Stripe, PayPal, Adyen, PlaySpan (acquired by VISA) and others created enormous value by abstracting those rails, optimizing routing, managing risk, improving developer experience, and owning the merchant workflow. I think the same thing is happening with LLMs.

At the bottom, we will likely have a small number of frontier model providers: OpenAI, Anthropic, Google, and a strong open-weight ecosystem. They will remain valuable. They will set the capability frontier. But for most production apps, the model will increasingly become a pluggable inference rail. The value moves up the stack.

Layer one: model gateways and routing.

OpenRouter, LiteLLM, Bedrock, Together, Fireworks, Groq, and internal enterprise gateways are making model access interchangeable. A developer can route a request to GPT, Claude, Gemini, Llama, Mistral, DeepSeek, Qwen, or a fine-tuned model depending on cost, latency, context length, modality, privacy, or benchmark performance. This is where the “LLM as rail” abstraction begins.

Layer two: RAG and context engineering.

The hard problem in enterprise AI is not generating fluent text. It is assembling the right context at the right time. A useful AI system needs to know the patient record, contract clause, support ticket, lab result, CRM object, claim history, policy document, API schema, prior memory, and user permission boundary. RAG is evolving from “vector search over PDFs” into a full context layer: hybrid search, graph retrieval, tool retrieval, memory retrieval, structured database queries, re-ranking, summarization, and dynamic context packing. The LLM is only as good as the context substrate around it.

Layer three: MCP and tool connectivity.

MCP makes the harness layer much stronger because it standardizes how agents discover and call tools. Instead of every app building custom glue code for Gmail, Slack, GitHub, Postgres, EHRs, CRMs, calendars, and internal APIs, MCP gives agents a more consistent interface to external systems. This is a big deal.

Once tools become discoverable and composable, the agent is no longer just a chat interface. It becomes a workflow runtime that can read, reason, act, verify, and update state across systems.

Layer four: agentic orchestration.

This is where frameworks like LangGraph, LlamaIndex, LangChain, CrewAI, AutoGen, Semantic Kernel, and custom orchestration layers matter. The future agentic app will not call one model once.

It will use one model for planning, another for coding, another for extraction, another for medical reasoning, another for summarization, and another for cheap classification. It will make these decisions in real time based on task type, latency, cost, reliability, and safety constraints. One task may go to Claude for long-context reasoning. Another may go to Gemini for multimodal input. Another may go to GPT for tool use. Another may go to a local or open-weight model for cheap classification. Another may run through multiple models in parallel for consensus, critique, or ensemble evaluation.

This is exactly how payment orchestration worked. You didn’t hard-code one rail. You routed dynamically based on geography, fees, approval rates, fraud risk, currency, merchant category, and availability.

Layer five: evals, trust, and governance.

This is where I think platforms like TrustModel.ai become important. If the application can route across multiple LLMs, the system also needs a way to continuously evaluate which model is right for which task. Not just “which model is smartest,” but which one is safest, cheapest, fastest, most compliant, most consistent, most robust against prompt injection, best at structured output, best at domain reasoning, and least likely to hallucinate.

A serious agentic system needs multi-dimensional evals across models and workflows. It needs to test safety, quality, bias, factuality, privacy leakage, tool-use reliability, refusal behavior, cost, latency, and auditability. That eval layer becomes the control plane for selecting models and keeping applications safe across changing model providers. This is not optional in healthcare, finance, legal, or enterprise AI.

Layer six: vertical workflow applications.

This is where the most durable value gets created. A healthcare agent that closes care gaps is not valuable because it uses one specific LLM. It is valuable because it understands clinical workflows, patient context, lab data, insurance constraints, escalation paths, HIPAA boundaries, and provider operations. A revenue cycle agent is valuable because it knows claims, denials, CPT codes, payer policies, appeal letters, and EHR workflows.

A legal agent is valuable because it knows contract structures, risk positions, fallback clauses, negotiation playbooks, and approval workflows. The model is necessary. But the system, data, workflow, distribution, trust, and feedback loop create the moat. This is why I do not think “which model wins?” is the most interesting question. The better question is: who owns the orchestration layer between the model and the workflow?

My bet is that most serious applications and agents will be multi-model by default. That is already how I’m building. I’m working on agents that use five different LLMs in parallel, each selected for the task where it performs best: reasoning, extraction, summarization, coding, evaluation, or low-cost classification. The system should optimize in real time, just like a payment router optimizes transaction success, cost, and risk across multiple rails.

LLMs are becoming intelligence rails. The value will accrue to the builders who turn those rails into reliable systems.