The agent stack is so boring where buyers keep looking: the loop.
Loop in sight, the rest of the system is elsewhere in the loop (the agent loop did not disappear). Even more, a layer has been created to manage durable records of work done by the agent: state (a durable record of the work done by the AI agent to service customer requests). This is the crux of enterprise work. The enterprise will configure a system (AI or not) to create and maintain such records and also decide who can read such records, how long to keep them, what they are allowed to do with such records, etc. With the introduction of the state layer, AI agent infrastructure now splits at the state layer. Model + a prompt-centered agent no longer represents the infrastructure argument for AI agents.
The state layer of an AI agent is now the dividing line between the agent being a product and the agent being a parlor trick that happens to access Salesforce etc.
The loop is the wrong buying surface
The harder questions live one layer down.
Where does the state live after the tab has been closed? What writes the checkpoint? What happens when a tool call is successfully completed but the model fails before a summary can be generated? Where do traces, policy decisions and permission grants enter the run? What can an enterprise export when the vendor contract expires? Who can delete memory? Who can subpoena memory? Who is responsible for the record when two agents are working with the same customer and writing to the same file?
The durable boundary is where the agent becomes operating infrastructure.
State turns agent behavior into infrastructure.
The database is becoming the backend
On Thursday, LangChain and MongoDB announced a partnership that frames agent systems as requiring retrieval, persistent memory, operational data access, observability, and reliable deployment. The real launch is a set of tools, and the proof that the partnership matters sits in the “more about” line: "Agents need more than a model and a prompt". A tool-centered agent (or prompt-centered builder) is not a product.
That sentence is doing more work than the launch language around it.
The official LangGraph documentation notes that durable memory should use a database-backed checkpointer, with examples for Postgres, MongoDB, Redis, and Oracle. The LangSmith deployment documentation exposes the state-layer switch directly: set LS_DEFAULT_CHECKPOINTER_BACKEND to mongo and provide LS_MONGODB_URI. That line belongs in the platform review, right next to storage, deployment, and data residency.
Boring. Also the point.
State is where lock-in turns real
O'Reilly's 2026 AI agent architecture diagram defines six layers between the LLM and an AI agent running real workloads. Memory / persistent state is a first level primitive in the architecture above the vector database. This architecture matches the actual experience of the market for the framework layer: easy to switch out when the application is well bounded. State is much harder to switch out.
TGVP outlined a similar argument for how stateful services create moats because memory graphs, identity stores, policy logic, and audit history compound over time. While I dislike moat arguments for infrastructure (such as how custom UIs in enterprise software create a barrier to entry that replaces simply offering a “connector catalog” which can then be replaced by a single adapter), there is merit in the increased memory graphs, identity stores, policy logic, and audit histories that compound over time.
I wrote this out originally as part of my vendor evaluation criteria. The first part was model neutrality. This seems to be something people want to hear. Durable memory should use a database-backed checkpointer, and implementation details for that checkpointer should be exposed to review (storage, deployment, data residency, etc) long before the memory format, the trace (or log) schema, the policy and decision-making portions of the state, and the audit export. Open model routing on top of closed state is a nicer cage with better lighting.
Governance follows the state
The state layer also pulls governance out of slideware.
Agent governance can sound abstract and be confined to slideware when it is stored in policy documents. But as soon as the platform is viewed as a series of layers, the team starts to ask concrete questions about a given run of an agent. For example: who was the principal that started the run off? What was the task that narrowed down the set of permissions that were granted to that principal? What was the tool call that was made that actually crossed the boundary of what that principal was allowed to do with that set of permissions, given by that task? What was the policy that actually allowed that tool call to be made by that principal, given by that task? What was the checkpoint that actually recorded that run of the agent? And what was the trace that actually proves that that run of the agent happened in that order.
Research into the security of Agent-based systems is only just beginning to explore the boundary between what the agent is granted permission to do and what the MCP server will actually allow the agent to do. AgentBound looked at 296 popular MCP servers and found that automatically generated permission manifests worked without modification 80.9% of the time, with 0.6 ms average enforcement overhead. It is clear that MCP tool access is heading in the direction of explicit manifests and runtime enforcement of access rights rather than the current model of trust-by-default and running of host processes.
Identity work also dovetails nicely with this analysis. A 2026 AI identity report defines AI identity as a continuous relationship between what an agent declares and what it is observed to do. This relationship is established through the declaration, the observation, and the confidence in that observation. Thus the same architecture that tracks the state of a productive AI also establishes and tracks the identity of that AI. Declaration without observation is a credential. Observation without a durable store of state is a log. Confidence in the system’s understanding of the relationship between an agent’s declared identity and its observed behavior requires that the system have a durable store of state and be able to compare what has happened to what the system expected to happen.
Task-scoped access control, a topic we’ve covered in the past, also falls under this topic. A writeup for TrueFoundry’s TBAC (Task-Based Access Control) function explained how current identity-centric access models can strain under the weight of agents, because each task may need to call for a different permission set. That can happen quickly, and is particularly exposed to prompt-injection attacks. In short, TrueFoundry argues that task-based controls bundle minimal permissions for the duration of work. And that’s a great architecture, even if the vendor packaging leaves a bit to be desired.
We have looked at the authorization side before through tool-call security and agent principals (e.g. running an agent as a user). From a state-layer perspective, these all tie together: authorization, identity, memory, checkpoints, traces and audit records. The record of an agent doing work.
The stateful loop is the product
A live agent run has a rhythm.
Load memory. Call tool(s) and enforce policy for each. Write trace for each tool call. Record audit event. Update run state. Later, resume run from evidence collected during run. As mentioned previously, each live agent run will have a history which can be inspected by the platform. Such a history provides much greater value than having the agent’s runs be driven by prompts to which the model may or may not answer correctly.
State compounds.
The work does not have to be “unlearned” when an agent run is abandoned for whatever reason. A new run of work can start from the exact point at which the previous run left off.
However, all of this work would be for naught if the evidence from past runs of an agent were not properly monitored. This is why agent monitoring is infrastructure. The stateful agent loop that we discussed earlier includes a section for “write trace and audit records” for a reason. This is the means by which all of the side effects of a run of an agent are recorded, so that they can be inspected by a human (or automated system) later.
It is therefore no surprise that agent platforms that can handle real workloads are focused on platform work (as illustrated before).
The model may still be the expensive line item. The state layer is where operational trust accumulates.
Platform evaluation moves to the state layer
The buying checklist should change.
Where does it store the checkpoints for previous runs? What is the format of the platform’s memory? How does it join together the information from thread ID, run ID, trace ID, and audit ID? Does it record permission decisions made by a tool call with the record of that tool call, or does it have to rebuild that information from elsewhere? Can it export the state information on the platform surfaces, or is that something that one has to ask support to do for them? (Note that just because something is exported as a state, it doesn’t mean that the state information exported is the same as the information that was stored as state information on the platform surfaces). Can the system delete the memory of one user without affecting the memory of other users that are stored in the same index? How does the system partition out the information that it stores for state information by tenants, by environment, by region, by retention rules, etc.
Then ask the uncomfortable version: what gets lost during migration?
Prompt history and agent builder configuration are one thing. Checkpoints, memory, traces, approvals, audit records, and policy history are another. If the vendor can’t migrate all of that to the customer's new backend then the vendor is in control of the live behavior of the agent. Yes, the agent runs on the customer's cloud. Yes, the customer pays for the model and has a model key that the customer uses to sign into the agent. But the memory, the record of the agent’s governance, the record of the agent’s work, that lives somewhere else.
Own that layer or the agent's memory, governance and operating history belong to someone else's backend.
Top comments (0)