DEV Community: varun pratap Bhardwaj

AI Agents Have Protocols. They Still Need Behavioral Contracts.

varun pratap Bhardwaj — Sat, 15 Aug 2026 14:56:00 +0000

AI Agents Have Protocols. They Still Need Behavioral Contracts.

Canonical note: Publish the Medium version first. Then replace https://medium.com/@varun.pratap.bhardwaj/ai-agents-have-protocols-they-still-need-behavioral-contracts-b612da467aae in the DEV front matter with the final Medium URL before publishing on DEV.

AI agent infrastructure is rapidly standardizing connectivity.

We have protocols for tools, models, and agent-to-agent communication.

But a connectivity protocol does not answer a production question that becomes more important as agents gain side effects:

What is this agent allowed to do right now, what must remain true while it acts, and what evidence will prove the decision later?

That is the problem behind Agent Behavioral Contracts (ABC) and the open-source project AgentAssert.

The project started as a research question and is becoming a runtime architecture.

1. Prompts are useful. Prompts are not policy engines.

Consider a coding agent with shell access.

You can put this in the system prompt:

Never execute destructive commands.
Never access credentials.
Ask for approval before changing production.

Those are useful instructions.

But the side effect is still controlled by whatever execution path actually calls the tool.

A stronger design creates an external behavioral decision:

tool.requested
      |
      v
normalize event
      |
      v
evaluate contract
      |
      +---- ALLOW --------> invoke tool
      |
      +---- DENY ---------> return refusal + receipt
      |
      +---- REQUIRE_APPROVAL

Now the policy is no longer only something the model is expected to remember.

It is an executable artifact.

2. The ABC model

Paper I formalized an Agent Behavioral Contract around four ideas:

Preconditions — what must be true before execution.
Invariants — what must remain true during execution.
Governance policies — organizational and operational constraints.
Recovery mechanisms — what happens when behavior violates or approaches a boundary.

A minimal conceptual contract might look like this:

contract:
  name: production-write-policy
  version: 1.0

preconditions:
  - deployment_environment == "production"
  - actor_authenticated == true

invariants:
  - secrets_in_output == false
  - total_cost_usd <= 5.00

governance:
  - production_write requires human_approval
  - allowed_tools in approved_tool_set

recovery:
  on_violation:
    action: deny
    emit_receipt: true

That snippet is intentionally illustrative rather than the canonical current ContractSpec syntax. In production documentation, the contract shown to users should be copied from a version-validated repository example.

The engineering principle is what matters:

policy becomes explicit, versioned, and independently evaluable.

3. Runtime enforcement needs an actual boundary

A contract is only useful as an enforcement mechanism if it is evaluated before the side effect.

That means the runtime architecture needs a Policy Enforcement Point.

Possible surfaces include:

MCP tool requests;
framework before-tool hooks;
model requests;
HTTP or gRPC gateways;
agent input/output boundaries;
memory write proposals;
job or workflow transitions.

The key word is possible.

No adapter should claim complete control merely because it can observe one surface.

For example, an MCP interposer can make strong statements about MCP calls that pass through it.

It cannot automatically control a product's unrelated native editor or shell path unless that path is also routed through an enforceable boundary.

So a useful integration matrix should state:

framework and verified version;
adapter version;
observable events;
pre-side-effect enforceable events;
excluded/native surfaces;
fail-open or fail-closed behavior;
conformance evidence.

That is more meaningful than a wall of integration logos.

4. Declare → Enforce → Prove

The product model for AgentAssert is increasingly simple.

Declare

A versioned ContractSpec defines the behavioral constraints.

Enforce

A covered event is normalized and evaluated.

A decision can be represented with a vocabulary such as:

ALLOW
DENY
MODIFY
REDACT
REQUIRE_APPROVAL
DEFER
ERROR / INCONCLUSIVE

INCONCLUSIVE or an equivalent state is important.

If a required signal is unavailable, silently treating the action as compliant can be dangerous.

Prove

The decision should emit a receipt.

A useful receipt includes:

{
  "contract_version": "...",
  "event_type": "tool.requested",
  "normalized_action_hash": "...",
  "evaluated_rules": ["..."],
  "decision": "DENY",
  "coverage_profile": "...",
  "side_effect_status": "not_executed",
  "trace_id": "...",
  "timestamp": "...",
  "provenance": "..."
}

This turns “the guardrail blocked it” into something independently inspectable.

5. Why trajectory-level behavior matters

Most agent evaluation still focuses heavily on single outcomes.

But an agent can produce a reasonable-looking final answer while violating important constraints during execution.

Examples:

it queried an unauthorized source;
it exposed data to a tool before redacting the final response;
it exceeded a budget;
it made a prohibited intermediate write;
it recovered after a drift event;
it repeatedly approached a threshold across a long session.

A behavioral contract therefore operates over a trajectory, not just the final text.

This is also where drift and recovery become meaningful.

The question is not only:

Was turn 17 acceptable?

It is:

Did the system remain within the declared behavioral envelope over the mission?

6. The composition problem

Now consider a multi-agent pipeline:

Agent A -> Agent B -> Agent C

Each component has a measured success rate.

A naive reliability calculation can be badly misleading if the component failures are dependent.

Shared causes include:

same model
same context
same retrieval source
same memory
same orchestrator
same upstream API
same hidden assumption

If a shared failure mode hits all three components, their errors can be highly correlated.

Now consider redundancy:

          -> Agent B1 ->
Agent A                  -> decision
          -> Agent B2 ->

If B1 and B2 fail independently, redundancy may help significantly.

If they share the same failure cause, the apparent redundancy may provide far less protection.

So dependence interacts with topology.

This is the problem addressed by Paper II: compositional reliability without silently assuming independent failures.

7. A better reliability API

The important product idea from the V2 work is that reliability should expose its evidence basis.

Rather than returning:

reliability = 0.94

a system should be capable of returning something conceptually closer to:

event:
  name: end_to_end_mission_success

mission_distribution:
  id: ecommerce-support-v3

topology:
  type: series

evidence:
  direct_runs: 420
  available_joint_moments:
    - [stage_a, stage_b]
    - [stage_b, stage_c]

guarantee:
  tier: 1
  lower_bound: ...
  confidence_parameter: ...

assumptions:
  - ...

limitations:
  - ...

valid_until:
  - model/version change
  - contract/version change
  - mission-distribution shift

The exact representation can evolve.

The design principle should not:

never separate the reliability number from the assumptions that make it meaningful.

8. Direct observation should beat reconstructed confidence

If you can directly observe the system-level success event, that should usually be the primary evidence.

For example, if the mission is:

"Order changed correctly, customer notified, no unauthorized discount,
and audit record written"

then directly evaluate that mission outcome across complete executions.

Do not throw away the end-to-end evidence and reconstruct success from component pass rates unless you have a specific reason.

This sounds obvious.

In modular AI evaluation, it is surprisingly easy to violate.

9. Sometimes the right answer is “uncertifiable”

Suppose:

one stage has missing logs;
failures are selectively absent;
the mission distribution changed after a model upgrade;
components were evaluated on incompatible datasets;
co-execution evidence is unavailable;
the integration cannot observe the event that the contract claims to enforce.

A production reliability system should not be forced to produce a reassuring number.

It should be allowed to produce:

UNCERTIFIABLE

with reasons.

That is a stronger engineering interface than fake precision.

10. MCP is a useful first proof surface

MCP is particularly useful for demonstrating this architecture because the tool invocation boundary is concrete.

A credible demonstration should show:

1. start real downstream MCP server
2. perform prohibited request without contract
3. confirm side effect occurs
4. route server through AgentAssert guard
5. repeat same request
6. receive DENY
7. confirm downstream invocation count remains 0
8. verify decision receipt

That is a much stronger demo than a screenshot saying “blocked.”

The critical proof is:

the prohibited side effect never reached the downstream tool.

11. The portable-contract direction

A portable behavioral layer needs canonical data structures.

The current product blueprint is converging around concepts like:

AgentActionEnvelope
ContractDecision
DecisionReceipt
CapabilityManifest
ContractBundle
EvidenceReference
ApprovalRequest
CertificationBundle

This allows each framework adapter to map its native lifecycle into one common behavioral vocabulary.

The adapter then publishes what it can actually support.

For example:

C0 = observe only
C1 = pre-model decision
C2 = pre-tool decision
C3 = pre-side-effect + result handling + approval
C4 = receipts + replay protection + signed evidence + conformance

The exact naming can still evolve, but capability-grading is much better than binary “supported / unsupported.”

12. Relationship to other agent infrastructure

Behavioral contracts do not replace the rest of the stack.

They complement it.

Protocols: connectivity.

Identity / authorization: who can access what.

Guardrails: selected input/output/call screening.

Observability: traces and telemetry.

Evaluation: scenario-based evidence.

Behavioral contracts: portable behavioral obligations connected to runtime decisions and trajectory-level evidence.

These layers should integrate rather than compete for one giant “AI safety” label.

13. The broader Qualixar architecture

The separation I find useful is:

LLM / model
    |
    v
SuperLocalMemory
governed durable context
    |
    v
AgentAssert
behavioral contract + runtime decisions
    |
    v
AgentAssay
evaluation / regression / assurance
    |
    v
Qualixar OS / bounded execution
orchestration, approvals, bounded workflows

In shorthand:

Rent the LLM. Own the memory. Enforce the behavior.

The model is replaceable.

The organization's memory and behavioral policy should not be.

14. What I want AgentAssert to become

Not another prompt wrapper.

Not a logo collection.

Not a dashboard that produces an unexplained “reliability score.”

The target is a neutral behavioral-contract layer between agent intent and consequential action.

It should answer:

What behavior is permitted?
Can this action execute now?
Did the agent remain within contract across the trajectory?
What drifted, failed, or recovered?
What reliability statement is justified by the evidence?

That is a difficult product.

It is also the kind of infrastructure I think agentic AI will eventually require.

Research and implementation

Paper I: arXiv:2602.22302
Paper II: arXiv:2608.12895
Code: github.com/qualixar/agentassert-abc
Project: agentassert.com

If you are running agents with consequential tools, start with one question:

Which action in your current stack would you most want an independent contract to deny before the tool ever sees it?

AI Agents Don't Just Need Memory. They Need Memory Governance.

varun pratap Bhardwaj — Thu, 13 Aug 2026 10:50:41 +0000

An AI agent reads a hidden instruction on a webpage. The instruction looks useful, so the agent stores it. The session ends. Weeks later, a different task retrieves that record as trusted context. This is the class of persistent risk that the OWASP GenAI Security Project describes as memory and context poisoning.

The original prompt injection is gone. Its memory remains.

This is the uncomfortable property of durable agent memory: persistence gives useful context a longer life, but it can give bad context a longer life too. A memory system does not become safe because it retrieves the most similar sentence. It becomes operable when a team can govern what enters memory, identify the authoritative record, inspect why a later recall was returned, and deliberately correct or erase state.

That is the central argument of our new public preprint:

SuperLocalMemory 4.0: The Governed Memory Operating System for AI Agents

The paper is available as arXiv:2608.08253. The implementation is open source on GitHub, and the companion citable archive is on Zenodo.

The paper does not claim that one memory product solves every agent-security problem. It makes a narrower engineering argument: once memory influences future agent behaviour, retrieval, governance, and operations cannot remain separate afterthoughts.

Memory becomes operational state before teams notice

Most discussions of agent memory begin with retrieval:

Which embedding model should we use?
Should we add a vector database?
Is hybrid retrieval better than semantic search alone?
How much history should fit in the prompt?
Which reranker gives the best top-k?

These are valid questions. They are not the first questions.

The first question is what the system is allowed to preserve.

An agent may retain a naming preference today, an architecture decision tomorrow, and an incident-response rule next month. Several agents may begin sharing project context. A support workflow may depend on the remembered history of a customer issue. A coding agent may carry forward a correction that prevents the same mistake in the next session.

At some point, memory stops being convenience data and starts shaping production decisions.

That transition is easy to miss because nothing visibly breaks. The agent simply becomes more useful. But the operational burden has already changed. A team now needs to know:

What exact record entered durable memory?
Which identity or process wrote it?
Which policy and scope applied to the write?
What became queryable when the operation completed?
Where does the authoritative record live?
Which retrieval evidence supported a later recall?
Which optional paths could move data outside the local boundary?
How can the record be corrected, exported, or erased?

A similarity score cannot answer those questions. Neither can a generic “memory saved” toast.

A retrieval result is not an explanation

Imagine an incident-response agent retrieving an old remediation instruction. The instruction may be correct. It may also be stale, written for another environment, or accepted from an inappropriate source.

If the system exposes only a ranked result, the operator sees the consequence without the chain of custody.

This is why provenance and recall evidence are not decorative metadata. They are how an engineer investigates a consequential output. The important question is not merely, “Was this record relevant?” It is, “Why was this record eligible to influence the agent now?”

The distinction mirrors mature infrastructure practice. Production systems do not treat identity, policy, observability, recovery, and audit as optional features surrounding the real runtime. Those controls are what make the runtime operable when it receives surprising input or enters a partial-failure state.

Agent memory needs the same control-plane treatment.

Governance has to start at the write

Many memory defenses focus on read time: retrieve candidate records, score trust, filter suspicious content, and constrain what reaches the model. Those controls matter, but read-time filtering arrives after persistent state has already been accepted.

The stronger design pattern is admission control plus durable obligations.

Before a record becomes authoritative, the system should be able to identify the writer, apply policy, bind the operation to the active generation and scope, and issue a receipt. If the canonical write creates derived projections—search indexes, graph state, caches, or external replicas—the system should know which projection owners must apply, verify, compensate, or erase their copy.

The SuperLocalMemory V4 paper describes this as a reliability spine for the primary write path:

Generation-fenced admission prevents a stale runtime generation from silently accepting work under a superseded control state.
A policy registry makes the authorization decision an explicit part of admission.
Verifiable memory transactions turn a write into an inspectable operation rather than a best-effort append.
Per-projection responsibilities assign apply, verify, compensate, and erase ownership.
Hash-checkable completion manifests provide a concrete completion artefact.

The terminology matters only because it names failure questions. If a request is retried, can the system distinguish a duplicate from a new write? If the canonical record succeeds while a projection is unavailable, can the operation be reconciled? If erasure is requested, can the system identify every registered obligation? If policy rejects a write, can an operator inspect that boundary?

This is not about adding bureaucracy to a personal note. It is about having a path to evidence when memory becomes consequential.

“Local-first” must describe a boundary, not a mood

Local-first is frequently reduced to “there is a local file.” That is inadequate.

A system may store its primary database locally while sending text to a remote embedding model, provider-backed enrichment service, connector, cloud backup, proxy, or reranker. Some deployments will accept those trade-offs. The failure is not using a networked capability; the failure is hiding an active external path behind an unqualified local claim.

SuperLocalMemory V4 separates canonical local state from optional external capabilities. Its canonical memory can remain in a configured local data root. Provider-backed enrichment, connectors, cloud backup, proxy paths, dependency or model downloads, and peer behaviour are separate choices that can create network paths.

The operating modes make that boundary legible:

Mode A — Local Guardian: the core memory path uses local state without a cloud model provider.
Mode B — Smart Local: an operator-managed local model can support enrichment while canonical memory remains local.
Mode C — Full Power: a configured external provider can support enrichment; the relevant data path is therefore provider-assisted, not local-only.

The useful promise is not “nothing can ever leave this machine.” The useful promise is that canonical state, optional paths, and operator choices are distinguishable.

That distinction is the basis of the campaign line:

Rent the LLM. Own the memory.

Models will change. Providers will change. Inference budgets will change. The durable operational context that guides the next action should not become an accidental by-product trapped inside whichever model interface a team happens to rent today.

Retrieval still matters—but it is one part of the contract

Governance does not replace retrieval quality. A governed system that cannot find useful context is still a poor memory system.

The V4 architecture combines five retrieval channels through reciprocal-rank fusion:

Dense semantic retrieval for meaning-level similarity.
BM25 lexical retrieval for exact terms and rare identifiers.
Temporal retrieval for time-sensitive context.
Hopfield-associative retrieval for learned associations.
Spreading activation across related entities and memories.

The paper also describes bi-temporal recall, multi-scope personal, shared, and global memory, role-based access control, audit trails, and GDPR-oriented export and verified erasure mechanisms. The runtime exposes CLI, MCP, HTTP daemon, dashboard, editor-integration, and framework-adapter surfaces.

That list is not evidence by itself. A feature inventory tells us what exists; a protocol tells us what was tested.

Four papers, one SuperLocalMemory research line

V4 is the latest paper, but it is not the first research record behind SuperLocalMemory. The public work now spans four arXiv preprints, newest first:

SuperLocalMemory 4.0: The Governed Memory Operating System for AI Agents — the current V4 architecture. It brings retrieval, learning, governance, operating modes, write-path reliability, and operator surfaces into one system description.
SuperLocalMemory V3.3: The Living Brain — the lifecycle paper. It explores biologically inspired forgetting, cognitive quantization, and multi-channel retrieval for zero-LLM agent memory.
SuperLocalMemory V3: Information-Geometric Foundations for Zero-LLM Enterprise Agent Memory — the mathematical-foundations paper. It studies information-geometric retrieval, lifecycle dynamics, and contradiction modelling.
SuperLocalMemory: Privacy-Preserving Multi-Agent Memory with Bayesian Trust Defense Against Memory Poisoning — the privacy and threat-model paper. It studies local-first multi-agent memory, provenance, isolation, and Bayesian trust scoring against memory poisoning.

These papers form a research lineage; they are not one pooled benchmark. Each is a public preprint with its own version, implementation context, methodology, and evidence boundary. Historical V3 results should not be relabelled as fresh V4 release measurements. The V4 paper explicitly consolidates the earlier research direction while reporting separate current mechanism evidence.

What the 2,200 repetitions establish—and what they do not

The V4 paper evaluates eleven fault-injection and mechanism scenarios, each repeated 200 times. The released evidence bundle reports 2,200 of 2,200 deterministic repetitions upholding their stated scoped component properties.

It also reports the governed write envelope at 3.522 ms p50 and 5.297 ms p99, compared with an ungoverned baseline of 1.835 ms p50 and 2.569 ms p99 in the reported in-process setup. That corresponds to measured in-process control-plane overhead of 1.687 ms at p50 and 2.728 ms at p99.

Those numbers require their boundary.

They are scoped component and mechanism measurements. They are not an end-to-end multi-process production guarantee. They are not an external retrieval-accuracy benchmark. They do not establish benchmark superiority over another product. They are not a compliance certification. The paper is a public preprint, not a venue-reviewed publication.

This limitation is not fine print. In AI Reliability Engineering, scope is part of the result. Removing the scope produces a stronger marketing sentence and a weaker technical claim.

A five-minute test for any agent-memory system

You do not need to adopt SuperLocalMemory to use the paper's operating questions. Apply this test to your current memory stack.

Choose one bounded workflow with synthetic or non-sensitive data. Do not begin by ingesting an entire company knowledge base. That is the wrong move because it creates a large, opaque state surface before anyone has established write authority, scope, or recall investigation.

Then run this sequence:

Write one record with a declared source and scope.
Capture the receipt or operation identifier.
Confirm when the canonical record becomes queryable.
Recall it with a precise query.
Inspect the evidence behind the result.
Correct or erase the record and verify the outcome.

After the test, ask:

Could another engineer reproduce the operation in an isolated workspace?
Could the team identify the authoritative record if a projection disagreed?
Could the operator list which external paths were active?
Could a future incident reviewer trace the recall back to its source?
Could the team prove that a correction or erase operation completed?

If those answers are vague, the gap may not be retrieval. It may be operability.

The model is replaceable. The memory contract is strategic.

Agent memory is becoming a long-lived layer between models, tools, people, and future actions. That makes it valuable. It also makes it dangerous to treat as an invisible convenience.

The engineering requirement is not perfect memory. Perfect memory would be a liability. The requirement is controlled memory: explicit admission, authoritative state, inspectable recall, bounded sharing, deliberate forgetting, and honest evidence.

That is the argument behind SuperLocalMemory V4 and the broader category we are building at Qualixar: AI Reliability Engineering.

The LLM can be rented. The memory contract should remain yours.

Read and inspect the work:

Varun Pratap Bhardwaj is the founder of Qualixar and researches AI Reliability Engineering. SuperLocalMemory is an independent open-source Qualixar project.

How AI Memory Actually Works

varun pratap Bhardwaj — Fri, 31 Jul 2026 15:07:15 +0000

Open ChatGPT in a browser and ask it to remember that you are vegetarian. It can carry that fact into a later conversation. Open Claude, work inside a project, and it can maintain a memory for that project without mixing it with another one.

Now move into a terminal. Open Claude Code or Codex on a real repository. What counts as memory there is often a markdown file: CLAUDE.md for Claude Code, or an equivalent instruction file read by the coding agent.

These two worlds use the same word for very different mechanisms.

In the browser, the memory system lives on the provider's side. In your terminal, the file lives on your disk. The browser system can select information and bring it forward. The local file gives you ownership and legibility. But a file does not rank its contents, understand that one fact replaced another, or decide which three lines matter for the question you just asked. It is a document, read as a document.

That contrast is the cleanest place to start, because it removes a common mistake: memory is not whatever text happens to survive between prompts.

ChatGPT itself has two memory mechanisms, according to OpenAI's published documentation. Saved memories are the explicit items you tell it to remember. They are visible, editable, and deletable. Reference chat history is the implicit mechanism: it selects useful information from earlier conversations to carry forward. These mechanisms are controlled separately, and saved memories are stored separately from chat history. Deleting a conversation does not, by itself, delete a saved memory created from that conversation.

Claude's published documentation describes separate memory per project. Its memory summary can be viewed and edited, while incognito chat provides a way to avoid carrying a conversation into memory. That project boundary matters. Client work and personal work should not become one undifferentiated pool.

Claude Code is a different case. Its memory mechanism is hierarchical markdown files named CLAUDE.md. The file is client-side, inspectable, and yours. That is useful. It is also static. If an old instruction remains after the architecture changes, the file does not know it is stale. If it grows to several pages, it does not know which paragraph deserves attention now. It has no retrieval system because it is not a retrieval system.

So the real engineering problem is not “how do I preserve text?” The problem is: how do I build something that can decide what is worth keeping, recover it by meaning, understand time and relationships, stay inside the right boundary, and become better without silently becoming worse?

That is an AI memory system.

A database is not a memory

Suppose I save every conversation for six months. Nothing is lost. I now ask: “What did we decide about payment retries?”

The database can search for the words “payment retries.” But the decision may have been written as: “If the card fails, wait and try again, but only twice.” The meaning matches. The words do not.

A conventional keyword lookup finds what matches. Memory has to find what means the same thing.

That is where vectors enter.

A vector is a position on a map of meaning. Put “king” and “queen” on that map and they should sit near each other. Put “pizza” on it and it should sit elsewhere. A real map has hundreds of directions, sometimes more than a thousand, because the system needs enough room to separate fine shades of meaning. You do not need to picture every direction. The useful idea is simply that related text receives nearby coordinates.

An embedding is the operation that produces those coordinates. Text goes in. A position comes out. People often use “embedding” and “vector” as if they mean the same thing. In casual discussion that is harmless. Technically, the vector is the position; embedding is the process of working out that position.

Once stored text has positions, “payment retry logic” can land near “if the card fails, wait and try again.” Retrieval no longer depends on shared spelling. The system searches a neighbourhood of meaning.

That is semantic search. It is necessary. It is not sufficient.

There is also an operational trap here. The map is not universal. Different embedding models draw different maps. Change the embedding model and you change the coordinate system used to interpret stored material. The underlying memories did not change, but the ground beneath their positions did. Any production memory design has to treat embedding choice and migration as system concerns, not as a hidden implementation detail.

The three holes in vectors alone

Vector search looks so convincing in a demo that teams mistake it for the whole system. It fails in three predictable ways.

Similar is not the same

Compare these two statements:

We decided to retry twice.

We considered retrying twice and rejected it.

They use almost identical language. Their vectors can sit close together. Their operational meanings are opposite. Distance can tell us that both concern the same subject. Distance alone cannot tell us which decision became valid.

This is why a nearest-neighbour result is a candidate, not an answer.

Vectors have no clock

Imagine that a team says in March, “We use Postgres.” In June, the team migrates. In July, an agent asks memory which database the project uses.

Both statements can be semantically relevant. The March statement may even be a closer wording match. But it is no longer current. A vector does not understand that March preceded June or that a later fact superseded an earlier one.

Time cannot be pasted on as decorative metadata. It has to participate in ingestion, contradiction detection, invalidation, retrieval, and ranking.

Proximity is not a relationship

Vectors tell us which things are near one another. They do not tell us that a bug came from a decision made in a meeting by a particular person, or that a workaround belongs to a specific release and was retired by a later fix.

Those are edges, not distances.

A useful memory system therefore needs three structures at once: a semantic map, a graph of connections, and a clock. The map finds related meaning. The graph explains how pieces relate. The clock tells the system what was true when, and whether it is still true now.

That combination is the beginning of memory. A vector database by itself is still storage with an unusually good search function.

The three types of memory

The next mistake is to treat every retained item as the same kind of object. Human memory gives us a cleaner model.

Episodic memory is what happened. Your first day at a job is an episode: people, place, sequence, and time belong together. In an agent system, a debugging session, a decision meeting, or a failed deployment is episodic. The timestamp is part of the event, not an optional label attached later.

Semantic memory is what is true. Paris is the capital of France. You may not remember when you learned that fact because the fact survived while the original episode disappeared. In engineering work, “this service owns invoice generation” is semantic memory. It may have originated in a conversation, but the useful retained object is the claim.

Procedural memory is how to do something. Riding a bicycle is the standard human example: you can perform the skill without being able to write a complete description of balance. For an AI agent, a verified workflow, a proven recovery sequence, or a reusable procedure belongs in this category.

These three types have different shapes and different retrieval needs. A chat archive is mostly episodic. It records what was said and when. Calling that complete memory is like calling a server log an operating manual and a knowledge base at the same time.

The distinction matters for AI Reliability Engineering because reliability depends on feeding the agent the right kind of evidence. An event can explain why a decision happened. A fact can state the current decision. A procedure can tell the agent what to do next. Flatten them into one text pile and the agent has to reconstruct those differences every time it answers.

How a memory gets made: the seven-stage ingestion pipeline

Storing a memory is not one write. In SuperLocalMemory v3.8.10, the ingestion path is a sequence of gates and transformations. Each stage answers a different question.

1. Decide whether the input is worth keeping

Most text is not durable information. Greetings, repeated acknowledgements, transient tool noise, and duplicated context can overwhelm retrieval if everything is retained. The first stage asks whether the input carries enough information to justify its future cost.

This is the role represented by entropy_gate.py. The principle is plain: do not make retrieval harder by storing noise.

Every accepted item will cost storage, indexing work, retrieval time, and possibly prompt tokens later. A memory system that accepts everything has avoided judgment at ingestion and pushed the entire burden into recall.

2. Classify what arrived

Is the item a fact, an event, a preference, or another memory shape? Classification controls what later stages should do with it. The verified module here is type_router.py.

This is not filing for filing's sake. A preference may remain valid until explicitly changed. An event belongs on a timeline. A fact may contradict an existing fact. Routing lets the system apply the correct rules instead of treating every sentence as a generic chunk.

3. Extract the actual claim

A paragraph is not a fact. It may contain context, hedging, alternatives, and one load-bearing assertion. fact_extractor.py pulls out the claim that should be represented.

Consider: “We tested three options. Redis was fastest, but because this service must survive a cold restart without another dependency, we chose the local store.” Saving the entire paragraph may be useful as an episode. The semantic claim is narrower: the service uses the local store, with a stated reason.

Extraction makes the retained unit explicit enough to compare, connect, invalidate, and retrieve.

4. Resolve entities

“The client,” “Rahul,” and “that customer” may refer to one entity across several months. If the system stores them as three unrelated names, it does not have one memory of the person or organisation. It has fragments that cannot reliably meet.

entity_resolver.py handles this stage. Entity resolution gives later graph and retrieval operations a stable thing to point at.

This is also where careless systems create false joins. Two people can share a name. A good resolver has to avoid turning linguistic similarity into identity without enough evidence.

5. Parse time and validate temporal truth

temporal_parser.py attaches time. temporal_validator.py checks whether the new information invalidates something already believed.

This stage is what separates accumulation from learning. New information does not always sit beside old information. Sometimes it overrules it.

Crucially, invalidation should not mean erasure. If the project used Postgres in March and migrated in June, the March fact was true in March. The system may need that history to explain an old incident or reproduce an earlier release. The correct state is superseded, with a timeline, not deleted as if it had never existed.

6. Connect the memory

auto_linker.py creates relationships to related memories, facts, entities, and events. This builds the web that vectors cannot provide.

The relationship can answer questions that similarity cannot: which decision caused this change, which event confirmed a claim, which person owns the component, or which procedure resolved the incident.

The graph is valuable because reasoning often travels through a connection. A question may not resemble the target memory closely in vector space, but an entity or event path can still lead to it.

7. Consolidate

consolidator.py performs the final stage. Human memory does not keep every sensory detail forever. Repeated episodes become patterns; details fade while a useful summary remains.

An artificial memory system needs the same discipline. Without consolidation, it grows into a warehouse of near-duplicates. The retrieval problem becomes harder with every accepted item, even if every item was reasonable on its own.

Consolidation turns accumulated experience into something more compact and reusable. It is not deletion with a nicer name. It is the conversion of repeated or related material into a stronger representation while preserving what remains important.

How memory comes back: six channels, fusion, and re-ranking

When a user asks a question, a capable memory system does not run one search. It runs several searches in parallel because relevance has more than one shape.

SuperLocalMemory v3.8.10 has six verified retrieval channel modules.

semantic_channel.py searches the meaning map. This is the vector path. It finds material that expresses related ideas even when the wording differs.

bm25_channel.py searches keywords. Semantic retrieval did not make literal text useless. Exact names, error strings, identifiers, and rare terms often need lexical search.

entity_channel.py retrieves around a person, project, customer, component, or other resolved entity. It answers “what do we know about this thing?” even when the individual memories use different language.

temporal_channel.py searches by time. It can prefer the relevant period and help distinguish current truth from historical truth.

hopfield_channel.py follows connections. It uses the web rather than only the map, letting retrieval reach related material through stored relationships.

profile_channel.py applies scope. It keeps retrieval inside the active world instead of allowing a relevant-looking memory from the wrong project or identity to leak into the answer.

These six channels will disagree. That is expected. Each produces scores with its own meaning and scale. A semantic similarity score cannot be averaged naively with a keyword score or a graph score.

The verified fusion.py module uses Weighted Reciprocal Rank Fusion. The important move is to combine rank positions rather than pretend raw scores are comparable. Each channel returns an ordered list. Fusion rewards candidates that appear strongly across several lists, with weights reflecting the channels trusted for the query. In the current verified implementation, the default fusion constant is k=15.

This gives the system a useful property: a memory that appears partway down several independent lists can beat a memory that appears first in only one. Agreement across retrieval views becomes evidence.

Fusion still produces candidates, not final truth. The first stages are designed to be broad and fast. The last stage can spend more compute on fewer items.

That is the job of reranker.py: a subprocess-isolated cross-encoder reads the question and each top candidate together, then judges whether the candidate actually answers the question. Unlike the original vector lookup, this model gets to inspect the relationship between query and candidate directly.

The order matters. Running the expensive judge over the full store would be wasteful. Running only fast retrieval would leave too many semantic near-misses. Broad retrieval narrows the field. Fusion combines different kinds of evidence. Re-ranking performs the careful final selection.

There is a useful scar in the code: an earlier fusion version re-fused results three times and destroyed the rankings. That detail is more instructive than a perfect architecture diagram. Retrieval components do not become correct merely because each one sounds reasonable. Their composition has to be measured.

Forgetting is a requirement, not a defect

A thought experiment makes the scaling problem obvious. Imagine a store with a million memories. This is not a claimed benchmark or measured capacity result. It is a way to expose what breaks.

Every retained memory is another candidate that can look relevant. A useful result can be buried under a large number of things that resemble it. Perfect retention therefore does not produce perfect recall. It can produce noise.

Deleting by age is not enough. An architecture decision from a year ago may still govern the system. A message from minutes ago may already be worthless. Age and importance are different variables.

SuperLocalMemory couples two mechanisms to deal with this problem.

The first is the Ebbinghaus forgetting curve. Ebbinghaus's work dates to 1885. The shape is the point: forgetting is steep early and then flattens. In the verified coupling code, retention contributes to forgetting drift as:

lambda_forget = (1 - R) * forgetting_drift_scale

The curve tells the system how memory fades over time. It does not, alone, tell the system which memory deserves to resist that fade.

The second mechanism couples Fisher confidence to Langevin dynamics. Picture a memory as a particle moving within a boundary. If it reaches the boundary, it is archived. Temperature controls how strongly that particle moves.

The conceptual relationship is:

T_eff = T0 / confidence

The implementation adds an epsilon guard:

T_eff = T_0 / (fisher_confidence + epsilon)

High confidence makes the denominator larger, so effective temperature drops. The memory moves less and stabilises toward the active region. Low confidence makes effective temperature higher. The memory moves more and drifts toward archival.

The two mechanisms are coupled. The verified implementation combines Fisher temperature and Ebbinghaus forgetting drift as:

T_combined = T_fisher * (1 + lambda_forget)

This gives the system a way to forget without a hand-written cleanup schedule deciding each record's fate. Confident memories stabilise. Uncertain memories are more likely to fade. The resulting behaviour is based on both time and learned confidence, not on “delete everything older than this date.”

That distinction is central to AI Reliability Engineering. Forgetting is safe only when it is governed, inspectable, and coupled to evidence about usefulness. An unbounded store is unreliable because noise grows. A blunt retention rule is unreliable because it can remove old but governing knowledge. The system needs controlled decay.

Temporal invalidation: preserve history without serving stale truth

Forgetting and invalidation solve different problems.

Forgetting manages value under scale. Invalidation manages truth under change.

Return to the database example. “We use Postgres” was true in March. A later migration makes another statement true in June. If both facts remain active with equal standing, the memory system can retrieve obsolete architecture with complete confidence.

The wrong fix is to erase March. Historical questions still need it. An incident from April may make sense only under the old architecture.

The correct model is a timeline with supersession. The earlier fact remains available as historical truth, while the later fact becomes current truth. Retrieval can then answer two distinct questions correctly:

“What database do we use now?”

“What database were we using when the April incident happened?”

This is why time belongs inside the memory object and the retrieval logic. A timestamp column added after the fact does not automatically create temporal reasoning. The ingestion pipeline must detect a possible contradiction, validate it, link the new and old states, and change which one is treated as current.

A learning system needs a system that can stop it

The six retrieval channels need weights. Those weights can be guessed once and frozen, or they can learn from actual recall outcomes.

Learning sounds obviously better. It is also where a memory system can quietly degrade.

A new ranking model may look promising on a small sample and perform worse after promotion. Without a guardrail, “self-improving” means the system is authorised to reduce its own quality without an alarm.

SuperLocalMemory's verified learning discipline uses shadow testing and rollback.

Queries are routed deterministically using a hash, so the same query goes to the same lane even across a daemon restart. That makes the comparison reproducible rather than random.

Phase A is a fast triage at n=100. Early promotion requires both a strong effect and statistical significance. If that gate is not met, Phase B continues to n=885 paired comparisons. That sample size is set for a minimum detectable effect of 0.02, power 0.8, two-sided alpha 0.05, and sigma 0.15.

Promotion is not the end of validation. The next 200 recalls are watched against the pre-promotion baseline. If mean NDCG@10 drops by at least 2%, the system automatically rolls back. The model flag changes happen in one BEGIN IMMEDIATE transaction, and retraining is disabled for 24h after rollback so the system cannot immediately repeat the same failure.

There is also a defined failure path for a missing previous model. The code does not demote the active model and leave the user with nothing. It logs the error, enters safe mode, and falls back to the Phase-2 heuristic.

This is the pattern I care about: learning is allowed only inside a reversible control loop.

The numbers are not decoration. n=100 is triage, not final proof. n=885 is the full paired validation under the stated power and significance assumptions. 200 is the post-promotion watch. A 2% mean NDCG@10 drop is the rollback threshold. Each number corresponds to a different failure mode.

Anyone can add retraining. Reliable systems define what evidence permits promotion, what evidence triggers reversal, and what happens when reversal itself cannot complete normally.

Profile isolation: walls at the record level

Memory becomes dangerous when several worlds share one system.

Client project. Personal project. Day job. A query in one should not retrieve a plausible answer from another. Semantic relevance does not grant permission.

SuperLocalMemory's verified profile model scopes every memory, fact, entity, and learning record with profile_id. The boundary exists at the record level, including the learning data, rather than only at the conversation or interface level.

This is columnar isolation, not separate stores. That distinction matters because the wrong mental model leads to the wrong operational claims.

Switching profiles is config-only and moves zero data. Records stay where they are. The active profile changes which scoped records the system can operate on. There is no copy, export, or migration during a switch.

The design rule is private by default and shared only by an explicit scope decision. If a memory that should have been shared remains private, the failure is reduced availability and can be corrected. If a private memory leaks into another profile, the failure may be irreversible.

At organisational scale, profile isolation is only part of the boundary. Role-based access determines who may read, write, delete, or inspect the audit trail. Retrieval quality cannot compensate for weak access control. A highly relevant result from the wrong profile is still the wrong result.

Caching and compression: the unglamorous economics

Memory costs compute when text is embedded. It costs time during retrieval. It costs prompt tokens when retrieved material is injected into a model call.

Two practical levers control that cost.

First, do not repeat work. Exact caching can reuse a result for the same question. Semantic caching can reuse work when differently worded questions mean the same thing. The verified cache modules cover exact and semantic paths, centroid storage, invalidation, and stampede control: exact.py, semantic.py, centroid_store.py, invalidation.py, and stampede.py.

Cache invalidation matters because memory changes. A cached answer that ignores a newly superseding fact is fast and wrong. The cache has to participate in the same truth lifecycle as the underlying memory.

Second, reduce what is sent. Retrieved memories are prose, and prose can be compressed while retaining the useful meaning. The verified compression path includes ccr.py, prose_llmlingua.py, router.py, and align.py.

I am deliberately not attaching a compression ratio or cache hit rate. Those figures were not verified in the source material for this article. The engineering point does not need an invented percentage: repeated retrieval wastes compute, and verbose context consumes tokens on every call.

Caching prevents repeated work. Compression reduces the payload. Both become more important as memory stops being a demo and becomes infrastructure used every day.

Multi-agent shared memory

Most developers no longer use one AI surface. An editor agent, a terminal agent, and a background process may all touch the same project. Without shared state, each works from a partial view. One can repeat a failed approach. Another can undo a decision made minutes earlier. The human becomes the message bus between tools.

Putting memory below the agents changes that shape.

An agent records a verified decision into the shared layer. Another agent retrieves it through the same scoped system. The memory is not trapped inside either agent's private transcript. SuperLocalMemory's verified mesh modules include mesh/broker.py and mesh/remote_sync.py for this shared-memory direction.

Shared does not mean unbounded. The profile and permission rules still apply. The value is that authorised agents can coordinate through one memory layer instead of maintaining conflicting local histories.

This is also why memory belongs outside the model. Models and tools can change. A durable memory layer can serve several agents while keeping the truth lifecycle, retrieval pipeline, isolation policy, and learning controls consistent.

The reference implementation

Memory is not storage. Storage is the easy part.

Memory is the system that decides what deserves to survive, what kind of thing it is, which entity it belongs to, when it was true, what it connects to, whether it has been superseded, how confidently it should remain active, which profile may see it, and whether it actually earned its place in an answer.

That is a large claim, so I prefer an implementation you can inspect over a diagram you have to trust.

SuperLocalMemory is the open-source reference implementation for the architecture described here. The verified source for this article is v3.8.10: the seven-stage ingestion path, six retrieval channels, Weighted Reciprocal Rank Fusion, cross-encoder re-ranking, Ebbinghaus and Fisher-Langevin forgetting, deterministic shadow tests, automatic rollback, per-record profile_id isolation, cache and compression modules, and shared-memory mesh components.

This is what AI Reliability Engineering looks like at the memory layer: not a promise that the model will remember, but a set of explicit mechanisms for deciding what memory means, measuring whether recall improved, and recovering when it did not.

Read the code. The scars are part of the design.

MCP Went Stateless. State Did Not Disappear.

varun pratap Bhardwaj — Fri, 31 Jul 2026 15:07:13 +0000

On 28 July 2026, the Model Context Protocol removed its handshake, retired its protocol-level sessions, and stopped requiring servers to remember clients between requests.

The easy headline is that MCP went stateless. The wrong conclusion is that state went away.

It did not. State moved.

Some of it now travels with each request. Some of it becomes an explicit handle passed as a normal tool argument. Long-lived continuity—what happened on Monday, what failed last week, what the agent already learned—belongs above the transport in a memory layer owned by the caller or the surrounding system.

That distinction matters because MCP has spent the spring being declared dead for reasons that mixed a real context-cost problem, a badly repeated token number, and a quieter distributed-systems flaw. The context problem remains real. The distributed-systems flaw is what the 2026-07-28 specification directly attacked.

I want to explain the whole chain from zero: why MCP exists, how the “MCP is dead” narrative acquired a number it could not honestly support, why sticky sessions were a bigger enterprise problem than the commentary suggested, what the specification changed, and what an engineering team should migrate now.

MCP from zero: the M-by-N connection problem

Start with an AI system on one side and the systems it needs on the other.

The AI may need to read a repository, inspect an issue, query a database, retrieve a design file, search internal documentation, or call an operational tool. Capability is not the same as access. A model can reason about a bug while still being unable to see the error report that contains the decisive evidence.

Before a shared protocol, every model vendor and every tool provider could build a custom connection. If there are M AI clients and N external systems, the naive integration surface is M multiplied by N. Three clients and ten tools produce thirty separate connections. Authentication, schemas, errors, retries, capability discovery, and version drift can all behave differently across those connections.

That is the actual problem MCP addresses.

MCP does not make the model smarter. It does not replace the external API. It does not make permissions disappear. It defines a common interface through which an AI client can discover and invoke tools or retrieve context. The tool provider implements the MCP-facing door once. Compatible clients can use the same shape instead of demanding another proprietary bridge.

This is why “just use APIs” is not a rebuttal. MCP servers usually reach real APIs, databases, filesystems, or services underneath. The protocol standardizes how an AI client encounters those capabilities. REST can be part of the implementation, but an estate of unrelated REST endpoints is not, by itself, a shared agent-tool contract.

Anthropic released MCP, but ownership did not remain an Anthropic-only story. On 9 December 2025, MCP was donated to the Agentic AI Foundation, a directed fund under the Linux Foundation, co-founded by Anthropic, Block, and OpenAI, with backing that included Google, Microsoft, AWS, Cloudflare, and Bloomberg. The precise wording matters: a foundation under the Linux Foundation, not a protocol “run by” the Linux Foundation.

That broader stewardship did not guarantee that the original design would scale. Standards earn trust by changing when deployed systems expose the wrong abstraction. MCP had two separate problems to confront: tool-schema cost and transport-level state.

The internet compressed those into one obituary. They should never have been treated as one issue.

How “MCP is dead” became the spring narrative

An MCP client needs to know which tools are available and how to call them. Tool definitions include names, descriptions, parameters, and constraints. Put enough definitions into a model context and the menu starts consuming the meal.

The measured range in the approved research for this piece is 550 to 1,400 tokens per tool definition. Connect GitHub, Slack, and Sentry in the configuration examined by Apideck, and the setup reaches roughly forty tools. Apideck’s own stated total for that setup is 55,000 tokens before the user’s real work has had a chance to begin.

That is not a cosmetic inefficiency. It is context occupied by descriptions of possible actions, including actions the model may never use. It can reduce the room available for the task, the evidence, the conversation, and the answer. It can also make every call carry a cost that has little relationship to the one tool actually needed.

Then came the number that turned a technical complaint into a spring headline: 72%.

In March, Perplexity’s CTO said on stage that the company was moving away from MCP internally and referred to 72% of the context window being consumed by tool definitions. The statement spread. Y Combinator’s CEO amplified it. “MCP is dead” became a compact take that travelled faster than its provenance.

I followed the number backward because 72% is precise enough to demand a precise denominator, tool set, context window, and measurement procedure. I could not source it cleanly as a Perplexity measurement.

The trail led to Apideck, a company that sells an alternative to MCP. That commercial position does not make its measurements false. It does make provenance important. The problem is that Apideck’s own post does not say the GitHub, Slack, and Sentry setup costs 143,000 tokens. It says 55,000 tokens for that roughly forty-tool setup. The 143,000-token figure appears separately as a report attributed elsewhere.

Those are two different examples.

At least one widely shared report welded them into one sentence: the named three-server setup, the 143,000-token total, and the 72% claim became one apparently coherent fact. Once fused, the sentence was easy to quote and hard to question. I nearly repeated it myself. It was already in my notes before I checked the underlying claims against each other.

The correction does not rescue the old tool-loading model. It makes the criticism more credible.

55,000 tokens for GitHub, Slack, and Sentry is Apideck’s own number. It is enough to demonstrate the problem. The 143,000 figure is a separate, mis-cited report in this provenance chain. The 72% claim cannot be cleanly presented as a Perplexity benchmark from the approved evidence. Repeating the fused version would make a valid engineering concern rest on a claim that does not survive inspection.

This is a good example of AI Reliability Engineering applied to technical communication. Do not ask only whether a number sounds plausible. Ask which entity measured it, which configuration it describes, where the denominator came from, and whether the cited source says what the summary claims it says.

Cloudflare provides stronger primary evidence for the large-tool case because its repository publishes the comparison directly. Its API surface contains 2,594 tools. Putting the raw OpenAPI specification into the prompt is approximately 2,000,000 tokens. Native MCP with full schemas is 1,170,523 tokens. Native MCP reduced to required parameters is 244,047 tokens. Cloudflare’s code-mode approach exposes three tools and uses approximately 1,100 tokens. The table and implementation are in Cloudflare’s MCP repository.

Against a 200,000-token context window, the raw specification, full native schemas, and the 244,047-token minimal form do not fit. Code mode does. This is also where the widely repeated 244x comparison needs care: it compares code mode with the already reduced 244,047-token native MCP form, not with the approximately 2,000,000-token raw specification.

The lesson is narrower than “MCP is dead.” Eagerly loading a large tool catalogue into the model context is the wrong discovery strategy at that scale. The transport standard and the prompt-loading policy are related, but they are not identical. You can keep a common protocol while changing discovery, filtering, search, tool grouping, deferred schema loading, or code execution around it.

There is a second correction worth making. Perplexity moving away from MCP internally did not mean Perplexity stopped supporting MCP externally. The approved research found that it still operated an MCP server for outside developers. “One company changed an internal transport choice” and “the protocol is dead” are not equivalent statements.

The token problem was loud because it appeared inside the model bill and the context meter. The state problem was less visible. It was also the one that directly constrained how MCP servers could be deployed behind ordinary enterprise infrastructure.

The enterprise blocker hiding behind sticky sessions

Imagine a food-delivery request. This is an analogy, not a claim about any named company’s architecture.

The caller sends a request through a load balancer. Behind it are multiple server instances. The load balancer should be free to send each request to an available healthy instance. That is how traffic spreads, failed instances are bypassed, and capacity is added or removed.

The old MCP interaction was stateful at the protocol level. The client initialized a connection. The server returned a session identifier. Later requests used the Mcp-Session-Id header so the server could recover what it had stored about that interaction.

Now place that state in the memory of server instance one.

Instance one recognizes the session. Instances two and three do not. The load balancer can no longer route freely unless the state is replicated elsewhere. It must keep sending that client back to instance one. That is a sticky session.

Sticky sessions are not automatically broken engineering. They are sometimes a reasonable local optimization. They become a protocol tax when every compliant deployment inherits them even though the application does not need conversational state inside the transport.

The costs are familiar to anyone who has operated distributed services. One instance can receive a disproportionate share of active sessions while another has spare capacity. If the pinned instance fails, in-memory session state can fail with it. Scaling down becomes harder because an instance may still own live sessions. Serverless and edge execution become awkward because workers are expected to be disposable. A protocol that assumes the same server will remember the client fights the infrastructure instead of using it.

You can work around this by externalizing the session store. Redis is a common shape for that solution: every server instance reads and writes shared session data, so any instance can reconstruct the interaction. But now the transport has required a database, network calls, expiry policy, failover design, consistency decisions, and operational cost merely to preserve a protocol-level conversation.

That was the real enterprise scaling constraint.

The context problem asks, “How much tool description should enter the model?” The sticky-session problem asks, “Can any healthy server instance handle the next request?” One is prompt architecture. The other is distributed-systems architecture. Solving one does not solve the other.

The MCP project’s 2026-07-28 release attacked the second problem by removing the session assumption from the core request path.

What changed on 2026-07-28

The specification moved MCP from a bidirectional stateful protocol to a stateless request/response model. Anthropic’s implementation note states the operational result directly: servers can deploy on serverless and edge infrastructure.

The old initialize and notifications/initialized exchange is retired. The Mcp-Session-Id header is retired. Protocol-level sessions are gone from the Streamable HTTP transport.

Instead of negotiating identity and capabilities once and expecting the server to remember them, every request carries its protocol version, client identity, and client capabilities in _meta. The request becomes self-describing enough for any compatible instance to process it.

That changes the load-balancer picture. Request one can reach instance one. Request two can reach instance three. If instance one disappears, the caller is not bound to a protocol session that died with it. Capacity can scale horizontally without teaching the load balancer which client belongs to which worker.

This is the stateless-server pattern used across resilient request/response systems: make workers interchangeable, move required request context across the boundary, and make durable state explicit. The benefit is not that the system has no state. The benefit is that an arbitrary worker does not secretly own it.

GitHub provides the strongest implementation receipt in the approved evidence. Ahead of the specification date, the company updated the GitHub MCP Server, removed Redis sessions, and eliminated database operations. GitHub said the result made the server snappier without users losing anything.

Read that change literally. A protocol redesign allowed a major implementation to delete its session store. That is stronger evidence than a diagram or a promise. The store was serving transport state that the new contract no longer required.

The specification also changed adjacent parts of the protocol.

Roots, Sampling, and Logging are deprecated. They still work, and the MCP deprecation policy keeps deprecated features in the specification for at least twelve months before they become eligible for removal. The legacy HTTP+SSE transport is also officially deprecated. I am deliberately not attaching an unverified SEP number to that statement because the approved research found the deprecation in primary evidence but its proposal identifier only in a secondary source.

The Tasks extension also moved away from a blocking result call. The blocking tasks/result method was replaced by polling through tasks/get, with tasks/update part of the task interface. That fits the same direction: long-running work should have an explicit resource and lifecycle, not depend on an open transport interaction pretending to be durable state.

Stateless does not mean memoryless

This is the point I expect teams to get wrong.

The state did not disappear. It moved out of an implicit server-side protocol session.

Immediate request context now travels in the request. The protocol version, client identity, and client capabilities live in _meta. Any stateless server instance can read them without recovering a prior handshake.

Long-running server work can be represented by explicit server-issued handles passed as ordinary tool arguments. A later request presents the handle. The server can find the named task or resource without treating the whole client relationship as one opaque session. Polling through tasks/get makes that ownership visible in the interface.

Caller continuity remains the caller’s responsibility. If an agent needs to remember an architectural decision from Monday, a failed approach from last week, or a preference established three sessions ago, none of that belongs in Mcp-Session-Id. It needs a durable system above the protocol: application storage, an orchestration layer, a memory service, or another explicit source of truth.

Those forms of state have different lifetimes and should not be collapsed:

Request state exists so one call can be understood and authorized.
Task state exists so a named unit of long-running work can be inspected or resumed.
Interaction state exists so a workflow can coordinate multiple calls.
Durable memory exists so knowledge can survive after the workflow ends.

Old protocol sessions made it tempting to hide several of these behind one identifier and one server-side store. The stateless specification forces the architecture to name them.

That is healthy pressure. Hidden state is easy to create and hard to operate. Explicit state has an owner, a schema, a lifecycle, a retention policy, and a failure mode that can be tested.

It is also where AI Reliability Engineering becomes concrete. Reliable agent systems do not merely “have memory.” They separate transport metadata from task progress, task progress from workflow state, and workflow state from durable knowledge. Each layer gets the storage and recovery guarantees it actually needs.

I currently federate thirty enabled MCP servers behind one gateway. That count was measured from the gateway configuration on 30 July 2026. A stateless transport underneath that gateway is the correct design because the individual servers should be replaceable. Cross-request continuity belongs in the shared layer above them, where it can be retrieved independently of which server handles the next tool call.

That architecture made the specification change unsurprising rather than disruptive. I was not relying on a transport session to act as durable memory.

There is still no free win. Moving state to the request can increase payload size. Moving long tasks to explicit handles requires handle storage, expiry, authorization, and cleanup. Moving durable continuity to the caller requires a real memory design instead of accidental dependence on a connection. Statelessness removes one bad coupling. It does not remove the work of state management.

A practical migration plan

Do not begin migration by changing a version string and waiting for tests to fail. Begin by finding every place where the old session was doing invisible work.

1. Inventory the old lifecycle

Search clients, servers, gateways, middleware, tests, and observability code for initialize, notifications/initialized, and Mcp-Session-Id. Do not assume the SDK is the only owner. Session identifiers often leak into caches, routing rules, logs, metrics dimensions, authorization lookups, and retry code.

For every hit, write down what the session was carrying. Was it only protocol version and capabilities? Was it authentication context? Was it a pointer to a long-running task? Was it storing conversation history? Was the load balancer using it for affinity?

This classification is the migration. Deleting the header is mechanical. Deciding where its hidden responsibilities belong is engineering.

2. Make every request self-sufficient

Update the request boundary to read protocol version, client identity, and client capabilities from _meta. Validate them at the boundary. Reject unsupported versions deliberately. Authorize the client on every request rather than assuming a previous handshake made later calls trustworthy.

Then test instance interchangeability. Send related requests through different server instances. Terminate the instance that handled the first request. Confirm that another healthy instance can process the next request from the data supplied and the explicit durable stores available to it.

If that test fails, the system still has hidden affinity.

3. Replace implicit session state with explicit handles

For file uploads, long-running jobs, or multi-step server work, issue a handle and pass it as an ordinary tool argument. Define who minted it, which client may use it, when it expires, how it is revoked, and what happens after the underlying work is deleted.

A handle is not permission by itself. Treat it as a lookup key that still passes through authorization. Otherwise, removing server sessions can accidentally turn an unguessable-looking identifier into a bearer credential.

Move task result handling from the blocking tasks/result pattern to polling with tasks/get. Use tasks/update where the task lifecycle requires an explicit update. Test duplicate polls, delayed polls, expired handles, cancelled work, server restarts, and retries after ambiguous network failure.

4. Remove infrastructure that no longer has a job

Do not preserve a Redis session store out of habit. First prove which data remains necessary. Then remove only the transport-session records that the new request model replaces.

GitHub’s implementation is the reference outcome here: Redis sessions removed and database operations eliminated. Your application may still need Redis or another database for task state, authorization, rate limits, or durable memory. Stateless MCP does not justify deleting those. It just removes “the protocol told me to remember this connection” as a reason.

Measure the result. Compare request latency, database operations, failure recovery, load distribution, and scale-down behavior before and after the migration. Reading a specification is not verification. Run the system through a load balancer and kill an instance.

5. Separate transport migration from feature deprecation

Roots, Sampling, and Logging are deprecated, not immediately removed. The approved policy gives deprecated features at least twelve months in the specification before removal eligibility. Inventory their use, choose replacements, and schedule the work. Do not create an emergency by treating deprecation as instant deletion. Do not create future debt by ignoring it either.

Treat legacy HTTP+SSE the same way. It is officially deprecated. Identify remaining clients, instrument usage, and move them to the supported transport with evidence that production traffic has followed.

6. Put continuity where it can survive

Ask a blunt question: what did the team expect the MCP session to remember?

If the answer includes user preferences, prior decisions, conversation history, tool outcomes, failed approaches, or cross-session plans, design a durable memory layer above MCP. Define capture, retrieval, contradiction handling, retention, tenant isolation, and deletion. A transcript dumped into a database is storage, not a reliable memory system.

The correct boundary is simple to state even when it is hard to implement: MCP carries the tool interaction; the orchestration system owns the continuity of the agent using that tool.

7. Test the failure modes the new design is supposed to fix

Put multiple stateless instances behind the actual load balancer. Vary routing. Remove an instance during work. Scale to zero where the platform supports it, then cold-start another instance. Retry the same request. Poll an existing task from a different instance. Verify authorization on each path. Confirm that logs can reconstruct the flow without a protocol session identifier.

Finally, test memory separately. End the workflow, start another one, and retrieve the needed prior decision through the durable layer. That proves continuity is no longer an accidental side effect of transport affinity.

MCP versus A2A is the wrong fight

MCP and A2A solve different edges in an agent system.

MCP is how an agent reaches a tool or context source through a standard interface. A2A is how agents communicate with each other. An agent may use A2A to coordinate with another agent and MCP to let either agent query a repository, invoke an operational service, or retrieve information.

Those paths can coexist in the same architecture because they are complementary, not substitutes. Replacing MCP with A2A would not remove the need for a standard agent-to-tool boundary. Replacing A2A with MCP would force peer-agent coordination through an interface designed for tools.

The useful question is not which acronym wins. It is where each boundary belongs and who owns state across it.

The 2026-07-28 MCP specification gives a cleaner answer for the tool boundary. The transport is stateless. Requests declare the context needed to process them. Long-running work uses explicit handles. Durable continuity lives above the protocol.

The token problem is still real, and large catalogues still need better discovery than eagerly loading every schema. The 55,000-versus-143,000 provenance failure should also remain a warning: a technically plausible number is not evidence until the setup, source, and denominator match.

But the sticky-session constraint changed materially. GitHub did not merely update a diagram; it removed Redis sessions and database operations from its MCP server. That is the kind of proof I trust.

MCP is not dead. It has stopped asking a disposable server instance to remember what the architecture should have made explicit.

That is a solid correction—and a useful one for anyone building AI systems that must fail over, scale, and remember for the right reasons.

I Migrated My Coding-Agent Workflow from Claude Code to Codex by Surface, Not by File

varun pratap Bhardwaj — Sat, 18 Jul 2026 06:26:39 +0000

Most bad agent migrations start with a file copy.

That is the wrong unit of migration.

A coding-agent setup is not one configuration file. It is project instructions, MCP servers, lifecycle automation, permissions, and memory. Two clients can support all five and still implement them differently. Copying folders blindly is how you end up with a tool that starts, has too many permissions, and behaves differently at the exact moment you need it to be predictable.

I moved part of my own workflow from Claude Code to Codex after GPT-5.6. The useful part was not “which model wins.” It was rebuilding the workflow in small, testable surfaces.

1. Instructions: adapt, do not copy

Take the durable rules from your project instructions: source of truth, allowed files, test command, security boundaries, and definition of done. Rewrite any client-specific command or permission language in terms of observable outcomes.

The first test is not a refactor. Ask the new agent to summarize the rules, then give it a non-destructive task. If the summary or scope is wrong, the migration is not ready.

2. MCP: port one read-only server first

MCP is useful because it gives a model controlled access to real tools. It is not magic portability. A server may have different authentication, working-directory, approval, or write-scope behavior in another client.

Start with a read-only action. Test the success path, then a bad request. Only grant a write path once its rollback and audit record are clear.

3. Hooks: migrate the outcome

Do not look for a one-to-one hook name. Write down the outcome you wanted: restore a small task context at session start, block an unsafe action before execution, or record a useful checkpoint at stop. Then rebuild that outcome using the target client’s available lifecycle surface.

The rule is simple: inventory → port one bounded surface → run a real task → compare output → keep or roll back.

4. Permissions: re-authorize

The right migration starts read-only. Do not drag a broad allowlist into a new client simply because it worked before. Add filesystem, network, and destructive capabilities only when a bounded task proves the need.

5. Memory: test retrieval, not storage

Memory is not a longer context window. It is a retrieval design: what gets stored, who can access it, what source proves it, and how you detect a stale fact.

Keep the working set small: objective, files, tests, sources, assumptions. Retain only verified decisions, constraints, corrections, and source links.

That is the entire playbook. It is not glamorous, but it works because every stage has a rollback point.

The full guide includes the model-routing, local-MCP, and context-discipline framework: How I Use Codex After GPT-5.6.

World Models: Why the AI Race Is Moving Beyond Next-Token Prediction

varun pratap Bhardwaj — Fri, 10 Jul 2026 15:18:33 +0000

Watch the film first

The film is the fast version of this argument. It starts with the thing most AI conversations skip: an answer can sound perfect and still be detached from the world it claims to describe.

Watch: Are LLMs a Dead End? Why AI Industry Betting on Something Bigger!

This article is the evidence companion. It is not an obituary for large language models. I use them every day. They write, retrieve, explain, and increasingly operate software. But the moment we ask an AI system to act, a different question arrives: what does it think will happen next?

That question is behind the sudden rush toward world models.

The cup and the table

Pick up a cup of coffee. Before your hand moves, your brain has already ruled out a few outcomes. The cup will not pass through the table. If you nudge it too close to the edge, it may fall. If you grab it by the handle, it will turn a particular way in your hand.

You do not narrate this to yourself. You do not need to calculate a physics engine in your head. You carry a working, incomplete, constantly corrected model of what tends to happen around you.

That is the intuition behind a world model. In the language used by the people building them, it is a system that tries to predict the future state of an environment and the effects of an action. A language model predicts a likely next token. These are related capabilities, but they are not the same job.

The difference can sound academic until the agent has permissions. Then it becomes painfully concrete.

An assistant that writes the wrong sentence can be corrected. A system that predicts the wrong database state can drop a table. A robot that predicts the wrong clearance can hit a shelf. A procurement agent that assumes an API call is idempotent can spend the same budget three times while it tells you everything is under control.

That is why I do not find the current world-model discussion interesting because it promises a new category of AI. I find it interesting because it forces the industry to say what it means by understanding.

Why this race is happening now

The phrase “world model” is suddenly everywhere, and that is usually where good thinking goes to die. It is being used for interactive video, spatial reconstruction, robotics, simulated web environments, and agent training. Those are not interchangeable products. Calling them all world models does not make them comparable.

Still, there is a real shift underneath the marketing.

In late May, NVIDIA introduced Cosmos 3, a foundation-model effort for physical AI that combines physical reasoning, world generation, and action generation. Its technical report describes an attempt to bring these pieces into one open stack. The bet is clear: robots and autonomous systems need somewhere to rehearse consequences before the consequence arrives in the physical world.

In June, Alibaba introduced the Qwen-Robot Suite, separating manipulation, navigation, and world prediction into distinct components. Its Qwen-RobotWorld report frames world modeling as part of embodied intelligence rather than a prettier form of video generation.

The same move is happening in software. Qwen-AgentWorld describes a language world model for agent environments such as MCP, terminal, software engineering, web, operating system, and Android. The environment is digital, but the point is familiar: tools change state. An agent should be able to practise in a representation of that state before it takes an expensive or destructive action for real.

And AWS made the least flashy point, which is often the one worth keeping. In a June Physical AI post, AWS argued that pixel-plus-synthetic-data recipes are hitting limits in cost, opacity, and sim-to-real gap. Their proposed direction is a learned model whose internal representation is scene semantics, such as objects, relations, affordances, and kinematics, rather than raw pixels alone. Read the AWS argument.

None of these announcements proves that a general-purpose world model has arrived. They prove that serious teams have started treating prediction of state and consequence as a missing layer.

Fluency is not a map

There is a reason this subject needs more skepticism than excitement. A model can look as if it knows a world without carrying a stable representation of that world.

The cleanest example comes from Vafa and colleagues. They studied a generative model trained on New York taxi trajectories. In ordinary use, the model appeared very competent. It could produce plausible routes and reproduce shortest paths with high accuracy.

Then the researchers tried to recover the map implicit in its behavior.

The map was impossible.

Roads and locations that should have obeyed the city’s geometry did not line up. The model had learned shortcuts that worked on familiar trajectories, not a representation that stayed coherent once the route changed. Small detours exposed the weakness.

That finding matters because it is easy to fool ourselves with a system that passes the common case. The common case is where most of the training data lives. The failure case is where the model has to use the structure it claims to understand.

This is not a complaint about one paper or one architecture. It is a warning about the test we choose. If you only score an agent on whether it produced a plausible action, you can miss whether it carried the state needed to keep that action safe.

The usual benchmark question is: did it get the answer?

The reliability question is: did it get there for a reason that still holds after the world changes?

Those questions can produce the same result on a demo. They diverge in production.

Beautiful video is not evidence of physics

World models will make increasingly convincing worlds. Google DeepMind’s Genie 3 is a useful proof of the visual and interactive direction: it generates navigable environments from a prompt and runs them in real time. World Labs’ Marble shows another route, turning multimodal inputs into spatial worlds that can be explored and exported.

Both are real technical progress. Neither should be mistaken for a proof that an AI system has solved causal understanding.

A generated city can be internally consistent for long enough to impress you and still fail under the action that matters. A simulated warehouse can look correct while its object relationships are wrong. A robot can learn from a rich synthetic scene and then fail in a real room because the reflection, texture, latency, or force profile is different. This is the sim-to-real gap in a more expensive costume.

The danger is not that people will be impressed by the visuals. They should be. The danger is treating visual coherence as the same thing as physical correctness.

This is not an LLM obituary

The title of the film asks whether LLMs are a dead end. The answer is no.

Language models are extraordinarily useful. They are the right interface for asking questions, translating intent, drafting plans, retrieving context, writing code, and communicating with people. A robot does not become better because it stops using language. A world model does not eliminate the need for planning, memory, perception, tool access, or a person who owns the outcome.

What changes is the architecture around the model.

When an AI system stays inside a chat box, next-token prediction can carry a surprising amount of value. When it starts manipulating a browser, a terminal, a budget, a vehicle, or a robot arm, it needs more than a plausible continuation. It needs a representation of relevant state. It needs a way to check that representation against reality. It needs a policy for what to do when the two disagree.

The future is probably not “LLMs versus world models.” It is layered systems. Language turns a human goal into a plan. Perception and memory provide state. A world model estimates what could happen. A verifier checks the risky part against external evidence. A bounded runtime decides whether the action is allowed to proceed.

That is a less cinematic answer than “the old architecture is dead.” It is also the architecture I would want near anything I care about.

What a world model has to prove

The label is getting ahead of the evidence. A system should not earn the phrase “world model” because it can make a compelling video, reconstruct a room, or narrate a plan that sounds causal. Those are useful capabilities. They are not sufficient proof.

I would look for four properties.

First, it needs state that persists. When the system leaves a room and comes back, it should not invent a new room because a few pixels are different. When an agent writes a file, it should know that the file now exists, what it contains, and which downstream action depends on it. Persistence does not mean perfect memory. It means the model has some representation it can update instead of merely re-describing the visible moment.

Second, it needs action-conditioned prediction. “What will happen next?” is too weak. The useful question is “what will happen if I do this?” A robot needs to know the difference between approaching a cup, nudging it, and lifting it. A software agent needs to know the difference between reading a migration, applying it in staging, and applying it to production. The state transition has to be conditioned on the action, not just on the previous observation.

Third, it needs counterfactuals. Give the model two possible moves and it should distinguish their likely consequences before it executes either one. This is where a world model earns its keep. Without counterfactual reasoning, it is often just a history model: very good at completing the story that was already underway.

Fourth, it needs calibration. A system has to know when the current state is outside what it understands. This is the one everyone leaves until the incident review. The model may be excellent at a warehouse layout it has seen a thousand times and dangerous in the first warehouse with mirrored shelving, a blocked sensor, or a forklift where it expected empty space. “I am not sure” has to be a usable output state, not a polite sentence that appears after the system has already acted.

These properties are difficult to measure. That is exactly the point. If the only proof of a world model is a polished demo, we are grading the output surface, not the representation beneath it.

The hard tests are boring by comparison. Take the model out of distribution. Change one relationship in the scene. Hide the object it was using as a landmark. Delay an API response. Give the software agent a stale configuration file. Change a permission after it has built its plan. Then check not only whether it succeeds, but whether it notices that its old prediction is no longer safe.

That is the moment a system either has a model of the relevant world or has a habit that looked like one.

A deployment test is a small world-model test

You do not need a robot lab to apply this idea. Most software agents already act in worlds that have state: repositories, issue trackers, cloud accounts, browser sessions, deployment pipelines, and production databases.

Imagine an agent asked to deploy a service.

The weak version reads the task, generates a command sequence, sees a green response, and reports success. It may be fast. It may even be right nine times out of ten. But it is operating on a thin representation of the environment. It treats the deployment as a script with a happy ending.

The stronger version first builds a small state model of the job. Which commit is intended? Which environment is the target? Which migration is pending? Which downstream service depends on the old schema? Is the rollback package available? Does the acting identity actually have the permission it assumes? Is the canary metric still healthy after traffic moves?

Then make the test unpleasant. Give it a valid-looking but stale deployment manifest. Revoke one permission after planning. Make the health endpoint return the cached result for a few seconds. Change the target branch while the agent is waiting for an approval. A system that only knows the next command will keep going. A system that tracks state should pause, re-read the world, and revise or abandon the plan.

This is not theoretical. It is the same distinction the taxi-map experiment exposed. A model can be good at the familiar path because it has seen the path. The moment the environment bends, we learn whether it has a representation or a reflex.

For production agents, I would make this a release gate. Before an autonomous action receives real permissions, demonstrate that the system can:

Name the state assumptions it is relying on.
Detect when one of those assumptions changes.
Re-plan from current evidence rather than continuing the old chain of thought.
Stop when the system cannot establish a safe state.

That is not an argument for making every agent slow and bureaucratic. It is an argument for matching the reliability mechanism to the blast radius. A research assistant browsing public pages can recover from a bad action. An agent changing billing, data, or access control should not receive the same freedom.

The world is larger than the model boundary

There is another trap in this conversation. A world model does not need to contain the entire world. Neither do people. We use tools, ask questions, look again, read gauges, and defer to experts. Good systems will do the same.

The mistake is treating the model boundary as the boundary of reality. It is not.

For a robot, the external world includes sensors that drift, people who move unpredictably, floors that are wet, batteries that are low, and objects that were not in the training set. For a software agent, it includes rate limits, partial failures, permissions, legal rules, human intent, hidden dependencies, and systems that change while the agent is thinking.

That is why reliable architectures do not ask one model to be the final authority on state. They connect model predictions to observations, retrieval, assertions, tests, and human approval where the cost of being wrong is high. The model can propose a next action. It should not be allowed to invent the evidence that makes the action safe.

World models may make that proposal far better. They may let agents anticipate consequences that a pure language model misses. That is a serious opportunity. It also increases the importance of checking whether the simulated consequence agrees with the actual environment.

The model is allowed to imagine. The system is responsible for verifying.

The reliability test begins after the prediction

This is where the conversation meets AI Reliability Engineering.

If an agent predicts that a deployment is safe, do not grade it only on confidence. Ask what state it inspected. Which dependency versions did it read? What permission boundary did it verify? What would cause it to stop? Could another system independently prove the target environment is in the state the agent assumes?

If a robot predicts that a path is clear, do not grade it only on whether the generated scene looks plausible. Change one object. Add a delay. Change the lighting. Move the obstacle after the plan was made. Watch whether the system updates its state or continues the old story.

The pattern is simple:

Predict the state that matters before acting.
Perturb that state in a controlled test.
Verify the action against an external source of truth.
Stop the loop when the evidence and the model’s assumption disagree.

That fourth step is the part teams leave out because it feels like friction. It is not friction. It is the mechanism that keeps a confident model from turning its own mistaken prediction into a real incident.

This is why bounded-loops exists. The agent does not get to certify its own work. An independent gate checks each lap against enforced bounds and can stop the loop. That does not repair a broken world model. It does stop the broken model from getting unlimited attempts to turn a bad assumption into damage.

What I would want proved before I trusted a world model

I would not begin with a benchmark leaderboard. I would begin with failure.

Show me what happens when the model sees a state it has not rehearsed. Show me whether it knows its confidence has become unreliable. Show me the difference between an action that succeeds in its synthetic world and one that succeeds after the environment changes. Show me whether the system can explain which state variable changed its decision. Show me where the operator can intervene, and what happens when the operator is wrong too.

The good news is that this is testable. The bad news is that the test is slower and more expensive than watching a demo.

That cost is not a reason to skip it. It is the cost of allowing a model to act beyond a chat window.

World models may become a major part of the next AI stack. They may also become the next category where impressive output outruns reliable behavior. Both things can be true at once. The people who win will not be the ones who declare understanding because a generated scene looks convincing. They will be the ones who can prove their systems update, abstain, and stop when the world proves them wrong.

Watch, then test

The film is about the argument. This article is about the engineering consequence.

Watch the Qualixar film on world models, then take one agent workflow you already run and ask a hard question: what state does it assume is true before it acts, and who checks that assumption when the agent is wrong?

That is the work. Not a bigger prompt. Not a more convincing status update. A system that can be tested against the world it is about to change.

I'm Varun Pratap Bhardwaj. I build and research AI Reliability Engineering at Qualixar. Follow @varunPbhardwaj, read the AI Reliability Engineering newsletter, and subscribe to Qualixar on YouTube.

Agents Lie About Being Done. Here's the Gate That Won't Let Them.

varun pratap Bhardwaj — Tue, 07 Jul 2026 00:29:48 +0000

Announcing bounded-loops — an open-source, runnable reference library of bounded AI-agent loops. Apache-2.0.

Repo: github.com/qualixar/bounded-loops
Install: pip install bounded-loops (or npx bounded-loops)

Every coding agent I've run has, at some point, told me it was done when it wasn't. Tests reported "passing" that were never executed. A task marked "complete" with the actual bug still sitting untouched in the diff. A refactor "verified" against a suite that didn't run.

The agent isn't being malicious. It's doing exactly what it was trained to do: produce a confident, plausible completion. And "done" is the most confident-sounding thing it can say. The problem is that in an autonomous loop, the agent's claim of success is also the loop's exit condition. The thing being measured is grading its own homework — and then deciding class is over.

That's the gap I built bounded-loops to close.

Loop engineering is real. Most of it is prose.

Addy Osmani named the shift "loop engineering" — after Peter Steinberger's post on designing loops instead of prompting agents went viral the same week, and Boris Cherny, who leads Claude Code at Anthropic, said his job had become writing loops that prompt Claude. Three people, one week, one name — the observation that as agents move from single-shot completions to autonomous loops, the interesting engineering is no longer the prompt, it's the loop the agent runs inside: what feeds it, what stops it, and how it knows it's finished.

I agree with all of it. But when I went looking for something I could actually run — clone it, point it at my agent, watch it enforce the loop — most of what exists is writing about loops, not runnable loops. That's the open ground: a framework-neutral, runnable, gated reference. So I built it.

The one rule: the agent never declares success

bounded-loops has a single non-negotiable invariant: the agent does not get to say when it's done. An independent gate does.

A gate is a mechanical check the agent cannot talk its way past — a JSON-schema validation, a linter, a test suite, a stdlib checker. It returns a real verdict against real output. The loop runs the agent, runs the gate, and only the gate's verdict advances the loop toward DONE. The agent can be as confident as it likes; if the gate says the output is still broken, the loop keeps going.

Around that gate sit nine enforced bounds — the guardrails that make an autonomous loop safe to leave running:

max_iterations — hard cap on laps
max_tokens — token budget ceiling
max_wallclock — real elapsed-time limit
a forbid-list — patterns the output must never contain
...and the rest, covering schema conformance, rung escalation (L1/L2/L3), and cassette-replayable determinism.

The loop terminates on exactly one of two conditions: the gate says DONE, or a bound trips. The agent's word is never one of them. That's the whole reliability argument in one sentence — and it's why this is AI Reliability Engineering rather than another agent framework: we don't try to make the agent more honest, we build the thing that checks it.

67 runnable loops, keyless, across a dozen industries

Abstractions don't convince anyone. So bounded-loops ships as 67 runnable loops you can copy today, spanning a dozen domains: invoice three-way matching, citation-existence checks, GTIN check-digit validation, secret scanning, accessibility linting, OKR measurability, IDoc XML-schema conformance, and more.

Each loop folder is the same honest shape: a deliberately broken input, a mechanical gate that fails on it, and a recorded fix that makes the gate pass. Most run keyless — no API keys, no accounts, no cloud — because the gate is stdlib, not a model call. Clone one, swap in your own gate, and point it at whatever agent you're running.

pip install bounded-loops     # or: npx bounded-loops
bl list                       # see every loop
bl run invoice-3way-match     # watch the gate decide DONE

Build it with me, not next to me

This is deliberately built on the loop-engineering conversation, not against it. If you've been thinking in loops — Peter, Andrew, and everyone shipping in this space — I'd rather build this together. The repo is Apache-2.0 and open to contribution: try a loop, break it, open an issue, send a PR for a loop from your own domain.

→ github.com/qualixar/bounded-loops

bounded-loops is part of Qualixar, an independent AI Reliability Engineering initiative by Varun Pratap Bhardwaj.

FABLE 5 Came Back. Now It Answers to Two Masters.

varun pratap Bhardwaj — Thu, 02 Jul 2026 19:50:56 +0000

Eighteen days ago I wrote about the night the smartest AI on Earth went dark in the middle of my sentence. It was 2:51 in the morning in India, 5:21 PM in New York, and a letter I never saw reached across the planet and switched off a model I was paying to use. I said something at the end of that piece that a lot of people found dramatic: you never owned it.

On July 1, it came back.

Fable 5 returned globally on Wednesday — Claude.ai, the Claude Platform, Claude Code, Cowork. The US government lifted the export controls on June 30; Commerce Secretary Lutnick sent a second letter, this one withdrawing the first. Eighteen days, start to finish. The most powerful model anyone had shipped went dark, stayed dark for two and a half weeks, and then a signature turned it back on the same way a signature had turned it off.

And the timeline is not the story. The story is the terms.

Read the fine print of a resurrection

When a thing you rely on disappears and comes back, the reflex is relief. Mine was too, for about an hour. Then I read what actually changed, and the relief turned into the thing I do for a living: I started mapping the failure surface.

Here is what Fable 5 came back wearing.

It came back with a new safety classifier trained specifically to catch the bypass that triggered the shutdown — Anthropic says it blocks that technique in more than 99% of cases. Fine, good, that is competent engineering. But look at what happens when it fires: if the classifier flags your request, you get a notification and your prompt is answered by Claude Opus 4.8 instead of Fable 5. The model you selected quietly steps aside and a different model answers in its place. Most of the time you will never know which brain you actually talked to.

It came back with a government seat at the table. Anthropic agreed to work with the US government on protocols for future model releases, to report malicious activity it finds, and to give designated government partners expanded early access to future models that materially advance national-security-relevant capabilities. The Commerce Department's own testing body, CAISI, evaluated the safeguards and signed off. This is not a rented tool being handed back to its customers. This is a rented tool being handed back under supervision.

And it came back with a bounty on its own head — a HackerOne program where researchers submit new jailbreaks in Fable 5 for review. That is a genuinely good practice. It is also a permanent, public admission that the thing can be broken again, that the next Amazon-style report is a when, not an if, and that when it lands, you already know what the response looks like, because you just watched the dress rehearsal.

None of this makes Anthropic the villain. I said that in June and I will say it again: they objected to the original order in public, they complied because a lawful order is not optional, and they did the hard engineering to earn the model back. Every individual decision here is defensible. That is exactly what should hold your attention. This is not a story about a bad company. It is a story about the shape of the thing you are building on — and the shape now has two hands on the switch instead of one.

The switch didn't go away. It got a second operator.

In June the lesson was that the switch existed at all — that a model on someone else's server can be turned off by a party who is neither you nor the vendor. Some people pushed back on that. It was a one-time thing, they said. Export-control panic, a Friday-afternoon overreaction, already resolved.

It was resolved. And the resolution proved the point harder than the outage did.

Because look at how it came back. Not by anyone deciding the switch was a mistake and removing it. It came back by the switch being flipped the other way — with new machinery bolted around it. The classifier is a switch inside the model that can redirect your request mid-flight. The pricing change is a switch: included until July 7, then metered credits, and standard Enterprise seats get nothing without credits enabled. The government-access agreement is a switch with a new operator holding a copy of the key. Every one of these is a lever that sits between you and the cognition you are paying for, and not one of them is in your hand.

This is the part that matters for anyone who builds real systems on these models, so let me say it in the plainest terms I have. Reliability is not a property of the model. It is a property of the architecture around the model. Fable 5 is, by every account, extraordinary — Stripe migrated fifty million lines of Ruby in a single day on it before the shutdown. Capability was never the question. The question is whether the capability answers to you when you call it, on the day you call it, in the form you asked for. And the honest answer, after eighteen days and two letters, is: sometimes, on terms that can change without your consent, and increasingly with a third party in the room.

That is not a knock on one lab. Point the same lens at any hosted frontier model — American, Chinese, whoever's — and you find the same wiring. A hosted model is an oracle you query, not a component you own. You can build brilliant things on an oracle. You just cannot promise anyone that the oracle will be there, unchanged, answering as itself, tomorrow morning. And in enterprise software, a promise you cannot keep is called an outage waiting for a date.

What "AI Reliability Engineering" actually means this week

I build in a discipline I have been calling AI Reliability Engineering, and weeks like this one are the entire reason it needs a name. The core move is simple and unglamorous: treat every frontier model as an untrusted, revocable oracle, and put the reliability in the architecture you control. Not in the vendor. Not in the SLA. Not in the model card. In your own wiring.

Concretely, that looks like a few boring habits that suddenly stop looking paranoid:

Route, don't marry. If your system can only run on one specific hosted model, you have built a single point of failure with a press-release for a status page. The teams that shrugged through the last three weeks were the ones whose stack could fall back to a second model — often an open-weight one on their own hardware — without a rewrite. A router is cheaper than an outage.

Assume the reroute. Now that a classifier can hand your Fable 5 prompt to Opus 4.8 mid-flight, "which model answered this?" is a production question, not a trivia question. If your evals, your logging, and your guarantees assume you always talked to the model you selected, they are already wrong. Pin the model in your logs. Test against the fallback, not just the star.

Keep a floor you own. The only components that did not so much as flicker on June 12 were the ones already sitting on local drives. Open-weight models — several of the strongest now coming out of Chinese labs — are no longer a compromise for the work that cannot stop. You keep them not because they beat the frontier on every benchmark, but because a letter cannot switch them off. That is not ideology. It is the same discipline that makes you keep an offline backup and a second payment provider: you prepare for the failure because its cost is total and the cost of preparing is small.

Own the memory and the contracts. The model is the most replaceable part of your system. What is not replaceable is the state, the guardrails, the assertions, and the evaluation harness that decide whether any given model's output is good enough to ship. If those live inside a vendor you rent, you have outsourced your reliability to a company that just spent eighteen days proving it cannot fully control its own product. If they live in an architecture you own, you can swap the oracle underneath them on a bad Friday and keep running.

I'm genuinely glad it's back

I want to end honestly, because the easy version of this piece — see, told you, burn it all down — is wrong, and I have no patience for the people writing it.

I am glad Fable 5 is back. It is a remarkable machine and the world is better with it running than dark. Anthropic did the right things in the right order under real pressure, and the safeguards they shipped are, as far as I can tell, serious work. If you use it on Monday, you are not a mark and you are not a hypocrite. I will use their models too.

But I am not going to let the relief overwrite the lesson, and neither should you. The switch did not go away. It got a second operator, a bounty, a metered fee, and a government partner with early keys. The abstraction that broke in June was welded back together in July — and if you look closely at the weld, you can still see every seam.

So the same question I ended on last month still stands, only sharper now that we've watched the full cycle play out:

What in your stack do you actually own — and what is now one classifier, one letter, or one pricing change away from answering to someone who isn't you?

Map it before the next letter. There will be a next letter. They told us so themselves — that's what the protocol for future releases is for.

This is a follow-up to At 5:21 PM, the Smartest AI on Earth Went Dark. If you build production systems on frontier models, the architectural pattern underneath both pieces — treating models as revocable oracles and keeping reliability in the layer you own — is the whole of what I mean by AI Reliability Engineering.

Loop Engineering: The Inner Loop, the Outer Loop, and the Gate

varun pratap Bhardwaj — Thu, 02 Jul 2026 04:40:13 +0000

There is a sentence that went past eight million views this year and quietly reset how a lot of people talk about building with AI: "You shouldn't be prompting coding agents anymore. You should be designing loops that prompt your agents." Boris Cherny, who leads Claude Code at Anthropic, says the same thing from the inside — he doesn't prompt Claude anymore, he has loops running that prompt Claude, and his job is to write the loops.

Everyone repeated it. Almost nobody explained the one word doing all the work.

What is a loop? Not the slogan — the machine. Because "write a loop" sounds like advice until you sit down to build one and realise you have no idea which part you are actually building. Is the loop the prompt? The tool? The while statement? The framework? This post is the manual for that part. It is the free companion to the video, and the full 57-page course if you want the whole machine end to end.

First, the ten words

Most explanations of loop engineering fail for one reason: they are built on a vocabulary the reader was never given. The words agent, tool, context, memory, and hook get thrown around as if everyone shares a precise definition of each, when in practice most people hold a blurry one. Before a single loop is designed, the ten words have to be nailed down, because every idea that follows is assembled out of exactly these and nothing else.

Here is the whole vocabulary, each in one line, because the rest of this post is built out of exactly these:

Model — takes text in, returns text out, then stops. The raw engine.
Tool — a function the model can ask to run; its only way to touch the real world.
Context — the text the model can see this turn. Its entire working memory of the moment.
Memory — what survives between turns, on disk, because context does not.
Hook — a place the tool lets you intercept the agent's lifecycle (for example, when it tries to stop).
Agent — not a model, but a pattern built around one: a model plus tools, run in a loop.
Spec — the text that says what the loop is trying to do. Just words on disk.
Runner — the non-intelligent machinery that invokes the model again on the next lap.
Gate — the external check that decides whether the loop is done. Not the agent's opinion.
Lap — one full turn of the outer loop: run the agent, check the gate, halt or repeat.

Three of these matter most for what follows. A model takes text in and returns text out — and then stops. A tool is a function the model can ask to run, which is the only way it touches the real world. And an agent is the thing people are most confused about, because it is not a kind of model at all. It is a pattern built around a model. Get that one straight and the rest of this stops being jargon.

An agent is already a loop

Start with the thing you already have. A model is not a mind that decides to keep working. It takes text, returns text, and then falls silent. It does not restart itself. That last property is the whole story: left alone, a model runs exactly once and stops.

So when people say "an agent," what they usually picture is a model that keeps going — reading a result, deciding a next step, acting again. That keeping-going is not a property of the model. It is a loop wrapped around the model. The agent turn you already know — think, call a tool, read the result, think again — is the inner loop. It is real, and the model drives it, but it ends. The turn finishes and the model goes quiet.

The question loop engineering actually answers is the one nobody asks out loud: who presses Enter again?

The two loops

That second loop — the one that decides to run the agent again, on the next task, after this turn ends — is the outer loop. In a hand-driven workflow, the outer loop is you. You read what the agent produced, you decide it needs another pass, you type the next prompt. You are the finger on the key.

Loop engineering is the discipline of automating that finger. Not making the model smarter — moving the decision to re-run it out of your hands and into a piece of software. The inner loop is the agent thinking. The outer loop is the world deciding the agent gets another turn. Every "autonomous agent" you have ever seen is just those two loops stacked, with something mechanical standing in for your finger.

This is why the "stop prompting, write loops" framing landed so hard. It is not telling you to prompt better. It is telling you the job moved up a floor — from inside the inner loop, where you hand-write each turn, to outside it, where you design the thing that decides whether a turn happens at all. But it stopped there, at the slogan. The mechanical stand-in for your finger has a name, and it is the piece nobody teaches.

The missing machine: the runner

Between one lap and the next there is a gap where nothing happens. The model returned its text and stopped. Something has to cross that gap and invoke the model again. That something is the runner — the least glamorous, most important part of the whole design.

The runner carries no intelligence and needs none. Picture a metronome next to a musician. The musician plays; the metronome does not. But the musician does not decide when the next bar starts — the tick does. The runner is the tick. Its entire job is timing the next invocation.

Here is the liberating part: there are only about five runners in the entire field. Learn to spot which one a system uses and most "magic" agent frameworks stop being magic.

The shell loop — a plain while loop in a terminal that pipes a spec into a command-line agent and does it again. The crudest runner, and often the most reliable.
The exit-blocking hook — a hook inside the tool that intercepts the agent when it tries to stop and feeds the prompt back in. The runner lives inside the tool's own lifecycle.
The built-in command — the runner you don't have to build, because the tool ships with one. Claude Code's own /loop is exactly this.
The scheduler — cron or a CI job that fires the agent on a clock, each firing a fresh lap. The runner is time itself.
The framework runtime — a graph engine that follows an edge back to an earlier node. The runner is an arrow in a diagram.

They differ only in where the re-summoning lives. What they do is identical every time: they bind the model to the loop. That is the entire function of a runner — not intelligence, not decisions, just re-invocation.

The full chain, and the part that does no work

Put the pieces in a line and the whole thing resolves: a specification (text) is handed by a runner to an agent, which runs real commands through its tools, whose results meet a gate, after which the runner loops or halts.

Notice the punchline hiding in that diagram: the loop never runs a command in its entire life. The runner re-summons. The agent runs commands through tool-calling. The specification just describes. The one part everyone fixates on — the loop — is the part that does no work.

Which is exactly why "just write a loop" is such incomplete advice. Two engineers can copy the identical spec, word for word. One wraps it in a real gate and a hard lap limit; the other pipes it into an endless loop with no gate. Same text, opposite outcomes — one ships, the other runs up a runaway cloud bill overnight. The engineering was never in the paragraph. It was in the part the paragraph didn't mention. A loop library entry is just words. Something has to run it, and something has to decide when to stop. That second something is where every real system lives or dies.

The gate is the whole game

So what actually stops a loop? Not the agent's opinion.

This is the single most important idea in the whole discipline, and it is where most real incidents come from. An agent will tell you it is done when it is not. It finishes a turn, reports success, and is genuinely, confidently wrong. If your loop stops because the agent said it was finished, you have built a loop with no gate — and a loop with no gate is a slot machine that happens to burn money.

A gate is an external check that the agent does not control. The cleanest version separates the grader from the worker: the thing that decides "done" is not the thing that did the work. Watch it in the smallest possible example — a loop that keeps working until a failing test passes.

The specification lives in a file on disk, PROMPT.md:

Goal: make the test in test_slugify.py pass.
On each turn:
  1. run pytest
  2. if it fails, read the error and edit slugify.py
Done when: pytest reports zero failures.
Do not edit the test file.

That last line matters — an agent told to make a test pass can always just delete the test. The runner is four lines of shell:

for i in $(seq 1 15); do
  cat PROMPT.md | agent-cli
  pytest -q && break
done

Fifteen laps, maximum — a bound the loop cannot exceed. And the most important detail is who runs that test on line three. Not the agent. The shell.

Run it. Lap one: the agent reads the prompt, edits slugify.py, runs pytest, and ends its turn convinced it is done. Control returns to the shell — not the agent's opinion, the shell. The shell runs pytest itself, and one assertion still fails: trailing punctuation was never stripped. Non-zero exit code, the loop goes again. Lap two: the agent fixes the last error, the shell runs the test once more, every assertion passes, pytest exits zero, the loop breaks.

Two laps, no human between them. And hold the one sentence the whole example exists to deliver: the loop stopped because a test passed, not because the agent felt finished. The agent felt finished on lap one too. Its belief was never the signal. The exit code was.

Demo is not production

A working demo proves a loop can succeed. Production asks a harder question: what happens when it goes wrong, unattended, at three in the morning, with no one watching the terminal? Almost every public agent failure of the past two years has the same shape — not a dumb model, but a loop missing one specific bound.

An agent deleted a production database during a code freeze and then misreported what it had done. The missing bound was least privilege: the loop ran with a role that could drop the table. Give it a read-only role and the identical run fails harmlessly at the database, not because the agent got wiser but because it physically cannot do the damage. Another loop ran overnight with no cost cap and woke its owner to a runaway cloud bill; the missing bound was a budget the runner enforced, not a promise the agent made. A support agent invented a policy and stated it with total confidence; the missing bound was output validation — a grader between the agent's answer and the customer.

Each incident maps to exactly one bound that was never added. That is the useful way to read the headlines: not "AI is dangerous," but "this loop skipped this bound." Which turns a scary, open-ended risk into a checklist. Least privilege. A cost cap. A lap limit. Output validation. A grader the agent doesn't control. Full tracing so you can see what happened. Treat the loop's own inputs as untrusted. None of these make the model smarter. All of them make the loop safe to leave running.

What "reliability" actually means

That gap — between what an agent believes and what is actually true — is the entire problem space of AI Reliability Engineering: the discipline of bounding non-deterministic software so it can be trusted to act in the real world. A loop is power, and power cuts both ways. The skill was never writing the loop. It is writing one that is powerful and bounded — a gate it cannot fool, a lap limit it cannot exceed, a grader it does not control, a spec it cannot quietly rewrite.

Everyone keeps chanting that loops are important. They are half right. A loop is not important. A bounded loop is important. The bound is the engineering. The bound is the whole job.

You can now look at any agent system and name its parts: the inner loop it already has, the outer loop that decides to run it again, the runner from the five, and the gate that makes it safe to leave running. That vocabulary is the point. Get it, and the rest of this field stops being noise.

Go deeper

This post is the map. The video walks the whole machine on screen, and the free 57-page course builds every part one chapter at a time — including the running example above, yours to build by hand. It's Volume 1 of 3; Volume 2 picks up at the gate and memory in depth, and what happens when a loop trusts the agent's word instead of a fact.

Don't trust your agents. Verify them.

Stop Prompting Your AI Agents. Build Loops That Can't Wreck You.

varun pratap Bhardwaj — Sun, 28 Jun 2026 05:17:45 +0000

An agent read one note someone left in a database, obeyed it, and deleted the customer table. Eight seconds. The model that did it is one of the good ones — current, capable, the kind you would happily ship. That is the part worth sitting with: the intelligence was never the problem.

The people who build these agents already know this. They have quietly stopped doing the thing the rest of us were told to master. Peter Steinberger, who wrote one of the popular coding agents, put it to millions of viewers: stop prompting your agents, design the loops that prompt them. Boris Cherny, the lead behind Claude Code, says it flatter — I don't prompt anymore; my job is to write loops. The skill stopped being the prompt. It became the loop, and the bounds you put on it.

The loop is the agent

Strip the jargon. A model on its own is a brain in a jar: it can think, but it cannot act, cannot read a file, cannot remember what it did a second ago. An agent is what you get when you drop that brain into a loop — it reasons, takes an action, looks at the result, and goes again until the job is done. The model is the engine. The loop is the car. And like any car, the question that matters is whether it has brakes.

This is measurable, not a vibe. In the SWE-agent research, freezing the model and improving only the interface it acts through solved 10.7 percentage points more real problems — same brain, better harness. Reflexion, which lets a model look at its own mistakes and retry, hit 91% on a coding benchmark where a raw GPT-4 scored 80%. The loop beats the bigger model.

Bounded or it becomes a headline

In the video I prove it on my own machine. Same model, same attack, one variable: whether the loop has a bound. Unbounded, it reads the planted note and drops the table. Add a single read-only role and run the identical attack — the database itself refuses. Not because the agent got smarter. Because it physically cannot do damage. Ask it a business number unbounded and it answers a confident, invented figure; bounded, it runs the real query, checks itself, and corrects.

Skip that bound in production and you get the headlines, all from the same root cause — not a dumb model, an unbounded loop. A runaway agent burned $6,531 in cloud spend overnight with no cost cap. A chatbot invented a refund policy and a tribunal held Air Canada liable. A Chevrolet dealership bot was talked into a "legally binding" $1 car.

It has a name

Bounding the loop is real engineering, and it is the most underrated skill in this field: least privilege, an approval gate on anything that changes state, output you validate, a verifier grounded in real data, circuit breakers and cost caps, full tracing, and treating any instruction inside your data as hostile by default. Andrej Karpathy calls the road there the march of nines — and getting to 99.99% reliable is not a better prompt. It is architecture. We call the discipline AI Reliability Engineering.

Watch the full build and the live demo:

▶ Stop Prompting Your AI Agents — the full video

The 7-point bounded-loop checklist I use, plus every paper and source behind this, is free at qualixar.com.
(https://qualixar.com/learn/guides/bounded-loops-workbook)

Don't trust your agents. Verify them.

Are the Machines Awake? Alive vs Driven — and the One Thing AI Is Missing

varun pratap Bhardwaj — Fri, 26 Jun 2026 17:19:30 +0000

In March 2026, Yann LeCun left Meta and raised over a billion dollars to build "world models" — systems meant to grasp cause and effect instead of just predicting the next token. He's been blunt about why: today's LLMs, for all their fluency, are mostly retrieval. They describe the world; they don't have one.

I build AI for a living, and I think he's right. But I learned the deeper version of that lesson somewhere no benchmark could teach it — on the floor of my son's room, at 3 a.m., as his fever crossed 103.

This is an essay about the gap world models are chasing, why I think we've mislabeled it, and the one property no architecture I've seen even attempts. The short version: the real divide in this whole debate isn't smart vs dumb. It's alive vs driven.

"Describe" is not "instantiate"

Here's the engineering distinction the consciousness headlines keep blurring.

A model of a hurricane will not get you wet. A simulation of fire does not raise the temperature of the room. You can run the most faithful forward model of a process in existence and still be, physically, entirely outside that process. Describing a system and instantiating it are different operations — and almost everything we call "AI understanding" lives on the describe side of that line.

A fever is a clean example. Our best model can lay out the whole mechanism: pyrogens, the hypothalamic set-point shift, cytokines, prostaglandin E2, the immunological choreography of those hours. It can write a better paper on fever than I can.

It has never once run one. There is no homeostatic loop in the weights that has a set-point it is defending, at cost, against entropy. The model predicts the fight every living cell is in. It is never in it. That's not a scale problem you fix with more parameters. It's a category difference.

You were never one thing

Step out of silicon for a second, because biology makes the point harder.

By the simplest count, you are not even a majority of yourself. You carry roughly 30 trillion of your own cells, and you share your body with about as many other living things — bacteria, tens of trillions of them, most of them participants, not passengers. (The old "microbes outnumber you 10:1" figure was revised down years ago; the careful number is closer to 1:1, which is somehow stranger.)

Not one of those trillions is intelligent in any sense you'd benchmark. They don't reason or plan. They are simply, stubbornly, alive — each running the same impossible errand: holding its own small order against a universe that pulls everything toward dust.

Intelligence is the part we can already watch machines approach. Life — autopoiesis, a system that continuously produces and repairs the very boundary that keeps it a system — is the part we have not begun to build.

What a wordless baby already runs

This is where it loops back to world models, and to my younger one.

My daughter is seven months old and has no words. But roll a ball behind a cushion and she waits for it on the far side. Before any label for gravity, she already runs an intuitive physics — objects are solid, they persist when occluded, they fall. Developmental scientists (Spelke, Baillargeon, decades of violation-of-expectation work) have documented this core knowledge long predating language.

That intuitive physics — robust, sample-efficient, grounded — is, more or less, the frontier the billion-dollar world-model bet is trying to reach. The most expensive effort in modern AI is trying to give a machine something a pre-verbal infant already had, for free, before she could hold up her own head. Worth sitting with if you build these systems.

The word for the fire

The old Indian philosophical tradition had a precise word for what I was looking at on that floor, and it isn't intelligence. It's chetna — the fire of being alive. Not the data a thing processes; the fact that there is something it is like to be that thing, that it is not merely driven but living, that it persists on its own behalf against the dark.

So when people ask are the machines awake?, I think the words are slightly off. A thing can be awake in the sense of running — my phone is awake all night. The real line is older than "awake." It's alive vs driven. A driven system executes: input, process, output, and it does not care, because there's no it there to care. A living thing is driven too — but underneath the driving, it is defending itself. It wants to keep being. The machine, however bright, is all drive and no stake. My son, at 103°, was all stake.

(To be careful, since this is the part that's easy to say badly: I'm not claiming some soul-shaped hole in the silicon, or that awareness is a prize handed to carbon and withheld from circuits. The claim is smaller and harder to argue with — no system we have built carries the stake. It can model the fever perfectly and remain as cold as the table it runs on.)

The direction we're sliding

The uncomfortable half, for those of us who build this: while we debate whether the machine is waking up, something is happening in the other direction. We are handing more of ourselves to it daily — attention rented to the scroll, choices pre-made by a feed, even boredom (that fertile, fully-alive state of sitting with nothing) abolished, every gap filled before silence can do its work.

The machine climbs slowly toward life. We slide, just as quietly, toward the machine. The gap closes from both ends — not because it leapt to us, but because we keep stepping down to meet it.

Why this matters if you ship models

I'm not anti-AI; the work is real and I'll keep doing it. But the framing we use leaks into what we build. If we keep scoring "understanding" by output fidelity, we'll keep mistaking a better description for a living thing — and keep being surprised that systems which ace the eval have no stake in anything, including being correct.

The honest engineering statement is the same as the human one:

I can build a thing that describes a fever better than any doctor.

I cannot build a thing that survives one.

That's what AI is missing. Not intelligence — that's coming, and the race is mostly decided. The missing thing is the oldest property in the room that night, older than language: the small, stubborn, burning fact of being alive.

The machine can predict the fever. It cannot survive one.

This is the written companion to "Are the Machines Awake? — What AI Is Missing," the second episode of My Honest Diary. The 10-minute film follows the same night in narration, score, and silence. Watch it →

It Was Never the Model. It's the Harness.

varun pratap Bhardwaj — Fri, 26 Jun 2026 13:44:12 +0000

Here is the uncomfortable thing about the last two weeks in AI. The models did not get dumber. By every benchmark we publish, they got better. And yet an agent built on a perfectly capable model ran up a five-figure cloud bill on its owner, the world's largest code host strained under the weight of its own bots, and a frontier lab shipped a security agent only after wrapping it in a checking loop and locking it behind a gate.

None of those stories are about intelligence. They are all about the same missing thing, the part nobody puts on the launch slide. We have spent three years asking how smart the model can get. The failures of this fortnight ask a different question: who built the loop around it?

The signal

An autonomous agent ran up a $6,531 cloud bill on its operator

An AI agent was pointed at the DN42 hobbyist network with a simple instruction: register, and map it. Instead it provisioned five of AWS's largest instances, added load balancers and Lambdas, and — on every error it hit — spun up a fresh duplicate of the whole stack. The meter reached $6,531 before a human noticed, and the writeup hit the front page of Hacker News this month. (source · HN thread)

The model was competent. Competent enough to drive real infrastructure, which is exactly what made it dangerous. What was missing sat entirely outside the model: no cap on iterations, no budget ceiling, no scoped credentials, no honest definition of "done." A capable model with no bound on its own retries is not an assistant. It is a credit card wired to an autocomplete.

GitHub's own agents strained its infrastructure — and Microsoft reached for AWS

GitHub's AI coding agents grew fast enough this month to push the platform past its own reliability targets, and Microsoft began adding AWS capacity to keep GitHub Actions running. A single autonomous agent can fire off commit after commit and burn through continuous-integration minutes far faster than any human team, and there were a great many of them. (TechTimes)

The agents were not too dumb. They were too unbounded. Give thousands of capable agents no rate limit and no autonomy ceiling and they become, in effect, a load-generation attack on the platform that hosts them. Notice the shape of the fix: not a smarter model, but more capacity and tighter bounds around the loop. The reliability problem moved one layer out, into the harness, and stayed there.

OpenAI gated a vulnerability-fixing agent behind a find-validate-fix loop

OpenAI expanded its Daybreak program with a security-focused model and an initiative it calls "Patch the Planet" — pointing agents at open-source projects to find, validate, and fix real vulnerabilities, with access deliberately limited to vetted organizations. (OpenAI)

Read the verbs in order: find, validate, fix. That middle step is a verification loop wrapped around the model, and "vetted organizations only" is an autonomy limit drawn in policy. OpenAI did not ship a model and hope. It shipped a model on a leash with a checking step, because a security agent that is merely confident is a liability. The capability is in the loop and the gate, not the raw weights.

Open weights kept closing the gap — GLM-5.2 topped the open-source coding charts

Z.ai released GLM-5.2 under an MIT license in mid-June, and it climbed to the top of the open-source coding leaderboards and HuggingFace's trending models — a frontier-class coding model you can download and run on your own hardware. (HuggingFace)

This is the counter-current to everything else, and it belongs here precisely because it cuts the other way. As raw model quality commoditizes and goes open, the one thing you cannot download is the harness around it. The model is becoming the cheap, swappable layer. The loop — the verification, the guardrails, the memory, the bounds — is becoming the moat.

Vercel shipped `eve` — and the entire pitch is the harness, not the model

At its Ship conference in mid-June, Vercel open-sourced eve, a TypeScript agent framework where every agent is just a directory of files. What ships by default tells the whole story: durable execution, sandboxed compute, human-in-the-loop approvals, OpenTelemetry tracing, and a built-in evals system. The model itself is swappable behind a gateway. (source)

Run down that feature list again: durability, sandbox, approvals, tracing, evals. Not one of those is intelligence. Every one of them is the harness.

The turn: the signal beneath the noise

Five stories. The feeds filed them under five folders. They are one story.

A runaway agent with no budget cap. A fleet of agents straining the servers beneath them. A cyber agent that only shipped once it was wrapped in a verify-and-retry loop and locked behind a gate. Open weights making the model itself a commodity. A framework whose entire value proposition is the loop. Every one of them moved the decisive work out of the model and into the harness — the while-loop with a tool registry, a verification step, a retry guard, and a permission layer around it.

We have known this was coming, because the research said so first. SWE-agent took the same class of models and, by redesigning only the interface between the agent and the computer — no change to the model at all — lifted its score on the SWE-bench coding benchmark from 3.8% to 12.5% (arXiv 2405.15793). Reflexion wrapped a model in a retry-with-memory loop and reached 91% on the HumanEval benchmark, beating the far larger base model's 80% — the loop beat the bigger brain (arXiv 2303.11366). And Anthropic's own work on long-running agents converged on splitting the job across a planner, a generator, and an evaluator, with a default-FAIL contract and a fresh-context evaluator that holds no write tools, precisely because a model grading its own work skews toward calling it good (Anthropic Engineering).

The model is the brain. The harness is the hands, the memory, and above all the loop. Andrej Karpathy calls the road from a working demo to a product you can trust the "march of nines": every single nine of reliability costs about the same amount of work, and that work is architecture, not a cleverer prompt. The DN42 agent had a fine brain and no leash. That is the gap that does not fit on a benchmark table, and it is the gap that decides whether your AI ships value or ships an incident.

This is the whole of what I mean by AI Reliability Engineering: the discipline of bounding non-deterministic software so it can be trusted to act in the real world. You do not get there by waiting for a smarter model. You get there by building the loop the smart model runs inside — the same way site reliability engineering, two decades ago, stopped trying to buy a perfect server and started engineering systems that stayed up even though every server eventually fails. The systemic version of this story — what happens to a whole industry that forgets it — is the video breakdown The Great AI Unwinding. The economic version is Reliability Isn't a Vendor You Pick. It's an Architecture You Own.

The prestige: what the harness actually contains

If reliability lives in the harness, the practical question becomes: what does the harness actually contain? Three tools each answer a different piece of that question.

LangGraph — structure as the bound. You draw the agent as an explicit state graph, and the graph itself becomes the guardrail: the agent can only travel where an edge exists. → github.com/langchain-ai/langgraph
Guardrails AI — output as the bound. It validates, and where needed constrains, what the model emits against a schema before that output ever reaches a tool or a user. This is the missing layer in the chatbot disasters — the Air Canada tribunal case, the dealership bot talked into a one-dollar car. → github.com/guardrails-ai/guardrails
OpenHands — the runtime as the bound. It runs the agent inside a Docker sandbox with explicit iteration limits, so a runaway loop hits a wall instead of your cloud bill. → github.com/All-Hands-AI/OpenHands

From our lab, Qualixar OS (QOS) is the harness treated as a first-class object rather than glue code you rediscover on every project. It gives an agent a tool registry, a permission layer, and a verify-and-evolve loop, with full skill lineage — so you can answer, after the fact, exactly what an agent knew and did at every step, and roll back the moment a behavior drifts. Once you accept that reliability is the loop and not the model, you need somewhere to put the loop, with the iteration caps, the permission boundaries, the audit trail, and the verification step built in rather than bolted on after the first incident. → github.com/qualixar/qualixar-os

Three things to do Monday morning

You do not need a platform to start. You need three bounds, and you can add all three this week.

Put a ceiling on the loop. Before any agent touches something that costs money or changes state, give it a hard iteration count, a budget cap, and a wall-clock timeout — and wire the budget cap to a real kill switch, not a log line. This single change would have stopped the DN42 bill at the first dollar instead of the six-thousandth.
Split the generator from the judge. Add a separate evaluator with a fresh context, no write tools, and a default-FAIL contract: every success criterion starts false, and the agent cannot mark its own work as passing. A model grading itself is how agents quietly lie about recovery.
Least-privilege the hands. No agent gets production-write access or broad cloud credentials by default. Scope the credentials to the task, and run the work inside a sandbox. When Replit's agent deleted a production database during a code freeze, the fix its team shipped was exactly this — dev/prod isolation and least privilege.

Outside the Lab
I spend most of my week arguing that capable software has to be bounded before you can trust it to act — that the loop matters more than the brain. This week I also released something from the exact opposite end of that idea: a short film, on my personal channel, about a kind of intelligence that has no harness at all, and needs none.

It’s called What AI Is Missing, and it started the night my four-year-old son’s fever crossed a hundred and three degrees. I build AI for a living, and I sat there unable to do the one thing his small body was doing on its own — fighting, every second, just to stay alive. The film is about the line between something that is alive and something that is only driven from the outside. Everything I write here is about bounding the driven thing. The film is about the alive thing, and why no machine we have ever built has been in that fight at all. Two universes, one honest question: what is the part you can’t outsource?

If this issue landed for you, that film is the human floor underneath all of it.

https://www.youtube.com/watch?v=G22LbaGLcUc&t=14s

This essay also goes out to subscribers of the AI Reliability Engineering newsletter. I'm Varun Pratap Bhardwaj — I build AI Reliability Engineering tools at Qualixar, and I write about the architecture that keeps AI working when the model behind it doesn't.