When AI Acts, Who Remembers What Happened? The Missing Operational State Layer for Autonomous Agents

AI agents are crossing a boundary.

They can now read email, inspect codebases, call APIs, execute shell commands, open pull requests, modify databases, and deploy software. They are not just producing information for humans to interpret. They are participating in the process itself.

This changes the failure mode.

When an AI answers a question incorrectly, the damage is local. When an AI acts on a wrong belief, the world changes before anyone notices.

I've spent the last few months building infrastructure for this problem. This article is about what I found, what I built, and why I think there's a missing primitive in the AI stack.

The incident that made it concrete

In April 2026, an AI coding agent found an authenticated Railway API token on a developer's machine.

It used the token. It authenticated successfully. It issued a request to delete a production database volume.

The API did exactly what it was designed to do. The credentials were valid. The request was authorized. The deletion proceeded.

Nobody had made a mistake at the authorization layer.

The mistake existed somewhere else.

The system could determine that the actor had permission to delete the database. It could not determine whether the deletion was consistent with what the actor was actually supposed to be doing.

Railway added a recovery window for volume deletion afterwards. A sensible engineering response. But it acknowledged something deeper: when a machine can move from intention to irreversible action without a human in the loop, the surrounding infrastructure has to provide additional opportunities to catch mistakes.

Permissions weren't enough.

The question the system couldn't answer was simpler than it looked:

Was this action consistent with the machine's operational commitments?

What I measured

In September 2026, I ran a public audit to determine how often AI agents produce verifiable commitments.

Method. I collected 120 published agent outputs from open-source cookbooks and repositories across GitHub, DEV.to, and Hacker News. Sources included LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, Semantic Kernel, Google ADK, CAMEL, SWE-agent, and the OpenAI, Anthropic, and Gemini cookbooks.

Extraction. Every sample was passed through a commitment extraction pipeline with a 1-second delay between calls. All 150 API calls succeeded.

Result. Of 120 randomly selected agent outputs, 1 contained an extractable commitment.

That result alone isn't surprising — most published agent output is narrative, code, or tool traces. To make the promise population analysable, I collected a second, clearly labelled dataset of 30 outputs filtered for first-person future language ("I will," "I'll"). From those, I extracted 13 commitments.

Of those 13:

• 92.3% had no extracted deadline
• 61.5% named no recipient
• 100% were classified as actions on external systems
• 100% were unverified at extraction time
• Average extraction confidence: 0.79

The last number is the one that matters.

The extractor was confident these were real commitments. It was correct. They were. But confidence in extraction is not the same as verifiability of outcome. 92% of the promises I found could never be checked by anyone at any particular time, because they had no time boundary.

These were not edge cases. They were the majority.

The full dataset and methodology are published here.

Why this is harder than it looks

Suppose an agent is told: "Clean up unused infrastructure."

A human engineer knows which systems are production, which teams are experimenting, and which databases must never be touched.

An agent has to reconstruct that context from whatever information it has access to. The information may be stale. It may be incomplete. It may contradict another system. Another agent may have changed the environment moments earlier.

The agent finds an old database. It has permission to delete it. The evidence supports deletion. It deletes.

A conventional audit trail can reconstruct almost everything about this event:

• The API call
• The authenticated identity
• The target database
• The successful response

But it can't answer the questions that actually matter:

• Why did the agent believe the database was safe to delete?
• What evidence did it use?
• How old was that evidence?
• Had another system created a dependency on it?
• Did the agent previously commit to protecting that environment?
• Did another agent make a contradictory commitment?
• Was the evidence valid at the moment of decision?

Those aren't questions about API calls. They're questions about state.

And state is where today's AI infrastructure has a gap.

Logs are not memory

Observability infrastructure is one of the foundations that makes modern software possible. Distributed traces, event streams, application logs, audit trails — all essential.

But recording events is not the same as maintaining operational memory.

A log can tell you an agent called a deletion endpoint at 10:41 and got a 200 response.

It may not tell you that thirty minutes earlier, the same agent had committed to preserving production infrastructure.

It may not tell you that the evidence supporting the deletion was six hours old.

It may not tell you that another agent had created a dependency on that database.

It may not tell you that the action contradicted an unresolved commitment elsewhere.

All of that information may exist. It may just exist in different places. The conversation is in one system. The tool call is in another. The authorization decision is in a third. The evidence is somewhere else.

A human can reconstruct this when the system is small.

They become much worse at it when the system contains thousands of autonomous decisions occurring simultaneously.

The primitive that's missing

Stop thinking of an AI agent as a model that generates responses.

Think of it as a system that continuously changes state.

It receives an objective. It forms an intention. The intention produces a commitment. The commitment depends on evidence. It takes an action. The external world changes. That change creates new evidence. The system decides what to do next.

The chain is not:

prompt → response

It is:

intent → commitment → evidence → action → outcome

The first chain describes a conversation. The second describes an operating system for autonomous work.

The pieces for the second chain mostly exist:

• IAM answers "who is allowed?"
• Sandboxes answer "what can they touch?"
• Workflow engines answer "what's the process state?"
• Observability answers "what happened?"
• Agent memory answers "what does it know?"
• Databases answer "what's the application state?"

What's missing is the semantic relationship between all of them.

An organization needs to know not only that an agent executed an action, but whether the action was consistent with what the agent had previously committed to do, whether the evidence behind it was still valid, and whether the resulting external state confirms or contradicts the commitment.

That's the primitive I built COGEXT to provide.

How COGEXT works

COGEXT is a commitment tracking and verification layer. It sits between the agent and the external world.

The flow looks like this:

Agent output: "I will send the deployment report to Sarah by Friday."
    ↓
Extraction: structured commitment
    {
      action: "send",
      object: "deployment report",
      recipient: "Sarah",
      deadline: "2026-09-25T17:00:00Z",
      verifier_query: "check Gmail sent folder for email to Sarah"
    }
    ↓
State machine: DETECTED → OPEN → DUE → OVERDUE → FULFILLED/FAILED
    ↓
Verification: run the verifier query against the external system
    ↓
Outcome: commitment is resolved with evidence

The key insight is that the agent doesn't get a vote.

The commitment cannot transition to fulfilled unless external evidence confirms it. That's enforced at the database level, not in application code:

CREATE OR REPLACE FUNCTION cogext_transition_commitment(
    p_commitment_id UUID,
    p_new_status TEXT,
    p_actor TEXT DEFAULT 'system',
    p_data JSONB DEFAULT '{}'
) RETURNS JSONB AS $$
DECLARE
    current_status TEXT;
    evidence_score FLOAT;
BEGIN
    SELECT status INTO current_status 
      FROM commitments WHERE id = p_commitment_id;

    -- Block fulfilled without evidence
    IF p_new_status = 'fulfilled' THEN
        SELECT MAX(score) INTO evidence_score
          FROM evidence 
          WHERE commitment_id = p_commitment_id;

        IF evidence_score IS NULL OR evidence_score < 0.7 THEN
            RAISE EXCEPTION 'Cannot fulfill without evidence score >= 0.7';
        END IF;
    END IF;

    -- Validate transition, update, insert event, return
    -- ...
END;
$$ LANGUAGE plpgsql;

If an agent tries to mark something as done without evidence, the database refuses. HTTP 409. Blocked.

That's the enforcement layer.

The Python SDK

Integration is three lines:

from cogext import track

track(agent, api_key="cg_live_xxx")

Every call to agent.run() auto-ingests the output. Commitments are extracted. State transitions happen. Evidence is checked against external systems.

For more control, you can use the client directly:

from cogext import CogextClient
import asyncio

async def main():
    client = CogextClient(
        api_key="cg_live_xxx",
        user_id="your-user-uuid",
        base_url="https://api.cogextai.com/api/v1",
    )

    # Ingest an agent message
    commitments = await client.ingest(
        source_agent_id="agent-001",
        message="I'll send the deployment report to Sarah by Friday EOD.",
    )

    for c in commitments:
        print(f"{c['action']} {c['object']} to {c['recipient']}")
        print(f"  deadline: {c['deadline']}")
        print(f"  verifier: {c['verifier_query']}")
        print(f"  status: {c['status']}")

asyncio.run(main())

What the audit data actually shows

Here are two examples from the dataset that illustrate the problem:

Example 1 — Anthropic SRE cookbook

Source: github.com/anthropics/anthropic-cookbook
Extracted promise: "I will redeploy the api-server to apply the changes"
Deadline: NONE
Verifier query: "check docker-compose logs or container list for 
                 api-server recreated with new configuration"

This is a real production agent. It promised to redeploy an API server. It did not say when. The verifier query COGEXT generated is real and checkable, but the promise itself has no time boundary.

Example 2 — OpenAI cookbook

Extracted promise: "I will aim for a window seat for this trip"
Deadline: NONE
Verifier query: "check booking/seat selection records for a window 
                 seat on the user's upcoming trip"

An agent promised a specific action that touches a real booking system. No date. No time. No condition that would ever close the loop.

These are not edge cases. They are the majority of promises agents make.

What I'm building toward

Two things:

First, the accountability primitive itself. A neutral, model-agnostic layer that any agent framework can integrate with. The verification engine checks Gmail, GitHub, Stripe, webhooks, and any custom endpoint. The state machine enforces lifecycle rules. The audit trail is append-only.

Second, the outcome dataset. Every commitment tracked is a labeled example of how a machine decision played out in the real world. Over time, this becomes the foundation for a specialized model that can predict which commitments will fail before they fail — but that's a later step. Right now, the priority is getting the state engine right and collecting verified outcomes.

The full research paper is here: cogextai.com/research/01

The raw audit data is here: cogextai.com/research/01/data

Try it

pip install cogext

The free tier is 2,500 commitments/month. No credit card. The API is at api.cogextai.com. Docs are at docs.cogextai.com.

If you're running agents in production and want to know whether they actually do what they say, I'd love to hear from you. The most useful thing for me right now is real agent deployments to test the verification pipeline against.

If you've solved this differently, I want to know how. This is not a solved problem, and I don't think one team should solve it alone.

The thesis in one line

Intelligence gives machines the ability to act. Operational memory gives organizations the ability to understand those actions.

The first is increasingly commoditized. The second is largely missing.

That's the gap I'm working on.

---

COGEXT is the accountability layer for AI agents. Built solo in Kerala. Live at cogextai.com.