MCP Went Stateless. Your AI Agent Still Needs State.

Robin Singh — Thu, 24 Sep 2026 01:02:01 +0000

One of the more interesting changes in the AI engineering ecosystem in 2026 is happening below the model layer.

The latest MCP specification moved the protocol toward a stateless core.

That sounds like agents should become stateless too.

They shouldn’t.

In fact, as AI agents become longer-running, more autonomous, and capable of executing real actions, application-level state becomes even more important.

The distinction is simple:

MCP should not need to remember your connection.

Your agent absolutely needs to remember its work.

This difference becomes critical once you move from demos to production.

What Changed in MCP?

The 2026-07-28 Model Context Protocol specification introduced a stateless protocol core.

Instead of depending on persistent sessions between an MCP client and server, requests can carry enough information to be handled independently.

That means an MCP request can potentially hit:

Client

|

v

Load Balancer

|

+------> MCP Server 1

|

+------> MCP Server 2

|

+------> MCP Server 3

without requiring the load balancer to keep routing a particular client back to the same server instance.

The new specification removed the old session-oriented initialize flow and Mcp-Session-Id, making requests more self-describing and much easier to scale using conventional HTTP infrastructure.

This is a good architectural change.

But there is an important trap here.

Stateless transport does not mean stateless workflow.

An AI Agent Is Usually a State Machine

Imagine an agent responsible for refunding a customer.

The workflow might look like:

What happens during those 30 minutes?

If your agent state only exists inside:

agent = Agent(...)

result = agent.run(...)

you have a problem.

The process might restart.

A deployment might happen.

The request could reach another Kubernetes pod.

The machine could disappear.

The approval request could arrive hours later.

The workflow therefore cannot depend on process memory.

You need durable state.

The Wrong Architecture

A common first implementation looks something like this:

pending_runs = {}

async def execute_agent(user_id, request):

result = await agent.run(request)

if result.requires\_approval:  
    pending\_runs\[[result.id](http://result.id)\] = result  

    return {  
        "status": "waiting\_for\_approval",  
        "run\_id": [result.id](http://result.id)  
    }

Later:

async def approve(run_id):

run = pending_runs[run_id]

return await run.resume()

It works perfectly...

until you deploy it.

Consider two instances:

         Load Balancer  
          /         \\  
         /           \\  
    Server A       Server B

The agent runs on Server A.

pending_runs["run_123"]

exists only in Server A's memory.

The user clicks:

Approve

The load balancer sends the request to Server B.

Server B asks:

pending_runs["run_123"]

and gets:

KeyError

Your AI model isn't the problem.

Your prompt isn't the problem.

Your distributed system is.

The Better Architecture

Persist the workflow state.

             ┌───────────────┐  
             │     Client    │  
             └───────┬───────┘  
                     │  
                     ▼  
             ┌───────────────┐  
             │ Load Balancer │  
             └───────┬───────┘  
                     │  
         ┌───────────┴───────────┐  
         ▼                       ▼  
   ┌───────────┐           ┌───────────┐  
   │ Server A  │           │ Server B  │  
   └─────┬─────┘           └─────┬─────┘  
         │                       │  
         └───────────┬───────────┘  
                     ▼  
           ┌─────────────────┐  
           │ Workflow State  │  
           │                 │  
           │ Postgres        │  
           │ Redis           │  
           │ Temporal        │  
           │ Durable Runtime │  
           └─────────────────┘

Now the agent runtime becomes replaceable.

Any server can reconstruct the current workflow.

Separate Three Different Types of State

This is where production agent architecture becomes interesting.

I usually think about agent state as three different layers.

1. Conversation State

This is what the model needs to understand the interaction.

For example:

{

"messages": [],

"summary": "...",

"user_preferences": {},

"retrieved_context": []

}

This state controls what the model knows.

2. Workflow State

This is what your application needs to understand where execution currently is.

For example:

{

"workflow_id": "refund_39281",

"status": "WAITING_FOR_APPROVAL",

"current_step": "refund_confirmation",

"order_id": "ORD_8821",

"refund_amount": 149.99

}

This is not prompt context.

It is distributed-system state.

3. Side-Effect State

This tells you what the agent has already done.

For example:

{

"email_sent": true,

"refund_created": false,

"crm_updated": true

}

Without this state, retries become dangerous.

Imagine:

Agent calls refund API

↓

Network timeout

↓

Agent doesn't know whether refund succeeded

↓

Agent retries

↓

Customer gets refunded twice

That is why production agents need idempotency.

Idempotency Becomes Extremely Important

Every side-effecting tool should ideally support something similar to:

refund(

order_id="ORD_8821",

amount=149.99,

idempotency_key="workflow_928_step_7"

)

Then:

Attempt 1

workflow_928_step_7

↓

Refund $149.99

If the workflow retries:

Attempt 2

workflow_928_step_7

↓

Already processed

↓

Return existing result

instead of creating another refund.

This applies to much more than payments.

Think about:

send_email()

create_ticket()

delete_resource()

publish_post()

update_crm()

book_meeting()

deploy_service()

transfer_money()

Once AI agents can perform actions, retry semantics become part of AI safety.

Human Approval Is Also a Distributed Systems Problem

Human-in-the-loop workflows are becoming common for sensitive actions.

For example:

The important word here is:

resume

You don't want to restart the entire agent.

You want to continue from a durable checkpoint.

Modern agent frameworks increasingly expose this kind of pause/resume model. OpenAI's agent documentation, for example, describes storing serialized state when human review happens later and continuing the same run once the decision arrives.

This changes how we should think about agent execution.

An agent isn't necessarily:

HTTP request

↓

LLM

↓

response

It may instead be:

Start

↓

Think

↓

Tool

↓

Think

↓

Tool

↓

Pause

↓

--- 4 hours later ---

↓

Resume

↓

Tool

↓

Think

↓

Complete

That is much closer to a workflow engine than a normal API request.

Durable Execution Is Becoming Part of the Agent Stack

Long-running agents introduce familiar distributed-system problems:

process crashes

network failures

duplicate messages

timeouts

retries

partial execution

concurrent updates

human approvals

scheduled execution

deployment during execution

These problems existed long before LLMs.

We're just rediscovering them inside agent systems.

A production architecture may therefore look like:

                    User  
                     |  
                     v  
                API Gateway  
                     |  
                     v  
             Agent Orchestrator  
                     |  
         ┌───────────┼───────────┐  
         │           │           │  
         v           v           v  
       Model       Tools        MCP  
         │           │           │  
         └───────────┼───────────┘  
                     |  
                     v  
            Durable Workflow  
                     |  
        ┌────────────┼────────────┐  
        │            │            │  
        v            v            v  
     State DB      Queue       Event Log

Frameworks are increasingly acknowledging this requirement. LangChain, for example, describes durable execution, memory, human-in-the-loop support, multi-tenancy, and observability as infrastructure needed underneath long-running production agents.

MCP and Durable Execution Solve Different Problems

This distinction is worth remembering.

MCP answers:

How does an agent communicate with tools and external systems?

Durable execution answers:

How does an agent reliably continue working over time?

They complement each other.

You might have:

Agent

|

| MCP

v

Salesforce

Agent

|

| MCP

v

GitHub

Agent

|

| MCP

v

Slack

while the overall workflow is managed separately:

Step 1: Fetch GitHub issue

Step 2: Analyze code

Step 3: Generate patch

Step 4: Run tests

Step 5: Wait for approval

Step 6: Create PR

Step 7: Post Slack notification

The MCP servers do not need to remember the entire workflow.

The orchestrator does.

Observability Also Changes

Traditional API monitoring might tell you:

POST /agent

200 OK

Duration: 4.2s

That isn't enough.

A production agent might execute:

Run #8291

├── Model call

├── retrieve_documents

├── Model call

├── search_customer

├── Model call

├── update_customer

├── approval_required

├── PAUSED

├── approval_received

├── update_salesforce

├── send_email

└── complete

You need to understand the complete trajectory.

That means tracking:

model calls

tool calls

tool arguments

tool responses

latency

token usage

retries

approvals

guardrail decisions

state transitions

errors

cost

Agent platforms are moving in this direction as well. Current OpenAI tooling, for example, exposes structured tracing across model calls, tool calls, handoffs, guardrails, and custom spans.

The Production Pattern

If I were designing a serious agent system today, I would separate it roughly like this:

┌───────────────────────────────┐

│ API Layer │

└───────────────┬───────────────┘

│

▼

┌───────────────────────────────┐

│ Agent Orchestrator │

│ │

│ Planning │

│ Reasoning │

│ Tool selection │

└───────────────┬───────────────┘

│

┌────────┴────────┐

│ │

▼ ▼

┌─────────────┐ ┌───────────────┐

│ MCP / Tools │ │ Workflow │

│ │ │ Runtime │

│ Stateless │ │ │

│ interface │ │ Durable State │

└─────────────┘ └───────┬───────┘

│

┌───────┼───────┐

▼ ▼ ▼

DB Queue Logs

Notice the separation:

MCP → tool interoperability

LLM → reasoning

Workflow layer → durability

Database → state

Queue → asynchronous execution

Tracing → observability

Guardrails → control

Trying to make the LLM responsible for all of these concerns is where agent architecture usually starts falling apart.

The Bigger Lesson

The AI industry spent the first phase of the LLM boom asking:

Which model should we use?

Then:

Which prompt should we use?

Then:

Which agent framework should we use?

The more important production questions are increasingly becoming:

How does the workflow recover?

How do we resume execution?

How do we prevent duplicate side effects?

Where does state live?

How do we authorize tool calls?

How do we trace a 50-step execution?

How do we roll out a new agent version safely?

How do we replay failed workflows?

How do we evaluate complete trajectories?

These are not fundamentally AI questions.

They are distributed systems questions.

And that might be one of the most important shifts happening in AI engineering right now.

MCP becoming more stateless doesn't remove state from agent systems.

It simply puts state where it belongs.

In the application and workflow layer, not the transport protocol.

Final Thought

The next generation of AI applications probably won't look like:

User → Prompt → LLM → Response

They will look more like:

User

↓

Agent

↓

Planner

↓

Tools / MCP

↓

Durable Workflow

↓

Events

↓

Approvals

↓

Retries

↓

Observability

↓

Result

The model may be the brain.

But production reliability still comes from good systems engineering.

And no amount of prompt engineering can replace that.

DEV Community: Robin Singh

MCP Went Stateless. Your AI Agent Still Needs State.

This is my post