One of the more interesting changes in the AI engineering ecosystem in 2026 is happening below the model layer.
The latest MCP specification moved the protocol toward a stateless core.
That sounds like agents should become stateless too.
They shouldn’t.
In fact, as AI agents become longer-running, more autonomous, and capable of executing real actions, application-level state becomes even more important.
The distinction is simple:
MCP should not need to remember your connection.
Your agent absolutely needs to remember its work.
This difference becomes critical once you move from demos to production.
What Changed in MCP?
The 2026-07-28 Model Context Protocol specification introduced a stateless protocol core.
Instead of depending on persistent sessions between an MCP client and server, requests can carry enough information to be handled independently.
That means an MCP request can potentially hit:
Client
|
v
Load Balancer
|
+------> MCP Server 1
|
+------> MCP Server 2
|
+------> MCP Server 3
without requiring the load balancer to keep routing a particular client back to the same server instance.
The new specification removed the old session-oriented initialize flow and Mcp-Session-Id, making requests more self-describing and much easier to scale using conventional HTTP infrastructure.
This is a good architectural change.
But there is an important trap here.
Stateless transport does not mean stateless workflow.
An AI Agent Is Usually a State Machine
Imagine an agent responsible for refunding a customer.
The workflow might look like:
User asks for refund
|
v
Agent investigates order
|
v
Checks refund policy
|
v
Calculates refund amount
|
v
Requires human approval
|
v
PAUSE
|
[30 minutes]
|
v
Human approves
|
v
Execute refund
|
v
Notify customer
What happens during those 30 minutes?
If your agent state only exists inside:
agent = Agent(...)
result = agent.run(...)
you have a problem.
The process might restart.
A deployment might happen.
The request could reach another Kubernetes pod.
The machine could disappear.
The approval request could arrive hours later.
The workflow therefore cannot depend on process memory.
You need durable state.
The Wrong Architecture
A common first implementation looks something like this:
pending_runs = {}
async def execute_agent(user_id, request):
result = await agent.run(request)
if result.requires\_approval:
pending\_runs\[[result.id](http://result.id)\] = result
return {
"status": "waiting\_for\_approval",
"run\_id": [result.id](http://result.id)
}
Later:
async def approve(run_id):
run = pending_runs[run_id]
return await run.resume()
It works perfectly...
until you deploy it.
Consider two instances:
Load Balancer
/ \\
/ \\
Server A Server B
The agent runs on Server A.
pending_runs["run_123"]
exists only in Server A's memory.
The user clicks:
Approve
The load balancer sends the request to Server B.
Server B asks:
pending_runs["run_123"]
and gets:
KeyError
Your AI model isn't the problem.
Your prompt isn't the problem.
Your distributed system is.
The Better Architecture
Persist the workflow state.
┌───────────────┐
│ Client │
└───────┬───────┘
│
▼
┌───────────────┐
│ Load Balancer │
└───────┬───────┘
│
┌───────────┴───────────┐
▼ ▼
┌───────────┐ ┌───────────┐
│ Server A │ │ Server B │
└─────┬─────┘ └─────┬─────┘
│ │
└───────────┬───────────┘
▼
┌─────────────────┐
│ Workflow State │
│ │
│ Postgres │
│ Redis │
│ Temporal │
│ Durable Runtime │
└─────────────────┘
Now the agent runtime becomes replaceable.
Any server can reconstruct the current workflow.
Separate Three Different Types of State
This is where production agent architecture becomes interesting.
I usually think about agent state as three different layers.
1. Conversation State
This is what the model needs to understand the interaction.
For example:
{
"messages": [],
"summary": "...",
"user_preferences": {},
"retrieved_context": []
}
This state controls what the model knows.
2. Workflow State
This is what your application needs to understand where execution currently is.
For example:
{
"workflow_id": "refund_39281",
"status": "WAITING_FOR_APPROVAL",
"current_step": "refund_confirmation",
"order_id": "ORD_8821",
"refund_amount": 149.99
}
This is not prompt context.
It is distributed-system state.
3. Side-Effect State
This tells you what the agent has already done.
For example:
{
"email_sent": true,
"refund_created": false,
"crm_updated": true
}
Without this state, retries become dangerous.
Imagine:
Agent calls refund API
↓
Network timeout
↓
Agent doesn't know whether refund succeeded
↓
Agent retries
↓
Customer gets refunded twice
That is why production agents need idempotency.
Idempotency Becomes Extremely Important
Every side-effecting tool should ideally support something similar to:
refund(
order_id="ORD_8821",
amount=149.99,
idempotency_key="workflow_928_step_7"
)
Then:
Attempt 1
workflow_928_step_7
↓
Refund $149.99
If the workflow retries:
Attempt 2
workflow_928_step_7
↓
Already processed
↓
Return existing result
instead of creating another refund.
This applies to much more than payments.
Think about:
send_email()
create_ticket()
delete_resource()
publish_post()
update_crm()
book_meeting()
deploy_service()
transfer_money()
Once AI agents can perform actions, retry semantics become part of AI safety.
Human Approval Is Also a Distributed Systems Problem
Human-in-the-loop workflows are becoming common for sensitive actions.
For example:
Agent
|
v
Draft action
|
v
Approval required
|
+---------- PAUSE ----------
|
|
Human approves
|
v
Resume job
The important word here is:
resume
You don't want to restart the entire agent.
You want to continue from a durable checkpoint.
Modern agent frameworks increasingly expose this kind of pause/resume model. OpenAI's agent documentation, for example, describes storing serialized state when human review happens later and continuing the same run once the decision arrives.
This changes how we should think about agent execution.
An agent isn't necessarily:
HTTP request
↓
LLM
↓
response
It may instead be:
Start
↓
Think
↓
Tool
↓
Think
↓
Tool
↓
Pause
↓
--- 4 hours later ---
↓
Resume
↓
Tool
↓
Think
↓
Complete
That is much closer to a workflow engine than a normal API request.
Durable Execution Is Becoming Part of the Agent Stack
Long-running agents introduce familiar distributed-system problems:
process crashes
network failures
duplicate messages
timeouts
retries
partial execution
concurrent updates
human approvals
scheduled execution
deployment during execution
These problems existed long before LLMs.
We're just rediscovering them inside agent systems.
A production architecture may therefore look like:
User
|
v
API Gateway
|
v
Agent Orchestrator
|
┌───────────┼───────────┐
│ │ │
v v v
Model Tools MCP
│ │ │
└───────────┼───────────┘
|
v
Durable Workflow
|
┌────────────┼────────────┐
│ │ │
v v v
State DB Queue Event Log
Frameworks are increasingly acknowledging this requirement. LangChain, for example, describes durable execution, memory, human-in-the-loop support, multi-tenancy, and observability as infrastructure needed underneath long-running production agents.
MCP and Durable Execution Solve Different Problems
This distinction is worth remembering.
MCP answers:
How does an agent communicate with tools and external systems?
Durable execution answers:
How does an agent reliably continue working over time?
They complement each other.
You might have:
Agent
|
| MCP
v
Salesforce
Agent
|
| MCP
v
GitHub
Agent
|
| MCP
v
Slack
while the overall workflow is managed separately:
Step 1: Fetch GitHub issue
Step 2: Analyze code
Step 3: Generate patch
Step 4: Run tests
Step 5: Wait for approval
Step 6: Create PR
Step 7: Post Slack notification
The MCP servers do not need to remember the entire workflow.
The orchestrator does.
Observability Also Changes
Traditional API monitoring might tell you:
POST /agent
200 OK
Duration: 4.2s
That isn't enough.
A production agent might execute:
Run #8291
├── Model call
├── retrieve_documents
├── Model call
├── search_customer
├── Model call
├── update_customer
├── approval_required
├── PAUSED
├── approval_received
├── update_salesforce
├── send_email
└── complete
You need to understand the complete trajectory.
That means tracking:
model calls
tool calls
tool arguments
tool responses
latency
token usage
retries
approvals
guardrail decisions
state transitions
errors
cost
Agent platforms are moving in this direction as well. Current OpenAI tooling, for example, exposes structured tracing across model calls, tool calls, handoffs, guardrails, and custom spans.
The Production Pattern
If I were designing a serious agent system today, I would separate it roughly like this:
┌───────────────────────────────┐
│ API Layer │
└───────────────┬───────────────┘
│
▼
┌───────────────────────────────┐
│ Agent Orchestrator │
│ │
│ Planning │
│ Reasoning │
│ Tool selection │
└───────────────┬───────────────┘
│
┌────────┴────────┐
│ │
▼ ▼
┌─────────────┐ ┌───────────────┐
│ MCP / Tools │ │ Workflow │
│ │ │ Runtime │
│ Stateless │ │ │
│ interface │ │ Durable State │
└─────────────┘ └───────┬───────┘
│
┌───────┼───────┐
▼ ▼ ▼
DB Queue Logs
Notice the separation:
MCP → tool interoperability
LLM → reasoning
Workflow layer → durability
Database → state
Queue → asynchronous execution
Tracing → observability
Guardrails → control
Trying to make the LLM responsible for all of these concerns is where agent architecture usually starts falling apart.
The Bigger Lesson
The AI industry spent the first phase of the LLM boom asking:
Which model should we use?
Then:
Which prompt should we use?
Then:
Which agent framework should we use?
The more important production questions are increasingly becoming:
How does the workflow recover?
How do we resume execution?
How do we prevent duplicate side effects?
Where does state live?
How do we authorize tool calls?
How do we trace a 50-step execution?
How do we roll out a new agent version safely?
How do we replay failed workflows?
How do we evaluate complete trajectories?
These are not fundamentally AI questions.
They are distributed systems questions.
And that might be one of the most important shifts happening in AI engineering right now.
MCP becoming more stateless doesn't remove state from agent systems.
It simply puts state where it belongs.
In the application and workflow layer, not the transport protocol.
Final Thought
The next generation of AI applications probably won't look like:
User → Prompt → LLM → Response
They will look more like:
User
↓
Agent
↓
Planner
↓
Tools / MCP
↓
Durable Workflow
↓
Events
↓
Approvals
↓
Retries
↓
Observability
↓
Result
The model may be the brain.
But production reliability still comes from good systems engineering.
And no amount of prompt engineering can replace that.
Top comments (0)