Machine-checked testing of LangGraph 1.2.9, CrewAI 1.15.2, and pydantic-graph 1.x reveals that none honor their documented resume contracts — and the failures aren't edge cases, they're systematic violations of exactly-once semantics. A machine-checked conformance contract tested five widely deployed frameworks and found that every single one answers differently to the same persistence question, with no framework exposing a verifiable contract. What I call the Durable Execution Gap is exactly this: vendors are racing to market durable agents, yet independent verification shows the underlying resume contracts are broken across the board.
That gap matters because agent infrastructure has shifted from orchestration syntax to persistence economics. The real product is no longer the graph syntax — it's the checkpoint and replay layer. But if the replay layer re-executes completed work, duplicates side effects, or silently persists invalid state, you're not buying durability. You're buying a fiction.
What's actually broken with agent checkpointing?
Durable execution is not the same as session memory. Saving chat history helps an agent remember a user's name, but it doesn't prove which shell command ran, which email was sent, or whether a retry would duplicate a side effect. Per durable execution research, production agents need execution journals with idempotency keys, not just transcript storage. Without deterministic replay and idempotent tool boundaries, recovery from crashes becomes improvisation rather than controlled continuation.
Here's why that distinction breaks in practice. The Agent Resumption pattern identifies two production approaches: deterministic replay of recorded effects (the Temporal or Inngest pattern) versus checkpoint snapshots of agent state (the LangGraph Cloud pattern). Both require deterministic idempotency keys passed to side-effect targets. Without them, a crash between an effect and its log produces duplicates. The research warns that without these keys, recovery is not a feature — it's a liability.
The contradiction is stark. Every major framework is marketing durable execution right now. CrewAI 1.14.3 added lifecycle events for checkpoint operations, checkpoint and fork support for standalone agents, and reduced cold-start time by roughly 29 percent. Mastra Durable Agents, announced in June 2026, persist streaming state in real time via a server cache and PubSub layer. Microsoft Agent Framework 1.14.0 added workflow checkpoint creation and resume support to AgentFrameworkWorkflow, plus Foundry-hosted session-state storage. Even Google ADK promotes durable memory schemas and event-driven dormancy gates for enterprise workflows spanning days or weeks.
Yet the same machine-checked study found that LangGraph durably records a second resume value and never consults it. It persists schema-invalid state silently. After a real SIGKILL, it re-executes durably recorded work — violating exactly-once semantics across crashes. CrewAI 1.15.2 re-executes completed effect-bearing methods against its own written claim. Pydantic-graph 1.x cannot resume after a mid-node crash. No two probed frameworks share the same conformance profile. So when you read about "durable agents," you're reading marketing, not mechanics.
If you're tracking state resilience more broadly, our post on Agent State Machines: Hidden Backbone of Production AI Agents covers why enterprise rollbacks stem from unmanaged state issues — and why deterministic finite state machines remain the required foundation.
How does LangGraph Cloud price durable execution?
LangGraph Cloud is generally available with persistent state across agent runs, human-in-the-loop checkpointing, and a time-travel debugger that lets developers replay or branch any execution from any historical state. Per Ship or Skip's review, pricing is step-based at $0.0025 per step on a usage-based model.
That sounds honest. It is — until it isn't. The same review notes that an agent running 10,000 steps debugging a code loop produces a $25 bill for one failed run. The pricing is transparent, but the cost is unpredictable at scale. You don't know how many steps a failure loop will consume until it consumes them. That's the same unpredictability users revolted against elsewhere, just framed differently.
Compare that to flat-rate models users understand. Replit Agent moved to effort-based pricing in 2026: a simple change is usually under $0.25, but a complex task is bundled into one checkpoint that may exceed $0.25, with cost revealed only after the task finishes. Users report being charged per checkpoint even when the agent fails, hangs, or errors — documented cases include a $1.15 charge for a non-existent method and weekly bills reaching approximately $1,000 after Agent 3 increased autonomy. The objection isn't average cost. It's that the cost of any given action is unknowable in advance and untethered from whether the work was useful.
Here's the contradiction: users rejected Replit's "pricing casino" as unpredictable, yet praised LangGraph's $0.0025/step as "the honest way to price infrastructure" — despite the same unpredictability at scale. One failed agent loop costs $25. A debugging session that spirals costs real money. The difference is framing, not economics.
You'll find that the LangGraph checkpointer mechanism — MemorySaver, SqliteSaver, PostgresSaver — saves the graph's state after every superstep boundary, keyed by thread_id. That enables crash recovery and resume without re-running completed steps. But the machine-checked testing shows that the resume value recorded is not the one consulted. So the mechanism exists, but the contract is broken.
Which replay and debugging tools are worth the cost?
If you're choosing between replay infrastructure, the landscape splits between managed hosted replay and self-hosted persistence. Retrace records every LLM call, tool call, and branching decision, then lets you fork from the exact broken step and re-run a corrected version. It is hosted-only — no self-host option — which means every recorded trace goes to Retrace's infrastructure. Per AIDiveForge, pricing starts at $29/month (Starter, 10,000 traces/mo), $99/month (Pro, 50,000 traces/mo), and $399/month (Teams, 500,000 traces/mo). A fork/replay add-on is available for $5/month on the free tier.
That hosted-only model is a real constraint. If your security policy requires traces to stay on-premises, Retrace is not an option. The tradeoff is clear: you get genuinely differentiated time-travel debugging — the review calls it "hard to build correctly" — but you surrender control of the trace data.
Other platforms are racing to fill the gap with different architectures. Shepherd, an open-source MIT-licensed Python substrate in early alpha, records agent runs as Git-like typed event traces, supports fork/replay/revert, and reports forks 5× faster than Docker with over 95% prompt-cache reuse on replay. It's not production-ready, but it's portable — exactly the kind of open substrate that avoids vendor lock-in.
Perplexity SPACE, shipped July 15, 2026, runs each agent task in an AWS Firecracker microVM, supports session pause/branch/resume for up to a week, keeps secrets outside the sandbox, and reports median sandbox creation of 60ms and p90 latency of 89ms. OpenComputer's serverless agent preview (Aug 4, 2026) uses hibernating VMs that preserve memory, filesystem, and working context, supporting checkpoints, VM forks, and hardware-level isolation via KVM.
The comparison below shows where pricing and capabilities stand based on verified research data:
| Tool | Pricing | Checkpoint Capability | Target Audience |
|---|---|---|---|
| LangGraph Cloud | $0.0025/step | Persistent state, time-travel replay, human-in-the-loop source | Production multi-step agents |
| Retrace | $29/mo Starter to $399/mo Teams | Fork/replay from exact broken step, CI gates source | Teams needing hosted replay |
| Replit Agent | Effort-based (<$0.25 simple, >$0.25 complex) | Checkpoint billing, no verified resume contract source | Rapid prototyping (high cost risk) |
| CrewAI 1.14.3 | — (free tier noted in research) | Checkpoint resume, fork, lineage tracking source | Open-source multi-agent workflows |
Notice what's missing from the table: verified resume contracts. Every tool offers persistence. None have submitted to machine-checked conformance verification.
For a deeper look at replay and control in production, see our analysis in Best AgentOps Tools: Control Beats Replay in 2026.
Why do framework resume contracts fail in practice?
The machine-checked study measured five frameworks at pinned releases using a deterministic, LLM-free harness. The results are not ambiguous. LangGraph 1.2.9 durably records a second resume value and never consults it. It persists schema-invalid state silently. After a real SIGKILL, it re-executes durably recorded work — exactly-once across interrupts, at-least-once across crashes, on one API. That means the same framework promises exactly-once semantics but delivers at-least-once behavior under crash conditions.
CrewAI 1.15.2 re-executes completed effect-bearing methods against its written claim. Pydantic-graph 1.x cannot resume after a mid-node crash. No two probed frameworks share the same conformance profile. The failure crosses hosts: two racers on separate machines duplicating effects in 10/10 repetitions.
Under concurrent delivery, the consume-once property fails entirely. When k processes resume one parked interrupt, the gated effect fires k times. Saturation hits 1.0 in 36 of 40 tested cells and never drops below 0.933 on durable backends. The admitting window tracks the gated node's own execution time, measured by dose-response. This isn't a theoretical race condition — it's a measured, reproducible failure.
So when vendors market "durable agents," they're describing a feature set, not a verified behavior. The Durable Execution Gap — the distance between marketed durability and machine-verified reliability — is the real infrastructure problem.
The Agent Resumption pattern makes the fix explicit: both deterministic replay and checkpoint snapshots require deterministic idempotency keys. Without them, crash-between-effect-and-log produces duplicates. The pattern also forbids non-serializable in-memory references in long-running paths — a constraint most frameworks don't enforce.
If you're coordinating long-running workflows across agents, our post on Agent Task Queues: Infrastructure That Decides If AI Ships explains why durable state and human checkpoints decide whether agents ship — not model scale.
What should production teams prioritize instead?
Don't treat checkpoint and replay features as reliable until vendors submit to machine-checked conformance contracts. That sounds harsh, but the data supports it. The hidden cost of a broken resume isn't just a failed run. It's duplicated side effects, unrecoverable state, and bills for work that should never have been repeated.
Prioritize platforms with explicit idempotency keys, budget guardrails, and circuit breakers over those marketing "time-travel" UX. The NAC harness from Arcee, released Aug 13, 2026, separates temporary action context from persistent episode summaries using a thread-and-episode architecture — addressing context rot on long-horizon tasks without pretending the transcript is the state. CrewAI 1.14.2 added checkpoint resume with from_checkpoint, checkpoint forking with lineage tracking, and deploy validation CLI commands (resume, diff, prune) — practical tools for managing checkpointed runs, not just replaying them.
Real-time continuous persistence through server caches (Mastra) offers faster recovery than periodic superstep snapshots (LangGraph), but both risk losing or corrupting mid-step state if the contract is broken. The simpler snapshot is easier to audit; the continuous stream is harder to verify. Choose based on your audit requirements, not vendor marketing.
For cost control, build guardrails before you build replay UIs. A $0.0025/step price looks cheap until a loop runs 10,000 steps. A $29/month Retrace Starter plan looks affordable until you realize it doesn't include self-hosting. The economics favor open substrates like Shepherd for portability, managed replay for speed, and explicit budget caps for survival.
If you're running multi-step agents today, treat every checkpoint as a best-effort snapshot, not a guarantee — and build your budget guardrails before you build your replay UI. What's your team's current checkpoint strategy: deterministic replay with idempotency keys, or snapshot-and-hope?
Originally published at SaaS with Alex
Top comments (0)