Why We Brought This Tool Into Our Lab
The failure that matters is not an agent returning a bad paragraph. It is an agent sending an email, charging an account, or provisioning infrastructure—and then losing the response before recording what happened.
Our infrastructure review starts at that boundary. A retry queue can restart the job. A checkpoint table can restore conversation state. Neither automatically tells us whether the payment completed just before the worker disappeared.
Restate is interesting because it moves recovery ownership outside the agent process. Its server drives application handlers, records durable operations in an execution journal, and supplies recorded results during replay. The agent SDK remains responsible for prompts, model interaction, and tool selection. Restate supplies the execution substrate underneath it.
Our evidence boundary is important: we reviewed the official integration guide and Pydantic AI template, but did not complete a local Restate deployment. Our sandbox did not have the server, and its permitted installation path could not provide one. We therefore have no independently measured crash-recovery time, retry count, duplicate-effect count, or timeout result to publish. The commands below are a reproducible walkthrough, not a disguised execution transcript.
That limits our verdict to an architectural and integration assessment. It does not make the failure model less useful.
I separate three promises when evaluating Restate AI durable execution:
- Recorded-result replay: a completed, journaled operation returns its stored result instead of executing again.
- Durable orchestration: recovery, waits, and service coordination do not depend entirely on the application worker remaining alive.
- External-effect safety: a payment, email, or database mutation does not occur twice.
The first two do not automatically establish the third.
Our central concern is the gap between an external system committing an effect and Restate recording its successful result. A durable runtime cannot erase that gap merely by wrapping an ordinary HTTP request.
For the integration details, our primary reference is the official SDK integration guide. For teams choosing infrastructure more broadly, we maintain related evaluations in our tools collection.
Pydantic AI Template: Setup and Illustrative Output
I would start with the smallest official Pydantic AI template rather than immediately porting a production agent. That keeps endpoint discovery, model integration, and tool durability distinguishable.
The template uses:
- A Restate server.
- A Python service listening on port
9080. - The Restate CLI for deployment registration.
-
uvfor running the Python project. - An OpenAI API key for the supplied weather-agent example.
The last point matters: the unmodified template is not a secret-free durability test. We can design server-and-handler tests without an LLM, but that is a separate fixture.
Here is the setup sequence for a machine with Homebrew, Git, and uv. Run each terminal block separately.
# Terminal 1: install the server and CLI, then start the runtime.
brew install restatedev/tap/restate-server restatedev/tap/restate
restate-server
# Terminal 2: obtain and run the official Pydantic AI template.
git clone https://github.com/restatedev/ai-examples.git
cd ai-examples/pydantic-ai/template
# Record the exact example revision before changing anything.
git rev-parse HEAD
export OPENAI_API_KEY="replace-with-your-key"
uv run .
# Terminal 3: register the service and invoke its handler.
restate -y deployments register localhost:9080
curl --fail-with-body \
http://localhost:8080/agent/run \
--json '{"message":"What is the weather in San Francisco?"}'
An illustrative response, adapted from the template’s example—not captured from our sandbox—is:
The weather in San Francisco is currently 23°C and sunny.
The model’s wording may differ. This response alone would establish neither recovery nor duplicate prevention.
I would inspect the invocation journal at http://localhost:9070 next. The important output is the durable operation history, not the weather sentence.
The Pydantic AI template uses RestateAgent to integrate the agent loop. Its tool implementation uses restate_context().run_typed() for recoverable work. The essential shape is:
from restate.ext.pydantic import RestateAgent, restate_context
# Wrap an existing Pydantic AI agent.
restate_agent = RestateAgent(weather_agent)
# Inside a tool, choose an explicit durable operation boundary.
result = await restate_context().run_typed(
f"Get weather {city}",
call_weather_api,
city=city,
)
This fragment belongs inside the template; it is not a standalone program. The distinction matters because the Restate context exists within an invocation managed by Restate, not in arbitrary Python execution.
I would not put an entire multi-effect tool into one durable block by default. If the tool looks up an account, sends an email, and updates a record, those are three recovery decisions. One enclosing block makes them one recoverable unit.
The acceptance tests I would run next
These are pending experiments, with explicit pass criteria—not claimed results.
| Test | Fault injection | Evidence required |
|---|---|---|
| Completed-step replay | Kill the application worker after step A is visibly journaled, before step B finishes | A’s external counter stays at one after recovery |
| Runtime restart | Restart Restate while preserving its data directory | The original invocation resumes without a new client submission |
| Ambiguous external commit | Commit a mock external effect, then terminate the worker before returning | Attempt count and committed-effect count are recorded separately |
| Coarse tool boundary | Perform two effects inside one block, then fail before completion | Identify which effects repeat when the unfinished block reruns |
| Retry budget | Use an always-failing operation and a version-verified finite retry policy | Confirm attempt semantics and behavior across worker restart |
| Timeout budget | Let a mock endpoint commit work but delay its response beyond the client timeout | Establish whether retries reuse the same operation identity |
For duplicate tests, I would use a separate mock HTTP service backed by persistent storage. A worker-local variable disappears during the very failure we are trying to measure.
For timeout tests, I would record at least four events: request received, effect committed, response attempted, and durable result observed. Without those timestamps, “the request timed out” tells us almost nothing about whether retrying is safe.
Deployment Blocker and Integration Constraints
Our immediate roadblock was environmental: no Restate server meant no valid crash experiment. Replacing it with a Python dictionary or a homegrown replay loop would test our implementation, not Restate.
The remaining findings are integration constraints identified during review. I would treat each as a deployment gate.
Exactly-once RPC is not universal exactly-once execution
I would reject an architecture diagram that labels every ctx.run() operation “exactly once” without identifying the destination.
Consider this sequence:
- Our handler sends a payment request.
- The payment provider commits it.
- Our worker dies before the successful result becomes durable.
- Recovery retries the unfinished operation.
The operation may run again. The destination needs a stable idempotency key, transactional deduplication, or another suitable protocol.
My preferred external API contract is a business-operation identifier such as order-184:charge:v1. Every retry uses the same identifier. The receiving system atomically associates that identifier with the committed effect and returns the original result on repetition.
Journal replay reduces unnecessary re-execution after recorded success. It does not turn an arbitrary remote API into a transaction participant.
Agent wrapping is not the same as fine-grained durability
We can choose a coarse integration boundary by wrapping the entire agent execution in one ctx.run() block.
I would use that only when repeating the whole unfinished execution is acceptable. A failure near the end can otherwise repeat earlier model calls and tool work within that block.
For a business workflow, I prefer explicit boundaries around individual model calls and meaningful tool operations. With Pydantic AI, RestateAgent supplies the integration layer, while tool implementations still need deliberate durable steps.
The debugging question becomes: what result was recorded before the failure? “The function returned” is not sufficient evidence.
Error handling can interfere with orchestration
Some agent SDKs convert tool errors into input for the next model iteration. That is useful for recoverable argument mistakes; it is dangerous when runtime control signals are treated as ordinary tool failures.
I would verify two paths:
- Terminal failures must remain terminal rather than becoming another opportunity for the model to retry.
- Suspension signals must propagate rather than being swallowed and described to the model.
The supplied material does not establish the exact Python terminal-error class or retry-option signatures for a pinned release. I would not invent them. Before implementation, I would lock the SDK version, inspect its supported API, and add explicit propagation tests.
Parallel tools and streaming change the design
The Pydantic AI template explicitly prevents parallel tool execution to avoid replay nondeterminism.
That is a material limitation for researcher agents or workflows built around independent tool fan-out. I would not restore concurrency by inserting asyncio.gather around durable operations. I would move concurrency to supported Restate primitives and test the resulting journal behavior.
We cannot use streaming responses directly as ctx.run() results. I would collect a complete model result before treating it as that block’s durable output.
That does not establish that every form of streaming UI is impossible. It means token delivery and durable result recording require separate design decisions.
Finally, IDs, timestamps, and built-in search results can affect replay just as much as model output. I would persist nondeterministic values or use supported deterministic facilities rather than regenerate them during recovery.
Compatibility remains an unresolved risk
We have not established a compatible pinned combination of server, CLI, Python SDK, and Pydantic AI versions. In our setup review, we found two invocation forms: /agent/run in the template README and /restate/call/agent/run in the quickstart. We have not established their release-specific compatibility.
I would follow the route supported by the installed release and service playground, not mix commands from unrelated revisions. Our reproducibility record would include the template commit, dependency lockfile, binary versions, service discovery output, and invocation route.
Scale, Latency & Cost vs. Alternatives
We have no measured throughput, memory footprint, recovery latency, or saturation curve. I cannot responsibly say Restate is faster or cheaper than Temporal or Inngest from this review.
I can compare why we would shortlist each option and what we would test before adopting it:
| Option | Why I would shortlist it | What I would investigate before adoption |
|---|---|---|
| Restate | Journaled operations integrated with ordinary service handlers and agent SDKs | Replay compatibility, tool concurrency, deployment lifecycle, external idempotency |
| Temporal | A broader workflow-platform evaluation, especially where the team already operates it | Worker/versioning strategy, SDK fit, migration effort, operational ownership |
| Inngest | An event-oriented application workflow evaluation | Hosting model, step behavior, limits, agent integration requirements |
| Queue plus checkpoint database | A small existing workflow with bounded recovery needs | Supervision, deduplication, timers, fencing, and retry-state ownership |
This is a selection framework, not a feature-parity benchmark.
I would load-test recovery under failure, not just healthy requests. A dependency outage can accumulate unfinished work; recovery then releases competing retries against the dependency that just returned. Backoff, admission control, concurrency limits, and tenant isolation become more important than the speed of a successful weather lookup.
Tool serialization also changes the critical path. If our workflow assumes concurrent independent tools, the template’s sequential behavior can lengthen completion time. We have not measured that penalty, and its size depends on the workload.
Retry budgets are cost budgets
I separate three controls:
- Transport timeout: how long an individual client operation waits.
- Operation retry policy: how many failures or how much retrying we permit.
- Business deadline: when the overall task stops being useful.
A finite retry count does not necessarily enforce a wall-clock deadline. A disconnected HTTP client is not proof that its durable invocation has been canceled.
Nested retries deserve special scrutiny. As a hypothetical example, three provider attempts inside three outer attempts can create nine outbound requests for one logical operation. That is arithmetic, not an observed Restate result.
We would cap model-call retries to avoid unbounded attempts and LLM costs. I would also account for agent turns, provider-side retries, tool retries, and recovery attempts before approving a spend ceiling.
A practical break-even model
I would calculate monthly value as:
Engineering time saved + duplicate work avoided − runtime cost − operational labor − migration amortization.
For illustration only:
- Twelve engineering hours saved at
$150/hour:$1,800. - Two hundred avoided repeat calls at
$0.04/call:$8. - Runtime infrastructure allowance:
$250. - Four operational hours at
$150/hour:$600. - Migration amortization:
$500.
That produces a hypothetical net benefit of $458 per month. These are planning assumptions, not Restate prices or measured savings.
The useful conclusion is that model-call savings alone may not justify a durable runtime. Recovery engineering and incident reduction are usually the larger variables. Teams with an established Temporal or Inngest deployment should include the cost of operating a second orchestration platform.
What We Could Not Verify
We did not deploy Restate locally or measure recovery, duplicate effects, retry exhaustion, timeouts, performance, or cost. The acceptance tests remain pending.
Our Final Verdict: When to Deploy, When to Skip
We would shortlist Restate for a controlled proof of concept, but the available evidence does not support production approval.
I like the architectural separation: the agent library handles agent behavior, while an external runtime owns durable execution. I also value the integration guide’s explicit warnings about replay determinism, error propagation, retry ceilings, streaming, and concurrency. Those constraints belong in architecture review, not in an incident retrospective.
I would deploy after acceptance testing if:
- Our workflows must survive worker loss without relying on a human to resubmit them.
- We can define useful durable boundaries inside tools.
- External mutations support stable idempotency keys or transactional deduplication.
- We can preserve runtime data and operate an appropriate availability configuration.
- We have verified error propagation, retry exhaustion, deadlines, and recovery on pinned versions.
- We have a deployment strategy for long-running invocations and compatible code.
I would hold off if:
- The application is a short, read-only request with little recovery value.
- We require parallel Pydantic AI tools without integration changes.
- We expect arbitrary streamed responses to become durable automatically.
- Our payment or provisioning APIs cannot safely tolerate ambiguous retries.
- We want to remove queues and checkpoint tables before proving which responsibilities Restate actually replaces.
Our checkpointing architecture checklist goes beyond state snapshots: we also require supervision, durable retry scheduling, session coordination, and safe deployment behavior. I would not apply that as a blanket indictment of every checkpoint-based framework; implementations and surrounding infrastructure differ.
Our final production gate is concrete: kill the worker in the ambiguous-commit window, restart the runtime with preserved storage, exhaust the retry budget, and let the caller time out while work continues. Then reconcile invocation history against the destination’s committed effects.
Until those tests pass, “durable” is an integration design—not an independently established operating guarantee. For help turning that design into an acceptance suite, talk with our infrastructure team.
Top comments (0)