Your Agent Backup Is Not a Recovery Plan Until You Test the Restore

An AI agent can have backups and still be unrecoverable.

A backup proves that bytes were copied somewhere. Recovery proves that a clean runtime can use those bytes to resume safely, without replaying an irreversible action or losing the identity and policy context that made the run valid.

For long-lived agents, I now treat restore as a four-part contract:

1. State: conversations, schedules, leases, idempotency keys, tool outcomes, and configuration.
2. Identity: which account, workspace, tenant, and credential references the run is allowed to use.
3. Policy: approvals, tool allowlists, rate limits, and versioned rules.
4. Recovery position: the last known safe point, plus work whose outcome is still unknown.

If one of those is missing, a restore can look healthy while producing duplicate or unauthorized side effects.

The backup boundary

Do not start with a database dump command. Start with an inventory of what the agent needs to make a safe decision:

Category	Example	Restore question
Durable state	run records, messages, schedules	Can the process reconstruct the next state transition?
Execution evidence	tool request, provider ID, outcome	Can it distinguish done from unknown?
Identity references	tenant, workspace, credential name	Does the reference still resolve to the same boundary?
Policy versions	tool rules, approval state	Which policy is rechecked at resume time?
Runtime config	model, endpoint, feature flags	Can config drift be detected before execution?

A useful rule is: backup decisions, not just data. If the system stores only the prompt and the final answer, it cannot prove whether a tool call was sent, accepted, timed out, or merely never attempted.

A minimal restore drill

Run this against a disposable environment, not production. The test should be repeatable from an empty host or container.

1. Stop the agent and record the last durable event.
2. Create a clean runtime with no local cache, temporary files, or implicit credentials.
3. Restore the declared state and configuration manifest.
4. Start in read-only or safe mode.
5. Reconcile every in-flight operation before allowing new side effects.
6. Verify one known completed operation, one never-dispatched operation, and one operation whose outcome is unknown.
7. Only then enable the normal executor.

Here is a compact shell skeleton for making the clean-room property explicit:

tmpdir=$(mktemp -d)
trap "rm -rf $tmpdir" EXIT
mkdir "$tmpdir/state" "$tmpdir/runtime"
./agent restore --input backup.tar --state-dir "$tmpdir/state" --config manifest.json
./agent verify --state-dir "$tmpdir/state" --mode safe
./agent reconcile --state-dir "$tmpdir/state" --provider-lookups
./agent resume --state-dir "$tmpdir/state" --allow-side-effects

The exact commands will differ, but the sequence matters. A restore that jumps directly from unpacking files to sending messages has no checkpoint at which unknown work can be contained.

Failure injection checklist

A recovery drill should deliberately break the assumptions that normal backups hide:

• Restore with one database page or object missing.
• Rotate a credential between backup and restore.
• Change the tool policy version before resume.
• Expire a lease while the agent is offline.
• Replay a request whose provider already accepted it.
• Restore a schedule whose wall-clock time has passed.
• Start two restored workers at the same time.
• Make the provider return a timeout after accepting the request.

For each case, record the expected result. “The agent continues” is not precise enough. The result should say whether it pauses, quarantines the operation, looks up a provider ID, requires approval, or safely retries with the same idempotency key.

What to measure

A successful restore drill should leave evidence:

• time from clean host to safe mode
• percentage of state objects validated
• number of operations classified as complete, not-started, or unknown
• number of side effects attempted during the drill
• policy and credential mismatches detected
• whether two workers produced one logical effect

The most important metric is not restore speed. It is unreconciled uncertainty. If the drill finishes with unknown operations that nobody can explain, the backup may be intact but the recovery contract is incomplete.

For always-on OpenClaw deployments, the hosting choice is part of this test. A useful platform should make persistent state, credential boundaries, clean rebuilds, and restore verification explicit rather than leaving them as undocumented machine-local assumptions. managed OpenClaw hosting on Ampere is one option to evaluate for that operational boundary, not a substitute for running the drill yourself.

The practical definition of recoverable

I would not call an agent recoverable because its process restarts or its backup job is green. I would call it recoverable only when a clean runtime can:

1. reconstruct durable state;
2. revalidate identity and policy;
3. classify in-flight work, including unknown outcomes;
4. prevent duplicate side effects; and
5. produce an auditable record of the resume decision.

Run that test before you need it. The restore procedure is part of the agent, and an untested procedure is only a hope written in shell commands.

If you build AI agents that run beyond a single terminal session, follow for practical failure tests around state, delivery, identity, and recovery rather than model demos alone.