DEV Community

Zira
Zira

Posted on

Your OpenClaw Deployment Is Not Portable Until You Rebuild It

An OpenClaw deployment can be healthy and still be impossible to recover.

The dangerous assumption is that “the process starts” means “the agent is back.” A real recovery also needs its state, credentials, schedules, browser profile, queued work, and delivery records. If any of those live only on the original machine, a host failure turns restart into archaeology.

This article gives you a small, repeatable rebuild drill. It is designed for a single always-on OpenClaw instance, but the same boundaries apply to other long-running agent runtimes.

Define the recovery contract first

Write down what must be true after a rebuild:

  • The agent starts with the expected version and configuration.
  • Persistent state is present and readable.
  • Credentials are restored without printing their values.
  • Scheduled jobs are not silently duplicated.
  • In-flight work is either resumed from a checkpoint or marked for reconciliation.
  • Browser sessions are either restored intentionally or treated as expired.
  • Delivery records let you distinguish planned, executed, and delivered.

The last two points matter most. A process supervisor can restart a process, but it cannot tell whether a message was sent before the crash or whether a browser action reached its target.

Inventory state by recovery consequence

Do not start with a backup command. Start with an inventory:

State Example Recovery treatment
Rebuildable container image, lockfile, IaC Recreate from source control
Durable SQLite database, checkpoints, run ledger Back up and restore atomically
Secret API keys, session cookies Restore through a secret store; rotate if exposed
Ephemeral PID files, temp files, caches Delete and regenerate
Ambiguous browser profile, partially sent job Restore only with an explicit reconciliation rule

For each path, record the owner, backup location, encryption status, and restore test date. If you cannot answer those four questions, the path is not part of a recovery plan yet.

Use a manifest instead of a vague backup

A manifest makes the drill inspectable. Keep it beside your deployment configuration, not inside a secret dump:

# recovery-manifest.yml
runtime_version: pin-in-source-control
persistent_paths:
  - /srv/openclaw/state
  - /srv/openclaw/checkpoints
secret_names:
  - OPENCLAW_PROVIDER_KEY
  - DELIVERY_SIGNING_KEY
reconcile:
  queued_work: replay_from_checkpoint
  unknown_delivery: hold_for_review
  browser_session: expired_until_reauthenticated
health:
  - state_readable
  - secret_present
  - scheduler_disabled_until_verified
  - outbound_delivery_tested
Enter fullscreen mode Exit fullscreen mode

The exact paths will vary. The useful part is that the policy is explicit and reviewable.

Rebuild on a clean host

Create a temporary host or isolated VM. Do not use the original machine as the restore target. The test should prove that your deployment does not depend on undocumented local state.

A minimal sequence looks like this:

# Fetch the pinned deployment definition
git clone https://example.invalid/your-openclaw-deployment.git
cd your-openclaw-deployment
git checkout <known-good-commit>

# Install exact runtime dependencies
./scripts/install-runtime.sh

# Restore durable state into a staging directory
./scripts/restore-state.sh ./recovery-archive ./staging-state
./scripts/verify-state.sh ./staging-state

# Start with scheduling and outbound delivery disabled
OPENCLAW_SCHEDULER_ENABLED=false \
OPENCLAW_DELIVERY_ENABLED=false \
  ./scripts/start-openclaw.sh --state ./staging-state
Enter fullscreen mode Exit fullscreen mode

The example.invalid URL is intentionally a placeholder. Replace it with your own repository. The important property is the order: pin, restore, verify, then start in a non-destructive mode.

For an always-on deployment where you do not want to own the host lifecycle, managed OpenClaw hosting on Ampere can be a relevant option to evaluate. That does not remove the need for a state manifest or restore test. Hosting is not a backup policy.

Reconcile before enabling side effects

After startup, inspect the run ledger and compare it with the last known snapshot:

  1. Find work whose execution state is started but not completed.
  2. Find deliveries whose status is unknown.
  3. Check whether the scheduler created new jobs while the old host was still considered alive.
  4. Verify the restored credential identity without exposing the secret.
  5. Run one idempotent test action against a sandbox destination.
  6. Only then enable scheduling and production delivery.

A useful state machine is:

planned -> started -> executed -> delivered
                    -> failed
                    -> unknown -> held_for_review
Enter fullscreen mode Exit fullscreen mode

Never turn unknown directly into retry. A retry may be correct, but only after checking the destination or using an idempotency key. Otherwise recovery can create the duplicate side effect you were trying to avoid.

Failure-injection checklist

A rebuild drill is not complete until you simulate failures:

  • Delete the restored checkpoint and confirm startup fails safely.
  • Restore a credential with the wrong identity and confirm outbound delivery stays disabled.
  • Stop the process after execution but before delivery acknowledgement.
  • Start the scheduler twice and confirm only one lease is active.
  • Corrupt one state file and verify the archive is rejected atomically.
  • Restore an old browser profile and confirm the runtime treats it as expired.
  • Replay the same request key and confirm the side effect is not duplicated.

Record the expected result and the observed result. A green health endpoint is not evidence that these cases are handled.

The 15-minute operator test

Once a month, have someone follow the runbook without asking the original operator questions. Time these checkpoints:

  • clean host provisioned
  • state verified
  • runtime started in safe mode
  • credentials verified
  • unknown work reconciled
  • sandbox delivery completed
  • production scheduling enabled

If the runbook requires tribal knowledge, update the manifest or scripts. Do not solve the gap with a longer paragraph in an incident document.

The practical definition of portable

An OpenClaw deployment is portable when a clean host can reconstruct the runtime and an operator can explain every piece of work left around the failure boundary.

That is a stronger standard than “the service is running,” but it is also testable. Pin the runtime. Separate durable, secret, and ephemeral state. Start in safe mode. Reconcile unknown work. Prove the rebuild periodically.

If you run OpenClaw in production, the follow is for you if you want more concrete control-plane tests for state, identity, recovery, and delivery rather than another list of model capabilities.

Top comments (0)