An agent can survive a process restart and still lose the plot after a schema change.
The dangerous migration is not the one that crashes immediately. It is the one that lets old workers read new state, or new workers silently reinterpret old state, then continue with a plausible but wrong plan.
This article gives a small migration contract you can apply to SQLite, Postgres, or a document store.
1. Version the state, not only the database
A database migration tells you that columns changed. It does not tell an agent how to interpret a saved run.
Put an explicit schema version on every durable run, checkpoint, and pending effect:
action_state = {
"run_id": "run_123",
"schema_version": 3,
"status": "WAITING_FOR_TOOL",
"next_step": "send_report",
"pending_effect_id": "effect_456",
"updated_at": "2026-08-17T12:00:00Z"
}
Treat schema_version as part of the execution contract. A worker must reject an unsupported version instead of guessing.
2. Use expand, migrate, contract
Do not deploy a reader and writer that change meaning at the same time.
Use three phases:
- Expand: add fields without removing or changing old meanings. New workers can write both representations.
- Migrate: backfill existing records and dual-read while comparing old and new interpretations.
- Contract: stop writing the old form only after the fleet and recovery tools understand the new form.
For a field changing from a string to a structured value, keep the old field until the comparison window is complete:
def read_destination(row):
new_value = row.get("destination_v2")
old_value = row.get("destination")
if new_value and old_value and normalize(new_value) != normalize(old_value):
raise StateConflict(row["run_id"])
return new_value or parse_legacy(old_value)
The important behavior is the conflict, not the parser. A mismatch should stop automation and create a reviewable state.
3. Never migrate an in-flight effect by implication
A saved agent run may be between intent and outcome. That record needs more than a new shape:
- stable effect_id and idempotency key
- old and new schema versions
- dispatch state: NOT_SENT, SENT, or UNKNOWN
- provider lookup information for reconciliation
- the policy and credential versions used at dispatch
If a migration sees UNKNOWN, it must preserve UNKNOWN. It must not mark the effect complete because the new schema has a default value.
That distinction prevents a restart or migration from sending a duplicate email, browser mutation, or webhook.
4. Make rollback a read problem
Application rollback is unsafe if the new writer has already emitted records that the old worker cannot parse. Before rollout, answer:
- Can the previous binary read every record the new binary will write?
- Can a clean host restore the backup and replay the migration deterministically?
- What happens to a worker paused halfway through backfill?
- Can you identify records written by each version?
A useful deployment gate is a compatibility matrix:
| Writer | Reader | Expected result |
|---|---|---|
| old | old | pass |
| old | new | pass |
| new | new | pass |
| new | old | reject safely or pass by contract |
If the last row is undefined, rollback is not a plan.
5. Test the migration like an agent failure
Create a disposable copy of production-shaped state and inject these failures:
- Kill a worker during dual-write.
- Pause a backfill after 10% of records.
- Change policy between read and effect dispatch.
- Restore a backup with mixed schema versions.
- Re-run the migration twice.
- Present an old worker with a new record.
For each case, assert invariants instead of only checking that the process exits:
- no duplicate effect for one idempotency key
- no record silently downgraded to a guessed default
- every UNKNOWN effect remains reconcilable
- migration is resumable and idempotent
- old state remains available until the contract phase
A practical hosting check
If your agent runs continuously, the state directory, migration lock, and backup schedule are part of the deployment surface. A managed runtime such as always-on OpenClaw hosting on Ampere can solve where the process runs, but it does not define your schema contract or make an unsafe migration reversible.
Keep the database backup and the migration manifest together. Test a clean-host restore, then start one worker in read-only or dry-run mode before allowing effects.
The checklist
Before shipping a state change, verify:
- [ ] every durable record has an explicit schema version
- [ ] old and new readers have a defined compatibility matrix
- [ ] expand, migrate, and contract are separate releases
- [ ] in-flight effects preserve UNKNOWN and idempotency keys
- [ ] backfill can pause, resume, and run twice safely
- [ ] rollback has been tested against records written by the new version
- [ ] a clean restore includes the migration manifest and lock state
A model can produce a better plan after a migration. It cannot repair state that your runtime silently misinterpreted. Make the state contract executable first.
Top comments (0)