My AI agent self-healing fix was embarrassingly simple once I stopped retrying the whole thing
I finally snapped after watching an agent die on step 8 of 12, then restart from step 1 like it had learned absolutely nothing.
The workflow was not exotic:
- pull leads from a form
- enrich them
- summarize the account
- draft outreach
- push results into a CRM
- notify the team in Slack
The failure was dumb: one HTTP timeout during enrichment.
But because I had wired the workflow to retry the entire run, it:
- re-did earlier model calls
- re-fetched records I already had
- almost created duplicate CRM updates
- burned compute on work that had already succeeded
In n8n, that meant another full execution.
In LangGraph, that meant replay unless I handled state correctly.
Either way, I was paying for my own bad design.
I thought the answer would be better prompts.
It was not.
The answer was teaching the agent how to resume.
The best ai agent self-healing pattern is not a smarter prompt. It is durable execution: save explicit state, retry only failure-prone steps, and resume with a stable execution ID.
If you are building long-running agents with n8n, LangGraph, Temporal, Make, Zapier, or custom workers, this is the line between "kind of works" and "survives production."
The real problem with full-workflow retries
Full-workflow retries feel safe when the workflow is tiny.
They become a disaster when your agent spans:
- multiple LLM calls
- external APIs
- human approvals
- writes into systems that really do not like duplicates
My original logic was painfully common:
try {
await runWorkflow(input)
} catch (err) {
await runWorkflow(input)
}
That is not resilience.
That is replay.
And replay causes a bunch of avoidable problems:
- successful LLM calls get repeated
- tool calls hit external systems again
- token usage and latency go up for no reason
- logs get noisier on every retry
- one flaky step turns into a full pipeline failure
If Clearbit times out, or Salesforce rate-limits you, or a webhook returns 502, a better system prompt does not help.
This is why I stopped asking:
"How do I make the model less fragile?"
And started asking:
"What already succeeded, and how do I avoid doing it again?"
The fix: stable execution IDs + explicit state + step-level retries
What finally worked was boring in the best way.
I gave every run:
- a stable
execution_id - explicit persisted state after each meaningful step
- hard retry boundaries around the steps that actually fail in production
The state shape was not fancy. It looked like normal application plumbing.
{
"execution_id": "lead_9f3d7c2a",
"current_step": "summarize_account",
"lead_id": "lead_123",
"artifacts": {
"enrichment": { "company": "Acme", "employees": 120 },
"summary": "B2B SaaS company expanding sales ops"
},
"side_effects": {
"crm_upserted": false,
"slack_notified": false
},
"idempotency_keys": {
"crm_write": "crm:lead_123:v1",
"slack_post": "slack:lead_123:v1"
},
"last_error": {
"step": "enrichment",
"message": "HTTP timeout"
}
}
That one change killed most of the chaos.
The new rule became:
- retry model or HTTP calls at the step level
- never repeat a side effect unless the step is idempotent
- resume from the last completed checkpoint using the same execution ID
That is what self-healing looked like in practice.
Not the agent becoming smarter.
The agent becoming less forgetful.
What step-level retry logic actually looks like
Here is the pattern in plain TypeScript.
type WorkflowState = {
executionId: string
currentStep: string
artifacts: Record<string, unknown>
sideEffects: {
crmUpserted: boolean
slackNotified: boolean
}
}
async function runLeadWorkflow(state: WorkflowState) {
if (state.currentStep === "start") {
state.artifacts.enrichment = await retryStep(() => enrichLead(), 3)
state.currentStep = "enriched"
await saveState(state)
}
if (state.currentStep === "enriched") {
state.artifacts.summary = await retryStep(() => summarizeAccount(state.artifacts.enrichment), 2)
state.currentStep = "summarized"
await saveState(state)
}
if (state.currentStep === "summarized" && !state.sideEffects.crmUpserted) {
await upsertCRM({
data: state.artifacts.summary,
idempotencyKey: `crm:${state.executionId}`
})
state.sideEffects.crmUpserted = true
state.currentStep = "crm_written"
await saveState(state)
}
if (state.currentStep === "crm_written" && !state.sideEffects.slackNotified) {
await postToSlack({
message: "Lead processed",
idempotencyKey: `slack:${state.executionId}`
})
state.sideEffects.slackNotified = true
state.currentStep = "done"
await saveState(state)
}
}
async function retryStep<T>(fn: () => Promise<T>, maxAttempts: number): Promise<T> {
let lastError: unknown
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
try {
return await fn()
} catch (err) {
lastError = err
if (attempt < maxAttempts) {
await sleep(500 * attempt)
}
}
}
throw lastError
}
The important part is not the syntax.
It is the boundary.
Each expensive or failure-prone step gets its own retry policy.
Each successful step gets checkpointed.
External writes are protected with idempotency keys.
Resume, don’t restart
The key mental shift is this:
Separate workflow state from step execution.
Once I did that, the tooling got much easier to reason about.
LangGraph: reuse thread_id and persist graph state
LangGraph gets a lot better once you stop treating every run like a fresh conversation.
If you use a checkpointer and keep the same thread_id, you can resume from existing state instead of replaying the graph from the top.
Conceptually:
from langgraph.graph import StateGraph
from langgraph.checkpoint.memory import InMemorySaver
checkpointer = InMemorySaver()
graph = builder.compile(checkpointer=checkpointer)
config = {"configurable": {"thread_id": "lead-123"}}
graph.invoke({"lead_id": "lead-123"}, config=config)
# later, after failure or human review
graph.invoke({"resume": True}, config=config)
If you pause with interrupt() for human review, you can continue with Command(resume=...) instead of rerunning the whole graph.
That is a much better model for real automations.
Temporal: retry Activities, not the whole business process
Temporal is probably the cleanest expression of this pattern.
The Workflow keeps deterministic state.
Activities do the flaky work:
- HTTP calls
- database writes
- LLM tool invocations
- queue operations
That split matters.
If the enrichment API times out, retry the enrichment Activity.
Do not rerun the entire lead-processing workflow.
Pseudo-example:
// workflow.ts
export async function leadWorkflow(input: LeadInput) {
const enrichment = await enrichLeadActivity(input)
const summary = await summarizeLeadActivity(enrichment)
await upsertCrmActivity(summary)
await notifySlackActivity(summary)
}
Then attach retry policy where failure actually happens.
const activities = proxyActivities<typeof import('./activities')>({
startToCloseTimeout: '2 minutes',
retry: {
maximumAttempts: 3
}
})
That is the whole point.
Retry the unstable edge, not the entire process.
n8n: manual, but still worth doing
n8n is great for orchestration.
But if you want resumability, you have to design it intentionally.
Useful pieces:
-
execution.retryOfto detect retries - Error Trigger for remediation flows
- external state in Postgres, Redis, Airtable, or another store keyed by execution ID
- branching logic to skip completed steps
A practical pattern in n8n is:
- create an
execution_idat the start - save progress after each major step
- on retry, load prior state
- skip steps that already completed
- only rerun the failed node or subflow
That gives you something much closer to continuation instead of "start over and hope."
Quick comparison: restart vs resume
| Approach | What happens after step 8 fails? |
|---|---|
| Full workflow retry | Steps 1-7 run again, model calls repeat, external writes may duplicate |
| Step-level retry + checkpoints | Only step 8 retries, prior outputs are reused, external writes stay protected |
The cost problem is not subtle
This part matters a lot if you run agents all day.
Every full replay multiplies model calls.
If your workflow has six LLM steps and one API timeout near the end, a naive retry can turn one failure into:
- six extra inference calls
- repeated tool work
- more queue time
- more logs to untangle
- more chances to duplicate side effects
Under per-token pricing, this gets expensive fast.
And even if you are on flat-rate AI compute, wasted retries still hurt throughput.
You are burning capacity on work that already succeeded.
That is one reason I care about this pattern so much.
If you are running agents in n8n, Make, Zapier, OpenClaw, or custom frameworks, step-level retries are one of the fastest ways to make the system both cheaper and less annoying.
And if your stack still points at the OpenAI API with usage anxiety hanging over every replay, it is worth looking at a drop-in alternative like Standard Compute.
Standard Compute gives you unlimited AI compute for a flat monthly price and works with existing OpenAI-compatible SDKs and HTTP clients. So when you fix your retry design, you are not also stuck babysitting per-token costs every time an automation gets noisy.
That does not replace durable execution.
It just means your cost model stops fighting your architecture.
The pattern I would use again from day one
If I were rebuilding that workflow today, I would do this immediately:
- Assign a stable execution ID at the start.
- Persist state after every expensive or meaningful step.
- Store outputs in a structured state object, not scattered logs.
- Put idempotency keys on every external write.
- Retry only failure-prone steps like LLM calls, HTTP requests, and queue operations.
- Resume from the last checkpoint instead of replaying the workflow.
- Send true failures to a human or remediation flow instead of blindly looping.
If you want a minimal checklist for production agents, use this:
# production sanity checklist
[ ] stable execution id
[ ] persisted checkpoint after each major step
[ ] idempotency key on every external write
[ ] step-level retry policy
[ ] resume path tested
[ ] duplicate-write protection tested
[ ] human escalation path for hard failures
My actual takeaway
My agent did not need a more inspirational prompt.
It needed:
- memory
- boundaries
- permission to continue where it left off
Once I stopped retrying the whole thing, the self-healing part was embarrassingly simple.
If your agent keeps "recovering" by replaying everything, it is not self-healing.
It is just forgetting more aggressively.
Top comments (0)