Last week I posted my retry budget rule: retry once with feedback, then route to a human. A reader replied with a sharper axis than the one I'd used. I was classifying failures — transient, repairable, deterministic. He said the real question is different: does this attempt leave something behind you can't take back?
He was right. So I rebuilt the policy around that question. This post is the implementation.
The tag, not the failure
The old flow asked "what failed?" and picked a response. The new flow asks "what can the failed node do?" before it asks anything about the failure. Every node in my production workflows now carries one of two tags:
- pure — the output goes nowhere but the next node. Validation, parsing, an LLM completion that only gets stored in context.
- effectful — the node touches the outside world. HTTP POST/PUT, Send Email, a payment call, a database insert, a file write.
Effectful nodes get a second field: idempotent — true if the call carries an idempotency key (or is naturally idempotent, like a PUT with a fixed resource id), false otherwise.
The tag is declared once, in a config Code node at the top of the workflow:
const NODE_POLICY = {
"Validate Lead": { kind: "pure" },
"Draft Outreach": { kind: "pure" },
"Score Fit": { kind: "pure" },
"POST /crm/leads": { kind: "effectful", idempotent: true, key: "crm-lead-{{ email }}" },
"Send Payment Link": { kind: "effectful", idempotent: false },
};
return [{ json: { policy: NODE_POLICY } }];
The Classify node then looks up the failed node's name in this table before it reads the error at all. The policy is fully determined by two lookups:
const failed = $input.first().json.nodeName;
const pol = $("Policy Config").first().json.policy[failed] ?? { kind: "pure" };
let action = "review"; // default: human looks at it
if (pol.kind === "pure") {
// the whole old three-bucket policy lives here:
// transient -> backoff retry, repairable -> one feedback retry, deterministic -> review
action = classifyByError(err); // retry_once | retry_backoff | review
} else if (pol.idempotent) {
action = "retry_identical"; // byte-identical, same idempotency key, exactly once
} // else: effectful, no key -> review, no retry at all
Three lines, no ambiguity. The old error-classification code didn't disappear — it moved inside the pure branch, where the retry budget post still applies exactly as written.
Why the tag is load-bearing
The failure class of an error can lie to you. A 504 from an effectful node is indistinguishable from "the server never saw it" and "the server saw it, response was lost." No amount of error classification resolves that; only the node's properties decide what you may do next. A pure node's failure is always recoverable by definition — nothing left the process. An effectful node's failure is only recoverable if someone on the server side agreed to collapse duplicates.
That makes the tag load-bearing in a way the failure class wasn't: a changed request against an effectful node isn't a retry, it's a second instruction the server can't distinguish from intent. So the rule for effectful nodes without a key is the strictest one in the whole SOP — zero retries, straight to the review queue, even if the error looks transient. The review queue is cheaper than a duplicate charge.
What the logs say so far
Two weeks on the new tags, ~180 failures logged across my workflows:
- 164 hit pure nodes → the old budget policy handled them: ~70% first-attempt success on repairable failures, same curve as before. Unchanged, as expected.
- 13 hit effectful nodes with idempotency keys → all byte-identical single retries, 12 succeeded, 1 deterministic failure went to review.
- 3 hit effectful nodes without keys → zero retries, all three went to review. One was a rate limit I'd previously have retried; it waited 20 minutes and went through manually. Zero duplicates. The other two were schema changes no retry would have fixed.
The sample is small, but the shape is what matters: the expensive mistakes — duplicate charges, duplicate emails — live entirely in the effectful-no-key bucket, and that bucket is now structurally incapable of retrying. The protection comes from the table, not from the person on call remembering the rule at 3am.
How it fits the SOP
This is one page of the same SOP as the guard, the crash boundary, the sweeper, and the retry budget. The pages compose in a fixed order: the tag decides whether a retry is allowed at all; the failure class decides which kind of retry, for pure nodes only; the budget decides how many; the weekly tally decides whether the policy itself needs changing. Each page answers exactly one question, and none of them re-answers another page's question.
What tags would your nodes carry? I'd especially like to hear from anyone running LLM steps at real volume — where does your first duplicate hurt?
Top comments (0)