One Bad Row Killed the Whole Batch: Partial-Failure Handling for n8n Loops
This is part of my build-in-public series on running LLM workflows in n8n in production. Everything here is from real templates I sell and support — the mistakes are mine, the fixes are tested.
The batch failed at row 47. Rows 48 to 500 never ran
A nightly workflow of mine processed a batch of new leads: enrich each one, score it, write it to the CRM. One night a single lead had a malformed phone field, the enrichment node threw, and the execution stopped. The other 453 leads sat unprocessed until the next morning, when a client asked why half the day's leads were missing.
Nothing about this was exotic. A batch loop that stops on the first error is the default behavior in most workflow tools, and it turns one bad record into a silent outage for every record behind it. The error itself was visible — the execution log showed the failure. What was invisible was the cost: hundreds of good records that never got processed because they happened to queue behind a bad one.
The fix is not "validate better." You will never validate away every malformed record from a source you do not control. The fix is to decide, per item, what failure means — and to make that decision before the loop runs, not after it stops.
Three rules I now build into every batch loop
First, isolate failures to the item that caused them. In n8n this means the per-item work runs where an error cannot kill the loop: a sub-workflow executed per item, or an error-handling branch that catches the failure and returns it as data instead of throwing. The loop itself never sees an exception — it sees a result, either success or a structured failure record. Once a bad row produces a row in a failure list instead of a stopped execution, the batch always completes, and "complete" stops being a lie.
Second, collect failures into a dead-letter list, not a log line. A failed item goes into a table with the item's business key, the error, and the raw payload — everything needed to reprocess it without re-running the whole batch. My rule: a batch that ends with failures still reports success for the good items and a separate count for the failed ones, and the failed list is the input to a retry workflow, not a note for someone to read someday. If a failure cannot be retried from the list alone, the list is missing a field, and that is the bug to fix.
Third, report four numbers, not one status. Processed, failed, skipped as duplicates, and retried. When failed jumps from zero to one, that is a data problem worth one look. When failed jumps to forty, that is a schema change upstream, and I want to know before the client does. A batch that reports only "done" hides exactly the distinction between "everything worked" and "everything after row 47 never ran" — which is the distinction the whole report exists to make.
The retry rule that keeps this honest
Retrying the whole batch is how a partial failure becomes a duplicate problem: the good items run twice. So retries run against the dead-letter list only, keyed by the business id, with the same dedupe check the main loop uses. A retried item that succeeds is marked done; one that fails again stays on the list with its new error. Nothing ever re-enters the main batch, and the main batch never re-processes a good item because a bad one showed up late.
The principle is simple: a batch loop's job is to finish the batch, not to stop at the first surprise. One bad row should cost you one row, one entry in a list, and one retry — never the 453 good rows standing behind it.
I'm building AI Automation Lab — production-ready n8n workflow templates. The free AI Hook Generator shows how I structure real workflows; my open-source templates live at github.com/zhp910318/n8n-ai-automation-lab.
Top comments (6)
I hit the other side of this with email. Each send was its own queued job, so one bad address never held up the rest. But an address with a cut-short domain, like
name@hotmail, kept getting retried until the job's window ran out, because the provider answered with a validation error and my job retried it like any other failure.The fix was sorting the errors. An address the provider calls invalid now fails once, gets marked bounced and is skipped after that. Rate limits and outages still retry.
That is the exact classification the batch version of this needs too, and your email case shows why the classes have to be three, not two. A provider validation error is permanent: fail once, mark bounced, never retry. A 429 or 5xx is transient: retry with backoff. But the third class is the one that burns people — accepted-then-dropped-response, where the send may already have happened and the response never arrived. That one must never auto-retry; it goes to review, because retrying it is how you send the same email twice.
Two practical notes from doing this in batch loops. First, the retry budget should be per class, not just per job: your cut-short domain kept retrying until the window ran out because the budget was time-based, and a permanent error spent the whole budget. Permanent errors should spend zero retries. Second, mark the bounce per address plus reason, not as a hard delete — if the user corrects the address later, you want a path back in, and the bounce record is what tells you the correction is new rather than another retry of the same bad input.
catching the error as data inside the loop instead of letting it throw is the only way that survives untrusted input. returning a structured result object with an explicit error field for that index lets the downstream aggregator write out the 453 good rows and dump the bad one to a dead letter table with the payload attached, so you can inspect row 47 without the batch grinding to a halt.
Exactly — and the detail that makes error-as-data work is that the result object has to carry the item's business key, not just its position in the batch. An index alone is useless once the aggregator reorders or retries: when row 47 fails, the dead-letter row needs the lead's own id plus the raw payload, so the retry workflow can reprocess that one lead without re-running the batch or re-fetching from the source.
One second-order trap worth naming: the aggregator itself must not throw on a malformed failure record. If the error object for one row is missing a field and the aggregator crashes on it, you have rebuilt the same bug one layer up — one bad failure record kills the 453 good results. I validate the failure record's shape separately from the item's, so a failure can never fail the aggregation.
one bad row shouldn't eat 453 good ones.
That is the whole point in one line. The part that stings is the silence: a batch that stops at row 47 usually still reports "failed" for the one row, but nothing reports the 453 that never ran. So the fix that matters most is arguably not the isolation itself but the report — processed, failed, skipped, retried — because "failed: 1" and "failed: 1, never attempted: 453" are two very different mornings.