DEV Community

marcelo lang
marcelo lang

Posted on

Why your n8n workflow can lose items and still show a green checkmark

#n8n #automation #devops

I spent a while chasing a bug where a scheduled n8n workflow was dropping items — not crashing, not erroring in a way I noticed, just… quietly not processing some of them. The execution list looked healthy. Re-running did nothing. Here's what was actually happening, because I think a lot of people have this bug and don't know it.

The setup

A really common pattern: you keep a list of "already processed" ids in workflow static data so you don't re-send things. So the flow is:

mark id as done  →  do the work (send message / call API / write row)  →  done
Enter fullscreen mode Exit fullscreen mode

Intuitive. Also wrong, and here's the specific reason.

The cause

n8n persists static data on the workflowExecuteAfter hook, and that hook fires whether the run succeeded or failed. I went and read it in the source to be sure, then reproduced it: a workflow that marks an id done and then throws still has the "done" write persisted.

So when the work step fails:

  • the id is marked done ✅
  • the work never happened ❌
  • the run shows as failed, so you think you'll just retry it
  • but on retry, the id is already "done" → it's skipped forever

The item is gone. Nothing tells you. The counter of "processed" even looks right.

The fix is ordering, not code

Commit after the side effect succeeds, per item:

check if done  →  do the work ✅  →  only now mark it done
Enter fullscreen mode Exit fullscreen mode

Concretely, in a Code node it's just moving the write to the end:

// runs only AFTER the side effect succeeded, once per item
const state = $getWorkflowStaticData('global');
state.done = state.done || [];
const key = String($json.id);
if (!state.done.includes(key)) state.done.push(key);
return $input.item;
Enter fullscreen mode Exit fullscreen mode

Two details that matter in practice:

  1. Per item, not per batch. If you process 10 items and mark the whole batch done at the end, one failure in the batch re-does the other 9 on the next run (and re-sends 9 messages). Commit each item right after its own side effect.
  2. It's at-least-once, not exactly-once. If the process dies between the side effect and the commit, that one item repeats. That's the right trade for most work — a visible duplicate beats a silent loss — but know that you're choosing it.

Why this one is worth your attention

The thing I keep coming back to: this isn't an exotic edge case. It's the default way most people wire dedup/idempotency in n8n, and it fails in exactly the situation you built the dedup for (something downstream breaks). The green checkmark is the trap.

If you have a scheduled workflow with a dedup list right now, go look at the order of those two steps. It takes thirty seconds and it's either fine or it's been quietly eating your data.

I write these up as I hit them while building production n8n workflows. This one, plus five other error-handling patterns (smart retry, cooldowns, dead-letter, circuit breaker, graceful degradation) — each with the experiment and the number behind it — ended up in a kit I put together: Error-Handling Starter Kit for n8n. There's also an invoice-processing template built on the same idea: validate the output, don't trust it.

Top comments (0)