DEV Community

Manoj Kumar
Manoj Kumar

Posted on

Your automation ran twice. Here's how to make n8n and Make workflows safe to retry

Every automation builder meets this bug eventually. A form submission comes in, the workflow creates a CRM contact, sends a welcome email, and makes a Drive folder. Then someone messages you: "Why does this lead exist three times, and why did they get two welcome emails?"

Nothing was "broken". A webhook was delivered twice, or an API call timed out after it had already succeeded, and the retry did the work again. Retries are a feature. Duplicates are what you get when the workflow isn't built to be retried.

Here is the pattern I use when I build n8n and Make workflows so a run can happen twice, or fail halfway and be replayed, without doing anything twice.

Why duplicates happen

Three causes cover almost every case I've seen:

  1. At-least-once delivery. Most webhook senders retry if they don't get a fast 2xx. If your workflow does slow work before answering, the sender gives up waiting and sends the same event again.
  2. Timeouts after success. Your HTTP call to the CRM times out, but the CRM actually created the record. Your retry creates a second one.
  3. Manual replays. Someone re-runs a failed execution from the top, and every step before the failure runs again, including the ones that already worked.

You can't stop any of these. You can make them harmless.

Rule 1: give every run a stable key, before anything else

The first thing the workflow does is decide what this event is, as a key that will be identical on every delivery of the same event:

  • the form tool's submission or response ID,
  • the payment provider's event ID,
  • or, if there's nothing better, a key built from fields that identify the event (for example the email plus the submission timestamp the form sent, not the time your workflow received it).

Never use "now" or a random UUID generated inside the workflow. Those change on every retry, which is exactly what you don't want.

Rule 2: record the key, and check it, before calling anything external

Keep a small run log: a data table, a Google Sheet, an Airtable base, or a Postgres table. One row per key:

key status crm_id folder_id email_sent_at error

At the start of the run:

  • If the key is new, insert it with status received.
  • If the key exists and is done, stop. This is a duplicate delivery. Answer the webhook with 200 and exit.
  • If the key exists and is partly done, continue from where it stopped (Rule 3).

In n8n this is a lookup node followed by an IF. In Make it's a search module followed by a router with a filter on each route. Either way, answer the webhook quickly (n8n's Respond to Webhook node, with the Webhook node's Respond option set to "Using 'Respond to Webhook' Node"; Make's Webhook response module) so slow steps later don't trigger sender retries.

Rule 3: save each external ID the moment you get it

After every create call, write the ID it returned to the run log before the next step:

create CRM contact   -> save crm_id
create Drive folder  -> save folder_id
send welcome email   -> save email_sent_at
Enter fullscreen mode Exit fullscreen mode

On a replay, each step checks its column first. crm_id already set? Skip the create and reuse the ID. email_sent_at set? Don't send again. This turns a half-finished run into a resumable one, and it's the difference between "replay is safe" and "replay is how we got three contacts".

Rule 4: prefer upsert and natural keys over blind create

Many APIs let you avoid the problem at the source:

  • CRMs keyed by email. Search by email first, then update or create. HubSpot's contacts API also has a batch upsert endpoint that takes idProperty: "email", so one call either updates the existing contact or creates it. One caveat from HubSpot's docs: when idProperty is email, partial upserts are not supported, so send every property you want set.
  • Idempotency keys. Stripe accepts an Idempotency-Key header on POST requests. On API v1, sending the same key twice returns the first result instead of a second charge (keys are kept for at least 24 hours). Pass a non-personal key here: your run key plus the step name, hashed if the run key itself contains an email or other identifier. Stripe asks you not to use personal data as the key.
  • Folders and files. Put the run key in the folder name and search for it before creating. A second create is then easy to detect.

The search-then-create pattern has a small race window if two deliveries arrive at the same moment. Rule 2's run log closes most of it. If it really matters, make the key column unique in your database so the second insert fails and that run stops.

Rule 5: retry only what is safe to retry

Both tools have built-in retries, and both should be used with care:

  • n8n: the per-node Retry On Fail setting is fine for reads and for steps that are already idempotent (upserts, steps with an idempotency key). For blind creates, add the check from Rule 3 first. Point the workflow's error workflow (an Error Trigger in a separate workflow) at something that records the failure against the run key, so a person can see it. The Error Trigger only fires on automatic runs, not on manual tests.
  • Make: error-handler routes decide what happens on failure. The current handlers are Resume, Skip, Retry, Commit and Rollback (these replace the older Ignore / Break names you may still see in older write-ups). Retry, with Store incomplete executions turned on in the scenario settings, keeps the failed run so you can retry it later, which is only safe if the steps before it check the run log.

Retry with backoff on 429 and timeouts. Don't retry 400-type validation errors: they'll fail the same way forever, so mark the row failed with the reason and move on.

Rule 6: make failures visible and replayable

A run that fails silently is worse than one that fails loudly. Every permanent failure should end up as:

  • a row with status failed, the error message, and enough of the payload to replay,
  • one alert to an internal channel (not the customer),
  • and a documented way to replay from that row, which is safe because of Rules 2 and 3.

Test it on purpose

Before you hand the workflow over, prove it:

  1. Send the same test event twice. Expect one record, one email.
  2. Kill the run after the CRM step, then replay. Expect the CRM step to be skipped.
  3. Force a 429 or timeout on the CRM call. Expect a retry and still one record.
  4. Send a bad payload. Expect a visible failed row and an alert, not a crash loop.

Put the results in the handover notes. People rarely ask for this, but it's the part that saves them from cleaning up duplicates three months later.

The short version

Key every run by the event, not the clock. Record the key before any external call. Save each external ID as soon as you get it. Use upserts and idempotency keys where the API offers them. Retry only safe steps. Make failures visible and replayable.

None of this is n8n- or Make-specific. It's the same at-least-once delivery problem every queue and webhook system has. Low-code tools just make it easy to forget, because the happy path works fine in testing.

Top comments (6)

Collapse
 
hayrullahkar profile image
Hayrullah Kar •

Rule 3's per-step columns let you reason about a replay row by row. One state in Rule 2 needs a guard. A row that is partly done because another run is still working on it looks the same as a row whose run died. If the sender retries while the first run is still waiting on the CRM, the second run takes the 'continue from where it stopped' branch, and both create the contact. I'd write a started_at, or a lease_until, when a run claims the row, and only resume a partial row when that time is older than the longest a run can take. A newer partial row gets a 200 and an exit, and the run that is still going finishes the job.

Collapse
 
fun_fun_19313eced54af3c68 profile image
SoloAgentFlow •

Rule 2 has one sneaky race: if the sender retries fast, two runs can both do the lookup, both see 'key is new', and both continue. With a Sheet or Airtable it's hard to fully close that gap. With Postgres, let the insert be the check: put a unique constraint on the key and use INSERT ... ON CONFLICT DO NOTHING RETURNING id. Only the run that gets a row back does the work; the other one just answers 200 and exits.

Collapse
 
challan116ux profile image
challan116-ux •

"Timeout after success" is the cause I'd put in bold — it's the one that survives code review because nothing in the logs looks like an error. EDI has lived with that exact failure for decades: the file arrived, the acknowledgment didn't, so the sender quite reasonably sent it again. The fix is the same one you describe — make the receiving side able to recognize the second arrival as the same work — because you will never get every sender to stop retrying.

Collapse
 
krusenas profile image
Karolis •

Timeouts after success bit us. A form sender gave up at about 5s and re-sent, so every slow Drive call turned into a double. I return the HTTP 200 as soon as the webhook arrives, before the slow Drive upload, and that cut duplicates way down before any duplicate cleanup mattered. fwiw I built Webhook Relay and it stores each incoming webhook, then retries failed deliveries with backoff.

Collapse
 
delayrun profile image
delay.run •

Rule 3 is the one I wish every workflow builder knew. Saving crm_id right after the create is what turns "replay from the top" from scary into boring. The gap I keep running into is a timeout after success on that very first create: the CRM made the contact, the ID never came back, so the run log is empty and the replay creates a second one. Searching by the natural key before retrying that one step, like your Rule 4, is the only fix that has held up for me.

Collapse
 
kevinhall profile image
Kevin Hall •

with paid enrichment a replay from the top bills every step again, so we save each paid result against the row id and skip rows that already have one