Updated November 5: added Rule 6 (your own send record) and a dedicated branch for the reused-key 409, after a sharp comment from a reader about the 24-hour window. Thank you.
Give an AI agent a send_email tool and, sooner or later, a customer gets the same receipt twice.
Nothing is broken when it happens. Every piece works the way it was built to:
- The HTTP call to your email API times out after 10 seconds. The email was actually accepted at second 9. Your tool returns an error, so the agent tries again.
- The model sees a tool error and re-plans. Its second attempt rewrites the subject line "to be safe" and calls the tool again.
- Your agent framework retries failed tool calls automatically. So does your HTTP client. So does the queue worker that runs the agent. Three layers of retries, none of them aware of the others.
- A long-running agent loses its context, restarts the task from its notes, and finds "send confirmation to Ana" still on the to-do list.
Each of these is a reasonable design choice on its own. Together they turn "exactly once" into "at least once", and with email, "at least once" means your customer sees it twice. Unlike a duplicated database row, you can't delete a sent email.
This post is the pattern I use to make email sends safe for agents: idempotency keys derived from the event, a strict retry policy, narrow tools, and an undo window for anything that isn't transactional. The code uses the Cresca email API (I'm building it, so that's the one I know best), but every idea works with any email API that accepts an Idempotency-Key header.
Why "just hash the arguments" doesn't work
The first fix everyone reaches for is to hash the tool's arguments and use that as the idempotency key. Same arguments, same hash, so the API ignores the duplicate. Clean.
It breaks the moment the agent is involved:
// Attempt 1, written by the model
const attempt1 = { to: "ana@example.com", subject: "Your order #1042 is confirmed", html: "<p>Hi Ana, ..." };
// Attempt 2, after a timeout. The model "improved" it.
const attempt2 = { to: "ana@example.com", subject: "Order #1042 confirmed!", html: "<p>Hello Ana, ..." };
Different arguments, different hash, two emails. A language model almost never produces byte-identical output twice, and when it re-plans after an error it is actively trying to do something different.
Hashing also fails in the other direction. If Ana places two orders that produce the same text, the hash matches and the second confirmation is silently dropped.
The key has to identify the thing that happened in the world, not the request that describes it.
Rule 1: derive the key from the event
An idempotency key answers one question: which real-world event is this email about?
order-confirmation/1042
password-reset/7f3c9a ← the id of this reset request, not of the user
invoice-paid/inv_2026_0042
weekly-digest/user_881/2026-W45
The pattern is <event>/<id>. Three properties matter:
- Stable across retries. Every attempt to send the confirmation for order 1042 produces the same key, whoever makes the attempt: your code, the framework or the model.
- New when the event is new. If Ana asks for a password reset twice, those are two reset requests with two ids, and she should get two emails. Key on the request id, not on her user id.
-
Never time-based or random.
Date.now()or a fresh UUID per attempt defeats the whole point, because each retry gets a new key.
With Cresca, a request with a key the API has already seen (with the same body, within 24 hours) returns the original response instead of sending again, and adds the header idempotent-replayed: true. Two other cases are worth knowing:
| Situation | Response |
|---|---|
| Same key, same body, within 24 hours | The first response again, with idempotent-replayed: true. Nothing is sent. |
| Same key, different body | 409 invalid_idempotent_request |
| Same key while the first request is still being processed |
409 concurrent_idempotent_requests. Retry shortly. |
| More than 24 hours later | The key has expired, and the request is treated as new. |
That second row is exactly the "model rewrote the subject" case. You get a loud 409 instead of a quiet duplicate, which is what you want, as long as your code reads that 409 as a decision rather than an error (Rule 3 shows how).
And look at the last row. The API remembers a key for 24 hours. That makes the key a cache, not a record: it stops retries that happen within a day, but an agent that picks a task back up tomorrow gets a fresh send. Rule 6 covers the part that has to live on your side.
Rule 2: don't let the model write transactional email
That 409 is useful, but you should rarely see it, because the model shouldn't be composing the receipt in the first place.
For transactional email (receipts, resets, verification codes, alerts), give the agent a tool that takes ids, not content:
// tools.js. The shape the model sees. (Field names differ slightly between
// SDKs: some call the schema `parameters`, others `input_schema`.)
export const tools = [
{
name: "send_order_confirmation",
description:
"Send the order confirmation email for one order. Safe to call more than once: " +
"the same order never gets two confirmations. Use only after the order is paid.",
schema: {
type: "object",
properties: {
order_id: { type: "string", description: "The order id, e.g. 1042" },
},
required: ["order_id"],
additionalProperties: false,
},
},
];
Your code looks up the order, builds the email from a template and owns the key. The model decides whether and when to send; it never decides what the email says or how it's identified.
This does three things at once:
- Idempotency stops depending on the model. The key comes from the order id, which your code controls.
- Hallucinated content is impossible. The model can't invent a discount, a wrong total or a link to the wrong account, because it never writes the body.
- The tool description can promise safety. "Safe to call more than once" is now true, and models do read tool descriptions. It reduces the nervous re-calling you see when a tool has failed before.
Rule 3: retry the right failures, and only those
Here's the send function behind that tool. It's plain Node 18+ with fetch, no SDK:
// send-email.js
const API = "https://api.cresca.xyz/api/v1";
const KEY = process.env.CRESCA_API_KEY; // a "sending access" key: it can send, nothing else
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
export async function sendEmail(payload, { idempotencyKey, maxAttempts = 5 } = {}) {
if (!idempotencyKey) throw new Error("idempotencyKey is required for every send");
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
let res;
try {
res = await fetch(`${API}/emails`, {
method: "POST",
headers: {
Authorization: `Bearer ${KEY}`,
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body: JSON.stringify(payload),
signal: AbortSignal.timeout(10_000),
});
} catch (err) {
// Network error or timeout: the email may or may not have been accepted.
// Retrying with the SAME key is safe, so this is exactly when to retry.
if (attempt === maxAttempts) throw err;
await sleep(backoff(attempt));
continue;
}
if (res.ok) {
const body = await res.json();
return { id: body.id, replayed: res.headers.get("idempotent-replayed") === "true" };
}
const error = await res.json().catch(() => ({}));
// In flight elsewhere (another retry layer got there first): wait and ask again.
if (res.status === 409 && error.name === "concurrent_idempotent_requests") {
await sleep(backoff(attempt));
continue;
}
// Same key, different body: an email for this event already went out (the
// model probably rewrote the content). That's a decision, not an error:
// report it as already sent so the agent stops instead of re-planning.
if (res.status === 409 && error.name === "invalid_idempotent_request") {
return { id: null, replayed: true, conflict: true };
}
// Rate limited: the server tells us how long to wait.
if (res.status === 429 && error.name === "rate_limit_exceeded") {
const wait = Number(res.headers.get("retry-after") || 1) * 1000;
await sleep(wait + jitter());
continue;
}
// Server-side trouble: back off and retry.
if (res.status >= 500) {
if (attempt === maxAttempts) break;
await sleep(backoff(attempt));
continue;
}
// Everything else is a bug in the request or a used-up monthly quota.
// Retrying can't fix either. Fail loudly.
throw new Error(`Email not sent: ${res.status} ${error.name}: ${error.message}`);
}
throw new Error(`Email not sent after ${maxAttempts} attempts`);
}
const jitter = () => Math.floor(Math.random() * 250);
const backoff = (attempt) => Math.min(1000 * 2 ** (attempt - 1), 8000) + jitter(); // 1s, 2s, 4s, 8s
And the tool handler that ties it to the order:
// handlers.js
import { sendEmail } from "./send-email.js";
import { getOrder } from "./orders.js";
export async function send_order_confirmation({ order_id }) {
const order = await getOrder(order_id);
if (!order) return { ok: false, error: `No order ${order_id}` };
if (order.status !== "paid") return { ok: false, error: `Order ${order_id} is not paid yet` };
const result = await sendEmail(
{
from: "Acme <orders@mail.acme.com>",
to: order.customerEmail,
template: {
id: "order-confirmation", // a published template, by alias
variables: { NAME: order.customerName, ORDER_ID: order.id, TOTAL: order.totalFormatted },
},
tags: [{ name: "category", value: "order_confirmation" }],
},
{ idempotencyKey: `order-confirmation/${order.id}` },
);
// Tell the model plainly what happened, including "you already did this".
if (result.conflict) {
return {
ok: true,
done: true,
note: `A confirmation for order ${order.id} was already sent. Nothing new was sent. This task is complete; do not retry.`,
};
}
return result.replayed
? { ok: true, done: true, note: `Confirmation for order ${order.id} was already sent earlier (id ${result.id}). Nothing new was sent.` }
: { ok: true, done: true, note: `Confirmation for order ${order.id} sent (id ${result.id}).` };
}
That last return matters more than it looks. When the agent calls the tool a second time, it gets a clear answer, "already sent earlier", instead of a generic success it might second-guess. A clear answer like that is what stops the retry loop at the model level: there's nothing left to fix.
Four details in the retry policy are deliberate:
- Network errors and timeouts are retried. They're the dangerous case: the email might have gone out. The idempotency key is what makes retrying them safe.
-
retry-afteris respected on 429. Retrying a rate limit immediately just burns your next request. -
A reused-key 409 is a decision, not an error. Most agent frameworks and HTTP clients sort 4xx responses into "retryable" or "not retryable" and nothing else. If
invalid_idempotent_requestreaches the model as a tool error, the model re-plans and you're back to the rewritten subject line, arriving by a different route. So it gets its own branch that ends the task with "already sent". - Other 4xx responses are never retried. A validation error or an exhausted quota will fail the same way every time, and a retry loop here is how you end up with a stuck agent and a page of logs.
Rule 4: give non-transactional email an undo window
Receipts are easy because the content is fixed. The hard case is the email the agent does compose: "draft a follow-up to everyone who attended Tuesday's webinar and send it."
You can't key that on an order id, and you probably don't want a model hitting "send" on a message to 300 people with no human in the loop. The pattern I like is to schedule instead of sending, then let a person cancel:
// The agent's "send" tool for composed email schedules 15 minutes ahead.
export async function send_followup({ to, subject, html, campaign_id }) {
const sendAt = new Date(Date.now() + 15 * 60 * 1000).toISOString();
const { id } = await sendEmail(
{ from: "Acme <hello@acme.com>", to, subject, html, scheduled_at: sendAt },
{ idempotencyKey: `followup/${campaign_id}/${to}` },
);
await notifyHuman(
`The agent scheduled "${subject}" to ${to} at ${sendAt}. ` +
`Cancel within 15 minutes: POST /emails/${id}/cancel`,
);
return { ok: true, note: `Scheduled for ${sendAt}. A human can cancel it before then.` };
}
Cresca accepts scheduled_at up to 30 days ahead, and POST /emails/:id/cancel works while an email is still scheduled or queued; once sending has started it returns 422. A canceled email doesn't count toward your monthly allowance, so a canceled mistake costs nothing.
Fifteen minutes is arbitrary. Pick a window that matches how fast someone on your team actually looks at notifications. The point is that the agent's action becomes reversible for a while, which is the property email normally lacks.
Rule 5: give the agent a key that can only send
The agent runs with whatever credentials you give its tools. For email, that should be a key that can send and nothing else.
Cresca's sending access keys can call POST /emails and POST /emails/batch, but can't read your email logs or change templates. Your undo-window tool needs a full-access key to cancel, so keep that on the human side, in the notification handler, not in the agent's tools.
The general rule: the agent's credentials should match the agent's job, not your admin's.
Rule 6: keep your own record of what was sent
The idempotency key protects you for 24 hours. Agents don't always finish within 24 hours. A long-running agent that loses its context and rebuilds its to-do list from notes the next day will find "send confirmation to Ana" still open, and by then the key has expired. Every piece behaves correctly and Ana still gets two receipts.
The fix is a record on your side that outlives the provider's window. Write it in the same database transaction as the state change that triggers the email, so the two can never disagree:
CREATE TABLE email_outbox (
event_key TEXT PRIMARY KEY, -- e.g. 'order-confirmation/1042'
status TEXT NOT NULL DEFAULT 'pending', -- pending | sent
email_id TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
sent_at TIMESTAMPTZ
);
// When the order is marked paid, record that its confirmation is owed, in the
// same transaction. If the transaction rolls back, no email is owed.
await db.tx(async (t) => {
await t.none("UPDATE orders SET status = 'paid' WHERE id = $1", [order.id]);
await t.none(
"INSERT INTO email_outbox (event_key) VALUES ($1) ON CONFLICT DO NOTHING",
[`order-confirmation/${order.id}`],
);
});
// The tool (or a worker) checks the record before it ever calls the email API.
export async function send_order_confirmation({ order_id }) {
const key = `order-confirmation/${order_id}`;
const row = await db.oneOrNone("SELECT status, email_id FROM email_outbox WHERE event_key = $1", [key]);
if (!row) return { ok: false, error: `No confirmation is owed for order ${order_id}` };
if (row.status === "sent") {
return { ok: true, done: true, note: `Already sent (id ${row.email_id}). This task is complete; do not retry.` };
}
const result = await sendEmail(buildConfirmation(order_id), { idempotencyKey: key });
await db.none(
"UPDATE email_outbox SET status = 'sent', email_id = $2, sent_at = now() WHERE event_key = $1",
[key, result.id],
);
return { ok: true, done: true, note: `Confirmation for order ${order_id} sent.` };
}
The two layers cover different gaps:
-
Your outbox row answers "was this ever sent?", for as long as you keep it. A restart a week later reads
sentand stops. -
The idempotency key covers the seconds between calling the API and writing
sent: if the process crashes after the email goes out but before the update, the retry reuses the same key and the API returns the original result instead of sending again.
That's the outbox pattern payment systems use for the same reason: the provider's deduplication window is a convenience, and your own ledger is the source of truth.
The same pattern in Python
If your agent runs on a Python stack, here's the core of send_email with httpx:
import os, random, time
import httpx
API = "https://api.cresca.xyz/api/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['CRESCA_API_KEY']}"}
def send_email(payload: dict, idempotency_key: str, max_attempts: int = 5) -> dict:
for attempt in range(1, max_attempts + 1):
try:
res = httpx.post(
f"{API}/emails",
json=payload,
headers={**HEADERS, "Idempotency-Key": idempotency_key},
timeout=10,
)
except httpx.TransportError:
if attempt == max_attempts:
raise
time.sleep(backoff(attempt))
continue
if res.is_success:
return {"id": res.json()["id"], "replayed": res.headers.get("idempotent-replayed") == "true"}
name = res.json().get("name", "")
if res.status_code == 409 and name == "concurrent_idempotent_requests":
time.sleep(backoff(attempt)); continue
if res.status_code == 409 and name == "invalid_idempotent_request":
return {"id": None, "replayed": True, "conflict": True} # already sent: stop, don't re-plan
if res.status_code == 429 and name == "rate_limit_exceeded":
time.sleep(float(res.headers.get("retry-after", 1)) + random.random() / 4); continue
if res.status_code >= 500 and attempt < max_attempts:
time.sleep(backoff(attempt)); continue
raise RuntimeError(f"Email not sent: {res.status_code} {name}: {res.json().get('message')}")
raise RuntimeError(f"Email not sent after {max_attempts} attempts")
def backoff(attempt: int) -> float:
return min(2 ** (attempt - 1), 8) + random.random() / 4
How to test that it actually works
Don't trust the pattern until you've watched it refuse to send twice. Three tests worth automating:
1. The replay test. Call your tool twice in a row for the same order and assert one email:
const a = await send_order_confirmation({ order_id: "test-1042" });
const b = await send_order_confirmation({ order_id: "test-1042" });
assert(a.ok && b.ok);
assert(b.note.includes("already sent"));
2. The timeout test. Point sendEmail at a proxy that accepts the request, waits past your 10-second timeout, then lets it through. Your code sees a timeout and retries; you should still end up with one email and replayed: true on the retry.
3. The model test. Run the agent with a tool that fails the first call on purpose ("network error"), and log every tool call it makes after that. You're checking two things: that the model's retries land on the same key, and that the "already sent" note stops it from calling a third time.
The checklist
Before you hand an agent a tool that sends email:
- Every send carries an idempotency key built from the event (
<event>/<id>), never from time, randomness or a hash of model output. - Transactional tools take ids, and your code builds the content from templates.
- The retry policy retries network errors, timeouts, 429 (after
retry-after) and 5xx, and nothing else. - A replayed send and a reused-key 409 both return a clear "already sent, task complete" result to the model, never an error.
- Your own outbox row records each send, written in the same transaction as the state change, so a restart days later still knows it was sent.
- Composed or bulk email is scheduled, not sent, with a cancel window a human can use.
- The agent's key can send, and only send.
- You've watched the replay, timeout and model tests pass.
None of this is new. Payment APIs have worked this way for years, because charging a card twice is the same kind of mistake as emailing someone twice. And that's the real argument for narrow tools: most mistakes in agent systems can be rolled back, but you can't delete a sent email. What's new is that the thing calling your API is now a model that retries creatively. Design the tool so that creativity can't cost your customer a duplicate email.
I'm building Cresca, an email platform with a transactional email API (idempotency keys, scheduling and cancel included) and AI-designed marketing campaigns in the same account. The idempotency docs cover the exact behaviour used above. If you've hit a duplicate-send bug with an agent, I'd like to hear how it happened in the comments.
Top comments (2)
Keying on the event rather than the request is the correct answer and payments arrived at it the hard way. The key belongs to the thing that happened, and the party that owns the thing is the party that owns the key. Your rule 2 is the same insight pushed one step further: the model decides whether, your code decides what, so the key never depends on anything the model wrote.
Two things I would add, both about the 24-hour window.
An API idempotency key is a cache, not a ledger. Your own fourth bullet is the case that breaks it: a long-running agent loses context, restarts from its notes a day later, finds send confirmation to Ana still open, and now the key has expired, so the request is treated as new and Ana gets her receipt twice from a correct implementation. The durable record has to live on your side. A row saying order-confirmation/1042 was sent, written in the same transaction as the order's state change, so the window is your retention rather than the provider's.
The 409 is only loud if something reads it as a decision. Most agent frameworks and most HTTP clients classify 4xx as retryable or non-retryable and nothing else, so invalid_idempotent_request becomes a tool error, the model re-plans, and you are back to the rewritten subject line arriving through a different door. That response needs its own branch that stops the task and says already sent, not an error the model gets to interpret.
I wrote up the same shape on the payment side, where the provider's dedupe window is 24 hours and the reconciliation gap opens the moment anything retries outside it: dev.to/mickyarun/your-payment-webh...
The part of your post I would make louder: you cannot delete a sent email. Everything else in agent reliability is recoverable and this class is not, which is the only real argument for narrow tools.
You're right that the key is a cache, not a ledger, and my own fourth failure mode is the case that breaks it. An agent that resumes a day later lands outside the 24-hour window, and a correct implementation sends Ana her receipt twice. I've added a Rule 6: an outbox row written in the same transaction as the order's state change, checked before any send. "Was this sent?" now depends on our retention, not the provider's. The key only has to cover the gap between the API call and marking the row sent.
The 409 point is sharp too. My code threw on
invalid_idempotent_request, which handed the model an error to re-plan around: the rewritten subject line arriving through a different door, as you put it. It now has its own branch that returns "already sent, task complete" instead of an error.And agreed on making "you can't delete a sent email" louder. It's now the closing argument. Thanks for taking the time to write this up. Reading your webhook post next.