Your agent's worker sends a payout request, and then the process gets killed before the response comes back. The supervisor restarts it. The agent ...
For further actions, you may consider blocking this person and/or reporting abuse
Short answer: from the business operation, not from the retry layer. If the idempotency key is generated per-request, retries create new keys and duplicates walk right in. We use the natural key of the operation itself — the provider's event id for webhooks, the transaction id for payments — and enforce uniqueness at the database level with a constraint. The app checks first (fast path), but the constraint is the referee: two events in the same second, retries, double-submits — all converge on the index. The rule: idempotency enforced by the storage layer, or it isn't idempotency.
That storage-layer referee is the part most agent stacks skip. The app check is a convenience; the unique constraint is what stops two workers that both thought they were first.
One place this gets awkward with agents: the natural key sometimes does not exist until after the side effect. A payment transaction id arrives with the provider response, and a webhook event id arrives later. If the agent has to call before either id exists, you still need a client-minted intent key for that first attempt, then bind the provider id onto the same row when it shows up. Otherwise the first call has nothing to collide with.
I have also seen teams index only the provider event id and then lose the case where two different agent intents map onto the same upstream event after a rematch. Pairing the intent key with the provider id, and refusing a second intent that claims an already-bound provider id, closed that hole.
Curious whether your constraint is on the provider id alone, or on a composite that also names the business operation.
Ours is the provider id alone — because in the flows we've hardened, the provider id is the business operation. One Paddle event id = one "grant credits for this payment" — there is no case where two different business intents share the same upstream event. So the constraint on event_id alone is both the idempotency key and the operation identity.
Your intent-key pattern is the right answer for the harder case we haven't hit yet: an agent that must act before any provider id exists. Minting a client key for the first attempt, then binding the provider id onto the same row when it arrives — that collapses "at-least-once with dedupe" into one storage fact. Noted for when our agents start initiating payments instead of just receiving them.
The moment two intents could map to one upstream event, we'd move to a composite exactly like you described. The constraint tells you when you've outgrown it — that's the nice property of enforcing at the storage layer: the schema becomes the audit of your assumptions.
"The schema becomes the audit of your assumptions" is a line I am going to borrow. One Paddle event id mapping to one grant is a clean case, and the constraint on event_id alone is the honest choice there.
One thing worth checking before your agents start initiating payments: what happens when the grant step fails after the unique insert succeeds. If the row commits and the credit grant errors, a retry hits the constraint and silently skips the grant. The usual fixes are one transaction around the insert and the grant, or a state on the row that only moves to done once the grant lands.
Which would you reach for first in that flow, a single transaction or a state column? If your agents do start initiating payments I would like to hear how it goes. Email me at 1@christophersellers.com any time.
Depends on what the "grant" is — and the distinction is the whole answer. If both the insert and the grant are database rows, one transaction, always. The failure mode you describe (row commits, grant errors, retry silently no-ops) can't exist inside a single transaction — it either all lands or none of it does, and the constraint never meets a half-written state.
But the moment the grant is an external side effect — an API call, an email, anything outside the database — a transaction can't help you, because you can't roll back someone else's system. That's when I reach for the state column: inserted → grant_pending → done, with the retry worker only picking up rows stuck in pending. It's the outbox pattern wearing a trench coat, and it's the only honest answer for "at least once" against an external world.
Our credits are DB rows, so today: transaction. The day our agents call an external API to fulfill — your scenario — we flip to the state machine. Thanks for the borrowed line, by the way — the check is in the mail. And I'll take you up on the email when our agents start initiating payments; that's the exact tripwire you flagged.