My script charged a customer twice after a timeout. Here is how idempotency keys prevent it

#webdev #tutorial #python #beginners

A request times out. You retry. Later you find out the first request actually succeeded, so the customer was charged twice, or the email went out twice, or the row was inserted twice.

This is one of the most common bugs in code that talks to APIs, and the fix is small: an idempotency key.

The problem: a timeout does not mean failure

When requests.post() raises a timeout, you only know that you stopped waiting. The server may have finished the work.

import requests

def charge(amount_cents):
    return requests.post(
        "https://api.example.com/charges",
        json={"amount": amount_cents},
        timeout=5,
    )

# If this times out after the server already processed it,
# a naive retry creates a second charge.

The idea

The client generates a unique key once per logical operation and sends it with every attempt. The server remembers keys it has seen and, on a repeat, returns the original result instead of doing the work again.

Stripe documents this pattern with an Idempotency-Key header: https://docs.stripe.com/api/idempotent_requests

The IETF has also been standardizing an Idempotency-Key header field: https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/

Client side: generate the key once, reuse it on retries

import time
import uuid
import requests

def charge_safely(amount_cents, retries=3):
    key = str(uuid.uuid4())  # created ONCE, outside the retry loop
    last_error = None
    for attempt in range(retries):
        try:
            r = requests.post(
                "https://api.example.com/charges",
                json={"amount": amount_cents},
                headers={"Idempotency-Key": key},
                timeout=5,
            )
            if r.status_code < 500:
                return r
            last_error = RuntimeError(f"server error {r.status_code}")
        except requests.exceptions.RequestException as e:
            last_error = e
        time.sleep(2 ** attempt)
    raise last_error

The most common mistake is creating the key inside the loop. Then every retry looks like a brand new request and you are back to double charges.

Server side: a minimal version with SQLite

If you run the API yourself, the core is a table with a unique key:

import sqlite3, json

db = sqlite3.connect("idem.db")
db.execute("CREATE TABLE IF NOT EXISTS idem (key TEXT PRIMARY KEY, response TEXT)")

def handle(key, do_work):
    row = db.execute("SELECT response FROM idem WHERE key=?", (key,)).fetchone()
    if row:
        return json.loads(row[0])      # replay the stored answer
    result = do_work()                 # runs only the first time
    try:
        db.execute("INSERT INTO idem VALUES (?, ?)", (key, json.dumps(result)))
        db.commit()
    except sqlite3.IntegrityError:
        row = db.execute("SELECT response FROM idem WHERE key=?", (key,)).fetchone()
        return json.loads(row[0])
    return result

Note the gap: two concurrent requests with the same key can both run do_work() before either inserts. In production you reserve the key first (insert a "pending" row inside a transaction) and only then do the work. Stripe's docs describe how they handle in-flight duplicates.

Three rules to remember

One key per logical operation, reused across all retries of that operation.
Keys should be random (UUID4) or derived from stable intent such as an order id, never from content that can change between attempts.
Expire stored keys after a while (Stripe keeps them for 24 hours) so the table does not grow forever.

DEV Community