Why did two deliveries with the same order JSON mint two Redis keys? Finance saw one checkout and two settlements. Your logs printed pretty bodies that a human would call identical. The idempotency store disagreed, and it was right. A readability pull request had rebuilt a dict, and json.dumps quietly emitted a different byte string.
An idempotency key is not a business phrase. It is a digest of whatever you actually concatenated last winter. Vendors retry. Queues replay. Your handler will see the same checkout again, and the only question that matters is whether the second pass hashes to the same hex. If it does not, you charge twice. If it matches a different order, you drop a real sale.
Think of the key as a luggage tag glued on in a noisy warehouse. You can polish the suitcase. You cannot swap the tag and still claim it is the same bag.
How a “stable” key is usually born
Someone needed a string before a long weekend. They stuffed an order id, an email, a total, and a list of line items into a dict, dumped JSON, and hashed it. Python 3.7 made insertion order part of the language, so the dump looked deterministic on one machine. That was enough to ship.
The construction order was never written down. Neither was the default of json.dumps: spaces after separators, ensure_ascii=True, no sort_keys, str on whatever total happened to be. Retry metadata sometimes rode along because the envelope was convenient. None of that is cryptography. All of it is now a protocol between your process and Redis.
A later cleanup feels virtuous. You alphabetize keys. You drop delivery_attempt because retries should not change identity. You switch separators to the compact pair. You format money with two decimal places. Each change can be correct as a product decision. Bundled together they are a new protocol, published without a migration.
A teaching handler, not a processor
The module below is a fixture for this article. It is not a payment stack, not a vendor SDK, and not advice about card data. It exists so you can watch hex drift on your own laptop.
# invoice_key.py — messy on purpose; teaching fixture only
import hashlib
import json
PREFIX = "inv"
def build_key(event):
body = event.get("body") or {}
customer = body.get("customer") or {}
items = body.get("items") or []
payload = {
"order": body.get("order_id") or body.get("id"),
"email": (customer.get("email") or "").strip().lower(),
"total": str(body.get("total") or "0"),
"items": items,
"attempt": event.get("delivery_attempt", 1),
}
blob = json.dumps(payload)
digest = hashlib.sha256(blob.encode("utf-8")).hexdigest()
return f"{PREFIX}:{digest}"
Read it once as a protocol, not as style. attempt is inside the hashed object, so every vendor retry is a new key. items keep whatever order the client sent. total is str() of a JSON number, so 10.5 and 10.50 already disagree. json.dumps inserts spaces. Two engineers can stare at the same pretty log and still miss those four facts.
Replay against a fake seen-store
Do not start by inventing a prettier function. Start with bodies you already captured, redacted until they hold no live customer data. The oracle is not “does this JSON look right.” The oracle is a map from hex to the first delivery id that claimed it.
# replay_keys.py
import json
from pathlib import Path
from invoice_key import build_key
CAPTURES = Path("fixtures/deliveries.jsonl")
SEEN = Path("fixtures/seen_keys.json")
def load_captures():
rows = []
for line in CAPTURES.read_text().splitlines():
if line.strip():
rows.append(json.loads(line))
return rows
def mint(rows):
minted = []
for row in rows:
minted.append({
"delivery_id": row["delivery_id"],
"order_hint": (row.get("body") or {}).get("order_id"),
"key": build_key(row),
})
return minted
def record():
table = {}
for row in mint(load_captures()):
table.setdefault(row["key"], row["delivery_id"])
SEEN.write_text(json.dumps(table, indent=2, sort_keys=True) + "\n")
def assert_stable():
want = json.loads(SEEN.read_text())
got = mint(load_captures())
by_key = {}
for row in got:
by_key.setdefault(row["key"], []).append(row)
if row["key"] not in want:
raise SystemExit(f"new key for {row['delivery_id']}: {row['key']}")
if want[row["key"]] != by_key[row["key"]][0]["delivery_id"] and row["delivery_id"] != want[row["key"]]:
# first claimant must remain the first claimant
if row["delivery_id"] == want[row["key"]]:
continue
for key, first in want.items():
if key not in by_key:
raise SystemExit(f"lost key {key} (was {first})")
print("stable", len(got), "deliveries")
if __name__ == "__main__":
import sys
if sys.argv[1:] == ["record"]:
record()
else:
assert_stable()
Seed the captures with ugly rows, not textbook ones. Include a retry of the same order, a second order that shares an email, a total written as 10.5, a list whose dict keys arrive in two orders, and an email with a combining mark.
{"delivery_id": "d1", "delivery_attempt": 1, "body": {"order_id": "ORD-9", "total": 10.5, "customer": {"email": "Ada@Example.com "}, "items": [{"sku": "A", "qty": 1}, {"sku": "B", "qty": 2}]}}
{"delivery_id": "d2", "delivery_attempt": 2, "body": {"order_id": "ORD-9", "total": 10.5, "customer": {"email": "Ada@Example.com "}, "items": [{"sku": "A", "qty": 1}, {"sku": "B", "qty": 2}]}}
{"delivery_id": "d3", "delivery_attempt": 1, "body": {"order_id": "ORD-10", "total": "10.50", "customer": {"email": "ada@example.com"}, "items": [{"qty": 1, "sku": "A"}]}}
Record once, then treat seen_keys.json as an external contract. You are not snapshotting a scorer for fun. You are asking whether Redis would still recognize the luggage tag.
python replay_keys.py record
python replay_keys.py
On this fixture the second delivery already mints a new key, because attempt is inside the dump. That is not a test bug. That is the production incident waiting in staging. Freeze it in the seen-store first. Decide later whether excluding attempt is a migration you are willing to run.
The cleanup that looks equivalent
The next block is a labeled proposal. Do not paste it into build_key until replay_keys.py has a baseline, and do not combine it with a helper extract in the same diff.
# proposal — unexecuted; each line is a protocol change
def build_key(event):
body = event.get("body") or {}
customer = body.get("customer") or {}
payload = {
"email": (customer.get("email") or "").strip().lower(),
"items": body.get("items") or [],
"order": body.get("order_id") or body.get("id"),
"total": f"{float(body.get('total') or 0):.2f}",
}
blob = json.dumps(payload, separators=(",", ":"), sort_keys=True)
digest = hashlib.sha256(blob.encode("utf-8")).hexdigest()
return f"inv:{digest}"
Alphabetized keys change the dump even when values stay put. Compact separators strip spaces the old hash still expects. Two-decimal formatting turns 10.5 into 10.50 and collides with a different order that already used the string form. Dropping attempt is the one change finance actually wanted, and it is buried under three unrelated serializer tweaks. A reviewer who reads only the pretty JSON will approve the lot.
Here is a compact map of those moves. Use it in the pull request, not as wallpaper in the module.
| Change | Key hex stays? | Identity still means “same checkout”? |
|---|---|---|
Extract _canonical_bytes with the same json.dumps call |
yes | yes |
Drop attempt only |
no | closer to yes, needs a seen-store migration |
sort_keys=True |
no | same checkouts, new tags |
separators=(",", ":") |
no | same checkouts, new tags |
Force total to two decimals |
no | can merge distinct totals |
Sort items by sku
|
no | retries match, but two carts with swapped lines collapse |
| Unicode NFC on email | no | may merge lookalike addresses |
Keep extract work on the first row. Put product changes on their own commit, with a plan for keys already sitting in Redis. Mixing those rows is how a “tiny refactor” double-bills.
Canonical bytes, said out loud
Say the protocol in comments or a one-page note before anyone touches helpers. Order id comes from order_id, then id. Email is stripped and lowercased, not normalized beyond that. Totals stay str() of the decoded JSON value. Items stay in arrival order. Retry counters do not belong in the hash once you migrate, and they do belong until that day. json.dumps uses library defaults unless the note names other flags. The digest is SHA-256 hex with a fixed prefix.
That paragraph is dull, which is the point. Dull text is cheaper than a second settlement. If you cannot write the paragraph, you are not ready to rename functions.
Watch two traps that dumps hide. A float 10.5 and a string "10.5" are different JSON tokens after str(). A combining accent in an email can survive .lower() and still change after an innocent unicodedata.normalize you copied from a slugify helper. Your seen-store will tell you. Your eyes will not.
Draft the helper off the laptop, on synthetic bodies
A coding model is useful after the seen-store exists. It is not a substitute for that map, and it should never see live webhook bodies. Redact until the fixture is synthetic. Then you can ask for one extract: a _canonical_bytes(event) that returns the exact dump the hash already uses.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option. That pair is a scratch place to draft the helper and run python replay_keys.py without mixing captured fixtures into the laptop you use for mail. It does not mint keys for you, does not certify a digest, and does not replace the seen-store. If the hex drifts on that host, you discard the diff. If the hex holds, you still read the patch like any other protocol change.
Ask for a single function with the current dumps flags named in the prompt. Do not ask the model to “make this idempotent.” That sentence invites dropping attempt, sorting line items, and pretty-printing money in one blob. Name the frozen hex file. Name the function. Keep Redis out of the prompt.
What a stable hex does not mean
A matching seen-store means those deliveries still tag the same luggage. It does not mean the tag was a good idea. You may have frozen the retry bug. You may have frozen a float that will change if a client starts sending strings. Unseen shapes remain unseen. A new field on items will flow into the dump because you hashed the list as-is.
Do not put secrets in the hashed object. A key that includes a bearer token or a raw card-shaped string becomes another copy of that secret in Redis and in every log that prints the key. This fixture uses order ids and emails on purpose. If your real captures still hold secrets, they are not fixtures yet.
Parallel tests will lie if they share a writable seen_keys.json. Record in one process. Assert in another. Do not append production deliveries into the same file you commit.
Who should not use this replay
Skip it when a vendor already dictates the idempotency header and you only echo it. Skip it as the only check on a security change. Skip it on payloads you cannot redact, and do not upload those payloads to a shared scratch server, free or otherwise. Skip it when two teams already share an OpenAPI field list that defines the key; extend that list instead of hashing leftover JSON.
Teams that need exactly-once money movement also need a store with compare-and-set, not only a hex function. This article stops at the tag. The warehouse still needs a lock.
Leave the protocol boring
If you take one habit from this writeup, take the pause before a dict tidy. Pretty construction order is not a contract. Compact JSON is not a contract. Lowercased email is a contract only because the current hash already lowercased it.
When a model or a teammate offers a cleaner helper, feed the helper the redacted captures and keep the seen-store on your side of the diff. Wipe the scratch clone after the hex matches so those bodies do not linger on a box you do not patch. The luggage tag either matches or it does not. Everything else is interior design.
Top comments (0)