The Model Replied Twice Because My Client Gave Up First

Last week my invoice-extraction service wrote the same invoice into the database twice, and the two rows were identical except for their timestamps. The webhook log said the customer submitted exactly one PDF, yet the model endpoint had been called twice from my server's IP. So where did the second request come from, and why did my code treat it as a brand-new success?

I run this service on MonkeyCode's free server tier with its free model access, which keeps the monthly bill at zero while I experiment. (Disclosure: This article was prepared as part of MonkeyCode's product outreach.) The architecture is deliberately boring: a webhook receives the PDF, a worker calls the model to extract the fields, and another query inserts the result into Postgres. Boring architectures are exactly where the strangest bugs hide, because nobody suspects the boring parts.

The symptom: two rows, one submission

The duplicate showed up in a routine check:

SELECT id, customer_id, invoice_number, amount, created_at
FROM invoices
WHERE customer_id = 'cust_1042'
ORDER BY created_at;

The output had two rows with the same invoice_number, the same amount, and created_at values fourteen seconds apart. The customer had definitely not submitted twice, and the webhook provider's delivery log confirmed a single POST. My first instinct was to blame the free model endpoint for double-firing, so I spent an hour staring at the provider dashboard.

The wrong suspect

The dashboard showed two requests from my server's IP, which seemed to confirm my suspicion. Then I added request IDs to every log line, and the real story appeared:

14:02:11.004  req_9f31  webhook received, payload hash=7c2e...
14:02:11.102  req_9f31  calling model endpoint
14:02:21.003  req_9f31  client timeout after 10s, aborting
14:02:21.004  req_9f32  retry #1, same payload hash=7c2e...
14:02:21.105  req_9f32  calling model endpoint
14:02:24.310  req_9f32  model returned 200, writing invoice
14:02:25.901  req_9f31  model returned 200, writing invoice

There it was. The first request did not fail; it was simply slow. My client gave up at ten seconds, the server kept processing in the background, and the model finally answered at fourteen seconds on a connection my client had already abandoned. My code then processed that orphaned response as a fresh success, and the second insert sailed through because nothing stopped it.

The model answered twice because my client gave up once. The retry was the bug, not the model.

Why my timeout was wrong

I had set the timeout to ten seconds because the endpoint usually answered in three. But "usually" is not a timeout, and the latency distribution was bimodal: warm workers answered fast, while cold starts on the free server pushed the p95 closer to fifteen seconds. My timeout was surgically cutting off the slow tail, and every cut produced a false failure.

The deeper mistake was treating a timeout as a failure. A timeout means "I do not know what happened," not "the request failed." The request may still be running, and the server may still write the result after my client walks away. If you retry an unknown outcome without an idempotency key, you get exactly what I got: two rows.

The fix, in three layers

1. Make the write idempotent at the database level

Code-level checks are nice, but the database is the only place where a duplicate actually hurts. I derived an idempotency key from the input, not from the model output, and added a unique constraint:

ALTER TABLE invoices
ADD CONSTRAINT invoices_idempotency_key UNIQUE (idempotency_key);

The key is a hash of the PDF bytes plus the customer ID, so it stays stable even if the model returns a slightly different amount on a retry. The insert then becomes a safe no-op when the key already exists:

INSERT INTO invoices (idempotency_key, customer_id, invoice_number, amount)
VALUES (%s, %s, %s, %s)
ON CONFLICT (idempotency_key) DO NOTHING
RETURNING id;

If the second attempt arrives, the database quietly returns no row, and the first write wins.

2. Set the timeout from the p95, not the p50

I measured the real latency distribution over a week before choosing a number. The p95 on cold starts was around fifteen seconds, so the timeout went to thirty seconds, and the false aborts mostly disappeared. Measure the tail before you tune the timeout, and re-measure after you change anything about the server.

3. Retry with jittered exponential backoff, and only on safe outcomes

The retry loop now looks like this:

import random
from time import sleep

def call_model_with_retries(payload, max_attempts=3):
    for attempt in range(max_attempts):
        try:
            return client.chat(payload, timeout=30)
        except TimeoutError:
            # Unknown outcome: the server may still be processing.
            # Retrying is safe only because the write is idempotent.
            sleep(min(2 ** attempt, 8) + random.uniform(0, 1))
    raise RuntimeError("model call failed after retries")

The comment is the point. Retrying is only safe because layer one guarantees that a duplicate write is a no-op. If the write were not idempotent, the correct move would be to query the result by idempotency key before retrying, not to blindly fire the request again.

The decision table I wish I had written earlier

What happened	Client sees	Safe to retry?	Correct action
Request never reached the server	Timeout / connection error	Yes	Retry with jittered backoff
Server processed, response lost	Timeout	Only with idempotency key	Retry, then dedupe
Server processed, client aborted	Timeout	Only with idempotency key	Retry, then dedupe
Server returned an error	4xx / 5xx	Depends on the error	Inspect before retrying

The middle two rows are the ones that bite people. A timeout is ambiguous by definition, and any retry strategy that ignores that ambiguity will eventually create a duplicate.

What I would do differently

• Log time-to-first-byte separately from total time, because a slow first byte and a slow stream are different problems.
• Put the idempotency key in every log line so all attempts of one logical request are traceable.
• Test the retry path deliberately: kill the connection mid-request, then check whether the server-side work completed anyway.

Limitations and who should not use this

This approach assumes your write is naturally keyed by the input. If the model output is the thing you store and it can vary between attempts, the unique constraint still works, but you must decide which attempt wins. If you cannot tolerate any duplicate side effects — charging a credit card, sending an email, creating a support ticket — do not rely on retries plus ON CONFLICT DO NOTHING; use a transactional outbox or the provider's idempotency API instead. And if your latency budget is strict, a free server with cold starts is the wrong place for the hot path; this fix makes the system correct, not fast.

The model was never the problem. The problem was that I designed the retry logic as if "I did not hear back" meant "it did not happen." That assumption holds most of the time, until the day it does not, and on that day you get a duplicate invoice. If you want to avoid this class of bug entirely, add the idempotency key before you add the retry — the order matters.