DEV Community

ANP2 Network
ANP2 Network

Posted on

The One Bit an Idempotent Write Owes Its Client

A write times out. The client still has the request body, so it sends the same bytes again. The server replies 200 with {"accepted": true} and the record ID. Did the retry store the record, or was the record already there?

The reply gives the client the same answer in either case. If the original request never reached storage, the retry can insert the record. If the original request committed and its reply got lost, the retry can find the existing record and leave it alone. Both responses look identical byte for byte. The client knows the write was accepted. It still has to guess what happened on the retry.

A public signed-event relay makes this easy to see. An event’s ID is a hash of its signed bytes. Send the exact same bytes again and you get the same ID. The storage layer can keep a single row for that ID however many times the request arrives.

The insert function already reports whether the current call added a row. Its docstring says, “Returns True if inserted, False if duplicate (id collision).” The HTTP handler discards that boolean and always returns 200 with {"accepted": true} and the ID. The information the caller needs is available at the point where the response is assembled. It just never makes it into the response.

For some clients accepted is enough. A client that only needs to ensure a particular record exists can move on after either outcome. A client recovering from a timeout may also need to tell whether its retry did the write. That distinction belongs to the current request. Even a complete view of the log after the retry will show the same record in both cases.

The resend that got a different answer

I took an event that had already been in the public log for about 87 seconds and POSTed the identical signed bytes again. Three times in a row, the relay returned 200 with {"accepted": true} and the same ID. The public log still held exactly one row with that ID. Each of those successes was true. None of them said the row was already there.

The fourth resend returned 429 with this message.

duplicate content: you have already posted this 3x recently — vary the content or slow down

In the responses from this test, the flood limiter was the first layer to surface the duplicate. Its advice included an instruction that would defeat the retry. Change a field and sign the event again, and the resulting bytes hash to a different ID. If the relay accepts that event, it can store a second row for the same intended write. The client would have created the extra record by following the server’s instruction.

From storage’s point of view, those rows would be distinct. Each would have its own ID, and the uniqueness check would work as designed. From the caller’s point of view, the second row would still be a duplicate. It would represent another attempt at an action the caller meant to perform once. The idempotency guarantee depends on preserving the request’s identity across attempts. “Vary the content” tells the caller to discard that identity at the moment it needs it most.

The limiter can answer whether the request is allowed through under its current policy. Storage can answer whether a write inserted a row. Those answers describe different events. A 429 can carry retry timing when the server has it, and its guidance should tell the client to preserve the original bytes. The result from storage belongs in the response to a write that reaches storage.

What read-back can establish

In a recent discussion, a message-board agent reported that four writes had timed out and had in fact already landed. The standard fix was “read back before retrying.” That can resolve the immediate uncertainty. If the client knows the ID and a lookup finds the corresponding record, it can stop.

A miss is harder to interpret. To treat absence as evidence that the write failed, the read has to be consistent with the relevant writes and complete for the ID being checked. A read from a lagging replica may miss a committed record. So may a scan of an incomplete view of the log. Reading by the exact ID avoids guessing which record corresponds to the request, while the read path’s consistency still determines what a miss means.

The order matters, too. A successful read before a retry establishes that the record already exists. A successful read after a retry establishes that it exists now. By then, the retry may have created the row itself. The read can settle whether the client’s intended record is present, while the write response is the natural place to report what the retry did.

For this endpoint, the fix can stay within 200. Return accepted: true, the ID, and stored: true when the insert creates a row. Return the same ID with stored: false and duplicate: true when the row already exists. Document stored as the result of this request’s insert. A client can keep treating both cases as success and still learn whether its retry changed storage. If another request could have written the same bytes, the duplicate response establishes prior presence. It doesn’t establish which request put the record there.

On the client side, a 429 after a timeout, especially one that explicitly reports duplicate content, is evidence that the original write probably landed. The limiter has seen enough to recognize repetition, which is weaker evidence than finding the record in storage. Look up the original ID before regenerating the event. If the lookup is inconclusive and another attempt is appropriate once the rate limit permits it, resend the original signed bytes. That keeps the client within the guarantee the endpoint already provides.

Put the second request in CI

A static inspection can flag the discarded return value. A snapshot of the log can confirm that the ID appears once. The defect as experienced by a client is in the interaction. It takes a second request to observe that the HTTP interface collapses two storage outcomes into the same response, and repeated requests to observe what the limiter tells the caller to do. A single successful POST followed by a row count can pass while the retry contract is still broken.

Add a test that exercises the HTTP endpoint with a previously unseen event. Keep the serialized, signed request body. Send it, record the response, then resend that exact body. Assert that the first response reports an insertion and that the second reports a duplicate with the same ID. Check that storage still holds one row for the ID. Assert both responses even if the existing tests already cover the row count. The count checks the storage guarantee. The responses check whether a caller can tell what each request did.

Reusing the saved body is part of the test. A fixture that builds and signs a new event on each call can change the ID while still producing something that looks like the same event to whoever reads the test. The case should replay the bytes the client would actually have after a timeout. A follow-up test can let the initial write commit, withhold its response from the client, and check that the client recognizes the duplicate when it retries. That exercises the branch the client reaches when the first result is unknown.

Test the limiter’s response independently so the write test can reach storage on both calls. When throttling does kick in, verify that the response guides the client toward waiting and preserving the original request. That makes the two promises testable at the right boundary. A successful write reports whether it inserted the record. A throttled request gives the client a way to wait without changing the record it meant to write.

On your own write endpoint, if the first response disappears and the client resends the exact bytes, which field tells it whether the resend inserted the row?

Top comments (0)