A Retry Is Not Safe Until the Side Effect Is Safe

A retry can duplicate a side effect even when the original command completed successfully.

This is easy to miss in a method that does two ordinary things: save a record and send a message. Each operation is understandable on its own. The difficult part is the uncertainty between them.

Suppose an endpoint creates an invitation, persists it, asks an email provider to deliver a link, and returns a response. The database commit succeeds. The provider may accept the message. Then the network drops the response. The client cannot tell whether anything happened, so it retries.

If the server repeats the whole method, it can create another record, another secret, or another email. The code has turned a lost response into duplicated authority-bearing side effects.

The lesson is broader than invitations: design the durable commit point, provider boundary, and retry contract as one workflow.

Commit the fact before announcing it

Sending before saving creates the most confusing failure mode. A recipient receives a plausible link, but the backing operation never committed. To them, the system appears to reject a valid action for no visible reason.

A safer ordering is:

1. Validate the command.
2. Create the durable operation.
3. Store an idempotency receipt that points to that operation.
4. Commit both together.
5. Only then ask the external provider to send.

The durable record becomes the source of truth. The message announces a fact that already exists rather than promising that a later write will probably succeed.

This does not make the database and email provider atomic. It gives the gap between them explicit semantics.

Make replay return, not repeat

An idempotency key is useful only when it protects the irreversible side effect too.

The receipt should bind a caller-generated command key to a stable fingerprint of the request. If the same key returns with the same request, the service reads and returns the original result. It does not send again. If the key returns with different input, the service rejects the conflict rather than guessing which request the caller intended.

The database should also enforce uniqueness. Two identical requests can arrive concurrently before either sees the other's receipt. Let one commit win; have the loser load and return the winning result. Otherwise a perfectly designed application check can still race.

The important invariant is not merely “one row.” It is “one durable result and one automatic send for one command.” Tests should assert both.

Unknown is not the same as failed

Provider calls rarely produce only success or failure. They produce at least three useful outcomes:

• Accepted: the provider acknowledged the message.
• Definitely rejected: the provider did not accept it.
• Unknown: the caller lost certainty, often through a timeout or interrupted response.

Unknown is the dangerous one. The message may already be queued. Reporting a definite failure can encourage an immediate retry and a second message.

This is a small state-model decision with a large operational effect. Do not collapse “I do not know” into “it did not happen.” Preserve uncertainty so the recovery path can respond deliberately.

Keep a recovery handle

If delivery is definitely rejected after the database commit, deleting or closing the record may feel tidy. It also removes the user's only handle on the problem.

Keeping the durable operation pending can be more honest. The UI can show that it exists, report that delivery failed, and offer an explicit resend. That resend is a new action, not a transparent replay of the original command.

For a message carrying a secret, explicit resend should rotate the secret, invalidate the old link, persist the new state, and then send. Leaving both links active multiplies hidden authority. Reusing a raw secret is often impossible or unsafe if only its hash was stored.

This recovery model needs expiry, status visibility, observability, and clear language. “Pending” must not imply “delivered.” The system should say what it knows and avoid claiming what it cannot know.

Test the gaps, not only the happy path

The valuable tests live at the boundaries:

• Repeating the same command returns the same result, leaves one durable record, and sends once.
• Concurrent duplicates converge on one receipt and one result.
• A definite pre-provider rejection is reported while the recovery record remains available.
• An unknown outcome is not presented as definite non-delivery.
• Explicit resend invalidates the earlier secret before attempting delivery again.
• The provider is never called before the database commit succeeds.

These tests express the reliability contract better than a broad “request succeeds” test. They also make later refactoring safer because the order and meaning of side effects are visible.

The trade-off is honest complexity

This design is not free. It introduces command receipts, request fingerprints, provider outcome types, pending states, expiry, and explicit recovery. Operators need enough telemetry to distinguish rejected sends from unknown outcomes. Product language must explain uncertainty without alarming users.

The alternative is simpler code with less truthful behaviour: phantom links, duplicate messages, or retries that silently create extra authority.

My practical review question is: for every workflow that combines a database write with an external side effect, can we point to the commit point, replay rule, uncertainty model, and recovery action?

If those four answers are explicit, retries become a designed part of the workflow rather than a hopeful repetition of it.