Designing an idempotent confirmation protocol for browser uploads

A direct-to-object-storage upload removes the application server from the byte path, but it creates a subtle distributed-systems problem at the end of the flow.

The browser can finish uploading the object, send a confirmation request to the API, and lose connectivity before receiving the response. From the user's perspective the result is ambiguous: did the server create the media record, or should the browser try again?

If the confirmation endpoint is not idempotent, retrying can create duplicate records. If the client refuses to retry, a successfully stored object can remain invisible to the application. The protocol needs a stable identity that survives network uncertainty.

Model the upload as two durable operations

The first operation stores bytes. The second operation creates or returns the canonical application record.

browser -> API: authorize attempt
API -> browser: attempt ID, object key, signed upload URL
browser -> storage: transfer bytes
browser -> API: confirm attempt ID and object metadata
API -> browser: canonical media record

The attempt ID is generated once and reused for every confirmation retry. It must not be regenerated when a request times out.

Give the attempt a server-owned scope

A random UUID is useful, but identity alone is not authorization. The authorization response should bind the attempt to the authenticated or temporary guest session, the target event, the expected object key, and relevant constraints such as maximum size or content family.

A minimal record might contain:

type UploadAttempt = {
  id: string;
  eventId: string;
  contributorSessionId: string;
  objectKey: string;
  state: "authorized" | "confirmed" | "rejected";
  createdAt: string;
  confirmedMediaId?: string;
};

The confirmation endpoint looks up this record rather than trusting event or object identifiers supplied again by the browser.

Make confirmation return the existing result

The endpoint should behave like a create-or-return operation.

async function confirmUpload(attemptId: string, input: ConfirmInput) {
  return database.transaction(async (tx) => {
    const attempt = await tx.uploadAttempts.lockForUpdate(attemptId);

    if (attempt.state === "confirmed") {
      return tx.media.findById(attempt.confirmedMediaId);
    }

    assertAttemptBelongsToSession(attempt);
    const storedObject = await inspectStoredObject(attempt.objectKey);
    validateObject(storedObject, input);

    const media = await tx.media.create({
      eventId: attempt.eventId,
      objectKey: attempt.objectKey,
      size: storedObject.size,
      contentType: storedObject.contentType,
      moderationState: "pending",
    });

    await tx.uploadAttempts.markConfirmed(attempt.id, media.id);
    return media;
  });
}

The transaction and uniqueness constraints matter. Two confirmation requests can arrive concurrently after a client retry or double tap. A row lock or compare-and-set transition ensures that only one media record wins.

Do not use the object key as the only idempotency key

Object keys are storage addresses, not necessarily user-intent identifiers. A multipart retry, a replacement workflow, or an administrative recovery may reuse or transform storage paths.

An explicit attempt record gives the application a place to store authorization context, expiry, confirmation state, and the canonical result. It also makes abandoned-object cleanup safer because the system can distinguish an authorized but unconfirmed object from an unrelated storage file.

Reconcile after the browser returns

Persist lightweight attempt metadata in IndexedDB or another durable client store:

• attempt ID;
• object key;
• filename and size;
• last known client state; and
• timestamps.

On page restore, query the API for each non-terminal attempt. The server response should distinguish at least:

• authorized: storage or confirmation may still be incomplete;
• confirmed: return the canonical media record;
• expired: the client needs a new authorization; and
• rejected: show a terminal explanation.

The browser should not infer failure merely because it never received the original response.

Separate upload success from moderation success

Confirmation means the application accepted the media record. It does not mean the item is approved for public display. Returning a separate moderation state avoids the common UX bug where guests retry a successful upload because it has not appeared on a live wall.

The same separation is useful in the browser-based event contribution flow used by Gathmo: transfer, canonical confirmation, and visibility are different states even when the happy path feels like one action.

Test the ambiguous moments

The most valuable protocol tests deliberately interrupt the flow:

1. Drop the response after the server commits confirmation.
2. Send the same confirmation twice concurrently.
3. Retry after the signed upload URL expires but the object already exists.
4. Confirm an object whose size or content type differs from authorization.
5. Restore the browser after confirmation completed on another device.
6. Run cleanup while a confirmation transaction is in flight.

A reliable upload protocol is not one that never encounters ambiguity. It is one that can answer the same question repeatedly and return the same durable result.