DEV Community

kongkong
kongkong

Posted on

Make Chunked Upload Finalization Idempotent Before the Network Retries It

A 12 MB artifact is split into three chunks. The client uploads chunk 2 twice, loses the response to finalize, and retries. A weak API creates two artifacts or assembles corrupted data.

Define the contract

type Chunk={sessionId:string,index:number,sha256:string,size:number};
type Finalize={sessionId:string,orderedHashes:string[],artifactSha256:string};
Enter fullscreen mode Exit fullscreen mode

PUT /sessions/:id/chunks/:index should be idempotent for identical bytes and reject a different hash at an existing index. POST /sessions/:id/finalize should compare-and-set uploading → finalizing → complete, returning the same artifact ID after a retry.

created -> uploading -> finalizing -> complete
                    \-> expired
Enter fullscreen mode Exit fullscreen mode
Injection Expected result
duplicate identical chunk 200, no extra storage
duplicate changed chunk 409
missing index 422 with missing list
finalize response lost retry returns same artifact
expired session 410 and cleanup scheduled

Compute every chunk hash server-side and verify the final artifact hash after ordered assembly. Store temporary parts outside the public artifact namespace. A sweeper should remove expired sessions only when they are not finalizing.

End-to-end tests must cross UI selection, API authorization, temporary storage, assembly, download, and cleanup. Rollback disables new sessions while preserving completed downloads and garbage-collecting partial data.

A local filesystem fixture does not reproduce S3 consistency, multipart APIs, or production throughput. Which state transition in your upload flow currently has no idempotency key?

Top comments (0)