DEV Community

Surhid Amatya
Surhid Amatya

Posted on

Multipart Image Upload, Part 5: Production Architecture and Cross-Platform Implementation

Multipart Image Upload: A Practical Guide for Web, Mobile and Backend
Part 1: Understanding Chunked Multipart Upload
Part 2: Chunking the Image Locally
Part 3: Building the Multipart Upload API
Part 4: Making Uploads Reliable and Resumable
Part 5: Production Architecture and Cross-Platform Implementation (this post)

We have the protocol, the chunking, the server and the retry logic. This last part puts them into something you could run in production, shows how each platform plugs in, and lists what else needs handling before real users touch it.

The architecture

        Web          Mobile
          \           /
           ▼         ▼
          API Gateway
                │
                ▼
         Upload Service
                │
      ┌─────────┴─────────┐
      ▼                   ▼
 PostgreSQL          Object Storage
 Upload state        S3 / Azure Blob
                          │
                          ▼
                     Final Image
Enter fullscreen mode Exit fullscreen mode

Each piece has one job.

The API gateway terminates TLS, authenticates, rate-limits and enforces body-size limits before traffic reaches your code.

The upload service is the Spring Boot app from part 3. It validates requests, owns the upload lifecycle and talks to the other two. Keep it stateless so you can run several instances.

PostgreSQL holds upload state: uploadId, owner, file name, declared size, totalParts, expected checksum, status, expiry, and one row per received part. This replaces "look at the files on disk" from part 3, which only works on one machine.

Object storage holds the bytes, both temporary parts and the final image. Local disk is fine on a single server and breaks the moment you add a second one, because a request for part 4 may hit a different instance than part 3 did.

A shortcut worth knowing: S3 and Azure Blob have native multipart upload (S3's CreateMultipartUpload and UploadPart, Azure's block blobs with PutBlock and PutBlockList). They use the same ideas: part numbers, idempotent parts, a final complete call. A common design has your service authorize the upload and hand out presigned URLs per part, so chunks go straight to storage and skip your servers. Building it yourself, as this series does, is a good way to learn how it works, and it's the right call when you need your own processing in the path.

How each platform fits

The server contract is the same for everyone. Only the way each client reads bytes and builds the request differs.

Web (React / Next.js): File.slice(), then FormData, then the multipart API.

const form = new FormData();
form.append("partNumber", String(partNumber));
form.append("totalParts", String(totalParts));
form.append("chunk", file.slice(start, end), file.name);

await fetch(`/api/uploads/${uploadId}/parts`, { method: "POST", body: form });
Enter fullscreen mode Exit fullscreen mode

Don't set the Content-Type header yourself. The browser adds it with the right boundary.

Mobile (Flutter): file stream, read a chunk, MultipartRequest.

final request = http.MultipartRequest(
  'POST',
  Uri.parse('$baseUrl/api/uploads/$uploadId/parts'),
)
  ..fields['partNumber'] = '$partNumber'
  ..fields['totalParts'] = '$totalParts'
  ..files.add(
    http.MultipartFile.fromBytes('chunk', chunk, filename: 'chunk'),
  );

final response = await request.send();
Enter fullscreen mode Exit fullscreen mode

Java (Spring Boot, server side): MultipartFile, temporary storage, object storage. This is the controller from part 3. In production, swap the local transferTo for a stream into object storage and record the part in PostgreSQL.

Python: requests with multipart/form-data.

import requests

requests.post(
    f"{base}/api/uploads/{upload_id}/parts",
    data={"partNumber": part_number, "totalParts": total_parts},
    files={"chunk": ("chunk", chunk)},
    timeout=30,
)
Enter fullscreen mode Exit fullscreen mode

.NET: HttpClient with MultipartFormDataContent.

using var content = new MultipartFormDataContent
{
    { new StringContent(partNumber.ToString()), "partNumber" },
    { new StringContent(totalParts.ToString()), "totalParts" },
    { new ByteArrayContent(chunk), "chunk", "chunk" }
};

var response = await httpClient.PostAsync(
    $"/api/uploads/{uploadId}/parts",
    content);
Enter fullscreen mode Exit fullscreen mode

Same endpoint and same three fields from five languages. That's what a clear protocol buys you.

What production needs

Security

  • Authenticate every call, and tie each session to its creator. User A must not be able to write parts to, read the status of, or complete user B's uploadId.
  • Check the declared content type, but don't trust it. Look at the assembled file's magic bytes to confirm it really is a JPEG or PNG.
  • Scan the final file for malware (ClamAV or a cloud scanning service) and keep it quarantined until it passes.
  • Generate uploadId on the server as a UUID, and never put client-supplied names in storage paths.

Limits and abuse

  • Declare the maximum file size at creation and enforce it as parts arrive. Reject any part that would push the total over.
  • Let the server dictate chunk size in the create response, so you can retune without shipping new clients.
  • Rate-limit by user and IP, on part requests and on the number of open uploads per user.
  • Cap simultaneous sessions per user.

Integrity

  • Verify a hash per part, to reject corruption early, and a hash of the whole file at completion. The client computes the expected whole-file hash up front and sends it when creating the upload.

Lifecycle and cleanup

  • Give every session an expiry (24 hours is common) and refuse parts after it.
  • Run a scheduled job that deletes the parts and rows of expired or abandoned uploads. Without one, storage grows forever and nobody notices. Object stores can also do this with lifecycle rules.

Operations

  • Track parts received per second, retry rate, completion rate, abandonment rate, time to complete and errors by status. Put uploadId on every log line so one upload can be traced end to end. A rising retry rate usually means your chunk size is too big for your users' networks.
  • Make state transitions explicit (CREATED, UPLOADING, ASSEMBLING, COMPLETED, FAILED, EXPIRED), and guard complete so two simultaneous calls can't both assemble the file.

The full lifecycle

          CREATE
             │
             ▼
         uploadId
             │
             ▼
        CHUNK FILE
             │
             ▼
       UPLOAD PARTS
             │
      ┌──────┴──────┐
   SUCCESS         FAIL
      │              │
      │            RETRY
      │              │
      └──────┬───────┘
             ▼
          STATUS
             │
             ▼
        ALL PARTS?
         /       \
       NO        YES
       │          │
       ▼          ▼
    Resume     COMPLETE
                  │
                  ▼
              CHECKSUM
                  │
                  ▼
            RECONSTRUCT
                  │
                  ▼
           OBJECT STORE
Enter fullscreen mode Exit fullscreen mode

There are two loops in there. The inner one uploads parts and retries failures. The outer one checks status and resumes until nothing is missing. Only then does the client call complete, and the server verifies the checksum before assembling the file into object storage.

The series, in short

Part Question Answer
1 Why do this? Small requests are cheap to fail and retry.
2 How do we split the image? Byte ranges via ceil(size / chunkSize), never smaller images.
3 How does the server identify parts? Metadata (uploadId, partNumber), not the bytes.
4 What when the network fails? Status, retry, resume, idempotent parts, bounded parallelism.
5 How does it run in production? Gateway, stateless service, database state, object storage, and the safeguards around them.

If two ideas stick, make them these: a part's identity lives in its metadata, and every operation can be repeated safely.

Top comments (0)