Stop Proxying Large Media Uploads Through Your App Server

Uploading a 4 GB video through your application server is an easy architecture to build.

It is also an easy architecture to regret.

The browser sends the file to your backend. Your backend receives the bytes and forwards them to object storage. That works fine for profile pictures and small documents. Once users start uploading RAW photos, long videos, or entire media collections, the application server becomes an unnecessary bottleneck.

A better pattern is to let the browser upload directly to object storage while keeping authentication and authorization inside your backend.

For large media files, multipart uploads make that pattern much more resilient.

The problem with proxying uploads

A traditional upload flow often looks like this:

Browser
   |
   v
Application server
   |
   v
Object storage

Your server now has to deal with:

• long-lived HTTP connections
• gigabytes of network traffic
• upload timeouts
• memory or temporary disk usage
• retries after interrupted connections
• horizontal scaling during upload spikes

The application server is performing expensive work without adding much value to the actual byte transfer.

Instead, move the data path away from your backend:

Browser ----------------> Object storage
   |
   |
   +----> Application server
           authentication
           authorization
           upload metadata
           signed upload URLs

The backend stays in control without becoming the pipe through which every file must pass.

Why multipart uploads matter

Uploading a large file as one request is fragile.

Imagine a user uploads a 3 GB video and their connection fails at 96%.

With a single PUT request, they may need to restart the entire upload.

Multipart uploading divides the file into smaller pieces:

video.mp4

part 1   10 MB
part 2   10 MB
part 3   10 MB
...
part 300

Each part is uploaded independently.

If part 187 fails, only part 187 needs to be retried.

That gives you:

• resumability
• parallel uploads
• better progress tracking
• smaller retry costs
• more tolerance for unstable mobile connections

Step 1: Initialize the upload on the server

The browser should never receive permanent cloud credentials.

Instead, it asks your backend to create an upload session.

POST /api/uploads
Content-Type: application/json

{
  "filename": "wedding-film.mp4",
  "contentType": "video/mp4",
  "size": 2873948291
}

Your backend verifies the user and decides whether the upload is allowed.

app.post("/api/uploads", async (req, res) => {
  const user = requireUser(req);

  const { filename, contentType, size } = req.body;

  if (size > MAX_UPLOAD_SIZE) {
    return res.status(413).json({
      error: "File too large"
    });
  }

  if (!ALLOWED_TYPES.includes(contentType)) {
    return res.status(400).json({
      error: "Unsupported file type"
    });
  }

  const objectKey =
    `${user.id}/${crypto.randomUUID()}/${filename}`;

  const upload = await storage.createMultipartUpload({
    key: objectKey,
    contentType
  });

  res.json({
    uploadId: upload.id,
    objectKey
  });
});

Notice that the browser does not decide the final storage path.

That matters.

Never trust a client-provided object key such as:

/users/admin/private/file.jpg

Generate storage keys on the server.

Step 2: Split the file in the browser

A File object can be sliced without loading the entire file into JavaScript memory.

function splitFile(file, partSize) {
  const parts = [];

  for (let start = 0; start < file.size; start += partSize) {
    parts.push(
      file.slice(
        start,
        Math.min(start + partSize, file.size)
      )
    );
  }

  return parts;
}

For example:

const PART_SIZE = 10 * 1024 * 1024;

const parts = splitFile(file, PART_SIZE);

Choosing the right part size is a tradeoff.

Very small parts mean many HTTP requests.

Very large parts reduce the benefit of retrying individual chunks.

For large photo and video uploads, chunks in the range of several megabytes to a few dozen megabytes are often a reasonable starting point.

Step 3: Request signed URLs

Before uploading each part, the browser asks your backend for a short-lived signed URL.

POST /api/uploads/:uploadId/parts
Content-Type: application/json

{
  "partNumber": 7
}

The server verifies that:

• the upload belongs to the current user
• the upload is still active
• the requested part number is valid

Then it generates a signed URL.

const url = await storage.signUploadPart({
  key: upload.objectKey,
  uploadId,
  partNumber,
  expiresIn: 900
});

The browser can now upload that specific part directly.

const response = await fetch(signedUrl, {
  method: "PUT",
  body: part
});

if (!response.ok) {
  throw new Error("Part upload failed");
}

const etag = response.headers.get("ETag");

Keep the returned identifier for every successful part. The storage provider typically needs those values when the multipart upload is finalized.

Add concurrency, but not too much

Uploading parts sequentially wastes bandwidth.

Uploading 200 parts simultaneously can overwhelm the browser, network, or storage API.

Use bounded concurrency.

const CONCURRENCY = 4;

A small worker pool can process parts:

async function worker(queue) {
  while (queue.length) {
    const part = queue.shift();
    await uploadPart(part);
  }
}

await Promise.all(
  Array.from(
    { length: CONCURRENCY },
    () => worker(queue)
  )
);

Four to eight concurrent transfers is usually much friendlier than firing every request at once.

Measure this on real devices. Desktop fiber connections and mobile networks behave very differently.

Retry individual chunks

Retries are where multipart uploads become especially useful.

async function retry(fn, attempts = 3) {
  let lastError;

  for (let i = 0; i < attempts; i++) {
    try {
      return await fn();
    } catch (error) {
      lastError = error;

      await new Promise(resolve =>
        setTimeout(resolve, 500 * 2 ** i)
      );
    }
  }

  throw lastError;
}

Now a failed chunk can be retried without restarting a multi-gigabyte transfer.

Be careful not to retry every error.

A 403 caused by an expired signed URL may require requesting a new URL.

A 400 caused by invalid metadata probably should not be retried at all.

Track progress from bytes, not part counts

If all chunks are equal in size, counting completed parts looks acceptable.

The last chunk usually is not equal.

Track uploaded bytes instead.

const progress =
  uploadedBytes / file.size * 100;

For detailed live progress, XMLHttpRequest still has one practical advantage over fetch: upload progress events are widely supported.

xhr.upload.onprogress = event => {
  if (event.lengthComputable) {
    updatePartProgress(
      partNumber,
      event.loaded
    );
  }
};

Aggregate progress across all active parts to display one total percentage.

Step 4: Complete the upload

Once every part succeeds, send the list of uploaded parts back to your backend.

{
  "parts": [
    {
      "partNumber": 1,
      "etag": "..."
    },
    {
      "partNumber": 2,
      "etag": "..."
    }
  ]
}

The server completes the multipart upload.

await storage.completeMultipartUpload({
  uploadId,
  key: upload.objectKey,
  parts: req.body.parts
});

Only after completion should your application mark the media object as ready.

That distinction prevents partially uploaded files from appearing in user-facing galleries.

Do not forget abandoned uploads

Users close tabs.

Laptops sleep.

Browsers crash.

Some multipart uploads will never finish.

If you ignore them, unfinished chunks may remain in object storage and generate unnecessary storage costs.

Store upload sessions in your database with states such as:

initialized
uploading
completed
aborted
expired

Then periodically clean up stale sessions.

A lifecycle rule at the storage layer can provide another safety net.

Security still belongs on the server

Direct-to-storage does not mean bypassing your backend's security model.

Your server should still control:

• who may upload
• maximum file sizes
• accepted content types
• storage keys
• tenant ownership
• signed URL expiration
• multipart completion
• database records

Also remember that browser-provided MIME types are not proof of file contents.

If uploaded media is later processed or distributed, validate it server-side before treating it as trusted.

The architecture scales better because the backend does less

The important insight is not multipart uploading itself.

It is separation of responsibilities.

Your application server handles decisions.

Object storage handles bytes.

The browser handles upload concurrency and retries.

That architecture removes a large amount of bandwidth and connection pressure from the application tier while making large uploads more resilient for users.

For media-heavy SaaS applications, that is a much healthier place to start than trying to make a traditional request-response upload endpoint survive increasingly large files.