The Only Blog You'll Ever Need for S3 Presigned URL Uploads

Stop treating your backend like an unpaid intern for file uploads. Routing massive videos through your server will just crash it. 🗣️

Presigned URLs. Your backend just hands the browser a ticket to upload straight to S3. Your server never touches the file.

For huge files, we split them into chunks and upload in parallel (multipart uploading).

Here's how to build the whole flow from scratch:

Table of Contents

• Prerequisites
• Part 0: Create Your S3 Bucket
• Part 1: IAM Permissions
  • Step 1: Create a dedicated IAM user
  • Step 2: Create a custom policy
  • Step 3: Attach the policy
  • Step 4: Generate access keys
• Part 2: CORS Configuration
• Part 3: Backend Setup
• Part 4: Single-Part Uploads
• Part 5: Multipart Uploads
  • Requesting the Upload (Backend)
  • Uploading the Chunks (Frontend)
  • Confirming the Upload (Backend)
• The ETag CORS Gotcha
• Wrapping Up

Let's dive in.

Prerequisites

Before we start, make sure you have:

• An AWS account
• A Node.js backend (the examples use Express, but the concepts apply anywhere)
• Basic familiarity with making API requests from a frontend
• Your will

Part 0: Create Your S3 Bucket

First, head over to the S3 Console and create a new bucket.

A few quick tips for bucket creation:

• Bucket name: Pick something clear, like dropdesk-uploads.
• Region: Choose a region close to your users.
• Block all public access: Keep this enabled. Since we are using presigned URLs, the bucket should remain completely private. Only users with a valid signed URL will be able to upload or download files.

Part 1: IAM Permissions

It is always tempting to just give your backend user AmazonS3FullAccess and move on. However, if those credentials ever get leaked, your entire bucket could be compromised. Instead, we are going to create a user that only has exactly the permissions it needs.

Step 1: Create a dedicated IAM user

Go to IAM > Users > Create user.

Give the user a clear name like s3-upload-service. Make sure you do not enable console access, as this user is meant strictly for your backend code.

Step 2: Create a custom policy

Navigate to IAM > Policies > Create policy and open the JSON editor. Paste in the following configuration, making sure to replace YOUR-BUCKET-NAME with your actual bucket name.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowUploadAndManage",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:HeadObject",
        "s3:ListMultipartUploadParts",
        "s3:AbortMultipartUpload",
        "s3:ListBucketMultipartUploads"
      ],
      "Resource": [
        "arn:aws:s3:::YOUR-BUCKET-NAME",
        "arn:aws:s3:::YOUR-BUCKET-NAME/*"
      ]
    }
  ]
}

Here is a quick breakdown of why we need these specific actions:

• s3:PutObject: Allows uploading files.
• s3:GetObject: Allows generating download links and reading files.
• s3:DeleteObject: Allows deleting files.
• s3:HeadObject: Allows us to verify a file was successfully uploaded.
• s3:ListMultipartUploadParts: Serves as a fallback to list parts if the frontend has CORS issues reading ETags.
• s3:AbortMultipartUpload: Allows us to cancel and clean up failed multipart uploads.
• s3:ListBucketMultipartUploads: Helps with listing incomplete uploads for background cleanup jobs.

Name the policy something like DropdeskS3UploadPolicy and save it.

Step 3: Attach the policy

Go back to your user, click Attach policies directly, and search for the policy you just created to attach it.

Step 4: Generate access keys

Finally, go to the user's Security credentials tab, click Create access key, and select Application running outside AWS.

Copy the Access Key ID and Secret Access Key into your .env file. You won't be able to see the secret key again after you close this page.

AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
S3_REGION=eu-north-1
S3_BUCKET=dropdesk-uploads

Part 2: CORS Configuration

If you skip this step, your browser will block the frontend from uploading files directly to S3.

Go to your S3 bucket, open the Permissions tab, and scroll down to the CORS configuration section.

Paste in this JSON:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["PUT", "POST", "GET", "HEAD"],
    "AllowedOrigins": [
      "http://localhost:3000",
      "https://0bhishek.com",
      "https://yourdomain.com"
    ],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3600
  }
]

Make sure to update AllowedOrigins with your actual frontend URLs. The ExposeHeaders: ["ETag"] line is incredibly important for multipart uploads, which we will discuss more later.

Part 3: Backend Setup

Let's get our Node.js backend ready. We need to install the AWS SDK packages.

npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner

Next, initialize the S3 client in your codebase.

// src/infrastructure/s3/index.ts
import { S3Client } from "@aws-sdk/client-s3";

export const s3 = new S3Client({
  region: process.env.S3_REGION,
  requestChecksumCalculation: "WHEN_REQUIRED",
});

export const BUCKET = process.env.S3_BUCKET;

We set requestChecksumCalculation to "WHEN_REQUIRED" because the default behavior can sometimes cause signature mismatch errors when the browser uploads the file.

Part 4: Single-Part Uploads

For files under 100MB, a single-part upload is usually fine. Your backend creates one presigned URL, and the frontend uploads the whole file in one request.

Here is what the backend code looks like to generate the URL:

import { PutObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

async function generateSinglePartUrl(key: string, contentType: string) {
  const command = new PutObjectCommand({
    Bucket: BUCKET,
    Key: key,
    ContentType: contentType,
  });

  return await getSignedUrl(s3, command, { expiresIn: 3600 });
}

And here is how the frontend uses it:

async function uploadFile(file) {
  // 1. Get the presigned URL from your backend
  const response = await fetch("/api/media/request-upload", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      fileName: file.name,
      mimeType: file.type,
      size: file.size,
    }),
  });
  const { url } = await response.json();

  // 2. Upload the file directly to S3
  await fetch(url, {
    method: "PUT",
    headers: { "Content-Type": file.type },
    body: file,
  });

  // 3. Notify the backend that the upload finished
  await fetch("/api/media/confirm-upload", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ key }),
  });
}

This works great for small files. But if a user is uploading a large video and their internet drops out at 99%, the entire upload fails. That is why we need multipart uploads.

Part 5: Multipart Uploads

Multipart uploading breaks a large file into smaller chunks, usually 5MB each. The browser uploads these chunks in parallel. If one chunk fails, it can just retry that specific chunk without starting over from scratch.

Here is a visual overview of how the process works.

Requesting the Upload (Backend)

When the frontend asks to upload a file, the backend needs to initiate a multipart upload with S3 and generate a unique presigned URL for every single chunk.

import { CreateMultipartUploadCommand, UploadPartCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

const CHUNK_SIZE = 5 * 1024 * 1024; // 5MB

export const requestUpload = async (req, res) => {
  const { fileName, mimeType, size } = req.body;

  const totalParts = Math.ceil(size / CHUNK_SIZE) || 1;
  const s3Key = `uploads/${Date.now()}_${encodeURIComponent(fileName)}`;

  // Step 1: Start the multipart upload
  const createCmd = new CreateMultipartUploadCommand({
    Bucket: BUCKET,
    Key: s3Key,
    ContentType: mimeType,
  });
  const { UploadId: uploadId } = await s3.send(createCmd);

  // Step 2: Create a presigned URL for each part
  const urls = await Promise.all(
    Array.from({ length: totalParts }).map(async (_, index) => {
      const uploadPartCmd = new UploadPartCommand({
        Bucket: BUCKET,
        Key: s3Key,
        UploadId: uploadId,
        PartNumber: index + 1, // S3 parts are 1-indexed
      });
      return getSignedUrl(s3, uploadPartCmd, { expiresIn: 3600 });
    })
  );

  return res.status(201).json({ uploadId, key: s3Key, urls });
};

Uploading the Chunks (Frontend)

On the frontend side, we slice the file up and upload the pieces. To avoid overwhelming the user's network connection, we can limit the concurrency to a few chunks at a time.

const CHUNK_SIZE = 5 * 1024 * 1024; // 5MB

async function multipartUpload(file) {
  const initRes = await fetch("/api/media/request-upload", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      fileName: file.name,
      mimeType: file.type,
      size: file.size,
    }),
  });
  const { uploadId, key, urls } = await initRes.json();

  const parts = [];
  const CONCURRENCY = 5;
  const queue = urls.map((url, index) => ({ url, index }));

  async function uploadChunk({ url, index }) {
    const start = index * CHUNK_SIZE;
    const end = Math.min(start + CHUNK_SIZE, file.size);
    const chunk = file.slice(start, end);

    const response = await fetch(url, {
      method: "PUT",
      body: chunk,
    });

    const etag = response.headers.get("ETag");
    parts.push({ PartNumber: index + 1, ETag: etag });
  }

  // Process uploads in batches
  for (let i = 0; i < queue.length; i += CONCURRENCY) {
    const batch = queue.slice(i, i + CONCURRENCY);
    await Promise.all(batch.map(uploadChunk));
  }

  // Tell the backend to finish the upload
  await fetch(`/api/media/confirm-upload`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      key,
      uploadId,
      parts: parts.sort((a, b) => a.PartNumber - b.PartNumber),
    }),
  });
}

Confirming the Upload (Backend)

Once all chunks are uploaded, the frontend sends the ETags back to the server. The server then tells S3 to stitch the file together.

import { CompleteMultipartUploadCommand, HeadObjectCommand, ListPartsCommand } from "@aws-sdk/client-s3";

export const confirmUpload = async (req, res) => {
  const { key, uploadId, parts } = req.body;
  let sortedParts = parts.sort((a, b) => a.PartNumber - b.PartNumber);

  // Fallback just in case the frontend couldn't read the ETags
  if (!sortedParts[0]?.ETag) {
    const listRes = await s3.send(new ListPartsCommand({ Bucket: BUCKET, Key: key, UploadId: uploadId }));
    sortedParts = listRes.Parts.map((p) => ({ PartNumber: p.PartNumber, ETag: p.ETag }));
  }

  // Complete the upload
  await s3.send(
    new CompleteMultipartUploadCommand({
      Bucket: BUCKET,
      Key: key,
      UploadId: uploadId,
      MultipartUpload: { Parts: sortedParts },
    })
  );

  // Verify the file was saved properly
  const head = await s3.send(new HeadObjectCommand({ Bucket: BUCKET, Key: key }));

  // Update your database here...

  return res.status(200).json({ success: true, size: head.ContentLength });
};

The ETag CORS Gotcha

There is a very common issue where the frontend uploads the chunks successfully, but response.headers.get("ETag") returns null. This happens when your S3 CORS configuration is missing the ExposeHeaders: ["ETag"] rule. If the browser isn't explicitly told it is allowed to read the ETag, it will hide it from your JavaScript code.

If you ever find yourself in a situation where you don't control the bucket's CORS rules, your backend can still fetch the ETags directly using the ListPartsCommand as shown in the confirm endpoint above.

Wrapping Up

By letting the browser upload straight to S3, your server saves memory and uploads go faster. Add multipart uploading on top, and you can handle massive files without breaking a sweat if the network drops.

I'm currently using this exact setup in production for Dropdesk (an open-source workspace file sharing app).

If you dug this, drop a follow on Dev.to and X/Twitter.

Wanna chat? Hit me up at [e-mail].

Peace!