AWS S3 Pre-Signed URLs: Your Key to Secure Media Uploads

Introduction

AWS S3 Pre-Signed URLs offer a powerful solution for managing media file uploads securely and efficiently. In this article, we'll discuss what pre-signed URLs are, why they're beneficial for uploading media files, how the upload process works, and key code best practices to keep in mind when implementing this approach.

What is a Pre-Signed URL?

A pre-signed URL is a time-limited URL granting temporary access to a specific AWS resource. Typically generated by AWS services or SDKs, it allows clients to perform specific actions on a resource without requiring permanent credentials.

Why Choose Pre-Signed URLs for Media File Uploads?

Reduced Load on Your App Server:

Allowing users to upload files directly to Amazon S3 without going through your application server. When using pre-signed URLs, the file upload is handled directly between the client and Amazon S3, bypassing your server. This can reduce the load on your server, making it more scalable, especially in scenarios with a large number of file uploads.

Security:

Providing time-limited access to resources for sharing purposes. Granting temporary access to private objects in Amazon S3 for downloading or viewing.

Now, let's explore what sets pre-signed URLs apart from other methods of uploading files directly to S3, such as multer-s3. Now, let's explore what sets pre-signed URLs apart from other methods of uploading files directly to S3, such as multer-s3.

Multer is a middleware for handling multipart/form-data , which is typically used for file uploads.

Comparison with multer-s3

Server-Side Handling:

When using multer-s3 or a similar library, the file upload is handled directly by your server. This can increase the load on your server, especially if you have a large number of file uploads.

Security Concerns:

multer-s3 involves passing the file through your server first, potentially exposing it to additional security considerations. While Pre-signed url enable direct client-side uploads to S3, bypassing your server.

Media Upload with Pre-Signed URLs​

1. User requests a pre-signed URL through API Gateway.
2. API Gateway triggers an AWS Lambda function.
3. Lambda generates a pre-signed URL and a unique key for the object.
4. User makes a PUT request including the file to upload to S3.
5. The unique key is then stored in the database.

Role of API Gateway and Lambda

API Gateway:

1. Acts as a centralized access point
2. Triggers Lambda upon a URL request
3. API Gateway can enforce authentication, authorization, and other security policies before allowing clients to generate pre-signed URLs.

AWS Lambda:

1. Implements business logic for generating pre-signed URLs.
2. Dynamically sets expiration time based on business rules.
3. Handles scaling automatically based on demand.

Code best practices

ContentType:

Define Content Type in the URL query parameters. If it is not defined, the object will be uploaded with the default content type, which may lead to incorrect interpretation or processing of the file.
For Example, image/jpeg or image/png for images and video/mp4 or video/mp3 for videos and audios.

Unique Object Name:

Ensure each object has a unique key in the specified bucket. If any duplicate name is used, it will result in overwriting the existing object with the same name in the s3 bucket. We can use a library such as uuid to generate unique id for every object.

Expiration Time:

It's crucial to set an expiration time to enhance security. The expiration time determines the period during which the URL is valid, and after this duration, the URL becomes inactive.

Here's a sample TypeScript code for a Lambda function that retrieves a pre-signed URL from Amazon S3.

import { APIGatewayEvent, Callback, Context } from 'aws-lambda';
import AWS from 'aws-sdk';
import { v4 as uuidV4 } from 'uuid';
import mime from 'mime-types';
import { ACL, GET_PRE_SIGNED_URL_OP } from '../lib/s3/enum';
import { GetSignedUrlParams } from '../lib/s3/interface';
import { messages } from '../messages/en';
import { Body, Headers, Response } from '../lib/response/interface';

const s3 = new AWS.S3();

const getExtensionFromContentType = (mimeType: string): string | false => {
  return mime.extension(mimeType);
};

export const getSignedUrl = (
  operation: string,
  { bucket, key, expires, ACL: acl, contentType }: GetSignedUrlParams,
): string => {
  let params: any = {
    Bucket: bucket,
    Key: key,
    Expires: expires,
  };

  if (operation === GET_PRE_SIGNED_URL_OP.PUT_OBJECT) {
    params = { ...params, ACL: acl, ContentType: contentType };
  }

  return s3.getSignedUrl(operation, params);
};

export const createResponse = (
  statusCode: number,
  body: Body,
  extraHeaders?: Headers,
): Response => {
  const headers = {
    'Content-Type': 'application/json',
  };
  return {
    statusCode,
    body: JSON.stringify(body),
    headers: { ...headers, ...extraHeaders },
  };
};

export const handler = (event: APIGatewayEvent, __: Context, callback: Callback): void => {
  const contentType = event.queryStringParameters?.contentType as string;
  const extension = getExtensionFromContentType(contentType);
  if (!extension) {
    const response = createResponse(400, {
      error: 'Bad request',
      message: messages.error.badContentType,
    });
    callback(null, response);
  }
  const key = `${uuidV4()}.${extension}`;

  const params: GetSignedUrlParams = {
    bucket: process.env.IMAGE_BUCKET as string,
    key,
    ACL: ACL.PUBLIC_READ,
    expires: Number(process.env.S3_SIGNED_IMAGE_URL_EXPIRY),
    contentType,
  };
  const s3ImageUrl = getSignedUrl(GET_PRE_SIGNED_URL_OP.PUT_OBJECT, params);

  const response = createResponse(200, {
    key,
    url: s3ImageUrl,
  });
  callback(null, response);
};

That's all for today. Thanks for reading.

Ehsaan Technologies is a Software Development & Consultancy firm providing drone inspection services to its customers across the Globe.