Securing S3 uploads and downloads with Origin Access Control

We can securely upload, download and delete objects from S3 without using the SDK. It's relatively easy to restrict S3 operations to authorized users only.

1. The scenario

Say we have an application where we allow authenticated users to upload and download objects from a dedicated private S3 bucket.

One solution would be to use presigned URLs.

Another viable solution is to take advantage of the new Origin Access Control (OAC) protection mechanism CloudFront offers.

By using OAC and a CloudFront distribution, we can upload and download objects using HTTPS requests with packages like axios. We don't have to install the S3 client of the SDK.

This method reduces the application's size and improves deployment time. We can also use custom domains with CloudFront and Lambda@Edge for request authorization.

2. Origin Access Control

AWS has recently announced an upgrade on the Origin Access Identity (OAI) feature, which restricted access to the origin S3 bucket to the CloudFront distribution. It means that we couldn't access OAI-protected objects via their direct links.

Origin Access Identity is legacy now, and we should use Origin Access Control instead.

OAC extends the OAI features in that it supports server-side encryption on the S3 objects, dynamic requests and some other features that I won't discuss here.

Let's see how we can use Origin Access Control for the scenario described above.

3. A solution

This article won't discuss how to

• create a CloudFront distribution
• create an S3 bucket
• add the S3 bucket as the origin to the distribution
• set up server-side encryption with a customer-managed KMS key.

I'll leave some links at the bottom of the page that describe these operations in detail.

3.1. Adding OAC to the distribution

We have to add Origin Access Control to the distribution.

We can easily do that in the Origin section in CloudFront. AWS recommends selecting the Sign requests setting too. In this case CloudFront will SigV4 sign all requests that go to the origin.

The next step is to allow OAC to access objects in the S3 bucket.

3.2. Adding OAC to the bucket policy

If we set up OAC in the Console, it will offer us to copy the bucket policy to the clipboard. Let's do that.

In this step, we'll secure the bucket so that it only accepts requests from CloudFront. The bucket policy looks like this:

{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Sid": "AllowCloudFrontServicePrincipal",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudfront.amazonaws.com"
      },
      "Action": [
        "s3:GetObject"
        "s3:PutObject"
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::BUCKET_NAME/*",
      "Condition": {
        "StringEquals": {
          "AWS:SourceArn": "arn:aws:cloudfront::123456789012:distribution/DISTRIBUTION_ID"
        }
      }
    }
  ]
}

There's nothing exciting or unusual with this resource-based policy. We use the Condition key to specify the CloudFront distribution that we allow to access the objects in the bucket.

After we have saved the bucket policy, we won't be able to get the objects via their direct URL from S3. It'll only be possible through CloudFront.

3.3. KMS key policy

This example uses server-side encryption with a customer-managed KMS key.

When CloudFront accesses the objects, it must decrypt them first. Then it will return them to the user (in this case, this is the application).

This flow means that the CloudFront distribution needs permission to decrypt (and encrypt for uploads) with the KMS key. In practice, we will add these permissions to the key's resource-based policy.

The new statement will look like this:

{
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::123456789012:root",
    "Service": "cloudfront.amazonaws.com"
  },
  "Action": [
    "kms:Decrypt",
    "kms:Encrypt",
    "kms:GenerateDataKey*"
  ],
  "Resource": "*",
  "Condition": {
    "StringEquals": {
      "AWS:SourceArn": "arn:aws:cloudfront::123456789012:distribution/DISTRIBUTION_ID"
    }
  }
}

The distribution will also need the kms:GenerateDataKey* permissions to generate a data key (surprise) that we use to encrypt the individual objects. We can enable Bucket keys in the bucket that will reduce the number of API calls to KMS saving us money on encryption costs.

3.4. Code

Let's see some code now.

I will use a Lambda function to represent the application.

const axios = require('axios');

exports.handler = async (event) => {
  try {
    const obj = { name: 'test' };
    const response = await axios.put('https://CLOUDFRONT_URL/test1.json', obj);
    console.log('response: ', response.data)
  } catch (error) {
    console.error('error: ', error.message);
  }
}

The HTTP PUT operation will upload an object to S3. As we can see, all we have to do is call the CloudFront distribution domain name (or a custom domain name) and specify the name of the object in the path (test1.jaon).

If we want to download an object from S3 through CloudFront, we can use a GET request:

const response = await axios.get('https://CLOUDFRONT_URL/test2.json')
console.log('downloaded: ', response.data)

In case we want the application to delete an object from S3 for whatever reason, we can use HTTP DELETE:

const response = await axios.delete('https://CLOUDFRONT_URL/test3.json')
console.log('downloaded: ', response.data)

In this example, we are not using the SDK in the code. Instead, we make use of simple HTTP calls.

CloudFront can cache the objects in the edge locations. The CDN will provide us with a faster response time. The solution is practical when we have a use case of accessing the same objects multiple times.

4. Errors

We might receive some errors while trying to interact with S3 through CloudFront.

The most frequent ones and their potential solutions are the following.

HTTP 403 Forbidden - The CloudFront distribution doesn't have permission to access either the bucket or the KMS key. Check the bucket and the key policies.

HTTP 405 Method not allowed - Use GET, PUT or DELETE operations. We should use PUT instead of POST to create and upload a new object to the bucket. The corresponding AWS API operation (s3:PutObject) can give us a hint.

HTTP 412 Precondition failed - It can come up when we forget to specify the object's name in the URL path for upload or download.

5. Summary

Origin Access Control is an improved version of Origin Access Identity in CloudFront. It will help us secure an S3 bucket so we can only interact with it through CloudFront.

We should set up the relevant bucket and key policies (if we want to server-side encrypt the objects). We can then use simple HTTP requests to upload and download objects from the bucket.

6. Further reading

Restricting access to an Amazon S3 origin - Origin Access Control in AWS documentation style

Creating a distribution - How to create a CloudFront distribution

Creating, configuring, and working with Amazon S3 buckets - How to create a CloudFront distribution