Authenticated S3 Downloads Without Passing Through Your Lambdas

In my previous story I covered how Uclusion lets users securely upload files to S3 without having to route the upload data through our Lambdas. In this installment, I’ll explain how to do the same when users download files.

Step 1: Block public access from your bucket.

S3 bucket permissions should be as restrictive as possible, and later in the article I’ll cover how your users will get files without needing direct access to the bucket. Therefore, block it all. Here’s what Uclusion’s public access policy looks like from the console:

Step 2. Set up a bucket CORS policy

Since the users are directly uploading via Presigned Post (or PUT if you don’t need fine control), and will be downloading through a CloudFront distribution, the bucket needs a CORS access policy that browsers won’t balk at. Here’s Uclusion’s:

<?xml version=”1.0" encoding=”UTF-8"?>
<CORSConfiguration xmlns=”[http://s3.amazonaws.com/doc/2006-03-01/](http://s3.amazonaws.com/doc/2006-03-01/)">
<CORSRule>
 <AllowedOrigin>*</AllowedOrigin>
 <AllowedMethod>GET</AllowedMethod>
 <AllowedMethod>PUT</AllowedMethod>
 <AllowedMethod>POST</AllowedMethod>
 <AllowedMethod>HEAD</AllowedMethod>
 <MaxAgeSeconds>3000</MaxAgeSeconds>
 <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

Step 3: Create a CloudFront distribution for the bucket

As we’ve blocked public access to the bucket we need a way for users to download files. To provide this, we will create a CloudFront CDN distribution for the bucket. This has the added benefit of allowing you to cache frequently accessed files near the users, and reduce their latency. There is one HUGE gotcha however, in that all CloudFront distributions must be in US-EAST-1 if you want them to be globally distributed.

Uclusion uses the Serverless Framework to specify our CloudFormation templates, and here’s the block that sets up the distribution. Also, the configuration below sets up the Origin Access Identity and IAM policy the distribution will need to read from the bucket.

Resources:
  FilesBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: ${your_bucket_name}
      PolicyDocument:
        Statement:
          - Sid: CloudFrontRead
            Effect: Allow
            Principal:
              CanonicalUser:
                Fn::GetAtt: [ S3FilesCloudfrontOriginAccessIdentity, S3CanonicalUserId ]
            Action:
              - s3:GetObject
            Resource: arn:aws:s3:::${your_bucket_name}/*
  S3FilesDistribution:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Origins:
          - DomainName: ${your_bucket_name}.s3.us-west-2.amazonaws.com *#all buckets in Uclusion are us-west-2
            *Id: S3FilesOrigin
            S3OriginConfig:
              OriginAccessIdentity:
                Fn::Join:
                  - ''
                  - - 'origin-access-identity/cloudfront/'
                    - Ref: S3FilesCloudfrontOriginAccessIdentity

        Enabled: 'true'
        Comment: S3 file distribution
        DefaultCacheBehavior:
          ForwardedValues:
            QueryString: 'false'
            Cookies:
              Forward: none
            Headers:
              - Origin
              - Access-Control-Allow-Origin
              - Access-Control-Request-Origin
              - Access-Control-Request-Methods
              - Access-Control-Allow-Methods
              - Access-Control-Request-Method
              - Access-Control-Request-Headers
          AllowedMethods:
            - GET
            - HEAD
            - OPTIONS
          TargetOriginId: S3FilesOrigin
          CachedMethods: [GET, HEAD, OPTIONS]
          ViewerProtocolPolicy: redirect-to-https
        PriceClass: PriceClass_200
        ViewerCertificate:
          CloudFrontDefaultCertificate: 'true'
  S3FilesCloudfrontOriginAccessIdentity:
    Type: AWS::CloudFront::CloudFrontOriginAccessIdentity
    Properties:
      CloudFrontOriginAccessIdentityConfig:
        Comment: "user-files-access-identity"

Step 4: Create an authorizer for the CloudFront distribution

Cloudfront, unlike S3, allows you to set up a Lambda as an access request authorizer via Lambda Edge. This allows you to authenticate your users when they want to download a file. Here’s the basic picture (shamelessly stolen from a related Amazon Blog):

Note: at this time Lambda Edge does not support Provisioned Concurrency, which means you might need the warmup plugin for Serverless Framework when launching it.

In our case we use uploads for Images, and we didn’t want to modify the normal fetching of images too much. Hence, modify the request on the way out of the browser to embed the token as a request param of the GET call if we recongize it as our file.

Here’s the Serverless Framework definition for the authorizer function, and most of it’s code

s3FilesAuthorizer:
  name: 'S3FilesAccessAuthorizer'
  handler: authorizers/s3_files_authorizer.lambda_handler
  runtime: python3.7
  memorySize: 128
  timeout: 5
  region: us-east-1
  lambdaAtEdge:
    distribution: 'S3FilesDistribution'
    eventType: 'viewer-request'
  retain: true

import jwt
from authorizers.key_data import get_key
from urllib.parse import parse_qs

def lambda_handler(event, context):
    *# get the request object
    *print(event)
    request = event['Records'][0]['cf']['request']
    method = request['method']
    if method == 'OPTIONS':
        return authorized(request)
    request_path = request['uri']
    print('Request Path: ' + request_path)
    query_string = request['querystring']
    print(query_string)
    parsed_query_string = parse_qs(query_string)
    token = parsed_query_string.get('authorization')[0]
    *# no token? unauthorized
    *if not token:
        return unauthorized()
    if not valid_token(token): # an exercise for the reader
        return unauthorized()
    return authorized

def authorized(request):
    headers = request['headers']
    *# add the origin header because otherwise s3 won't do cors things
    *print('Authorizing request')
    origin = headers.get('origin')
    if origin is None:
        request['headers']['origin'] = [{'key': 'Origin', 'value': 'http://www.uclusion.com'}]
    return request

def unauthorized():
    return {
        'status': '403',
        'statusDescription': 'Unauthorized'
    }

Step 5: Extra credit, a service worker

Uclusion uses an authorization request param, but we didn’t want to worry about token expiration in links users may embed in their content. Therefore we need to intercept the request for the image on the way out, and add the param if needed. Fortunately Service Workers allow us to do just that.

Below is the code for Uclusion’s image service worker. It firsts looks for a specific pattern in the URL that matches our CloudFront image paths, and then fetches the token out of token storage and adds the authorization request param with that token (You may notice if it fails to get a token, it tries the user’s account key, as some files are accessible to the users’ account).

const OUR_FILE_PATTERN = /https\:\/\/\w+.cloudfront.net\/(\w{8}(-\w{4}){3}-\w{12})\/\w{8}(-\w{4}){3}-\w{12}.*/i;
self.*importScripts*('localforage.min.js');
self.addEventListener('fetch', (event) => {
  const { request } = event;
  const { method, url } = request;
  if (method === 'GET') {
    const match = url.match(OUR_FILE_PATTERN);
    if (match) {
      const promise = self.localforage.createInstance({ storeName: 'TOKEN_STORAGE_MANAGER' }).getItem(fileKey)
        .then((token) => {
          if (token) {
            return token;
          } else {
            return self.localforage.createInstance({ storeName: 'TOKEN_STORAGE_MANAGER' }).getItem(homeAccountKey);
          }
        }).then((token) => {
          const newUrl = new URL(url);
          newUrl.searchParams.set('authorization', token);
          return *fetch*(newUrl);
        });
      event.respondWith(promise);
    }
  }
});

See source on Github for this service worker here.