When we retrieve objects from S3, we can customize their content based on geographic location with S3 Object Lambda. This way, we might not need to upload multiple versions of the same object. Instead, the Object Lambda function will dynamically transform the response based on criteria that we define.
1. Problem statement
Bob has an application that should return the same object with some of its elements adjusted to the caller's geographic location.
Say for the sake of this example that the object name is location.json, and Bob stores it in an S3 bucket.
location.json looks like this:
{
"location": "Test country"
}
Users will retrieve this object from S3. The returned response should contain the caller's country in the location and the currency they use in that country in the currency property. Something like this:
{
"location": "Germany",
"currency": "EUR"
}
One solution is to create multiple versions of the same object and store them all in the bucket. The application will then return the version corresponding to the user location.
Another solution is to use S3 Object Lambda that dynamically changes the content. This way, we will write slightly more code and provision more resources but eliminate the need to store tens of versions of the same object.
Let's take a look at the second solution in more detail below.
2. Architecture
Creating an Object Lambda is slightly more complex than provisioning a simple resource because it involves several components. I will skip the details about creating an Object Lambda (references are at the bottom of this page). Instead, let's review a simple but working architecture:
The key parts of the architecture are the following.
2.1. S3 bucket
First, we need an S3 bucket where we store the content. We can call this bucket test-bucket.
2.2. Access Point
We must then create an S3 Access Point.
It will be a supporting access point, and the Lambda function will use it to retrieve the object from S3 via a pre-signed URL. Lambda will receive the URL as an input, and the application needs to call the endpoint.
2.3. Lambda function
It comes as no surprise that we need a Lambda function too. Let's call it object-lambda-function.
The function will receive a config object with the pre-signed URL in the input event. It downloads the file from the bucket, changes the response based on the relevant config object properties, and then passes the modified response to a GetObject operation. The S3 GetObject API call will then return the modified object. More details on the process (including some sample codes) will come below.
2.4. Object Lambda Access Point
The next resource we must create is an Object Lambda Access Point. Its alias will become the entry point to our application
. In this case, it will serve as the origin of a CloudFront distribution.
If the name of the access point is test-object-lambda-access-point, its alias will look like this:
test-object-lambda-a-SOME_CHARACTERS--ol-s3
Other services (e.g., other Lambda functions, ECS containers) can also invoke the endpoint alias.
2.5. CloudFront distribution
Finally, a CloudFront distribution will receive the requests from end users. Let's emphasize some key points here.
Origin
As discussed above, the Object Lambda Access Point will be the origin. We can't specify the access point's alias (see above) as is, but we should format it properly:
test-object-lambda-a-SOME_CHARACTERS--ol-s3.s3.eu-central-1.amazonaws.com
The origin must have the ALIAS.s3.REGION.amazonaws.com format. Otherwise, CloudFront will throw an error.
Origin Access Control
Object Lambda Access Points cannot be public, so we must add an origin access control to the origin. The origin type should be S3.
Cache policy
In this example, the Lambda function will modify the content based on the location information. For the logic to use the correct location, we should add some built-in CloudFront headers to the cache key. Let's add the CloudFront-Viewer-Country and CloudFront-Viewer-Country-Name headers. There are many more headers available. I have attached a link at the end of this post where you can find more information about them.
We can create a custom cache policy called geographic-locations, and add the above headers to the Cache key settings. If we do so, CloudFront will store the responses for various viewer requests in the corresponding edge location cache.
3. Permissions
Everything on AWS is about permissions. Event one service cannot call another without proper permissions.
We must add some IAM statements to the elements of architecture.
3.1. Object Lambda Access Point
Our test-object-lambda-access-point is just a regular access point, so we can attach an access point policy to it. Because the access point is the origin of the CloudFront distribution, we must allow it to invoke the endpoint:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "cloudfront.amazonaws.com"
},
"Action": "s3-object-lambda:Get*",
"Resource": "arn:aws:s3-object-lambda:eu-central-1:123456789012:accesspoint/test-object-lambda-access-point",
"Condition": {
"StringEquals": {
"aws:SourceArn": "arn:aws:cloudfront::123456789012:distribution/DISTRIBUTION_ID"
}
}
}
]
}
It is a very similar resource-based policy to the one we have in an S3 bucket when the origin directly points to the bucket. The difference is the Action (s3-object-lambda) and the Resource, which is the access point in this case.
3.2. Supporting access point
The supporting access point called test-access-point should have a policy similar to this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "cloudfront.amazonaws.com"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:eu-central-1:123456789012:accesspoint/test-access-point",
"arn:aws:s3:eu-central-1:123456789012:accesspoint/test-access-point/object/*"
],
"Condition": {
"ForAnyValue:StringEquals": {
"aws:CalledVia": "s3-object-lambda.amazonaws.com"
}
}
}
]
}
This policy allows CloudFront to call S3 actions through the access point only if the request comes from S3 Object Lambda.
3.3. Bucket policy
test-buckets policy should allow every call from all access points (including test-access-point) for the given account:
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "*",
"Resource": [
"arn:aws:s3:::test-bucket",
"arn:aws:s3:::test-bucket/*"
],
"Condition": {
"StringEquals": {
"s3:DataAccessPointAccount": "1232456789012"
}
}
}
3.4. Lambda function resource policy
We should allow CloudFront to invoke the Object Lambda function, which is a regular function, so it has a resource-based policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "cloudfront.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:eu-central-1:123456789012:function:object-lambda-function"
}
]
}
3.5. Lambda function execution role
As mentioned above, Object Lambda comes with a separate Action set. Because the function will write the modified content to the GetObject operation, we should Allow the s3-object-lambda:WriteGetObjectResponse action in the execution role.
4. Code
Finally, some sample Lambda function code!
The function will do the following:
- Get the object context and user location info from the event.
- Download the object from S3 using the pre-signed URL.
- Convert the content based on the required business logic.
- Send the modified content to the `GetObject` operation.
4.1. Object context and location info
When a user wants to retrieve the location.json object from S3 using our architecture, the event object will look like this:
getObjectContext: {
outputRoute: ROUTE_ID,
outputToken: TOKEN,
inputS3Url: 'https://test-access-point-123456789012.s3-accesspoint.eu-central-1.amazonaws.com/location.json?SOME_INFO'
},
configuration: {
accessPointArn: 'arn:aws:s3-object-lambda:eu-central-1:123456789012:accesspoint/test-object-lambda-access-point',
supportingAccessPointArn: 'arn:aws:s3:eu-central-1:123456789012:accesspoint/test-access-point',
payload: '{"accessPoint": "test"}'
},
userRequest: {
url: 'test-object-lambda-a-SOME_CHARACTERS--ol-s3.s3.eu-central-1.amazonaws.com/location.json',
headers: {
'CloudFront-Viewer-Country': 'GB',
'CloudFront-Viewer-Country-Name': 'United Kingdom',
// some other headers
}
},
// other properties
Let's quickly go over the properties.
getObjectContext
The function will need outputRoute and outputToken when it sends the modified content. It will use the inputS3Url pre-signed URL to download the original location.json object from S3.
configuration
The payload property is an optional object we can set for the transformation at the given Object Access Point. In our case, the transformation is GetObject (because we want to retrieve a specific object from S3). We can specify other transformation types too, and flag them in payload. Or, we can add some custom fields here.
This example won't use this option.
userRequest
This object contains information about the geographic location of the user. The function will extract these headers and use them to create custom content.
4.2. Downloading the original object from S3
The code uses axios to retrieve location.json from S3 via the pre-signed URL defined in getObjectContext.
4.3. Converting the original object
Similar to location.json, the business logic is rather basic. The currencyMap object contains the currencies for some countries. If the user is from a country not specified in the map, we will add the not specified value to the currency field.
We can also have a database call here instead of the map or any other logic that the business requires.
4.4. Sending the modified content
We can use the SDK v3 to send the modified object. The RequestRoute and RequestToken fields are required, and we can get them from getObjectContext.
4.5. Putting everything together
So the sample code can look like this:
import { S3Client, WriteGetObjectResponseCommand } from "@aws-sdk/client-s3";
import axios from 'axios';
const s3Client = new S3Client();
const currencyMap = {
DE: 'EUR',
US: 'USD',
GB: 'GBP',
};
export const handler = async (event) => {
const { getObjectContext: objectContext, userRequest } = event;
const countryName = userRequest.headers['CloudFront-Viewer-Country-Name'];
const country = userRequest.headers['CloudFront-Viewer-Country'];
let data;
try {
const response = await axios.get(objectContext.inputS3Url);
data = response.data;
} catch (error) {
console.error('Axios error: ', error.message);
throw error;
}
const converted = {
...data,
location: countryName,
currency: currencyMap[country] ?? 'not specified',
};
const input = {
RequestRoute: objectContext.outputRoute, // required
RequestToken: objectContext.outputToken, // required
Body: JSON.stringify(converted),
};
const command = new WriteGetObjectResponseCommand(input);
try {
await s3Client.send(command);
} catch (error) {
console.error('Object Lambda error: ', error.message);
throw error;
}
return {
statusCode: 200,
};
};
Let's now try to retrieve location.json via the distribution URL from a browser window, and we should get the location-specific response.
5. Summary
S3 Object Lambda can dynamically modify the content of the downloaded object based on some business logic. We can use the Object Lambda Access Point alias as a CloudFront origin and make our application available for users in different geographic locations.
We must add the relevant IAM permissions to each element of the architecture.
6. Further reading
Getting started with Amazon S3 - How to create a bucket in S3
Getting started with Lambda - How to create a Lambda function
Creating Object Lambda Access Points - No further info is needed here
Creating a distribution - This one is straighforward, too
Creating a new origin access control - Straight-to-the-point titles in the AWS documentation
Creating cache policies - What can I say
Headers for determining the viewer's location - CloudFront headers we can use to identify the location of the user
Top comments (0)