Lambda@Edge allows you to run your own business logic as part of the CloudFront request flow. You can intercept all requests that arrive at CloudFront, all connections to the origin and the responses from the origin and to the client. Lambda@Edge functions come with a series of constraints that make them different from regular Lambda functions, but also a lot of commonalities. One of these is the concurrency quota, which you should check before a production deployment.
Lambda@Edge is executed in so-called regional edge caches, which are, at the time of writing this, 13 AWS regions around the world. I explored this in a previous blog: Which AWS Regions are Lambda@Edge functions executed in?
When a request enters an edge location, it contacts its upstream regional edge cache and runs the Lambda functions there. Lambda@Edge ensures that the functions are replicated to all regional edge caches. With Lambda@Edge as part of the request flow, you're typically limited in the amount of requests your distribution can handle, because edge functions share the regional concurrency limits with the regular Lambda functions in that region.
AWS advertises a default limit of 1000 concurrent lambda execution contexts per region in most accounts, but in reality new accounts are frequently throttled to much smaller numbers than that. I've seen plenty of examples, where an account starts out with 10 concurrent execution contexts per region, which is then gradually increased.
That situation is less than ideal if you're trying to deploy a new CloudFront distribution that caters to a global audience. If you receive more concurrent requests than your quota allows, your users will get an HTTP 503 error code, with a response that looks something like this:
503 ERROR
The request could not be satisfied.
The Lambda function associated with the CloudFront distribution was throttled. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
The solution is relatively straightforward, though. You need to create a quota increase request in each of the 13 regional edge cache regions. The quota that needs to be increased is called Concurrent Executions with the code L-B99A9384. Since doing this 13 times is a bit tedious, I've asked Claude to help out.
Here's a script that I reviewed, which uses the API to list the current quota values across all regions that are also regional edge caches. To run it, you should ensure that your current shell is configured to connect to your target account, e.g. through the AWS_PROFILE environment variable or setting the credentials directly. It uses the GetServiceQuota-API to get the current values for each region.
#!/bin/bash
# Script to check Lambda concurrent execution quota (L-B99A9384) across regions
# Requires AWS CLI configured with appropriate permissions
# Define regions to check
REGIONS=(
"ap-south-1" # Asia Pacific (Mumbai)
"ap-northeast-2" # Asia Pacific (Seoul)
"ap-southeast-1" # Asia Pacific (Singapore)
"ap-southeast-2" # Asia Pacific (Sydney)
"ap-northeast-1" # Asia Pacific (Tokyo)
"eu-central-1" # EU (Frankfurt)
"eu-west-1" # EU (Ireland)
"eu-west-2" # EU (London)
"sa-east-1" # South America (Sao Paulo)
"us-east-1" # US East (N. Virginia)
"us-east-2" # US East (Ohio)
"us-west-1" # US West (N. California)
"us-west-2" # US West (Oregon)
)
# Service quota code for Lambda concurrent executions
QUOTA_CODE="L-B99A9384"
SERVICE_CODE="lambda"
echo "Checking Lambda Concurrent Execution Quotas"
echo "==========================================="
echo "Quota Code: ${QUOTA_CODE}"
echo ""
# Function to check quota in a region
check_quota() {
local region=$1
echo -n "Region: ${region} - "
# Get the quota value
quota_value=$(aws service-quotas get-service-quota \
--region "${region}" \
--service-code "${SERVICE_CODE}" \
--quota-code "${QUOTA_CODE}" \
--query 'Quota.Value' \
--output text 2>/dev/null)
if [ $? -eq 0 ] && [ "${quota_value}" != "None" ]; then
printf "%.0f\n" "${quota_value}"
else
echo "ERROR (check permissions/availability)"
fi
}
# Check each region
for region in "${REGIONS[@]}"; do
check_quota "${region}"
done
It's output will look something like this:
$ ./check_lambda_quota.sh
Checking Lambda Concurrent Execution Quotas
===========================================
Quota Code: L-B99A9384
Region: ap-south-1 - 10
Region: ap-northeast-2 - 10
Region: ap-southeast-1 - 10
Region: ap-southeast-2 - 10
Region: ap-northeast-1 - 10
Region: eu-central-1 - 1000
Region: eu-west-1 - 10
Region: eu-west-2 - 10
Region: sa-east-1 - 10
Region: us-east-1 - 10
Region: us-east-2 - 10
Region: us-west-1 - 10
Region: us-west-2 - 10
As you can see, for most regions the quota values need to be increased to handle a lot of traffic. To do that, you can use the following script. It has a TARGET_QUOTA variable, which it will request as the new quota should the current value be below it. There's also a DRY_RUN variable that you can set to "true" if you just want to see what it would do. Once you run it and you're satisfied with it's proposed actions, you can set DRY_RUN="false" and re-run the script.
It will then use the RequestServiceQuotaIncrease API to trigger the quota increases. You can safely re-run this script multiple times, because that API will prevent you from having multiple concurrent quota increase requests for the same quota in the same region. In that case you'll see an appropriate message in the output.
#!/bin/bash
# Script to check Lambda concurrent execution quota and request increase to 1000 if below threshold
# Requires AWS CLI with servicequotas:GetServiceQuota and servicequotas:RequestServiceQuotaIncrease permissions
# Define regions to check
REGIONS=(
"ap-south-1" # Asia Pacific (Mumbai)
"ap-northeast-2" # Asia Pacific (Seoul)
"ap-southeast-1" # Asia Pacific (Singapore)
"ap-southeast-2" # Asia Pacific (Sydney)
"ap-northeast-1" # Asia Pacific (Tokyo)
"eu-central-1" # EU (Frankfurt)
"eu-west-1" # EU (Ireland)
"eu-west-2" # EU (London)
"sa-east-1" # South America (Sao Paulo)
"us-east-1" # US East (N. Virginia)
"us-east-2" # US East (Ohio)
"us-west-1" # US West (N. California)
"us-west-2" # US West (Oregon)
)
QUOTA_CODE="L-B99A9384"
SERVICE_CODE="lambda"
TARGET_QUOTA=1000
# Set to "true" to actually submit requests, "false" for dry-run
DRY_RUN="false"
echo "Lambda Concurrent Execution Quota Increase Script"
echo "================================================="
echo "Target quota: ${TARGET_QUOTA}"
echo "Dry run mode: ${DRY_RUN}"
echo ""
# Function to check and potentially request quota increase
process_region() {
local region=$1
echo -n "Region: ${region} - "
# Get current quota value
quota_value=$(aws service-quotas get-service-quota \
--region "${region}" \
--service-code "${SERVICE_CODE}" \
--quota-code "${QUOTA_CODE}" \
--query 'Quota.Value' \
--output text 2>/dev/null)
if [ $? -ne 0 ] || [ "${quota_value}" == "None" ]; then
echo "ERROR (unable to retrieve quota)"
return 1
fi
current_quota=$(printf "%.0f" "${quota_value}")
echo -n "Current: ${current_quota} - "
if [ "${current_quota}" -lt "${TARGET_QUOTA}" ]; then
echo -n "NEEDS INCREASE - "
if [ "${DRY_RUN}" == "true" ]; then
echo "Would request increase to ${TARGET_QUOTA} (DRY RUN)"
else
# Submit quota increase request
request_output=$(aws service-quotas request-service-quota-increase \
--region "${region}" \
--service-code "${SERVICE_CODE}" \
--quota-code "${QUOTA_CODE}" \
--desired-value "${TARGET_QUOTA}" \
--query 'RequestedQuota.Id' \
--output text 2>&1)
request_exit_code=$?
if [ $request_exit_code -eq 0 ] && [ "${request_output}" != "None" ]; then
echo "Request submitted (ID: ${request_output})"
elif echo "${request_output}" | grep -q "ResourceAlreadyExistsException"; then
echo "Request already pending"
elif echo "${request_output}" | grep -q "InvalidParameterValueException"; then
echo "Invalid request (quota may already be >= ${TARGET_QUOTA})"
else
echo "FAILED to submit request: ${request_output}"
fi
fi
else
echo "OK (already >= ${TARGET_QUOTA})"
fi
}
# Process each region
for region in "${REGIONS[@]}"; do
process_region "${region}"
done
echo ""
if [ "${DRY_RUN}" == "true" ]; then
echo "To submit actual requests, change DRY_RUN to 'false'"
echo ""
fi
This is what the output could look like.
$ ./increase_lambda_quota.sh
Lambda Concurrent Execution Quota Increase Script
=================================================
Target quota: 1000
Dry run mode: false
Region: ap-south-1 - Current: 10 - NEEDS INCREASE - Request submitted (ID: ...)
Region: ap-northeast-2 - Current: 10 - NEEDS INCREASE - Request submitted (ID: ...)
Region: ap-southeast-1 - Current: 10 - NEEDS INCREASE - Request submitted (ID: ...)
Region: ap-southeast-2 - Current: 10 - NEEDS INCREASE - Request submitted (ID: ...)
Region: ap-northeast-1 - Current: 10 - NEEDS INCREASE - Request submitted (ID: ...)
Region: eu-central-1 - Current: 1000 - OK (already >= 1000)
Region: eu-west-1 - Current: 10 - NEEDS INCREASE - Request submitted (ID: ...)
Region: eu-west-2 - Current: 10 - NEEDS INCREASE - Request already pending
Region: sa-east-1 - Current: 10 - NEEDS INCREASE - Request submitted (ID: ...)
Region: us-east-1 - Current: 10 - NEEDS INCREASE - Request already pending
Region: us-east-2 - Current: 10 - NEEDS INCREASE - Request already pending
Region: us-west-1 - Current: 10 - NEEDS INCREASE - Request submitted (ID: ...)
Region: us-west-2 - Current: 10 - NEEDS INCREASE - Request submitted (ID: ...)
Once these requests are created, it's time to wait. I've found that the best way to monitor progress is through the list of support cases - the service quotas console shows only regional requests and clicking through all regions is a bit annoying.
It can sometimes take multiple days for AWS to increase the quotas, so I suggest you plan ahead before your go-live.
That's it, I hope this will help you! If you want to learn a lot more about CloudFront and HTTP Caching, check out my course on Udemy in which we dive a lot deeper!
Cheers
— Maurice
Cover Photo by Lee Tianxian on Unsplash
Top comments (0)