How we solved AWS Lambda timeouts... and improved performance

Matthias Sondermann — Wed, 06 Oct 2021 10:08:07 +0000

TL;DR

CDK sets default lambda memorySize to only 128MB

CPU power of lambda scales with memorySize

Node needs quite some RAM

Performance of Lambdas can be improved considerably by increasing memorySize

Our AWS tech stack

One of our microservices uses a Node-v14 runtime and runs on AWS Lambda using AWS API Gateway.

We build the infrastructure with AWS CDK, which is a Infrastructure as Code (IaC) framework to create AWS resources, and use AWS SAM with its CLI to do local development.

This local development unfortunately is far from resembling the cloud environment, which has caused us already quite some trouble (thinking of CORS handling, binary media types, DynamoDB permissions, ...), and now we had another issue with a new Lambda even though it ran fine locally in a couple of seconds.

The lambda timeout... again

When investigating why the newest function did not work as expected, we realized through the CloudWatch Logs, that the lambda timed out after 60 seconds.
That is way too long for our lambda, which was just fetching some data from DynamoDB and creating and storing some of it in Google Sheets. So we spent quite some time figuring out whether there were configuration issues with DynamoDB or the newly introduced googleapis had any problems connecting.

But everything was setup correctly - so what could it have been? At the very end, we realized that we had been using all of the available 128 MB RAM allocated to the lambda, you can see that in your CloudWatch Logs in the last REPORT entry.

Increasing the memory size

128 MB seemed rather low anyway, so while looking up how pricing scaled up, I also found out that lambda CPU power scales proportionally with memory size (see this).

So it was about time to give that Lambda some more power to play with. By simply increasing the memorySize from 128 to 384 the functions were running fine, and even saved us money, since they ran a lot faster. A simple function that just created an entry in DynamoDB now ran 4x faster than before, so overall being even cheaper than the previous, slow and low RAM version.

Now that we have this in mind, we are experimenting with different memory sizes to see what fits best in terms of performance/price for our use case.

We never had issues with this prior, because our previous services were built with Serverless Framework which sets a default memory size of 1024 MB.

For CDK, you can set memorySize as a property in your lambda.Function constructor props.

DEV Community: Matthias Sondermann

How we solved AWS Lambda timeouts... and improved performance

TL;DR

Our AWS tech stack

The lambda timeout... again

Increasing the memory size