ほうき星

Posted on Jul 24 • Originally published at qiita.com

Benchmarking AWS Lambda Functions with Side Effects Using AWS Lambda Power Tuning

#lambda #aws

This article is a machine translation of the contents of the following URL, which I wrote in Japanese:

副作用のある Lambda を AWS Lambda Power Tuning でチューニングする方法 #lambda - Qiita

はじめにこんにちは、ほうき星 @H0ukiStar です。 AWS Lambda は、割り当てるメモリのサイズを増やすことによって CPU をはじめとする実行リソースも比例して割り当てられる特徴があります。そのため、割り当てるメモリサイズを変更することにより処理時間も...

qiita.com

Introduction

Hello, I'm @H0ukiStar.

One of the characteristics of AWS Lambda is that increasing the allocated memory also proportionally increases the CPU and other execution resources available to the function.

As a result, changing the memory allocation can significantly affect execution time. Depending on the workload, allocating more memory may reduce the execution time enough to lower the overall execution cost.

However, the optimal memory size depends on the workload, and manually evaluating every memory configuration from 128 MB to 10,240 MB is impractical.

One tool that helps solve this problem is AWS Lambda Power Tuning.

In this article, I'll show you how to use AWS Lambda Power Tuning with Lambda functions that have side effects, such as writing to or deleting data from a database.

What is AWS Lambda Power Tuning?

AWS Lambda Power Tuning is an open-source tool for finding the optimal memory configuration for AWS Lambda functions.

As mentioned earlier, Lambda functions can be configured with memory sizes ranging from 128 MB to 10,240 MB. Increasing the memory allocation also proportionally increases the CPU and other execution resources.

Since Lambda pricing is based on allocated memory × execution duration, increasing the memory allocation can actually reduce the total execution cost if it shortens the execution time sufficiently.

Manually running the same function across multiple memory configurations and comparing the results is tedious and time-consuming.

AWS Lambda Power Tuning automates this process by executing your Lambda function with multiple memory settings and visualizing the performance and cost of each configuration.

This allows you to choose the optimal memory size based on actual measurements rather than intuition.

The project was created by Alex Casalboni and is available on GitHub:

alexcasalboni / aws-lambda-power-tuning

AWS Lambda Power Tuning is an open-source tool that can help you visualize and fine-tune the memory/power configuration of Lambda functions. It runs in your own AWS account - powered by AWS Step Functions - and it supports three optimization strategies: cost, speed, and balanced.

AWS Lambda Power Tuning

AWS Lambda Power Tuning is a state machine powered by AWS Step Functions that helps you optimize your Lambda functions for cost and/or performance in a data-driven way.

The state machine is designed to be easy to deploy and fast to execute. Also, it's language agnostic so you can optimize any Lambda functions in your account.

Basically, you can provide a Lambda function ARN as input and the state machine will invoke that function with multiple power configurations (from 128MB to 10GB, you decide which values). Then it will analyze all the execution logs and suggest you the best power configuration to minimize cost and/or maximize performance.

Note

Please note that the input function will be executed in your AWS account and perform real HTTP requests, SDK calls, cold starts, etc. The state machine also supports cross-region invocations and you can enable parallel execution to generate…

View on GitHub

Deployment

AWS Lambda Power Tuning can be deployed in several ways, such as cloning the GitHub repository and deploying it with AWS SAM, or deploying it directly from the Serverless Application Repository (SAR).

In this article, we'll use the SAR deployment because it's the quickest way to get started.

Open the AWS Lambda Power Tuning application in the Serverless Application Repository and click Deploy.
In the AWS Management Console, review the application parameters, acknowledge IAM role creation, and deploy the application.
Wait until the deployment status becomes Create complete.

Running AWS Lambda Power Tuning

AWS Lambda Power Tuning is implemented as an AWS Step Functions state machine.

To start tuning, execute the state machine with the ARN of the target Lambda function and the desired configuration.

Open powerTuningStateMachine and choose Start execution.
Enter the following JSON as the input, then choose Start execution.

  {
    "lambdaARN": "<Target Lambda ARN>",
    "powerValues": [128, 256, 512, 1024],
    "num": 10,
    "payload": {}
  }

Parameters

powerValues: Memory configurations to test

num: Number of executions for each memory configuration

payload: Event payload passed to the target Lambda function

Viewing the Results

After the execution finishes, open the Execution input and output tab.

The output contains a visualization URL, which displays an interactive report showing execution time, cost, and the recommended memory configuration.

In the example below, 512 MB provides the lowest execution cost.

Why Side Effects Become a Problem

AWS Lambda Power Tuning executes the target Lambda function as follows:

Executes the function num times sequentially for each memory configuration (default behavior)
Executes different memory configurations in parallel

Because of this behavior, Lambda functions that interact with external systems—such as databases—may not execute under identical conditions.

For example, a function that:

Reads from a database
Inserts records
Updates records
Deletes records

changes the external state every time it runs.

To obtain reliable benchmark results, you must restore the execution environment before every invocation.

Depending on your workload, simply preparing the required data may not be sufficient.

Each execution may require its own isolated dataset to ensure that previous executions do not affect subsequent ones.

For example, if multiple executions share the same database records, updates or deletions performed by earlier executions may change the behavior of later executions, resulting in inaccurate benchmark results.

Fortunately, AWS Lambda Power Tuning provides two properties specifically designed for this purpose:

preProcessorARN
postProcessorARN

These allow you to execute arbitrary Lambda functions immediately before and after every benchmark execution.

Preparing and Cleaning Up Data with preProcessorARN / postProcessorARN

AWS Lambda Power Tuning allows you to specify Lambda functions for preprocessing and postprocessing by using the preProcessorARN and postProcessorARN properties.

This makes it possible to prepare and clean up database data before and after every benchmark execution, even when the target Lambda function has side effects.

{
  "lambdaARN": "<Target Lambda ARN>",
  "powerValues": [128, 256, 512, 1024],
  "num": 10,
  "payload": {},
  "preProcessorARN": "<Pre-processing Lambda ARN>",
  "postProcessorARN": "<Post-processing Lambda ARN>"
}

When preProcessorARN and postProcessorARN are specified, every benchmark execution follows this sequence:

One important thing to note is how payload is handled.

According to the executor logic:

function Executor:
  iterate from 0 to num:
    [payload = execute Pre-processor (payload)]
    results = execute Main Function (payload)
    [execute Post-processor (results)]

The original payload is first passed to the pre-processing Lambda.

The return value of the pre-processing Lambda then becomes the input to the target Lambda function.

Therefore, if your target Lambda requires the original payload, your pre-processing Lambda must return it (or return a modified payload that includes the required information).

Note

This behavior only occurs when preProcessorARN is specified.

Without a pre-processing Lambda, the original payload is passed directly to the target Lambda function.

Conclusion

AWS Lambda Power Tuning is an excellent tool for optimizing Lambda memory allocation.

However, if your Lambda function modifies external resources such as a database, you may assume that Power Tuning cannot be applied safely.

As shown in this article, using preProcessorARN and postProcessorARN allows you to prepare and clean up your environment before and after every benchmark execution, ensuring that each run starts from the same state.

It's also important to understand how payload is passed between the pre-processing Lambda and the target Lambda, as this behavior can be surprising the first time you use it.

If you're planning to benchmark a Lambda function with side effects, I hope this article helps you apply AWS Lambda Power Tuning with confidence.

DEV Community