DEV Community: James Miller

Rust Lambda Hello World Tests

James Miller — Sun, 27 Mar 2022 13:09:17 +0000

This is my third and final post in this series about getting started with writing Lambda's in Rust. So far, we have created a new lambda, cross compiled it to run in the AWS Lamdba environment, and added some logging so we can debug issues and see how the code is executing.

In this final post, we will add tests and setup Github Actions to verify these test on any Pull Request.

You can check out the previous posts here

https://millerjam.github.io/posts/rust-lambda-hello-world/

https://millerjam.github.io/posts/rust-lambda-hello-world-logging/

Let's add a test

Now that we have some code and logging in place, there is one final thing I want to cover before we could consider adding more code. We should quickly look at how to test this code.

The AWS Lambda Rust Runtime makes it easy to test your handler code. When we look at the existing main() method we see that we are just registering our handler function func with the lambda_runtime.

This forces a nice decoupling between the code wiring up the lambda environment, and the business logic in the handler function.

Business logic

In the "hello world" example, the business logic is as follows:
1) Check for a "firstName" value in the incoming event.
2) If firstName is found, return "Hello, [firstName]!"
3) If firstName is NOT found, return "Hello, world!"

Writing a test

The easiest way to add test code, is to create the tests in the same file as the function we are trying to test.

In our example we will add a new test directly in the src/main.rs file.

Here is the code we will add as the bottom of the file

main.rs

#[cfg(test)]
mod tests {
    use super::*;
    use lambda_runtime::Context;

    #[tokio::test]
    async fn test_func() {
        let context = Context::default();
        let payload = json!({"firstName": "James"});
        let event = LambdaEvent { payload, context };
        let result = func(event).await.unwrap();
        assert_eq!(result["message"], "Hello, James!");
    }
}

You may notice that we are using #[tokio::test] and that the test itself is marked async, this is because the lambda rust runtime is using the tokio framework and when testing async function we need to using these constructs.

First, we create the test Event using a default Context, and a JSON event with the proper field. Once we have created our test event, we are able to use that as the payload to call our function func(event). We use the await.unwrap() to get the result since our handler method func is an async function.

The test_func itself is a "happy path" test, verifying that we will get the expected output when the "firstName" field is found in the event. The final step of the test is to assert that the returned value matches the expected "Hello, James!"

Adding a second test

Now that we covered the "happy path" logic, we should add one more test to cover the case when "firstName" is not found

In the same mod test block, we can add a second test.

#[tokio::test]
    async fn test_func_bad_request() {
        let context = Context::default();
        let payload = json!({"something": "wrong"});
        let event = LambdaEvent { payload, context };
        let result = func(event).await.unwrap();
        assert_eq!(result["message"], "Hello, world!");
    }

This test follows the same format as the previous, with the difference being the payload is updates with different JSON value. We expect that the handle will return "Hello, world!" in this case.

Running test

Now that we have created some tests, lets run them with cargo

cargo test

We should get output like this

running 2 tests
test tests::test_func_bad_request ... ok
test tests::test_func ... ok

Success! We can see that cargo found the 2 tests, and both are passing.

Automating test with Github Actions

Now that we can run tests locally, it would be nice to have them running automatically when we create a new PR for our project.

Adding github actions

This is really easy to setup, using actions-rs.

This project provides a toolkit of GitHub Action for Rust projects.

We need to create a rust.yml file in .github/workflows/rust.yml

The yaml file will contain "jobs" for the cargo commands check, test, fmt, and clippy. This will make sure that all PRs made against our repo, are properly formatted, have passing test, and passing all standard rust linting checks.

The entire file is here
https://github.com/millerjam/rust_lambda_hello_world/blob/main/.github/workflows/rust.yml

One you have committed this file into your github repo, you will see it populated in the "Action" tab

Passing PRs will show a green check, and if any of the checks fail the PR will be marked as failed. And selecting the failed action will show the 4 jobs and which has failed

Conclusions

Now, we have a completed "hello world" lambda example. We created:

Final GitHub Project

You can reference the complete project here in my github repo https://github.com/millerjam/rust_lambda_hello_world

Rust Lambda Hello World Logging

James Miller — Sun, 13 Mar 2022 18:35:38 +0000

In my previous article, I started a tutorial on creating a lambda in Rust. If you missed it, you can catch up on that article here -> Rust Lambda Hello World

Lets add some logging

Before we go much further with our Rust code, let's add some logging. We are take advantage of the tracing library which is already integrated into many of the lambda dependencies.

Details about the tracing library are out of scope for this article, but you can learn more from the tracing crate docs

Init the logger

Setting up the logger requires initializing the tracing subscriber library, the AWS Lambda Rust Runtime Examples has a examples with some details about how to get started.

Here is the code that we will have to add:

fn init_lambda_tracing() {
    tracing_subscriber::fmt()        
        .with_max_level(tracing::Level::INFO)
        // this needs to be set to false, otherwise ANSI color codes will
        // show up in a confusing manner in CloudWatch logs.
        .with_ansi(false)
        // disabling time is handy because CloudWatch will add the ingestion time.
        .without_time()
        .init();
}

I have created a small function init_lambda_tracing() with all the details for setting up the logging. This function will setup the default tracing_subscriber which prints logs to stdout. The additional methods called on the builder, set the default logging level to "info" and disable some of the defaults to make things nicer in the Lambda environment. Finally, I added additional code to the main() to call this function before the lambda runtime setup.

Fix the compile issues

Even though we did not add any additional import ... lines, we still need to add our new tracing library dependencies to Cargo.toml, before our code will build.

The reason we did not add import.. statements is that we fully qualified the crate directly in the code with the prefix tracing_subscriber::....

Lets go ahead and add the 2 dependencies: tracing, and tracing-subscriber

cargo add tracing

cargo add tracing-subscriber

Lets test it!

Ok now that we have logging setup, let's add some logs and try it. The tracing library provides logging macros such as, debug!, info!, warn!, error!

We will add an new "info" log to the func method.

info!("Going to say hello to {}!", first_name);

We will also need to import

use tracing::info;

Now we can update the function and test.

Compile & update the function

Just like in the previous article, we need to compile for the lambda env and update the function.

Compile

cargo zigbuild --release --target x86_64-unknown-linux-gnu

Update the existing function

aws lambda update-function-code --function-name rustTest \
  --zip-file fileb://./lambda.zip

Test invoke & check the logs

Now that the lambda is update with the new code, we can go ahead and test and check for our new logs.

Invoke

aws lambda invoke \
  --cli-binary-format raw-in-base64-out \
  --function-name rustTest \
  --payload '{"firstName": "James"}' \
  output.json

Now check the cloudwatch logs again

START RequestId: 93bcbe8c-4b2e-4af3-b00d-3b4b27083ebe Version: $LATEST
INFO lambda_test: going to say hello to James
END RequestId: 93bcbe8c-4b2e-4af3-b00d-3b4b27083ebe
REPORT RequestId: 93bcbe8c-4b2e-4af3-b00d-3b4b27083ebe  Duration: 1.16 ms   Billed Duration: 33 ms  Memory Size: 128 MB Max Memory Used: 17 MB  Init Duration: 31.20 ms 
XRAY TraceId: 1-622553a3-5644934610a4952626f10403   SegmentId: 3b7b79993cba7894 Sampled: true

Success! We now have our log,

INFO lambda_test: going to say hello to James

Reconfigure the logger

Update the code for dynamic log levels

Now that we have logging working, let's reconfigure it so that it reads the log level
from an environment variable. This is will be much more flexible and give us the ability to turn logging levels up and down while we are developing and debugging, without having to recompile the code.

We can update the init_lambda_tracing like this:

fn init_lambda_tracing() {
    tracing_subscriber::fmt()
        .with_env_filter(EnvFilter::from_default_env())
        // this needs to be set to false, otherwise ANSI color codes will
        // show up in a confusing manner in CloudWatch logs.
        .with_ansi(false)
        // disabling time is handy because CloudWatch will add the ingestion time.
        .without_time()
        .init();
}

We replaced the hardcoded set_max_level... with

.with_env_filter(EnvFilter::from_default_env())

this will read the logging level from the
environment variable RUST_LOG. Now we can change the log level dynamically without having the recompile and update the running code.

We will also have to update the Cargo.toml to make sure the "EnvFilter" feature is available from the tracing_subscriber library.

tracing-subscriber = {version = "0.3.9", features = ["env-filter"]}

Compile & update the function configuration

Now that code is update, we need to rebuild, update the function, and then also the config
to add the new "RUST_LOG" environment variable. Let's set the logging level to "DEBUG" for testing.

Compile

cargo zigbuild --release --target x86_64-unknown-linux-gnu

Update the existing function

aws lambda update-function-code --function-name rustTest \
  --zip-file fileb://./lambda.zip

Update the existing function configuration

aws lambda update-function-configuration \
    --function-name  rustTest \
    --environment Variables="{RUST_BACKTRACE=1,RUST_LOG=debug}"

Test invoke again

aws lambda invoke \
  --cli-binary-format raw-in-base64-out \
  --function-name rustTest \
  --payload '{"firstName": "James"}' \
  output.json

Check the cloudwatch logs again

Logs are much more verbose, we can see lots of new logs from the included dependencies.

START RequestId: 9ebad5dc-3cee-4302-a0f8-0e592e432f41 Version: $LATEST
DEBUG hyper::client::connect::http: connecting to 127.0.0.1:9001
DEBUG hyper::client::connect::http: connected to 127.0.0.1:9001
DEBUG hyper::proto::h1::io: flushed 109 bytes
DEBUG hyper::proto::h1::io: parsed 7 headers
DEBUG hyper::proto::h1::conn: incoming body is content-length (21 bytes)
DEBUG hyper::proto::h1::conn: incoming body completed
DEBUG hyper::client::pool: pooling idle connection for ("http", 127.0.0.1:9001)
INFO lambda_test: going to say hello to James
DEBUG hyper::client::pool: reuse idle connection for ("http", 127.0.0.1:9001)
DEBUG hyper::proto::h1::io: flushed 198 bytes
DEBUG hyper::proto::h1::io: parsed 3 headers
DEBUG hyper::proto::h1::conn: incoming body is content-length (16 bytes)
DEBUG hyper::client::connect::http: connecting to 127.0.0.1:9001
DEBUG hyper::proto::h1::conn: incoming body completed
DEBUG hyper::client::pool: reuse idle connection for ("http", 127.0.0.1:9001)
DEBUG hyper::proto::h1::io: flushed 109 bytes
END RequestId: 9ebad5dc-3cee-4302-a0f8-0e592e432f41
REPORT RequestId: 9ebad5dc-3cee-4302-a0f8-0e592e432f41  Duration: 1.46 ms   Billed Duration: 38 ms  Memory Size: 128 MB Max Memory Used: 17 MB  Init Duration: 36.11 ms 
XRAY TraceId: 1-622551be-6d58480b496655162937c59c   SegmentId: 03a6c5576dc28250 Sampled: true   
DEBUG hyper::client::connect::http: connected to 127.0.0.1:9001
DEBUG hyper::client::pool: pooling idle connection for ("http", 127.0.0.1:9001)

Conclusions

We were able to successfully add logging to our "hello world" lambda, and make it possible to dynamically change the logging level based on an environment variable. Using the tracing library made this easy, and the code is straight forward.

Finally the Source Code

The complete project is here in my github repo https://github.com/millerjam/rust_lambda_hello_world

Next Time

Next time, we will look at adding a unit test and finally try switching from building x86 to ARM for even more savings on our lambda executions.

Rust Lambda Hello World

James Miller — Mon, 07 Mar 2022 22:06:12 +0000

Getting started with AWS Lambda Rust Runtime

I just finished a quick tutorial to create a hello world lambda using the AWS Lambda Rust Runtime

Github repo for the tutorial

If you're in a rush...

You can view the completed source code here: rust_lambda_hello_world

Overview

I think that Rust is a great fit for serverless and I wanted to start documenting how to use Rust in a lambda. Lambdas are a great opportunity to test out a new language without making a huge commitment in terms of time or resources. Additionally, the explicit connection between execution time and cost highlight the benefit of Rust's speed, both for for cold starts and warm execution time.

Before getting started here some prerequisites that you should already have working...

Prerequisites

Install rust/cargo/etc
Code editor
AWS Account

Create a new rust binary

Let's get started with creating a new rust binary

cargo new lambda_test

Here I am using cargo to create a new binary with the name "lambda_test". This will create a new directory and create a scaffold of a rust project.

Getting started

Now we are ready to get started writing our first lambda in rust. We will create a simple "hello world" lambda that will read an event that contains a first name, and respond with "Hello [first name]." Along the way we will cover writing the code, adding the crates, cross compiling for the lambda environment, deploying, and finally testing our code.

Let's get started...

Add the code

To make this easy, we'll copy the code from the AWS Lambda Rust Runtime README example. (Note: make sure you've selected the tag for version, v0.5 at the time I'm writing this) Copy the code snippet below, and replace the existing code in your project's lambda_test/main.rs file.

use lambda_runtime::{service_fn, LambdaEvent, Error};
use serde_json::{json, Value};

#[tokio::main]
async fn main() -> Result<(), Error> {
    let func = service_fn(func);
    lambda_runtime::run(func).await?;
    Ok(())
}

async fn func(event: LambdaEvent<Value>) -> Result<Value, Error> {
    let (event, _context) = event.into_parts();
    let first_name = event["firstName"].as_str().unwrap_or("world");

    Ok(json!({ "message": format!("Hello, {}!", first_name) }))
}

This code has 2 methods. The main() which initializes some lambda related scaffolding. And the func(event: LambdaEvent<Value>) which is our code's handler function. This is where all the business logic for our lambda lives.

You'll notice that the AWS Lambda Rust Runtime uses tokio so the main function is preceded with a tokio macro and our handler function func needs to be async. The details of tokio are out of scope for this article. For now it is enough to just know it is an async framework used in some rust projects.

Fix the copy pasta errors

Currently the lambda will not build, your IDE may show some errors related to crates not found.

Lets update the Cargo.toml with the required libraries.

I use Cargo Edit which extends Cargo with commands to manage dependencies without having to manually edit your Cargo.toml

cargo add lambda_runtime

cargo add serde_json

cargo add tokio

Here we have added the AWS Lambda Rust Runtime, Serde JSON, and tokio

Finally, the project should be ready to build.

How to build

We will need to build a binary for the lambda environment, which might be different from our laptop. I'm using MacOS, so I'll need to cross compile. If you are using linux you might be able to build for our target architecture without these extra steps. You are are using a mac, you can follow along to add the zig linker and appropriate target for the Lambda environment.

Check that normal local builds are working, and fix any dependency issues if needed.

Before we get started with the cross compile steps, lets make sure a normal build works without and issues.

cargo build

Ok that should succeed and let us know that the code and dependencies are correct.

Build for AWS Lambda install x86 target

We are going to target the "Amazon Linux2 x86" for our Lambda environment. In order to achieve this from a MacOS we will need to cross compile. The AWS Lambda Rust Runtime Github repo has instructions for cross compiling that we will follow here. They use zig and a cargo binary zigbuild to accomplish the build process.

First install Zig

brew install zig

Install cargo zigbuild

cargo install cargo-zigbuild

Add the target for x86

rustup target add x86_64-unknown-linux-gnu

Finally, we build the release

cargo zigbuild --release --target x86_64-unknown-linux-gnu

Package and Create the Lambda

Ok, now we have a binary that we can run on the AWS Lambda infrastructure. Next step, we need to bundle the binary and upload to our AWS account.

Create a zip archive

In order to install and test our lambda, we are going to create a zip file with the binary. The binary is expected to have the name "bootstrap", so the following command renames it and creates a zip file that is ready to be uploaded to AWS.

cp ./target/x86_64-unknown-linux-gnu/release/lambda_test ./bootstrap && zip lambda.zip bootstrap && rm bootstrap

Create new function

Note: Currently, I'm missing details about how to create the role and copy the ARN

Now that we have the zip file with the "bootstrap" binary, we are ready to create a new lambda function. In the following command we use the aws cli to create a new lambda function with the name "rustTest".

aws lambda create-function --function-name rustTest \
  --handler doesnt.matter \
  --zip-file fileb://./lambda.zip \
  --runtime provided.al2 \
  --role arn:aws:iam::{PUT_AWS_ACCT}:role/lambda_basic_execution \
  --environment Variables={RUST_BACKTRACE=1} \
  --tracing-config Mode=Active

Note you will need to modify this command with your own "--role" name including account number.

Once we have run this command the new rust lambda function exists in our AWS account and we are ready to run a test.

Test and Check the logs

Test invoke

We can invoke for testing directly from the cli using this cli command.

aws lambda invoke \
  --cli-binary-format raw-in-base64-out \
  --function-name rustTest \
  --payload '{"firstName": "James"}' \
  output.json

Check output

The output of the lambda is written to a file specified in the "invoke" command.

> cat output.json
{"message":"Hello, James!"}%

Check logs

Note: would be good to add how to check cloudwatch logs from cli

After invoking our "rustTest" lambda, we will want to check the cloudwatch logs and see the execution time.

START RequestId: 2b0a5ad0
Version: $LATEST
END RequestId: 2b0a5ad0
REPORT RequestId: 2b0a5ad0  Duration: 0.85 ms   Billed Duration: 1 ms   Memory Size: 128 MB Max Memory Used: 15 MB  
XRAY TraceId: 1-6213be54    SegmentId: 282cd    Sampled: true

The "hello world" lambda doesn't do much, but it is still impressive to see a Billed Duration: 1 ms

Conclusions

We were able to successfully create a Lambda using Rust! Using the AWS Lambda Rust Runtime made this easy, the code is straight forward. And using Zig made it easy to cross compile. The final result is a small binary that runs super fast.

Finally the Source Code

The complete project is here in my github repo https://github.com/millerjam/rust_lambda_hello_world

Next Time

Next time, we will look at moving past "hello world" by adding some more functionality that we would need for any real world projects.

My First Post

James Miller — Tue, 22 Feb 2022 16:09:05 +0000

I started learning Serverless with Rust.

Here is a quick tutorial I started creating, rust_lambda_hello_world. This is a simple "hello world" lambda that uses the AWS Lambda Rust Runtime.

It walks through creating the new project, adding crates, and how to cross compile for Amazon Linux 2 on x86. Finally, create a new AWS lambda function and invoke using the cli.

It was fun to get this started and there are a lot of additional things I can add around testing, logging.