"Ok, how do I run this thing locally", is the first thing most developers from a non-serverless background ask when they're tasked to work on a serverless stack. It's a valid question which seasoned serverless developers have worked around in various ways by:
- Using local emulators like sam local, serverless-offline, localstack, etc.
- Wrapping Lambda handlers in unit or integration tests.
While these approaches sort of work, they're not ideal:
- Local emulators are not always accurate and can be a pain to setup and maintain. Also, they don't work well with other AWS services like S3, SQS, etc.
- Unit tests are great for testing individual functions, but they don't test the entire stack and you'll need to keep test payloads up to date.
What you really want is to be able to run Lambda functions on your local machine under the same context and IAM permissions as in the cloud, but have them triggered by real event sources, such as API Gateway, SQS, EventBridge, etc.
I became inspired to do something about this while listening to Sebastian Bille talk about sst.dev at a recent AWS User Group meetup in Stockholm. SST includes local debugging out of the box and Sebastian's demo looked great.
With my FOMO nerve tightly pinched, I felt determined to deliver a similar experience to SAM/CDK users.
My goals for this were:
- No need to install any additional dev dependencies (given that you already have
samp-cliinstalled). - No need to change your existing SAM template or CDK code.
- No need to change your existing Lambda code.
- It should work for all runtimes supported by Lambda. (so far it supports JavaScript/TypeScript, .NET and Python as of v1.0.40. Java, Go and Ruby is coming)
Get debugging
In this example I will debug a CDK stack since it's slightly more complicated than a SAM. The same approach works for SAM templates as well.
I will use dynamodb-streams-to-eventbridge-outbox-pattern by David Boyne as an example.
Install samp-cli
First, install the latest version of samp-cli
$ npm install -g samp-cli
Deploy CDK stack to AWS if you haven't already
$ cdk deploy
Configure launch.config
Note that automatic launch configuration is currently only supported for vscode.
From your project root, run:
$ samp local --debug
For SAM stacks, this step is fast and easy, as it finds all information it needs in the samconfig.toml file, but for CDK you have to give it some additional information.
This one-time step per project guides you through a short wizard to create (or append to) a launch.json file in your .vscode folder.
Start debugging
Now you're ready to start debugging, but before you press F5 it's a good idea to understand what's happening under the hood. This will help you understand why YOU SHOULD NEVER RUN THIS ON PRODUCTION.
The first thing to understand is that the tool replaces your function code in AWS with a relay proxy for the duration of the debug session. Once you stop debugging, it will restore to the last deployed code artifact.
It also updates your function configuration to set MemorySize to 128mb and Timeout to 60 seconds (it can be increased to 15 minutes by adding environment variable SAMP_TIMEOUT=900 in launch.json). The reason for this is to allow you to sit for longer on breakpoints without the function timing out. Also, the relay proxy doesn't require much memory, so setting it to 128mb gets a bit cheaper.
Both the local machine and the relay proxy lambda function establish an MQTT connection to AWS IoT Core. When the relay function gets triggered, it publishes the event payload, context object and environment variables to the MQTT topic. A local event router receives the message and routes it to the correct function on the developer's machine. Finally, the response it returned to the cloud over MQTT.
Video demonstrating setting breakpoints in each of the function in the above architecture as well as hot-reloading code changes:
The following sequence diagram visualises the flow of interactions (high resolution image here):
Stop debugging
When debugging is stopped, a clean-up script is run. This restores each function according to the last deploy in CloudFormation. Once this is done, functions are invoked as normal in AWS Lambda.
If this should fail for whatever reason, like if your AWS credentials have expired, you can re-run it using samp local --force-restore. If that fails, just redeploy your stack.
Limitations
- CDK can be set up in many different ways and there will be approaches out there that won't work with this tool. It's been tested with the TypeScript/CDK patterns in Serverless Land. I'd like to hear about your setup though, so please take a minute to raise an issue
- Functions using Lambda Layers could cause issues. If the layer is in the same stack, this workaround might work. If you include third party layers, like Lambda Powertools, just make sure you have it included in your
devDependencies. - If your function accesses resources in a VPC, you'll need to establish a tunnel to the VPC resources. See the AWS docs. You can also mock these calls by checking the presence of environment variable
LOCAL_DEBUG.
Cost
The cost of running this should be small, but please make sure you understand the pricing for Lambda and IoT Core.
In short:
- A 128mb Lambda invocation is charged at $0.000126 per minute
- $1 per 1 million AWS IoT messages
About samp-cli
Samp-cli was previously knows as sam-patterns-cli. It was born in a response to the announcement of serverlessland.com/patterns back in the spring of 2021. Since then, it's grown into a versatile productivity tool for SAM users, hence the rename. Here's a list of what it supports at the time of writing:
Conclusion
Every way to improve the developer experience for Lambda is a win for all the serverless developers out there. Getting fast feedback on the code we write has always been important and has not changed with the serverless movement.
I hope this will speed up development for you as much as it has for me. If not, please let me know any improvements you'd like to see by raising an issue.
Oldest comments (0)