AWS re:Invent 2025 - Debug Lambda functions deployed in cloud like never before (CNS367)

🦄 Making great presentations more accessible.
This project aims to enhances multilingual accessibility and discoverability while maintaining the integrity of original content. Detailed transcriptions and keyframes preserve the nuances and technical insights that make each session compelling.

Overview

📖 AWS re:Invent 2025 - Debug Lambda functions deployed in cloud like never before (CNS367)

In this video, Shridhar Pandey and Harold Sun introduce three new AWS Lambda developer experience features that accelerate serverless development by 40%. They demonstrate seamless console-to-IDE transition that reduces setup time from 35 minutes to 30 seconds, LocalStack integration for 75-80% faster local testing of multi-service workflows, and remote debugging that cuts debugging time by 80-85% by allowing developers to debug cloud-running functions directly from VS Code with breakpoints and variable inspection. Harold showcases troubleshooting a VPC security group issue in an order processing application. Best practices include using SAM sync for continuous deployment, applying IAM policies to prevent production debugging, and leveraging the newly public Lambda roadmap on GitHub for community feedback.

; This article is entirely auto-generated while preserving the original presentation content as much as possible. Please note that there may be typos or inaccuracies.

Main Part

Introduction: The Challenge of Debugging Lambda Functions and the Serverless Development Workflow

Welcome to CNS 367 and good afternoon, everyone. Debugging Lambda functions can seem challenging, but it doesn't have to be. Today we'll share some exciting new features that transform your development experience and make it a breeze to code, test, and debug your Lambda functions. I'm Shridhar Pandey, a principal product manager, and I lead developer experience in AWS Serverless. With me today is Harold Sun, senior software engineer on AWS Serverless. Let's get started.

How many of you spent 30 minutes or more debugging a Lambda function recently? It looks like many of you. Does this sound familiar? Debugging Lambda functions has traditionally been a time-consuming process where you have to repeatedly add logging, deploy, invoke, and check logs. This can disrupt your flow and impact productivity. But what if there was a better way? What if you could seamlessly test Lambda functions on your local device without even deploying to the cloud? And what if you could debug Lambda functions in the cloud as easily as your local code? That's the desired state we've been working toward.

Here is what we will cover today. First, we'll discuss the current serverless developer experience and its challenges. Then we'll introduce new features designed to transform this experience. We'll share best practices to make the most out of these features. Finally, we'll look at what's coming next. Before diving into the session, let's start by understanding the serverless development workflow.

Serverless developer experience can be conceptualized as two distinct but interconnected loops. The first is the inner loop. The inner loop is where you spend most of your time on your local development environment. It's all about rapid iterations: write code, test locally, debug, and repeat. The goal here is fast feedback cycles. The outer loop begins when you push code from your device to a production, staging, or integration testing environment. It includes CI/CD pipelines, staging, deployments, testing, and observability. The cycles are less frequent, but the stakes are much higher. The outer loop involves team collaboration and coordination, and this is where works on my machine meets reality.

Let me ask you this: what do you think is the most important requirement for the inner loop? It is speed. Developers need fast feedback cycles to maintain flow and productivity. When the inner loop slows down, developer productivity plummets. There are many challenges that can slow down the inner loop. Anything that slows down your inner loop can be frustrating.

Three Core Challenges in the Inner Loop and AWS's Solution Features

Today we'll focus on three main challenges in the inner loop for your serverless developer experience: setup friction, local testing limitations, and issues that occur only in the cloud. Let's dive into each one of these in more detail. The first challenge is setup friction. Today, setting up a local development environment for Lambda can take 30 to 40 minutes, and that's being generous. You need to find and install the right set of tools, manually copy code, configure credentials, and set up dependencies on your local device. This creates a significant barrier to getting started with building on your local device, and a major chunk of this effort needs to be repeated every time you create a new application.

The second challenge is limitations with local testing. Serverless applications are generally made up of multiple AWS services. But testing multi-service workflows locally is quite difficult. You need to get the integration right and also set up IAM roles and permissions. Most developers end up deploying to the cloud for testing, which slows down iterations because of the complexity of using services in the cloud and the cost of using services in the cloud, even though it's generally not significant.

The third challenge is debugging what's running in the cloud. Some issues, as we know, manifest only in the cloud environment. For example, VPC configurations, IAM permissions, and service-specific behaviors cannot be fully replicated locally.

Traditional debugging approaches can take 20 to 30 minutes for every bug, if we're being generous. This significantly improves productivity and the time required for resolution. To address these challenges, we've introduced three new features that complement each other really well and come together to form one unified experience: seamless code to console to IDE transition, local stack integration, and remote debugging for functions running in the cloud.

Together, these features significantly transformed the developer experience for serverless applications. First, console to IDE reduces your setup time from 30 to 40 minutes to just under 30 seconds. Local stack integration enables 75 to 80% faster iterations for local testing. And remote debugging reduces debugging time by about 80 to 85%. Overall, these capabilities accelerate development cycles by 40%.

Seamless Console to IDE Transition: Eliminating Setup Friction

The first challenge we addressed was a set of friction. Traditionally, setting up takes 30 to 40 minutes with multiple manual steps. You need to install tools, copy your code, set a project structure, install dependencies, and so on. This creates significant friction, especially for new members on your team or your organization. We solved this with a slick new feature: seamless console to IDE transition.

With one click you can transfer code and configuration to your local VS Code IDE. This completely eliminates any need for manual setup. You can also easily manage dependencies for your functions using package managers like NPM and PIP and use development tools like linters and formatters, all without any setup overhead. Now you can follow the link in the QR code to learn more about this launch, and we have embedded such QR codes throughout the presentation.

Let me show you how this works in practice. I've created a simple hello world function written in Node.js that returns hello from Lambda. Now I'll click the Open in Visual Studio Code button, and this takes me to VS Code on my device. As you can see, it downloads and sets up the code and all configuration.

Now I'll go ahead and change some text in the body. It prompts me if I want to deploy this to the cloud, to which I say yes. Now let's go back to the console and see what this did. Look at that. The change is now automatically synced back in your function in the cloud. As you can see, the process is incredibly streamlined. And if you're authenticated, all it takes is one click.

So the time savings are obviously dramatic. 95% less time to start using your local IDE. Earlier you had to go through a series of steps, which would take on average 35 minutes. Now, one click and you're ready in about 30 seconds. This removes a major productivity barrier.

LocalStack Integration: Accelerating Local Testing of Multi-Service Workflows

The second challenge we'll address today is testing application workflows that are made up of multiple services. For example, here is an application architecture. In this, we have an SQS queue triggering a Lambda function which then writes to a DynamoDB table, which triggers another Lambda function invocation. Now this function sends an event to an EventBridge bus, which writes to an SNS topic. Traditional approaches require deploying all of these services to the cloud, setting up event sources, and configuring IAM roles and policies. This obviously results in slower iterations, typically 10 to 15 minutes per cycle.

To address this friction, we'll launch a new integration with LocalStack, a local cloud emulator in the VS Code IDE. This integration makes it easy to set up LocalStack in your IDE. One click is all it takes. And you get access to emulated AWS services and resources to test against.

This significantly accelerates your local feedback loop for server development. Let me show you how the LocalStack integration works. I'm building a simple vote counting application. The SQS pushes messages about voter data to a Lambda function, which tallies the data and stores the results in a DynamoDB table. As you'll see, I'm in my LocalStack profile and I kick off a SAM build command. The build will take a second. Once it is complete, I'll go ahead and fire a deploy command again in my LocalStack profile. All right, looks like the deployment is now complete. I go ahead and run list commands to validate that the resources are actually created in the LocalStack environment. First the Lambda function, then the SQS, and finally the DynamoDB table. All right, we can see that everything looks good. Now let's run this application. What I've done is created a script to send five test votes to the SQS. I'm going to go ahead and run that script. There you go. As you can see, it has started processing and it's waiting on the five second delay as we desired. Here are the results. They're looking good. So you saw how simple and streamlined all of that was. No more deploying to the cloud just to test your app and see if it works.

The benefits are clear: 75 to 80 percent faster local testing iterations. Essentially, you can now bring the cloud to your local device. Locally emulated resources let you discover issues early in the development cycle. No more deploying to the cloud after each change. Simply deploy and test on your local device using LocalStack.

Remote Debugging for Lambda: Troubleshooting Cloud-Specific Issues in Real Time

Now on to the third challenge, which is issues that occur only in the cloud. There are a whole host of bugs that will only surface once you have your app running in the cloud. For example, VPC configuration related issues. This could be in VPC endpoints, security groups, or a number of other areas. Similarly, IAM permissions related issues like service control policies or role assumption chains and so on. And then there are service specific behaviors like DynamoDB throttling, SQS timeouts, and so on. Now these cannot be replicated locally. This means that every time you have to debug such issues, you have to resort to detailed logging, deploying, testing if the change worked, and repeating it all over again. Obviously, that slows things down, especially when you're trying to rapidly iterate through your application development process.

So presenting remote debugging for Lambda functions. This is exciting and truly a unique feature. Now you can debug functions running in your account in the AWS cloud directly from the local VS Code IDE on your device. You can debug directly against the real function setup and the data it handles, not emulated resources, without requiring any setup. So debug issues that occur only in the cloud like VPC and IAM related issues and interaction between services. This delivers faster time to resolution for cloud-specific issues. Remote debugging also bridges the inner and outer loops that we talked about earlier. You get the IDE experience with breakpoints, step-through debugging, and variable inspection combined with the production fidelity of real AWS services, VPC, and IAM.

It's truly the best of both worlds—lightning-fast iteration with production fidelity. Let me show you how remote debugging works in action. Here we have a simple authentication application that we built using API Gateway and Lambda. The logic is straightforward. If the user's name is Harold, it returns a login successful response; otherwise, login failed. Now let me do a curl against this API with Jason as the input. No offense, Jason. No points for guessing this login failed. We knew it would. And let me do another curl with Harold. Looks like that is failing too. All right, I guess I need to debug to find out what happened.

So for that, I will go to the Application Builder interface in your VS Code editor. This is something we built, and I start a remote debugging session. Now, as you can see, it picked up the local route path on its own. I just provide the POST event payload. And run it. I've also set up a breakpoint for it to step into and stop at the function code. All right, looks like the user info field is not getting the value. Yeah, I think the field name in the request payload has an underscore, whereas the field name in the code does not. So that was thankfully an easy one to find. So I stopped the debug session and fixed it. And now since I have SAM sync running, it automatically deploys this change to the cloud. And let's see if it works this time. Looks like it is working. There you go. You can see the success 200 response code.

Remote debugging reduces debugging time by 80 to 85 percent. Traditional methods require logging, multiple deploy cycles, and sometimes looking through volumes of logs. With remote debugging, you set breakpoints once and get the exact state information to go to the heart of the issue fast. This reduces debugging time from some 20 to 30 minutes to just 3 to 5 minutes per bug. We saw an example of remote debugging in action earlier with a simple example. I would now like to invite Harold to show us how you can use remote debugging with some complex scenarios.

Thanks, Srida. So we have seen remote debugging in action. Now, let's see a more complex example here. In this architecture diagram, we are showing an order processing service application, which reads order information from SQS and a Lambda function processes it and sends it to an external payment API to process that. This application uses a parameter store to store the external payment API configurations like URL or API keys. The Lambda function is put into a private subnet in a VPC and has a VPC endpoint connecting to the parameter store. They also use a NAT gateway in the public subnet to reach the external payment APIs. This is a pretty typical setup.

The problem is this application works in US-East-1. But it recently deployed to EU-West-1, and the application is not working. So let's go see the demo. How do we troubleshoot this issue? So in here, we have a script that can send test messages to the SQS and then it tells the Lambda logs to see the processing.

So first, let me walk you through the logs to see the processing. We're going to send an order to the EU US one. The scripts generated a sample order and put it into the SQS queue in US one. You can see the Lambda logs showing that it has started processing it, and we can see the results. It actually successfully processed it and sent it to the external API. This is a successful scenario. Let's look at a failure scenario. If you want to send an order to US EU West one, from here, we should be able to see from the log that the orders sent to EU West one just came out. From the Lambda logs, we see that the order information is read from SQS, but it was stuck somewhere and the function just exited. There is no more log information, so we can troubleshoot and figure out where this function got stuck.

What we can do is actually fire up the remote debugging. This is our function, which handles the messages from SQS. It's a pretty typical function that reads from the queue and processes orders one by one. We can use AWS SAM to initiate the remote debugging session. AWS SAM works with SAM templates. If your function uses SAM templates, it automatically reads that template and knows a lot of information about it. We set up the remote debugging, loaded the test message events, and invoked it. We set a breakpoint at the line to process a single order. When we hit the remote invoke, it stops at the breakpoint. We can step into that function. We can see it's logging and go to the next step. Here is getting configuration. We can step into that function again. We can see there is a creating a client and trying to reconfigure and get parameters. But once we step through that, the function seems stuck. There must be a problem there. I think we have identified where the function got stuck. It was trying to read the configuration from Parameter Store.

How do we get deeper to find what the actual problem is there? Since we have this architecture, we see the Lambda function tries to read from Parameter Store and it didn't get through. The Parameter Store is having a VPC endpoint. The Lambda function will reach that to read the data from Parameter Store. It's a Parameter Store VPC endpoint that has security groups as well. It's a premise that VPC endpoints have security groups and IAM roles. Those are two common problems that can cause this kind of issue.

So we can quickly look at the function's configurations. We can see this function is attached to a VPC which has a 10.2 CIDR range. We can open that VPC configuration and go to the VPC endpoint sections. To check the Parameter Store VPC endpoint, first we want to check the security groups. Let's open the security group's detailed information. We can see the inbound rules are allowing the 10.10 IP range. This doesn't match the Lambda function's 10.2 range, so this is the reason that actually caused the problem. We can quickly identify that our VPC endpoint actually has the wrong security group inbound rules, which caused our problem.

This works in US-East-1, but I think during the deployment in US-West-2, the Lambda's VPC's CIDR range changed, but the Parameter Store endpoint's security inbound rule was not changed as well. So that caused the problem. We can quickly fix this. It's a very simple fix. We just change the CIDR range of the security group. Then we can send another test order to verify that this actually got fixed. Yes, you can see the test order is generated, Lambda is running, pulling the order from the queue, and processing it and sending it to the external Payment API. It still takes some time, but finally it succeeded.

I think I will end the demo now. You can see during this more complex example, we can use remote debugging to set breakpoints and quickly identify where the problem is and quickly resolve the issue. This is much easier than adding logs and going through a deploy and troubleshooting the logs multiple times. It's much quicker. From here, actually, if we want to do more, we can troubleshoot more because from the logs we can see that the response time from external payment providers is much higher in US-West-1 than in US-East-1. But that's something we can still troubleshoot. I'll leave it there for now. But if you want to see the remote debugging in action, this is how Lambda remote debugging works.

It basically contains three components. On your desktop local device, you have your VS Code IDE. You can install the AWS Toolkit. The AWS Toolkit contains components that set up a local tunnel to AWS IoT Secure Tunneling service that runs on AWS. This is the middle piece. It's very critical because it enables both sides to connect seamlessly. On the Lambda side, we provide a Lambda layer, which is attached to the Lambda function when you set up the remote debugging. That layer will intercept your invokes and also set up a local proxy that connects to AWS IoT Secure Tunneling. Once both sides connect to the tunnel, your local debugger in VS Code can use that tunnel to connect to the runtime in debug mode.

Then you can use all the familiar debugging tools to debug your Lambda function.

Best Practices, Security Considerations, and the Path Forward with Lambda's Public Roadmap

In summary from the previous demos and features, you can say that with all these things, we are able to achieve much faster debugging time and solve issues much faster, improving serverless developers' productivity. We got several quotes from our users showing their enthusiasm about these new features. This one is from Luke, who is a principal engineer at Postman. There's another one from Ominiki, a software engineer at Omminiki. They also think these features are game changers for serverless developers.

Now that you have seen all these features, we want to share with you a few best practices on how to use them better and get the most out of them. We see a lot of tools that you can use in combination at each stage of your development so that you can get better results. If you start from the Lambda console, you can use the console to quickly move your function to the local environment. If you want to add more dependencies and use SAM, you can do all that. Once you move your code and set up your projects locally, you can use SAM to test it. You can use SAM local to invoke your function and SAM local API to stand up local API endpoints to test the function. You can also use the new LocalStack integration to test multi-service interactions locally without even deploying to the cloud.

Once your project gets into progress and you deploy into the testing environment in the cloud, you can use remote debugging to quickly debug issues, identify problems, and quickly fix them. Once your application goes into production, you can use CloudWatch logs and traces to monitor your system and identify different issues. Putting everything together will help you improve your developer experience.

Another thing is security. Friends don't have friends debug in production. Please use remote debugging in your development or testing environment. You can use this IAM policy to stop debugging features in your production account or use SAP to stop that at the organizational level. Or if you want to be more specific, you can apply this to a specific IAM user or IAM role. In the production environment, you can also lock down function update actions and only allow SAM roles to do that. There are a lot of things you can do.

When you do remote debugging, remote debugging allows you to check the status around the code, see the variables, and step through, but it does not allow you to change code. If you want to change code, you would have to do a deployment. There is a tool. SAM has a command called SAM sync. It will continue to watch your code, and if there is any change, it will compile the code and sync that code quickly into the cloud so that you can start another debugging session to try to see if your fix has actually worked. In the previous demo, we actually already showed that. In the QR code, you can see more information about SAM sync.

So for remote debugging, when we hit that remote debug button,

the toolkit actually attaches a layer to your function and makes some changes, like extending the timeout to 15 minutes. It then publishes a new temporary version of your function, and the invoke goes to that temporary version. We make these changes so that we want to avoid disruptions to your other traffic. However, there are some scenarios where you may not want to publish a version. If your function has SnapStart configured or provisioned concurrency configured, publishing a version takes time, like 2, 11, or 22 minutes. So that actually slows you down. If you are debugging in an environment where you are sure it is just you debugging it or you are sending traffic to it, you can check or uncheck that box to not publish a version. You can just debug the latest, which will be much faster.

If your function has event triggers like SQS or even raw API gateways, and if you want to test the function using those event triggers directly without sending test payloads, you can also do this. If your SQS queue actually tries to trigger your function's latest version, then you set it up and uncheck the published version. You can send a message to SQS, then the queue will trigger the function, and it will stop at your breakpoint. So those are a few best practices I want to share with you. Next, I want to invite Shridhar to talk about what will come in this space. Thank you, Harold.

So let's summarize what we saw today. We saw three new Lambda developer experience features that meaningfully transform serverless testing and debugging experience. These capabilities complement each other really well and help bridge the inner and outer loop experience for serverless developers. These new capabilities also work seamlessly with existing capabilities and accelerate local serverless development through unified tooling and integrations. Put together, they can accelerate your serverless development time by 40 percent or more.

Now, this is new. I am super excited to present to you the Lambda roadmap on GitHub. Go scan the QR code and check it out to see what is coming next. We are continuously working on improving Lambda's developer experience based on your feedback. We invite you to take a look at it, share your feedback, and take a peek into what we are building and what things we are exploring. This goes beyond just developer experience. This applies to everything in Lambda.

So if there is anything that you want to see us build next, you are absolutely welcome to provide your feedback because we want to take your input into consideration while we determine what to build next. Previously, there were mechanisms for folks to send this to us through individual channels or through AWS support and various other channels. But now we have decided that we are going to make our roadmap public. So once again, you can visit the roadmap where you can learn more and contribute your ideas, vote on any existing ideas, and tell us how you plan to use something that others are asking for.

Now that we have made it so easy to build serverless applications and debug and test and locally develop your Lambda functions, there is no reason to wait, is there? You can get started in three easy steps. It is that simple to transform your serverless development experience. Follow the link in the QR code. It will take you to our documentation page with a lot of additional details. With that, thank you for taking the time out of your busy schedule to attend this session. We are excited to see how these new capabilities transform your serverless development experience, and please complete the session survey in the mobile app. Your feedback helps us improve.

---

; This article is entirely auto-generated using Amazon Bedrock.