Local Development with AWS Lambda and NestJS: Docker Debugging and Hot Reload

Venya Brodetskiy — Sun, 22 Oct 2023 19:15:06 +0000

If you’ve ever worked with C#, Azure Functions, and Visual Studio, you know the comfort. Click here, debug there, drop in Docker with a single click — it’s smooth sailing. Visual Studio does a lot behind the scenes, making our dev life so easy.

But step into AWS Lambdas and Node.js, and the scene changes. Suddenly, things aren’t handed to you on a silver platter. You’re configuring, tweaking, and piecing together bits to make things click.

In this guide, we’re diving into that setup. How to integrate NestJS and AWS Lambda, dockerize it, and even get debugging and hot reload to play nicely inside Docker — all to mimic the ease Visual Studio provides for Azure Functions, but in the Node.js and AWS domain.

Why Integrate NestJS with Lambda?

AWS Lambda provides a serverless computing environment, enabling easy scaling and operational cost savings

NestJS offers a progressive environment for developing scalable server-side applications with Node.js, blending various programming paradigms.

Combining AWS Lambda and NestJS infuses the developer-friendly and flexible environment of NestJS with the serverless, auto-scaling capabilities of AWS Lambda, offering a robust solution for efficient development and deployment.

Advantages of Lambda + NestJS:

Enhanced Scaling: NestJS’s effective HTTP request handling merged with Lambda’s effortless scaling.
Boosted Developer Productivity: Harness NestJS’s developer-friendly aspects in a serverless architecture.
Optimized Costs and Resources: Develop using NestJS’s comfortable environment and deploy with Lambda’s resource-effective strategy.

In the upcoming sections, let’s break down in the steps involved:

Wrapping NestJS with Lambda
Configuring Hot Reload with Nodemon
Setting Up Docker-Compose
Enabling Debugging in VS Code

You can also find demo application in this repo: https://github.com/VenyaBrodetskiy/Lambda-NestJS-Demo

In demo you can find:

configured app according to this guide
how to implement communication between lambdas to work both locally and after deploy
how to add queues (SQS) to your solution

1. Wrapping NestJS with Lambda

In this part I will explain how to wrap the NestJS application within an AWS Lambda function. I used guide from NestJS official documentation, so I will not dive too deep in it here. By link you can also find how NestJS influence on lambda cold start and how to optimize it.

In my case I use serverless-http library, which facilitates the translation of the Lambda event to an HTTP request that a framework (like Express, which NestJS sits on top of) integrates with.

main.ts:

import { NestFactory } from '@nestjs/core';
import serverless, { Handler } from 'serverless-http';
import { AppModule } from './app.module';
import { LogLevel } from '@nestjs/common';

let server: Handler;

async function bootstrap(): Promise<Handler> {
  const isDev: boolean = Boolean(process.env.IS_OFFLINE);
  const logLevels: LogLevel[] = isDev
    ? ['error', 'warn', 'log', 'verbose', 'debug']
    : ['error', 'warn', 'log'];

  const app = await NestFactory.create(AppModule, {
    logger: logLevels,
  });
  await app.init();

  const expressApp = app.getHttpAdapter().getInstance();
  return serverless(expressApp);
}

export const handler: any = async (event: any, context: any) => {
  server = server ?? (await bootstrap());
  return server(event, context);
};

To locally run the Lambda, I’m utilizing the Serverless framework, notable for both emulating Lambda locally and streamlining deployments to AWS.

serverless.yaml:

service: notificationaccessor
frameworkVersion: '3'

provider:
  name: aws
  runtime: nodejs18.x

functions:
  notificationaccessor:
    handler: dist/main.handler
    timeout: 300 # Incremented to prevent timeouts during debugging
    memorySize: 128
    events:
      - httpApi: '*'

plugins: 
  - serverless-offline

custom:
  serverless-offline:
    host: '0.0.0.0' # Vital when operating from within a docker container
    httpPort: ${env:HTTP_PORT, '3002'}
    lambdaPort: ${env:LAMBDA_PORT, '4002'}

notificationaccessoris name of function, feel free to substitute it with your preferred function name
pluginsand custom: Utilize serverless-offline to emulate AWS Lambda and API Gateway on your local machine, essential for local development and testing. Ensure the npm package is installed as a dev dependency.

Post-completion of this stage, you should be capable of running your app utilizing the serverless offlinecommand.

2. Configuring Hot Reload with Nodemon

Adopting Nodemon ensures smooth and efficient local development by automatically restarting application when file changes are detected. Additionally, we’ll be utilizing its features to set the stage for seamless debugging — particularly from within a Docker container.

The configuration for Nodemon is stored in a JSON file, typically named nodemon.json. Here’s how it can be set up for our project:

nodemon.json:

{
  "watch": ["src/**/*.ts", "src/**/*.yaml"],
  "ext": "ts",
  "ignore": ["node_modules"],
  "exec": "nest build && node --inspect=0.0.0.0 node_modules/serverless/bin/serverless offline",
  "legacyWatch": true
}

exec: The command to execute each time an observed file is modified.
nest build: Compiles your NestJS application.
node --inspect=0.0.0.0: Enables the Node.js debugger, binding it to all network interfaces. Crucially, --inspect=0.0.0.0 is essential to permit debugging from inside Docker in forthcoming steps.
node_modules/serverless/bin/serverless offline: Initiates the Serverless Offline plugin.
legacyWatch: A fallback mode which can be useful if native file system events are unreliable.

To invoke Nodemon with this configuration, adjust your npm run start script in your package.json to simply call the nodemon command. For instance:

"scripts": {
  ...
  "start": "nodemon",
  ...
}

With these settings, each time you modify a TypeScript file within the src directory, Nodemon will rebuild your NestJS app and restart the Serverless Offline plugin, while keeping the Node.js debugger accessible for Docker-based debugging in upcoming segments.

3. Setting Up Docker-Compose

Guide to install docker on your machine

Docker is hailed in the development realm for its ability to create a consistent environment across all stages of a project, which simplifies debugging and facilitates collaborative development by keeping the software environment consistent among all developers.

Before we hop into Docker-Compose, let's ensure our Dockerfile is prepared.

Dockerfile:

FROM node:18.18.2-slim

# Installing curl to facilitate health checks if necessary.
RUN apt-get update && apt-get install curl -y

WORKDIR /app

# Copy package manifests and install dependencies
COPY package*.json ./
RUN npm install

# Copy app source to the container
COPY . .

EXPOSE 3000

CMD [ "npm", "start"]

In this Dockerfile, Node 18 is utilized as the base image, ensuring an LTS version to run our app. Dependencies are installed before copying the source code, optimizing Docker cache utilization during builds.

In the docker-compose.yml, one service is defined as an example:

version: '3.8'

services:
  notificationaccessor:
    build: 
      context: ./accessors/accessor.notification
    ports:
      - "3000:3000"
      - "9229:9229"
    environment: 
      - HTTP_PORT=3000
      - LAMBDA_PORT=4000
    volumes:
      - ./accessors/accessor.notification:/app
      - notificationacccessor_nodemodules:/app/node_modules
    # healthcheck to warmup the lambda 
    healthcheck: 
      test: ["CMD", "curl", "-f", "http://localhost:3000"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s

volumes:
  notificationacccessor_nodemodules:

Ports:

3000:3000: Mapping the HTTP port inside Docker to our local to be able to call API Gateway from Postman
9229:9229: Facilitates debugging by mapping the debug port inside Docker to a local port.

Environment Variables:

HTTP_PORT and LAMBDA_PORT: These are correlated with parameters in your serverless.yaml, ensuring consistency between local and Docker-run environments.

Volumes:

./accessors/accessor.notification:/app maps your local code into the Docker container, enabling hot reload and live debugging
notificationacccessor_nodemodules:/app/node_modules ensures that the installed node_modules are utilized and not overwritten by the local volume, aiding in consistency, and speed during local development.

Healthcheck:

While optional, a health check to warmup your Lambda can ensure it's primed and ready. Adjust intervals as needed.

4. Enabling Debugging in VS Code

Now let's configure VS code debugger.

Inside your project's .vscode directory, you'll need to create a launch.json file. This file instructs VS Code on how to manage the startup of your debug environment.

launch.json:

{
  "version": "0.2.0",
  "compounds": [
    {
      "name": "Debug Backend",
      "configurations": ["Debug Manager.Plan", "Debug Accessor.Plan"]
    }
  ],
  "configurations": [
    {
      "type": "node",
      "request": "attach",
      "name": "Debug Manager.Plan",
      "address": "localhost",
      "port": 9229,
      "localRoot": "${workspaceFolder}/managers/manager.plan",
      "remoteRoot": "/app",
      "protocol": "inspector",
      "restart": true
    },
    ...
  ]
}

Breaking down launch.json:

Compounds: Compound configurations allow you to group multiple debug configurations for simultaneous (or sequential) launch in VS Code.

"name": "Debug Backend": Simply a friendly name for the compound configuration.
"configurations": [ "Debug Manager.Plan", ...]: An array holding all configurations to be launched. If you have more than a single service, you can add here all of them and attach debugger to each of them in one click

Configurations: Array of configurations for attaching the debugger to different containers. Specific points to note:

"address": "localhost" and "port": 9229": Tell VS Code where to find the debugging session to attach to. When you have more than a single container running, you will need to use unique port for each service
localRootand remoteRoot: Paths ensuring VS Code maps local code files to their corresponding paths inside the Docker container.
"restart": true: Ensures the debugger will try to reattach if the connection is lost.

How to Add Configuration for New Service

When expanding your services, you might need to add new debugging configurations. Simply add a new object inside the configurations array. Update the name, port, and localRoot accordingly, ensuring they correspond to the new service context.

To group multiple configurations into a single debug instance, simply add them to the configurations array inside your compound.

Attaching the Debugger

After running docker-compose up, ensure your application is running inside Docker. Now:

In VS Code, navigate to the Run and Debug sidebar.
Select the appropriate compound or individual configuration.
Click on Start Debugging (or press F5).

If configured correctly, VS Code will attach to the debug session running inside your Docker container. You'll be able to set breakpoints, inspect variables, and utilize all the features of the VS Code debugger - right inside your running container.

Optional Setup: Running Backend with a Button in VS Code

To fully embrace a Visual Studio-like experience in VS Code, particularly running your Docker-encapsulated backend with a single-click functionality, you may also configure VS Code to run your Docker-encapsulated backend with a single click, instead of using docker-compose commands directly. This is achievable with the help of VS Code tasks defined in tasks.json within the .vscode folder.

Here's a concise version of tasks.json:

{
  "version": "2.0.0",
  "tasks": [
    {
      "label": "Start backend with docker-compose",
      "type": "shell",
      "command": "docker-compose -f ./docker-compose.yml up",
      "isBackground": true,
      "problemMatcher": []
    }
  ]
}

You will also need to change launch.json, adding new configuration:

{
  "type": "node",
  "request": "attach",
  "name": "Run backend",
  "address": "localhost",
  "preLaunchTask": "Start backend with docker-compose",
}

With the above setup, here's how to run your backend with a click:

In VS Code, navigate to the Run and Debug sidebar.
Select the Run backend configuration

Click on Start(or press F5).
Now select the appropriate compound or individual configuration for debugging and click Start Debugging to attach debugger. Once debugger is attached, you will see it in logs of docker containers.

Now your Docker-encapsulated backend should begin to build and run, directly from within VS Code, simplifying the local development process and ensuring a consistent environment among all developers.

In Conclusion

Thank you for embarking on this journey through local AWS Lambda development with NestJS, leveraging Docker for consistency and hot reload, and enriching the development process with seamless debugging using VS Code.

Your local development with AWS Lambda should now be as smooth and comfortable as possible, mimicking the ease experienced in other development environments.

🔗 Explore the Demo Repository with the practical implementation of the concepts discussed.

🤝 Your feedback is invaluable! Feel free to drop comments, ask questions, or share your insights and optimizations. Every contribution helps to enhance our collective knowledge and build a resourceful developer community.

Happy Coding! 🚀

DEV Community: Venya Brodetskiy