DEV Community: Eduardo Conti

Ensuring Idempotence: A Guide to Implementing Idempotent Endpoints with NestJS Interceptors

Eduardo Conti — Thu, 11 Jan 2024 04:30:51 +0000

Introduction

Idempotence refers to the property of an operation where applying it multiple times has the same result as applying it once. In the context of APIs, an idempotent operation ensures that making the same request multiple times produces the same outcome as making it once.
Idempotence is related to the effect that an operation has when executed multiple times, and this varies depending on the HTTP method used. Let's analyze the GET and POST methods.

GET:
Idempotent: The read operation does not change the server state. Requesting the same data multiple times does not cause changes to the server. Therefore, the GET method is naturally idempotent.

POST:
Non-idempotent: The operation of creating or sending data via POST usually changes the server state, creating a new resource. Each POST request can create a new resource or have different effects, so it is not idempotent.

The main distinction is that idempotent operations can be repeated without causing different side effects, while non-idempotent operations can lead to different results with each execution. This is crucial to ensure consistency and predictability in API design and server-side data manipulation.

Why Do I Need Idempotency?

Imagine you're managing an online booking system for concert tickets, and your application allows users to reserve tickets for their favorite band's upcoming show. Here's the situation:

A user initiates a ticket reservation by sending an HTTP POST request to the server, specifying the number of tickets and other relevant details.

The server receives the request and successfully reserves the tickets for the user, updating the database accordingly.

However, due to a temporary network issue or a sudden app crash, the client does not receive the successful response from the server.

The user, unaware that their reservation was successful, assumes it failed and decides to retry the reservation by sending another identical HTTP POST request.

The server processes the second request, not recognizing that it's a duplicate, and reserves the same set of tickets again.

Now, when the user checks their reservation history, they are surprised to see two identical reservations, and they have been charged for both sets of tickets.

This undesirable situation occurred because the system lacked idempotence. Ideally, an idempotent system would recognize that the second reservation request is a duplicate and would respond with a status indicating that the request has already been processed or the same response of first request. This way, the user wouldn't inadvertently end up with multiple reservations and duplicate charges.

Implementing idempotence in this scenario would ensure that even if the client retries the reservation due to a lack of response, the server would handle it safely, preventing unintended consequences such as duplicate reservations and charges.

Get Started

To implement an idempotent endpoint, I will use an API that I recently developed for this article: NestJS with RabbitMQ in a Monorepo: Building a Scalable Credit Card Payment System with Decoupled API and Consumers. As we are dealing with credit card charges, we don't want to risk duplicates.

Let's Go To The Implementation

Why Am I Using an NestJS Interceptor?
Because they make it possible to bind extra logic before / after method execution.
In this approach, I will initially pre-save the idempotence entity. Subsequently, after the controller execution, I will update it with the response.

Create class IdempotencyKeyInterceptor



import {
  Injectable,
  NestInterceptor,
  ExecutionContext,
  CallHandler,
  BadRequestException,
  Inject,
} from '@nestjs/common';
import { Observable, of } from 'rxjs';
import { tap } from 'rxjs/operators';
import { IdempotencyRepository } from '../repositories';
import { IIdempotencyRepository } from '@api/idempotency/domain/repositories';

@Injectable()
export class IdempotencyKeyInterceptor implements NestInterceptor {
  constructor(
    @Inject(IdempotencyRepository)
    private readonly idempotencyRepository: IIdempotencyRepository,
  ) {}

  async intercept(
    context: ExecutionContext,
    next: CallHandler,
  ): Promise<Observable<any>> {
    const ctx = context.switchToHttp();
    const request = ctx.getRequest<Request>();
    const idempotencyKey = request.headers['x-idempotency-key'];

    if (!idempotencyKey) {
      throw new BadRequestException(
        "Header 'x-idempotency-key' is required for this request.",
      );
    }

    if (!this.isValidUUID(idempotencyKey)) {
      throw new BadRequestException(
        "Header 'x-idempotency-key' must be a UUID.",
      );
    }

    const idempotencyModel = await this.idempotencyRepository.find(
      idempotencyKey,
    );

    if (idempotencyModel) {
      return of(idempotencyModel.response);
    }

    await this.idempotencyRepository.preSave(idempotencyKey);

    return next.handle().pipe(
      tap(async (data) => {
        await this.idempotencyRepository.update(idempotencyKey, data);
        return data;
      }),
    );
  }

  private isValidUUID(uuid: string) {
    const uuidRegex =
      /(?:^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[a-f0-9]{4}-[a-f0-9]{12}$)|(?:^0{8}-0{4}-0{4}-0{4}-0{12}$)/u;
    return uuidRegex.test(uuid);
  }
}

Dependencies Injection:
The class constructor injects an implementation of the IIdempotencyRepository interface through the IdempotencyRepository. This is a clear example of dependency injection, allowing for the flexibility to swap out different implementations of the repository.
Interception Logic:
The intercept method is the core logic of the interceptor. It checks for the presence of an 'x-idempotency-key' header in the incoming request. If not present, it throws a BadRequestException. It also ensures that the idempotency key is a valid UUID.
It then queries the IdempotencyRepository to check if there's an existing entry for the idempotency key. If found, it short-circuits the request by returning the stored response.
If no existing entry is found, it pre-saves the idempotency key using the repository before allowing the request to proceed to the controller.
After the controller execution, it updates the idempotency entry with the actual response from the controller.
Idempotency Repository Usage:
The class utilizes methods from the injected IdempotencyRepository to interact with the underlying storage. This repository handles tasks such as finding, pre-saving, and updating idempotency entries.
Dependency Inversion:
By injecting the IdempotencyRepository interface, the class adheres to the dependency inversion principle. This design pattern allows for a more modular and testable codebase, where the specific implementation details of the repository can be swapped out without affecting the functionality of the interceptor.

you can see the creation and implementation of this interceptor and the repository (in memory) in this commit.

Be Careful

Please exercise caution, as I haven't applied this approach in a production environment. If you identify any issues with this implementation or have suggestions for enhancement, I welcome and appreciate your valuable insights. Feel free to share your opinions and recommendations.

Conclusion

In the pursuit of ensuring idempotence in API endpoints, this article navigated through the fundamental concepts, explored the distinctions between idempotent and non-idempotent operations, and delved into the importance of maintaining consistency and predictability in API design. Drawing attention to the potential pitfalls of lacking idempotence, a real-world scenario illuminated the need for robust solutions.

The journey continued with a practical implementation using NestJS Interceptors, showcasing the development of an IdempotencyKeyInterceptor. Leveraging the principles of dependency inversion, the interceptor seamlessly integrated with an Idempotency Repository, providing a modular and testable solution.

The detailed breakdown of the interceptor's logic, from dependency injection to interception strategy, highlighted its role in handling idempotency gracefully. The careful design allowed for pre-saving idempotent entities and updating them post-controller execution, introducing an effective approach to address the challenges posed by non-idempotent scenarios.

As we embark on implementing idempotent endpoints, the cautionary note underscores the importance of prudence. The class, though well-architected and designed, has not been battle-tested in a production setting. It beckons the community to scrutinize and contribute, welcoming opinions, identifying potential pitfalls, and offering suggestions for refinement.

In the spirit of collaborative improvement, this article encourages readers to share their experiences and insights, fostering a collective endeavor towards more robust, reliable, and idempotent API solutions. With a foundation laid in understanding, implementation, and community collaboration, the journey towards ensuring idempotence is well underway.

repo: nestjs-rabitmq-example
links:

NestJS with RabbitMQ in a Monorepo: Building a Scalable Credit Card Payment System with Decoupled API and Consumers

Eduardo Conti — Sun, 07 Jan 2024 18:17:37 +0000

Introduction

In this article, we will explore the creation of a credit card payment application using NestJS and RabbitMQ to handle billing generation with a Payment Service Provider (PSP). Additionally, we will incorporate Docker and Docker Compose to streamline container management.
Initially, we will create an endpoint simulating the billing creation process, performing a database insert and an HTTP request. We will observe that the response time for this endpoint is unacceptably high, around 1100 ms. Next, we will introduce a message broker service to asynchronously process the HTTP request for billing generation, resulting in a significant reduction in the endpoint's response time.
To conclude, we will address the separation of the consumer from the API in a monorepo, enabling both to scale independently. I want to highlight that I developed this project using the Clean Architecture, but you have the flexibility to choose the architecture that best suits your needs.

Why should I use asynchronous processing and RabbitMQ?

Asynchronous Processing for Improved Performance:
The introduction reveals that the initial response time for the billing creation process is unacceptably high, around 1100 ms. Introducing a message broker allows you to shift to asynchronous processing, significantly reducing the endpoint's response time. RabbitMQ excels in handling asynchronous communication, enabling your application to scale more efficiently.
Enhanced Scalability and Responsiveness:
By leveraging RabbitMQ, you can decouple the billing generation process from the API endpoint. This decoupling facilitates improved scalability, as both the API and the billing generation service can scale independently. This flexibility ensures that your application remains responsive, even under increased load.
Reliable Message Delivery:
RabbitMQ provides reliable message delivery mechanisms, ensuring that messages are successfully delivered even in the event of system failures or network issues. This reliability is crucial in financial applications like credit card payment processing, where data integrity and consistency are paramount.

In summary, utilizing RabbitMQ in conjunction with NestJS for a credit card payment application brings tangible benefits such as improved performance, scalability, reliability, and flexibility, making it a well-rounded choice for handling asynchronous communication and optimizing your application's overall architecture.

1. Start new nestjs app

$ nest new nestjs-rabbitmq-example
See these changes in the commit.

After installation, I removed app.controller.ts and app.service.ts
See these changes in the commit.

2. Include entity, controller, and services to simulate the creation of a charge using a payment service provider such as Pagarme

$ nest g module credit-card

You can see the classes created in this commit.

Now we have an endpoint that simulates the creation of a credit card charge, as if making an insertion in the database (100ms) and then an http request to pagarme (1000ms).

In this scenario, the client is not required to linger on the HTTP request, awaiting a response from Pagarme; we can efficiently handle this process asynchronously. To achieve this, we will leverage RabbitMQ to queue the creation requests for charges.

3. Add the necessary libs to implement rabbitmq

$ yarn add @nestjs/microservices amqplib amqp-connection-manager
See these changes in the commit.

4. Dockerize application

Dockerfile

FROM node:16

WORKDIR /usr/src/app
COPY package*.json ./

RUN yarn
COPY . .

RUN yarn build

EXPOSE 3000
CMD [ "yarn", "start:prod" ]

docker-compose.yml

version: '3.7'

services:
  credit-card-api:
    container_name: credit-card-api
    restart: on-failure
    build:
      context: .
    volumes:
      - .:/usr/src/app
    ports:
      - 3000:3000
    command: yarn start:dev
    depends_on:
      - rabbitmq

  rabbitmq:
    image: rabbitmq:3.9-management
    container_name: rabbitmq
    restart: always
    hostname: rabbitmq
    ports:
      - 5672:5672  
      - 15672:15672  
    volumes:
      - rabbitmq_data:/var/lib/rabbitmq  

volumes:
  rabbitmq_data:

See these changes in the commit.

4. Connect application with rmq

update file main.ts to use app.connectMicroservice() and app.startAllMicroservices()
I'm not going to delve into rabbitmq settings.

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { RmqOptions, Transport } from '@nestjs/microservices';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  app.connectMicroservice<RmqOptions>({
    transport: Transport.RMQ,
    options: {
      urls: [`amqp://rabbitmq:5672`],
      queue: 'create_charge_psp',
      prefetchCount: 1,
      persistent: true,
      noAck: false,
      queueOptions: {
        durable: true,
      },
      socketOptions: {
        heartbeatIntervalInSeconds: 60,
        reconnectTimeInSeconds: 5,
      },
    },
  });

  await app.startAllMicroservices();

  await app.listen(3000);
}
bootstrap();

See these changes in the commit.

5. Start application

$ docker-compose up --build

Access rabbitmq panel in http://localhost:15672 with default crendentials login: guest and pass: guest

6. Create publisher and consumer

create-charge.publisher.ts

import { Inject } from '@nestjs/common';
import { ClientProxy } from '@nestjs/microservices';
import { catchError, firstValueFrom, throwError } from 'rxjs';
import { CreateChargeInputProps } from 'src/credit-card/domain/contracts/psp-service.interface';

export class CreateChargePublisher {
  constructor(
    @Inject('create_charge_publisher')
    private readonly clientProxy: ClientProxy,
  ) {}

  async publish(data: CreateChargeInputProps): Promise<void> {
    await firstValueFrom(
      this.clientProxy.emit('CREATE_CHARGE_PSP', data).pipe(
        catchError((exception: Error) => {
          return throwError(() => new Error(exception.message));
        }),
      ),
    );
  }
}

create-charge-on-psp.event-handler.ts

import { Controller, Inject } from '@nestjs/common';
import { Ctx, EventPattern, Payload, RmqContext } from '@nestjs/microservices';
import {
  CreateChargeInputProps,
  ICreateCharge,
} from 'src/credit-card/domain/contracts/psp-service.interface';
import { Pagarme } from 'src/credit-card/infra/psp/pagarme/pagarme.service';

@Controller()
export class CreateChargeOnPSPEventHandler {
  constructor(
    @Inject(Pagarme)
    private readonly pspService: ICreateCharge,
  ) {}
  @EventPattern('CREATE_CHARGE_PSP')
  async handle(
    @Payload() payload: CreateChargeInputProps,
    @Ctx() context: RmqContext,
  ): Promise<void> {
    console.log(payload);
    const channel = context.getChannelRef();
    const originalMsg = context.getMessage();
    try {
      await this.pspService.createCharge(payload);
    } catch (error) {
      console.log(error);
    }
    channel.ack(originalMsg);
  }
}

credit-card.module.ts

import { Module } from '@nestjs/common';
import { Pagarme } from './infra/psp/pagarme/pagarme.service';
import { CreateChargeUseCase } from './app/use-cases/create-charge.use-case';
import { CreateChargeController } from './presentation/controllers/create-charge.controller';
import { CreateChargeOnPSPEventHandler } from './presentation/event-handler/create-charge-on-psp.event-handler';
import { CreateChargePublisher } from './infra/rmq/publisher/create-charge.publisher';
import {
  ClientProxy,
  ClientProxyFactory,
  Transport,
} from '@nestjs/microservices';
import { ICreateCharge } from './domain/contracts/psp-service.interface';

@Module({
  providers: [
    Pagarme,
    {
      provide: CreateChargeUseCase,
      useFactory: (pspService: ICreateCharge) => {
        return new CreateChargeUseCase(pspService);
      },
      inject: [Pagarme],
    },
    CreateChargePublisher,
    {
      provide: 'create_charge_publisher',
      useFactory: (): ClientProxy => {
        return ClientProxyFactory.create({
          transport: Transport.RMQ,
          options: {
            urls: [`amqp://rabbitmq:5672`],
            queue: 'create_charge_psp',
            prefetchCount: 1,
            persistent: true,
            noAck: true,
            queueOptions: {
              durable: true,
            },
            socketOptions: {
              heartbeatIntervalInSeconds: 60,
              reconnectTimeInSeconds: 5,
            },
          },
        });
      },
    },
  ],
  controllers: [CreateChargeController, CreateChargeOnPSPEventHandler],
})
export class CreditCardModule {}

See these changes in the commit.

7. Change the use case to send charge data to the rabbitmq queue instead of making the call to pagarme directly

create-charge.use-case.ts

import {
  CreateChargeInputProps,
  CreateChargeOutputProps,
} from 'src/credit-card/domain/contracts/psp-service.interface';
import { CreditCardChargeEntity } from 'src/credit-card/domain/entities/credit-card-charge.entity';
import { ICreateChargeUseCase } from 'src/credit-card/domain/use-cases/create-charge.use-case';
import { IPublisherCreateCharge } from 'src/credit-card/infra/rmq/publisher/create-charge.publisher';

export class CreateChargeUseCase implements ICreateChargeUseCase {
  constructor(private readonly publisher: IPublisherCreateCharge) {}

  async execute(
    props: CreateChargeInputProps,
  ): Promise<Omit<CreateChargeOutputProps, 'pspId' | 'value'>> {
    const entity = CreditCardChargeEntity.newCharge(props);
    console.log(entity);
    await new Promise((resolve) => setTimeout(resolve, 100)); //simulate database access
    await this.publisher.publish(props);
    return { ...props, status: 'PENDING' };
  }
}

See these changes in the commit.

Now our endpoint for creating a credit card charge has an acceptable response time.

Several might halt their progress at this juncture; I've witnessed products in production following this approach. While it's not inherently incorrect, it hinders our ability to scale both the consumer and API horizontally and vertically independently. Additionally, the processing load on the consumer can adversely affect API response times. To address this, we aim to decouple startAllMicroservices() from listen(), instigating a shift towards a monorepo structure. Simultaneously, we'll develop a dedicated app for the consumer, fostering improved scalability and performance.

8. Switch from standard mode to monorepo mode

Let's change the project structure to monorepo.
$ nest generate app credit-card-consumer

See these changes in the commit.

It is imperative to modify the scripts responsible for building and launching the application. Additionally, we must craft a Dockerfile tailored for the consumer, configuring it to spawn three replicas, and subsequently, fine-tune the docker-compose.yml configuration accordingly.

See these changes in the commit.

9. Start application again

$ docker-compose up --build

Now, running the $ docker ps command we can see 3 instances of the consumer and 1 of the api running

10. Decouple the consumer from the api

Currently, the consumer remains tightly coupled to the API, as we've only created the new credit-card-consumer app without implementing any modifications. The next step involves decoupling these components by transferring the responsibility of initiating the consumer to the credit-card-consumer application.

credit-card-consumer/src/main.ts

import { NestFactory } from '@nestjs/core';
import { CreditCardConsumerModule } from './credit-card-consumer.module';
import { RmqOptions, Transport } from '@nestjs/microservices';

async function bootstrap() {
  const app = await NestFactory.create(CreditCardConsumerModule);
  app.connectMicroservice<RmqOptions>({
    transport: Transport.RMQ,
    options: {
      urls: [`amqp://rabbitmq:5672`],
      queue: 'create_charge_psp',
      prefetchCount: 1,
      persistent: true,
      noAck: false,
      queueOptions: {
        durable: true,
      },
      socketOptions: {
        heartbeatIntervalInSeconds: 60,
        reconnectTimeInSeconds: 5,
      },
    },
  });

  await app.startAllMicroservices();
}
bootstrap();

See that here we do not need to invoke the app.listen() method.

nestjs-rabbitmq-example/src/main.ts

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(3000);
}
bootstrap();

See that here we only keep the app.listen() method.
Moreover, I have eliminated the services and controllers produced by the $ nest generate app credit-card-consumer command.

You can see that changes in this commit.

To finalize the decoupling process, it's essential to relocate the event handler to the credit-card-consumer, as it's responsible for both removing and processing messages from the queue. However, a challenge arises as the event handler relies on the Pagarme class within the nestjs-rabbitmq-example structure. To resolve this issue, let's transfer all shared components between the API and the consumer to the 'libs' folder, utilizing the 'nest generate library' resource. Importantly, we must refrain from importing any class from the API into the consumer under any circumstances.
I moved all the shared code to the credit-card lib, basically i kept only the presentation layer for API and consumer.
Now our API is completely decoupled from the consumer, allowing us to scale each component independently.

See these changes in the commit.

11. Limit cpu and memory for containers

Establishing a controlled environment serves to restrict the RAM and CPU usage of containers.

See these changes in the commit.

12. Conclusion

In conclusion, this article outlined the journey of creating a credit card payment application, emphasizing the efficient integration of technologies such as NestJS, RabbitMQ, Docker, Docker Compose, and the use of a monorepo. When addressing the initial challenge of inadequate response time in our endpoint, we implemented a message broker service to enable asynchronous processing of HTTP requests, resulting in significant improvements in system efficiency and responsiveness.

Additionally, we explored separating the consumer from the API within a monorepo, providing flexibility and individual scalability for both parts of the system. It is worth noting that, while I opted for the Clean Architecture, the choice of architecture and implementation within a monorepo remains at the developer's discretion.

This approach not only enhances application performance but also provides a solid foundation for future adaptations and customizations as needs evolve. By seamlessly integrating these technologies within a monorepo environment, we hope this article serves as a valuable guide for those seeking to optimize efficiency and scalability in their payment applications.

repo: nestjs-rabitmq-example

links:

Automating Tag and Release Generation with Semantic Release and GitHub Actions for Node.js Applications

Eduardo Conti — Sat, 06 Jan 2024 20:15:35 +0000

In an increasingly automation-focused software development landscape, streamlining repetitive processes is essential. A common practice involves creating tags and releases to mark different versions of an application. However, this process can become tedious and prone to human errors. It is in this context that tools like Semantic Release and GitHub Actions come into play, offering an automated and robust approach to managing your project's versions.

What is Semantic Release?

Semantic Release is a tool that automates semantic versioning and release creation based on the content of changes made to the source code. By adopting this approach, the tool analyzes commits since the last version, automatically determines the version type (major, minor, or patch) based on the changes made, and then generates a new tag and release. This not only streamlines the process but also ensures consistency and accuracy in your software versions. See default release rules.

The Importance of Conventional Commits

Central to this process is the use of Conventional Commits, a standardized commit message convention. Conventional Commits enable Semantic Release to accurately determine the type of version change based on commit messages. By following this convention, you ensure that your commits provide clear and consistent information for automated versioning.

Get started

Let's install the libs with the necessary plugins to use semantic release with github.
npm i @semantic-release/commit-analyzer @semantic-release/git @semantic-release/github @semantic-release/release-notes-generator @semantic-release/npm --save-dev

Update package.json with



"release": {
    "branches": [
      "main"
    ],
    "repositoryUrl": "https://github.com/your_user/your_repository.git",
    "plugins": [
      "@semantic-release/commit-analyzer",
      "@semantic-release/release-notes-generator",
      "@semantic-release/npm",
      [
        "@semantic-release/git",
        {
          "message": "chore(release): ${nextRelease.version} \n\n${nextRelease.notes}"
        }
      ],
      "@semantic-release/github"
    ]
  }

Update your permissions for GITHUB_TOKEN to Read and write

Add the repository secrets NPM_TOKEN; it is essential for the @semantic-release/npm plugin to update your package.json version.

Create file release.yml in folder .github/workflows



name: Release
on:
  push:
    branches:
      - main
jobs:
  release:
    name: Release
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: "lts/*"
      - name: Install dependencies
        run: npm install
      - name: Release
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
        run: npx semantic-release

Now I'm going to add a feature to my application that implements an endpoint for health check.

$ git commit -m "feat: add health check endpoint"
$ git push origin main

We can see that the first version of the application was generated.
Now, I'm going to add one more feature, an endpoint to return a list of users.

$ git commit -m "feat: add endpoint to list all users"
$ git push origin main

We can see that a new tag was generated with a minor change.

Now I'm going to do a bug fix.
$ git commit -m "fix: bug fix example"
$ git push origin main

We can see that a new tag was generated with a patch change.

To conclude, I will implement a feature that includes breaking changes, resulting in the generation of a tag reflecting a major version update.



$ git commit -m "feat: add authentication for get users route

BREAKING CHANGE: route /users needs authentication"

repository: semantic-release-example

links:

NestJS APM with Elastic and Docker

Eduardo Conti — Thu, 04 Jan 2024 00:12:41 +0000

Introduction

Observability is a crucial aspect of modern software development, offering a comprehensive view into the performance and behavior of applications. As applications become more complex and distributed, understanding how they function in real-time becomes paramount. Application Performance Monitoring (APM) plays a pivotal role in observability, providing developers with the tools to track, analyze, and optimize their applications.

In this article, we'll delve into the implementation of APM with NestJS and Elasticsearch. By harnessing the power of these technologies, developers can gain valuable insights into their application's health, identify bottlenecks, and enhance overall performance.

Pre requirements

NestJs CLI
Docker

1. Init NestJS app

$ npm i -g @nestjs/cli
$ nest new apm-example

2. Add elastic-apm-node lib

$ yarn add elastic-apm-node

3. Create env file

.env
ELASTIC_APM_SERVICE_NAME=apm-example ELASTIC_APM_SERVER_URL=http://apm-server:8200

4. Add @nestjs/config

$ yarn add @nestjs/config

5. Import ConfigModule

app.module.ts



import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { ApmModule } from './apm/apm.module';
import { ConfigModule } from '@nestjs/config';

@Module({
  imports: [
    ApmModule,
    ConfigModule.forRoot({
      envFilePath: '.env',
      isGlobal: true,
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

6. Compiler options

Add compiler options configurations for correct functioning of the agent
Elastic config agent with ts
tsconfig.json
"esModuleInterop": true, "moduleResolution": "node"

7. Create apm module and service

$ nest g module apm
$ nest g service apm

apm.module.ts



import { Module } from '@nestjs/common';
import { ApmService } from './apm.service';

@Module({
  providers: [ApmService],
})
export class ApmModule {}

apm.service.ts



import { Injectable } from '@nestjs/common';
import 'elastic-apm-node/start';

@Injectable()
export class ApmService {}

8. Create files Dockerfile and docker-compose

Create Dockerfile and docker-compose.yml in src folder:
Dockerfile



FROM node:18.12.0-alpine AS build

WORKDIR /app

COPY package.json yarn.lock ./

RUN yarn 

COPY . .

RUN yarn build

FROM node:18.12.0-alpine

WORKDIR /app

COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist
COPY --from=build /app/package.json ./package.json
COPY --from=build /app/yarn.lock ./yarn.lock

EXPOSE ${PORT}

CMD ["yarn", "start:prod"]

docker-compose.yml



version: '3.7'

services:

  apm-example:

    hostname: apm-example

    restart: on-failure

    build:

      context: .

      dockerfile: ./Dockerfile

    volumes:

      - .:/app

    env_file:

      - .env

    ports:

      - "3000:3000"

    command: npm run start:dev

    depends_on:

      - apm-server

    cpus: 1

    mem_limit: 1024M

  elasticsearch:

    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1

    container_name: elasticsearch

    environment:

      - discovery.type=single-node

      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

    ulimits:

      memlock:

        soft: -1

        hard: -1

    volumes:

      - es-data:/usr/share/elasticsearch/data

    ports:

      - 9200:9200

    healthcheck:

      interval: 30s

      retries: 10

      test: curl -s http://localhost:9200/_cluster/health | grep -vq '"status":"red"'

  kibana:

    image: docker.elastic.co/kibana/kibana:7.12.1

    container_name: kibana

    ports:

      - 5601:5601

    environment:

      ELASTICSEARCH_URL: http://elasticsearch:9200

    cpus: 0.1

    mem_limit: 256M 

    depends_on:

      elasticsearch:

        condition: service_healthy

    healthcheck:

      interval: 40s

      retries: 20

      test: curl --write-out 'HTTP %{http_code}' --fail --silent --output /dev/null http://localhost:5601/api/status

  apm-server:

    image: docker.elastic.co/apm/apm-server:7.17.16

    depends_on:

      elasticsearch:

        condition: service_healthy

      kibana:

        condition: service_healthy

    cap_add: ["CHOWN", "DAC_OVERRIDE", "SETGID", "SETUID"]

    cap_drop: ["ALL"]

    ports:

    - 8200:8200

    command: >

       apm-server -e

         -E apm-server.rum.enabled=true

         -E setup.kibana.host=kibana:5601

         -E setup.template.settings.index.number_of_replicas=0

         -E apm-server.kibana.enabled=true

         -E apm-server.kibana.host=kibana:5601

         -E output.elasticsearch.hosts=["elasticsearch:9200"]

    healthcheck:

      interval: 10s

      retries: 12

      test: curl --write-out 'HTTP %{http_code}' --fail --silent --output /dev/null http://localhost:8200/

    cpus: 0.1

    mem_limit: 256M 

volumes:

  es-data:

Run docker containers

$ docker-compose up --build -d --timeout 180

10. GET /

access the application endpoint to collect the first metrics
http://localhost:3000

11. Open elastic app

http://localhost:5601

you should see something similar to this

12. Collect docker metrics with Metricbeat

http://localhost:5601/app/home#/tutorial/dockerMetrics

After install and configure Metricbeat, you should run
$ sudo metricbeat setup $ sudo service metricbeat start

you should see something similar to this

Click in Docker metrics dashboard

you should see something similar to this

Now you can explore Elastic metrics, happy learning!

repo: apm-example