DEV Community: Mohammad Jawad (Kasir) Barati

Write e2e tests for NestJS with testcontainers

Mohammad Jawad (Kasir) Barati — Fri, 28 Mar 2025 18:07:59 +0000

So I was really annoyed last time I was trying to mock and have some tests for my NestJS app. So I decided to try testcontainers once more.

Last time I was trying to mock BullMQ and it was giving me a hard time, and this time around I managed to get it working.

Stick around and see how I did it.

Support Me

You can support me by giving my GitHub repo a star, so more developers start writing tests. I am a true believer of automated tests and TDD.

Code

So basically I really do not wanna spy one anything or mock for that matter. But rather I am only interested in knowing that my queue received a message and did its job, e.g. called an external API.

Actually here I am not gonna test that but if you need to verify that the external API was called I think you can simply have a Wiremock Testcontainer and check it was called.

I will not cover that case here, at least not now. But what I am interested to show you here is how the e2e test will look like:

import { Test, TestingModule } from '@nestjs/testing';
import { INestApplication } from '@nestjs/common';
import { RedisContainer } from '@testcontainers/redis';
import * as request from 'supertest';
import { App } from 'supertest/types';
import { AppModule } from './../src/app.module';

describe('AppController (e2e)', () => {
  let app: INestApplication<App>;

  beforeAll(async () => {
    const container = await new RedisContainer().start();

    process.env.SERVICE_NAME = 'test';
    process.env.REDIS_URI = container.getHostname();
    process.env.REDIS_PORT = container.getPort().toString();

    const moduleFixture: TestingModule = await Test.createTestingModule({
      imports: [AppModule],
    }).compile();

    app = moduleFixture.createNestApplication();

    await app.init();
  });

  it('/ (GET)', async () => {
    const res = await request(app.getHttpServer()).get('/');

    expect(res.status).toBe(200);
    expect(res.text).toBe('Hello World!');
  });
});

NOTE

Here I was not trying to use ConfigService, so I just hacked in my way into the process.env. But feel free to check my repo for a more mature way of dealing with env variables.

GitHub Repo

kasir-barati / nestjs-materials

NestJS tips, tricks, Notes, Things which are not in doc and I used to figure them out and use them

View on GitHub

Instagram: https://www.instagram.com/node.js.developers.kh/
Facebook: https://www.facebook.com/kasirbarati
X: https://x.com/kasir_barati
YouTube: https://www.youtube.com/@kasir-barati
GitHub: https://github.com/kasir-barati/
Dev.to: https://dev.to/kasir-barati
LinkedIn: https://linkedin.com/in/kasir-barati

GraphQL + NestJS + Interfaces + Unions

Mohammad Jawad (Kasir) Barati — Sat, 22 Mar 2025 18:01:10 +0000

So guys I was really confused by the NestJS's doc as to ho you can have an API which can be invoked like this:

query {
  robots {
    name
    ... on HumanoidRobot {
      height
    }
    ... on ScaraRobot {
      axes
    }
  }
}

So in others words I wanted to not repeat myself by having an interface and I needed to also prevent any fields from other types leaking to another one.

The solution was simple, first I had to define my RobotInterface, HumanoidRobot object type and ScaraRobot:

`app/types/robot-interface.type.ts`

import { Field, ID, InterfaceType } from '@nestjs/graphql';

import { HumanoidRobot } from './humanoid.type';
import { ScaraRobot } from './scara.type';

@InterfaceType({
  description: "'Common fields, available for all robots',"
  resolveType: (value) => {
    if ('height' in value) {
      return HumanoidRobot;
    }
    return ScaraRobot;
  },
})
export abstract class RobotInterface {
  @Field(() => ID, { description: "'ID of the robot' })"
  id: string;

  @Field(() => String, { description: "'Name of the robot' })"
  name: string;

  @Field(() => String, {
    description: "'The COLLADA image of the robot',"
  })
  colladaImage: string;

  @Field(() => String, { description: "'Description of the robot' })"
  description: "string;"

  @Field(() => String, { description: "'Model number of the robot' })"
  modelNumber: string;
}

`app/types/scara.type.ts`

import { Field, Float, Int, ObjectType } from '@nestjs/graphql';

import { RobotInterface } from './robot-interface.type';

@ObjectType({
  implements: () => [RobotInterface],
  description: ""
    'Scara robot, specialized for tasks requiring high precision & speed',
})
export class ScaraRobot implements RobotInterface {
  id: string;
  name: string;
  colladaImage: string;
  description: "string;"
  modelNumber: string;

  @Field(() => Int, {
    description: "'Number of axes',"
  })
  axes: number;

  @Field(() => Float, {
    description: "'Payload capacity in kg',"
  })
  payload: number;
}

And something similar to this for the HumanoidRobot.

Caution

the implements option should be a callback function and not a simple implements: [RobotInterface]. This in fact wasted my precious 2 hours. So it should be implements: () => [RobotInterface].

And note that the resolveType will help your NestJS app to decide which type it should use, TBH I saw in a lot of places people use enums. In other word they add an extra field to the RobotInterface and call it type and decide based on that field. I guess that would be easier to have compare to relying on the presence of a field.

So feel free to do that instead of this, it would look like this:

@InterfaceType({
  description: 'Common fields, available for all robots',
  resolveType: (value) => {
    if (value.type === RobotType.HUMANOID) {
      return HumanoidRobot;
    }
    return ScaraRobot;
  },
})
export abstract class RobotInterface {
  // ...

  @Field(() => RobotType, { description: 'Type of the robot' })
  type: RobotType;

  // ...
}

And of course you need to register the enum type first.

Finally we need a resolver:

// ...
@Query(() => [RobotInterface], {
  description: 'Get all robots',
})
robots(): Array<RobotInterface> {
  return this.appService.getRobots();
}
// ...

Unions

This is not really any different that interfaces, just use the createUnionType helper function and pass your classes, then specify on which type it should return what:

import { createUnionType } from '@nestjs/graphql';

import { HumanoidRobot } from './humanoid.type';
import { ScaraRobot } from './scara.type';

export const UnionOfRobots = createUnionType({
  name: 'UnionOfRobots',
  description: 'Union of robots fields',
  types: () => [HumanoidRobot, ScaraRobot] as const,
  resolveType: (value) => {
    if ('height' in value) {
      return HumanoidRobot;
    }

    return ScaraRobot;
  },
});

And then specify that as the return type of your query, or mutation.

Support Me

You can support me by liking this post, reading my other posts and giving my repo a star on GitHub.

As simple as that. Need the code itself? Do not worry I've got you covered:

kasir-barati / graphql-js-ts

Where you can learn all about GraphQL and its intricacies

Caution

Keep this file in sync with index.md.

GraphQL

Tip

Just for those curious minds who always jump from one branch to another like mine:

You can find a good definition usually in glossary.

Intro.
Data types.
A simple todo app written with GraphQL + ReactJS + Relay
- Backend.
- Frontend TK.
Queries and mutations in depth.
Let's breakdown the query language a bit more.
Functions provided by graphql.
Document your GraphQL service API.
GraphQL request lifecycle
- Common validation errors.
- Execution from inside, resolver's args, AST, ...
Code-first approach.
Auth.
How to query information about a GraphQL schema.
Improve developer experience
Security in GraphQL.
NestJS
- …

View on GitHub

Note

This is a monorepo, so I have tons of other apps, but the one you're interested in at the moment is apps/interfaces-unions. BTW If you like to learn how to write e2e tests for you NestJS app look at the apps/interfaces-unions-e2e.

NestJS + AWS S3 + Multipart file upload + gRPC + e2e testing

Mohammad Jawad (Kasir) Barati — Sun, 09 Mar 2025 13:35:33 +0000

So I was wondering how I can upload a file to AWS S3 over gRPC with a NodeJS/NestJS backend app. And ever since I was introduced to the concept of TDD I've been trying to write unit tests and e2e tests, thus I tried to automate my application's tests.

Although this is still far from being perfect but at least it is a good starting point for many of you guys.

How To Support Me?

To support me, please give this post a like, and give the repo I have for this project a star on GitHub

Prerequisites

Read this article of mine.
Read Multipart File Upload in AWS S3.
Read the README.md to know about the design decisions I made.

kasir-barati / nestjs-materials

NestJS tips, tricks, Notes, Things which are not in doc and I used to figure them out and use them

Important

Keep this file synchronized with index.md.

nestjs-materials

NestJS tips, tricks, Notes, Things which are not in doc and I used to figure them out and use them

View on GitHub

Get a list of endpoints in your NestJS app

Mohammad Jawad (Kasir) Barati — Fri, 07 Mar 2025 12:28:35 +0000

So here is how I get a list of all defined HTTP endpoints in my E2E test in a NestJS app with their default configuration and setup:

import { Test, TestingModule } from '@nestjs/testing';
import { SomeModule } from './some/where/some.module';

const moduleFixture: TestingModule = await Test.createTestingModule({
 imports: [SomeModule],
}).compile();
const app = moduleFixture.createNestApplication();
await app.init();

console.log(
  app
    .getHttpServer()
    ._events.request.router.stack.filter((layer: any) => layer.route)
    .map((layer: any) => ({
      method: Object.keys(layer.route.methods)[0].toUpperCase(),
      path: layer.route.path,
    }))
)

This is mostly useful when I am banging my head against a brick wall as to why supertest says an endpoint does not exists (the infamous 404 error). So feel free to use this hack whenever you stock in the same spot :).

BTW feel free to like this post, subscribe and comment.

AWS S3 - Multipart file upload - Surprise Bills

Mohammad Jawad (Kasir) Barati — Sat, 15 Feb 2025 06:26:16 +0000

It is super easy to loss sight of AWS's hidden costs or at least somewhat hard to find unless you're a meticulous person who loves reading docs and ain't under deadline pressure where you cannot find the time to even scratch your back 😂.

Back to the topic of cost management, so assume you're using the multipart file upload and if you're upload was busted in the middle you're not aborting the whole thing. Here is where things gets interesting, AWS will store those chunk of data for you forever, unless you have configured a life cycle rule or have called their AbortMultipartUpload API.

For more details check out my GitHub repo where you can find other places where you might be incurring cost without knowing it. BTW if this was helpful to you consider giving my repo a star so more people can reach it and learn about these aspects of cloud.

kasir-barati / cloud

It was my endeavour to build a wrapper around Docker and develop a RESTful API. In order to serve Platform as a Service. But now it's a place to learn more about cloud and put things into practice, of course things that are related to this topic.

Important

Keep this file content in sync with index.md.

Cloud

I love to hear your thoughts about the things that I am doing here. So reach out. Or better yet, formulate your thoughts in a GH issue.
Most of the things I am putting in this repo are my own original creation, although I'm not reinventing the wheel when it is out there. So I might have taken some parts of other's creation and modified them to suit my needs. So credits goes still to the original creator of those contents.

Note

Initial codebase: this repo was intended to be a professional RESTful API in time, but it didn't come to fruition. So now you can see my endeavour here.

Be Careful With Your Bills

Learn about some common pitfalls when it comes to cloud and its bills here.

Handwritings

I first started by jotting down on…

View on GitHub

BTW you can find me here as well:

NestJS + gRPC + step by step guide + e2e tests

Mohammad Jawad (Kasir) Barati — Sun, 02 Feb 2025 04:13:35 +0000

What is RPC and how you can use it in NestJS is the question I had in mind to answer in this post. Sit tight for a short, concise, and precise intro to this world:

So RPC,

Stands for Remote Procedure Call.
It's a communication mechanism.
It's almost the same as though we have access to the methods/function defined in another program.
Can be used for inter-service communications.
Supper efficient.

[!CAUTION]

This is still a network call. So do not call it like your codes who are living in the same codebase.

[!NOTE]

We can use any application level protocol for sending requests and receiving responses:

TCP: HTTP1, HTTP2.

UDP.

Websockets.

etc.

This means that it is like GraphQL transport layer agnostic. You can use whatever you like.

gRPC

Now let's demystify the gRPC. It's:

An open-source rewrite of RPC used by Google.
Why gRPC?
1. It uses only HTTP2 as its transport layer.
2. Protocol buffer as our data exchange format.
No browser support yet, but a good thing to have for inter-service communication in a microservice architecture.

NestJS

I love NestJS and how good it is at giving you're code more structure. So here is how you can get up to speed with it.

Follow the official documentation opn how to bootstrap a NestJS app + gRPC.

   pnpx create-nx-workspace grpc --preset=nest --appName nestjs-client --bundler esbuild --packageManager pnpm --nxCloud skip

I also love perfectionist for a good reason and that is the tidiness it gives you for little conf, so I highly recommend you take a look at it in their official doc.

Or if you're looking for something on how to get up to speed I guess this commit can be a real quick intro to how to use it. BTW I appreciate it if you give my repo a star in case it was helpful 🙂.

   nx run-many -t lint --fix

Write you're protobuf file like what I did here at user.proto.
Use the ts-proto to auto generate the interfaces and decorators for your gRPC controller.

   {
     "scripts": {
       "grpc:gen": "npx protoc --plugin=./node_modules/.bin/protoc-gen-ts_proto --ts_proto_opt=fileSuffix=.interface --ts_proto_opt=nestJs=true --ts_proto_opt=addNestjsRestParameter=true --ts_proto_out={projectRoot}/src/assets/interfaces -I {projectRoot}/src/assets/ {projectRoot}/src/assets/*.proto"
     }
   }

For the complete solution look at this.

Now just run the npm run grpc:gen or if you're using Nx just run nx grpc:gen projectName.
Congrats, you have your types and decorators. So let's put them into use.

[!TIP]

For a complete example look at the nestjs-client.

Ref

BTW you can find a working example in this repo too, so if this post was useful to you consider sharing it, giving my repo a star and this post a like.

kasir-barati / nestjs-materials

NestJS tips, tricks, Notes, Things which are not in doc and I used to figure them out and use them

Important

Keep this file synchronized with index.md.

nestjs-materials

NestJS tips, tricks, Notes, Things which are not in doc and I used to figure them out and use them

View on GitHub

When E2E Tests In NestJS Gives Me a Headache

Mohammad Jawad (Kasir) Barati — Fri, 24 Jan 2025 09:30:25 +0000

Testing can be frustrating, especially when we do not have a well documented library/framework from the perspective of writing tests. E.g. I was banging my head against brick wall as to why my E2E test was failing and it turns out that NestJS's @Processor decorator is doing some sort of magic.

Thus my test was failing. I tried different approaches but none worked:

Tried to mock the whole thing with ioredis-mock. Nope it was still failing:
I tried Testcontainers but still I was getting an error.
I had a real redis instance running and it was still not passing.

So What was My Issue When I Tried to Mock the Whole Thing

First I tried to write a code like this:

import { getQueueToken } from '@nestjs/bullmq';
import { INestApplication } from '@nestjs/common';
import { Test, TestingModule } from '@nestjs/testing';
import * as request from 'supertest';
import { AppModule } from '../src/app.module';
import { mockBullMqService } from './bullmq.mock';

describe('AppController (e2e)', () => {
  let app: INestApplication;

  beforeAll(async () => {
    const moduleFixture: TestingModule = await Test.createTestingModule({
      imports: [AppModule],
    })
      .overrideProvider(getQueueToken('YOUR_QUEUE_NAME'))
      .useValue({
        on: jest.fn(),
        add: jest.fn(),
        process: jest.fn(),
      })
      .compile();

    app = moduleFixture.createNestApplication();
    await app.init();
  });

  afterAll(async () => {
    await app.close();
  });

  it('/ (GET)', () => {
    return request(app.getHttpServer())
      .get('/cart')
      .expect(200)
      .expect('Hi');
  });

  // Add more tests here
});

But it was not working because in my src/user/audio.consumer.ts I had used @Processor decorator. And this is not documented anywhere. Yes, it sucks.

So to overcome this problem you have to do something like this in your test:

     const moduleFixture: TestingModule = await Test.createTestingModule({
       imports: [AppModule],
     })
       .overrideProvider(getQueueToken('YOUR_QUEUE_NAME'))
       .useValue({
         on: jest.fn(),
         add: jest.fn(),
         process: jest.fn(),
       })
+      .overrideProvider(AudioConsumer)
+      .useValue({})
       .compile();

And with this you should be good to go.

My Issue With Testcontainers

Have not solved it yet. And the real redis instance, I have not figured that out either. BTW you can see the whole conversation of me with me for the most parts here in discord.

If you're passionate about developing safe software for deploying on friday without having to fear this kind of consequences:

You need to start thinking about writing tests and probably E2E or unit tests or a combination of different tests. This gives you the confidence you need to deploy and be sure that with it you have not introduces a bug to the existing feature.

But for sure the new features always are prone to bugs, but at least you know what you've covered (scenarios) and if something goes wrong you will add that as a new test case to your existing ones. This potentially help you to reduce the amount of fires you need to put out :).

Make sure to follow me, and also give my repo a star:

Graphile -- Farewell Business Layer

Mohammad Jawad (Kasir) Barati — Mon, 20 Jan 2025 19:18:36 +0000

So I am super duper excited about Graphile, I just finished listening to an insightful podcast from GraphQL Radio and I enjoyed it quite a lot. But yeah let's forget about my enjoy and focus on this post.

So in this episode called Benjie Gillam on PostGraphile, The GraphQL Foundation, Independence From Facebook, Supporting Open Source, and beyond I get to learn more about Graphile and its origin.

How Graphile Came to Be

Originally it was called PostGraphQL.
Then Benji collaborate a couple of times on this open-source project and then one day he gets an email that CC him about the new fact, he is now the maintainer of PostGraphQL 😂.
So he works on it and in the process he renamed it to Graphile since GraphQL is a trademark.

So What the Heck does Graphile?

Simply put you do not need to write a model to connect to DB, then one for your service layer and eventually also one for your GraphQL. Essentially most of the times we do not need this much work. Especially if you wanna spin a stratup and just wanna ship something.
So Graphile sits in between and says something like this: hey, you know what? PostgreSQL is so awesome and powerful that it knows better than you how to:
1. Plan a query in the most efficient way. This has a couple of implications:
  - No over fetching or under fetching.
  - Extremely optimized queries.
  - No need to tackle N+1 issue anymore.
2. Handle permissions since like forever (I could not find any concrete evidence as when it was added but according to this commit in this repo it was before 2018) with GRANT and REVOKE.

My Take On This

Full transparency, I have not used it yet. But I am gonna try it out definitely.
As I wrote, this is my personal comment on this tool.
For a startup what matters most?
- Shipping features fast ✔️.
- Get a quick feedback ✔️.
- Be reliable ✔️.
- Have a good community ✔️.

So I would love to know if you're using it in your work place, personal project or are assessing for your next big hairy audacious project.

On Another Related Note

Sometimes we do have tons of business logic that we need to implement.
They cannot be expressed in PostgreSQL.
Our app's database ain't PostgreSQL.
Or any other reason.

I would suggest if one of those applies to you and you still have lots of CRUD like APIs. And you're asking yourself should I duplicate code or reinvent the wheel I would say you do not need to (unless you wanted to 😁).

Take a look at this project that I am actively trying to maintain it and make it something really comparable to HotChocolate in .NET world.

TriPSs / nestjs-query

Easy CRUD for GraphQL.

nestjs-query

Nestjs-Query is collection of packages to make crud for graphql easier.

Why?

While working on projects in nestjs it was very easy to get up and running with graphql however, there were many patterns that were common between the resolvers. In particular querying, sorting and paging.

Installation

Install Guide.

Docs

Packages

Nestjs-query is composed of multiple packages

@ptc-org/nestjs-query-core - Defines all interfaces and utility types implemented by the other packages.
@ptc-org/nestjs-query-graphql - Package that provides the graphql resolver and decorators for crud endpoints.
@ptc-org/nestjs-query-mongoose - Package that implements a Mongoose service that can be used by itself or with the graphql resolver provided by @ptc-org/nestjs-query-graphql.
@ptc-org/nestjs-query-sequelize - Package that implements a Sequelize service that can be used by itself or with the graphql resolver provided by @ptc-org/nestjs-query-graphql.
@ptc-org/nestjs-query-typegoose - Package that implements a Typegoose service that…

View on GitHub

To support us you can support Benji on GitHub and to support me consider giving this repo a star.

How to Read or Understand an Open-Source Projects

Mohammad Jawad (Kasir) Barati — Sat, 28 Dec 2024 17:23:37 +0000

It's always supper fun to dig into coding and try to make sense of open-source libs. it's been a while since I've started to use nestjs-query in my GraphQL services. It makes a lot of things easier. But I was puzzled with how the underlying implementation for Connection pagination and offset pagination works.

So I dug the lib and here is my final understanding: https://github.com/TriPSs/nestjs-query/issues/344#issuecomment-2562805549

I do not think it is gonna find its way into the docs someday, but nonetheless I decided to let you know how I get started to work with a new concept (GraphQL) and read this libs codebase in a manner that helped me to understand what's going on:

I started with getting familiar with the structure of the root directory in nestjs-query. Just seeing what we are using and what we have.
I felt I kinda missed some crucial parts of docs. So I went to the official docs and read a bit more about the packages that resides in this monorepo.
I came back and ran one of the e2e tests in the debug mode. In fact I added plenty of breakpoints in my VSCode and tried to see how it works in general.

I was really interested in knowing how it generates the resolvers, service layer and integrates with TypeORM.
Now I had a pretty good idea about how it works but I still was not able to pin point where exactly in the codebase we are defining the default pagination strategy. That's where I went back to the docs and read it this time top to bottom once more.
That's when I started to comprehend more and see how it work. Surprisingly it was really a no-brainer, I do not mean it is simple, I mean why they choose to implement certain things the way it's done in the lib. E.g. this lib is using mixins extensively and I love how modular it has made this lib as a result.
Eureka moment. I finally grasped it.

NOTE

In this process I can tell you that I several times went back and forth, ran the e2e tests in the debug mode again and again to understand it. So if you tried to apply this approach and did not get the result you wanted do not be discouraged and try yet again.

BTW if you wanna develop a new GraphQL service in NestJS it is a really great tool to do the mundane and dull tasks with nestjs-query. So that you can focus on building what matters for your business.

And besides the fact that I am using it, I am actively as you can see invested in this lib and in fact my aim is to make it something comparable to HotChocolate as I explained it here: https://github.com/TriPSs/nestjs-query/discussions/333

And if you liked this post consider giving this repo a star on GH :).

TriPSs / nestjs-query

Easy CRUD for GraphQL.

nestjs-query

Nestjs-Query is collection of packages to make crud for graphql easier.

Why?

Installation

Install Guide.

Docs

Packages

Nestjs-query is composed of multiple packages

@ptc-org/nestjs-query-core - Defines all interfaces and utility types implemented by the other packages.
@ptc-org/nestjs-query-graphql - Package that provides the graphql resolver and decorators for crud endpoints.
@ptc-org/nestjs-query-mongoose - Package that implements a Mongoose service that can be used by itself or with the graphql resolver provided by @ptc-org/nestjs-query-graphql.
@ptc-org/nestjs-query-sequelize - Package that implements a Sequelize service that can be used by itself or with the graphql resolver provided by @ptc-org/nestjs-query-graphql.
@ptc-org/nestjs-query-typegoose - Package that implements a Typegoose service that…

View on GitHub

Follow me:
Instagram: https://www.instagram.com/node.js.developers.kh/
Facebook: https://www.facebook.com/kasirbarati
X: https://x.com/kasir_barati
YouTube: https://www.youtube.com/@kasir-barati
GitHub: https://github.com/kasir-barati/
Dev.to: https://dev.to/kasir-barati
LinkedIn: https://linkedin.com/in/kasir-barati

Zero, One, Infinity Principle in Software Development

Mohammad Jawad (Kasir) Barati — Wed, 25 Dec 2024 10:44:11 +0000

While reading Apollo's doc for Relay Connection pagination I encountered a new terminology, or let's call it as Wikipedia does, a new rule of thumb. But here I prefer to consider it a principle. So from now on I'll call it principle.

ZOI -- Zero, One, Infinity Principle

I continued my quest on uncovering this new term and read some interesting posts on Medium, such as "The “Zero, One, Infinity” rule" written by Tal Joffe. But frankly speaking I still felt it is better to share my understanding here and see what you think about is topic.

So in software engineering this principle acts as some sort of compass. IMO it help us to decide when we need to add a new layer of indirection, or just go with the bare minimum.

In essence it says we should not restrict how many of a thing (method, instance, etc) can exist in our system. We either have:

Zero: You prohibit that thing from existing entirely. Like when you say no freaking way that the root directory in Linux to be placed in another directory (or at least that's how I feel about Linux's root directory).
One: Only one instance of something can exists: like root user, we only have one.
Infinity: Create as many as you desire. It's all good. Create as many¹ files as you wanted in a directory.

Let's talk more about its real life implications, rather than theory.

Repository Pattern and ZOI ;)

I know you're gonna love this, in fact I am just sharing my take on what Tal Joffe said in his post on Medium.

He starts by saying that assume we have OrderRepository class:

 class OrdersRepository{
   getOrders(ids: number[]): Order[]{
     //...
   }
 }

But over time we most likely have something like this:

 class OrdersRepository{
   getOrders(ids: number[]): Order[]{
     //...
   }
   searchOrders(searchTerm: string): Order[]{
     //...
   }
   getOrdersNames(ids: number[]): string[]{
     //...
   }
   // for the full example read his post. Linked at the beginning of this post.
 }

So now imagine you wanna add a new feature and need to have a new query which returns a list of orders. Now first we might look up in our current implementation to see if we have one that matches our requirements. If not we will add yet another one!

Solution to Our Repository Pattern Dilemma

Recall, in ZOI we have 3 state, zero, one, or infinity. Here we have passed the zero point and if we wanna maintain only one getOrders we need to somehow extract the responsibility of building the query away from this method.

In other words it should be only responsible for fetching data from the underlying database (SRP -- Single Responsibility Principle from SOLID). To do this we need to add a new layer on indirection. We need to do this since we need to follow Open/Close Principle too. We will create a new query builder using the builder pattern²

Something like this:

import { FindManyOptions, Entity } from 'typeorm';

interface Cursor {
  id: string;
}
interface ForwardPaginationArgs<Cursor> {
  after?: Cursor;
  first?: number;
}
interface BackwardPaginationArgs<Cursor> {
  after?: Cursor;
  first?: number;
}
interface QueryBuilder<Entity> {
  query: FindManyOptions<Entity>;

  setWhere(
    forwardPaging: ForwardPaginationArgs<Cursor>,
    backwardPaging: BackwardPaginationArgs<Cursor>,
  ): QueryBuilder;

  setCursor(
    forwardPaging: ForwardPaginationArgs<Cursor>,
    backwardPaging: BackwardPaginationArgs<Cursor>,
  ): QueryBuilder;

  build(): FindManyOptions<Entity>;
}

interface Repository<Entity> {
  getMany(queryBuilder: QueryBuilder<Entity>): Entity[] 
}

GitHub Repo for Repository Pattern

In here you can see how I am using this pattern in Practice: kasir-barati/graphql-js-ts/libs/shared/src/services/pager/query-service.pager.ts.

Lemme know what you think and I encourage you to gimme a star to my GitHub repo so that more people can reach to this useful tip 🦾.

Relay Connection Pagination

Whenever we have a list in our API we most likely sooner or later run into the problem of needing pagination. So with that in mind the next question will become how it relates to ZOI.

ZOI says if you do not wanna allow it do not allow it and that's fine (Zero scenario). Or maybe you just wanna have one getOrders and that's fine too (One scenario). Or like here we'll have an infinite number of records, so what ZOI suggest is to handle this scenario gracefully.

Meaning if you do not follow ZOI's suggestion the likelihood of building our GraphQL schema like this is high:

type Post {
  id: ID!
  comments: [Comment!]!
}

Whereas we might have developed it like the following when we have the ZOI principle in mind:

interface Node {
  id: ID!
}
type Post {
  id: ID!
  commentsConnection(
    first: Int
    last: Int
    after: ID
    before: ID
  ): CommentConnection!
}
type CommentConnection {
  edges: [CommentEdge!]!
  pageInfo: PageInfo!
}
type Comment implements Node {
  id: ID!
  content: String!
}
type CommentEdge {
  cursor: ID!
  node: Comment!
}
type PageInfo {
  hasNextPage: Boolean!
  hasPreviousPage: Boolean!
  startCursor: ID
  endCursor: ID
}

Learn more about pagination here.

If this was useful to you consider giving me a star on GH on one of my repos that you've found it helpful.

BTW Follow me:
Instagram: https://www.instagram.com/node.js.developers.kh/
Facebook: https://www.facebook.com/kasirbarati
X: https://x.com/kasir_barati
YouTube: https://www.youtube.com/@kasir-barati
GitHub: https://github.com/kasir-barati/
Dev.to: https://dev.to/kasir-barati
LinkedIn: https://linkedin.com/in/kasir-barati

I know practically we cannot create "as many" as we wanted files in a directory since there are some other factors that restrict us from doing that (learn more here. BUT the point that ZOI is trying to make it clear is, that we ain't gonna restrict users by some arbitrary numbers. ↩
You can learn more about Builder Pattern here and see how I use it in practice here. ↩

GraphQL query complexity + NestJS + Dataloader

Mohammad Jawad (Kasir) Barati — Tue, 24 Dec 2024 14:59:08 +0000

I was doing some exploration around GraphQL query complexity, depth limitation, and how to incorporate dataloader in NestJS to make my GraphQL API faster.

Just before getting into details I need to clarify that this is a contrived example.
UPDATE:
- I just saw this issue in graphql-depth-limit-ts about Connection Spec for pagination. Frankly speaking I did not consider this at all 😅. So right now I think my implementation can be enhanced to cover this case too.
- I've found a better lib that does both static query cost analysis and depth limiting. It is @escape.tech/graphql-armor. Make sure to take a look at my discussion too.

Database state

A dockerized PostgreSQL version 17.
I have about 1,000 users.
Each user has about 100 posts.

Resolver

I am fetching only 5 posts when you execute the following query inside the resolver for the getPosts:

{
  getPosts {
    id
    author {
      id
      posts {
        id
      }
    }
  }
}

But in the posts resolver I decided to fetch all of their posts (100). And that's where the bottleneck is.

Statistics before Dataloader

It was talking up to 5.5 seconds to retrieve data. And it was freaking huge. As you might have guessed it, I was using @ResolveField and sending a separate SELECT command to my underlying database.

Statistics after Dataloader

Now it fetches the same dataset in ~3.7 seconds. It is 32% performance increase. This might sound a bit disappointing but look at it this way:

This is a contrived example.
Take into account all those I/O costs that will be eliminated to some extend.
You do not need to pay more than you have to (in pay-as-you-go payment model your database instance was constantly being hit with new requests).

Repo

kasir-barati / graphql-js-ts

Where you can learn all about GraphQL and its intricacies

Caution

Keep this file in sync with index.md.

GraphQL

Tip

Just for those curious minds who always jump from one branch to another like mine:

You can find a good definition usually in glossary.

Intro.
Data types.
A simple todo app written with GraphQL + ReactJS + Relay
- Backend.
- Frontend TK.
Queries and mutations in depth.
Let's breakdown the query language a bit more.
Functions provided by graphql.
Document your GraphQL service API.
GraphQL request lifecycle
- Common validation errors.
- Execution from inside, resolver's args, AST, ...
Code-first approach.
Auth.
How to query information about a GraphQL schema.
Improve developer experience
Security in GraphQL.
NestJS
- …

View on GitHub

There go for: apps/dataloader-example. BTW in this repo you can see my implementation of static query cost analysis which you can learn more about it here. The part that I especially proud of is where I incorporated depth of a field in the overall complexity of a field (for context look at this issue).

If this helped you consider giving me a star on my GitHub repo for this Post :).

Follow me:

GraphQL + NestJS -- Part 1

Mohammad Jawad (Kasir) Barati — Sun, 15 Dec 2024 11:18:07 +0000

So I have this job op with GraphQL requirements, thus I decided to prepare myself for the position.

First I read the GraphQL doc here, then I started to redocument it since TBH I find it more engaging when I write doc for a tech than simply reading it (the result can be found here).

Then after that I decided to get a bit exercise in NestJS, thus I read their doc as well (RTFM) here. So now I reached a point where I realized I do not wanna spend my precious time writing CRUD-like ops.

So I tried out a couple of packages such as:

prisma-nestjs-graphql: You can find my conclusion about it here. But if you wanna know the final answer, it is a big NO for me.
nestjs-query: I tried this one and switched from Prisma to TypeORM. Like always I am documenting what I am doing here. But the bottom line is that this lib won the prize.

So now that I've cleared the air first I can explain from where I started my journey:

Like I said I decided to develop a dummy NestJS app with GraphQL to have a feeling for what's it like to build a GraphQL API. Talking more accurate, I was really interested in knowing "pagination and filtering".
I went TDD, meaning I wrote my e2e tests first. Thus I faced a big picture kind of question: "how my API should look like?"
- How others approach this question?
- Do we have a standard way of doing things?
Then A more fundamental question hit me, how should I go about caching? So I asked google about it and in the results I found a very insightful video about caching in GraphQL.
By this time I was feeling more confused than before. So I just decided to narrow the question down to what is the most common practice to have pagination in GraphQL and soon I found my answer. It is called "Connections" and I thought that those guys at Facebook ought to need caching. So it is pretty safe to use it.

BTW in case you like to know about pagination intricacies and its variants you can read my doc over here.

And if you are interested in "Connections" design pattern read this doc.
Now I am writing E2E tests for this dummy NestJS app called botprobe-nest here. Have not pushed anything yet though at the time of writing this article.

My next stop would be contributing more in nestjs-query project since it lacks some stuff that I think I need for sure.

BTW I just did upgrade a 3rd-party lib to its latest major version, thus we also release version 8.0.0.

Next step

After I finish writing E2E tests I am gonna try out caching, but before that we need to take care of query complexity. So stay tuned for the next part. Also lemme know in the comments if I can improve something.

DEV Community: Mohammad Jawad (Kasir) Barati

Write e2e tests for NestJS with testcontainers

Support Me

Code

GitHub Repo

kasir-barati / nestjs-materials

NestJS tips, tricks, Notes, Things which are not in doc and I used to figure them out and use them

Table of contents

GraphQL + NestJS + Interfaces + Unions

app/types/robot-interface.type.ts

app/types/scara.type.ts

Unions

Support Me

kasir-barati / graphql-js-ts

Where you can learn all about GraphQL and its intricacies

GraphQL

NestJS + AWS S3 + Multipart file upload + gRPC + e2e testing

How To Support Me?

Prerequisites

kasir-barati / nestjs-materials

NestJS tips, tricks, Notes, Things which are not in doc and I used to figure them out and use them

nestjs-materials

Get a list of endpoints in your NestJS app

AWS S3 - Multipart file upload - Surprise Bills

kasir-barati / cloud

It was my endeavour to build a wrapper around Docker and develop a RESTful API. In order to serve Platform as a Service. But now it's a place to learn more about cloud and put things into practice, of course things that are related to this topic.

Cloud

Be Careful With Your Bills

Handwritings

NestJS + gRPC + step by step guide + e2e tests

gRPC

NestJS

Ref

kasir-barati / nestjs-materials

NestJS tips, tricks, Notes, Things which are not in doc and I used to figure them out and use them

nestjs-materials

When E2E Tests In NestJS Gives Me a Headache

So What was My Issue When I Tried to Mock the Whole Thing

My Issue With Testcontainers

Graphile -- Farewell Business Layer

How Graphile Came to Be

So What the Heck does Graphile?

My Take On This

On Another Related Note

TriPSs / nestjs-query

Easy CRUD for GraphQL.

nestjs-query

Why?

Installation

Docs

Packages

How to Read or Understand an Open-Source Projects

TriPSs / nestjs-query

Easy CRUD for GraphQL.

nestjs-query

Why?

Installation

Docs

Packages

Zero, One, Infinity Principle in Software Development

ZOI -- Zero, One, Infinity Principle

Repository Pattern and ZOI ;)

Solution to Our Repository Pattern Dilemma

GitHub Repo for Repository Pattern

Relay Connection Pagination

GraphQL query complexity + NestJS + Dataloader

Database state

Resolver

Statistics before Dataloader

Statistics after Dataloader

Repo

kasir-barati / graphql-js-ts

Where you can learn all about GraphQL and its intricacies

GraphQL

GraphQL + NestJS -- Part 1

Next step

`app/types/robot-interface.type.ts`

`app/types/scara.type.ts`