DEV Community

Advanced NestJS: How to build completely dynamic NestJS modules

John Biundo on September 16, 2019

John is a member of the NestJS core team So you've noticed the cool ways in which you can dynamically configure some of the out-of-the-box NestJS ...

Read full post

gimyboya • Mar 28 '20

Great article! i think it's worth showing the interface of MassiveConnectAsyncOptions as i feel it's a missing piece

John Biundo • Mar 28 '20

@gimboya, Thank you for the feedback.

I am in the midst of a large project so won't have time to update the article at the moment, but appreciate your suggestion. I'm sure you probably found it, but just in case, you can view the interface here: github.com/nestjsplus/massive/blob.... Here's what it looks like:

export interface MassiveConnectAsyncOptions
  extends Pick<ModuleMetadata, 'imports'> {
  inject?: any[];
  useExisting?: Type<MassiveOptionsFactory>;
  useClass?: Type<MassiveOptionsFactory>;
  useFactory?: (
    ...args: any[]
  ) => Promise<MassiveConnectOptions> | MassiveConnectOptions;
}

gimyboya • Mar 29 '20

Yup got it! and very much appreciate it. I am exited that I am starting to grasp dynamic modules

gimyboya • Mar 28 '20

In the section about Supporting Multiple Async Options Providers Techniques, in the first block of code, I think there a slight mistake in the code.

This part:

public static registerAsync(connectOptions: MassiveConnectAsyncOptions): DynamicModule {
    return {
      module: MassivconnectOptions.imports || [],eModule,
      imports:
      providers: [this.createConnectProviders(connectOptions)],
    };
  }

should be:

public static registerAsync(connectOptions: MassiveConnectAsyncOptions): DynamicModule {
    return {
      module: MassiveModule,
      imports: MassivconnectOptions.imports || [],
      providers: [this.createConnectProviders(connectOptions)],
    };
  }

John Biundo • Mar 28 '20

Good catch! Thanks for reporting!

Frédéric Woelffel • Sep 18 '19

Thanks for this post @johnbiundo 👏
It surely provides a good example of how to develop our own dynamic modules. I'm wondering how you would handle multiple database connections with this MassiveModule? Wouldn't you have to provide multiple MassiveService instance? If so, how would we inject the right one at the right place?

John Biundo • Sep 18 '19

@fwoelffel Thank you for your feedback!

Right, I didn't really cover that in this article, but good question. The Massive library uses pg-promise under the covers and the db object I briefly mention actually represents a connection pool. So the recommended pattern for Massive (and any pg-promise library) is to use a singleton connection object which manages the connection pool under the covers. The full Massive integration library I built has additional options to configure the connection pool size and some other parameters to let you fine tune.

Oh, and there's a tiny bit more plumbing in the full library. Basically, it builds an injection token representing the shared connection object, and to use it in any module, you just inject that token. I tried to cover that in the @nestjsplus/massive docs, but let me know if it's still not clear. I felt like this detail - while relevant to understanding the full MassiveJS library - was a bit too distracting to cover in this article, but I'm not surprised you picked up on it!

Hope that answers the question!

Костя Третяк • Dec 11 '22 • Edited

Wow, that's a lot of code. And what is the benefit of this? In my opinion, it is better to transfer the metadata for the database connection synchronously:

import { ConnectionConfig, TypeCast } from 'mysql';

const {
  MYSQL_HOST,
  MYSQL_PORT,
  MYSQL_USER,
  MYSQL_PASSWORD,
  MYSQL_DATABASE,
  MYSQL_CHARSET
} = process.env;

export class MySqlConfigService implements ConnectionConfig {
  charset = MYSQL_CHARSET;
  host = MYSQL_HOST;
  port = MYSQL_PORT ? +MYSQL_PORT : 3306;
  user = MYSQL_USER;
  password = MYSQL_PASSWORD;
  database = MYSQL_DATABASE;
  typeCast: TypeCast = (field, next) => {
    if (field.type == 'JSON') {
      return JSON.parse(`${field.string()}` || '');
    }
    return next();
  };
}

Pass this config to DI:

providers: [
  MySqlConfigService
  //...
]

And when it's time to execute requests to the database, then make an asynchronous connection to the database. This approach allows, in addition to simplifying the code, to reconnect to a database when needed:

import { createPool, Pool, PoolConnection, MysqlError, OkPacket } from 'mysql';
import { Injectable } from '@nestjs/common';

import { MySqlConfigService } from './mysql-config.service';

@Injectable()
export class MysqlService {
  private pools: { [database: string]: Pool } = {};

  constructor(
    private config: MySqlConfigService,
    // ...
  ) {}

  getConnection(dbName?: string): Promise<PoolConnection> {
    return new Promise((resolve, reject) => {
      const config = { ...this.config };
      const database = (dbName || config.database) as string;
      config.database = database;
      if (!this.pools[database]) {
        this.pools[database] = createPool(config);
      }

      this.pools[database].getConnection((err, connection) => {
        if (err) {
          this.handleErr(this.serverMsg.mysqlConnect, err, reject);
        } else {
          resolve(connection);
        }
      });
    });
  }
  // ...
}

Mick • Feb 8 '21 • Edited

@johnbiundo I cant find your e-mail address anywhere so I hope you read this comment. There was a comment already asking for the question I have, but you didnt answer it. This already took me hours and hours, days and days, but I cant find anything online, so I hope you might help!

When my module has other imports that need configuration (some examples are HttpModule, CacheModule, ClientsModule) and I want them to be configured by the options that are provided to my module (as you described in the article) but the provided options are not available to them.

Maybe try to import HttpModule to your MassiveModule and try to configure a value of it (e.g. timeout) by using one of your 3 providers. But I also made a simple sandbox repository to figure out a solution. Feel free to try it out there or even make a PR: github.com/MickL/nestjs-dynamic-mo...

There is also a question on Stackoverflow asking the exact same question 6 months ago: stackoverflow.com/questions/633564...

THANK YOU VERY MUCH! :)

Mustapha Aouas • Sep 16 '19

Such a great article, thanks!

John Biundo • Sep 16 '19

Glad you found it helpful @mustapha !

Matthew J. Clemente • Jul 22 '20

Thank you for this!

Your NestJS module for Massive was actually my introduction to MassiveJS, which was incredibly impressive. It really does hit the sweet spot for database interaction - avoiding the heaviness of an ORM while still providing some elegant interactivity.

I was wondering, do you have a preferred approach for utilizing TypeScript with Massive in your NestJS projects?

jostFT • Aug 10 '20

when using an import (say HttpModule) from within the register.
How would you achieve passing your options parameters to the HttpModule import on the registerAsync method?

Mick • Feb 5 '21 • Edited

I have the exact same question / problem and struggling for hours and hours now... :(

Question SO: stackoverflow.com/questions/633564...
Repo to try things out: github.com/MickL/nestjs-inject-exi...

Pale King • Apr 4 '20

thank you for the great article, I spend a lot of time to create the share module with async style.

John Biundo • Apr 4 '20

Glad it helped! Thanks for the feedback.

Monirul Islam • Sep 27 '21

Is it possible to log knex queries and insert into database or any sort of logger file?

microud • Apr 8 '20

Thanks for sharing. I have translated it into Chinese. Can I post it on my blog？

Sathish vinayk • May 4 '21 • Edited

Thanks for the post!. Is there a way where we can pass configuration thru api and dynamically create multiple database.