DEV Community: Raphael Häcker

What I learned about external configuration of the database connection in NestJS

Raphael Häcker — Fri, 07 May 2021 00:51:13 +0000

As part of my 100 day of code challenge, I am building a web app using NestJS (and a not yet specified frontend framework). So one of the first things I tried setting up was the database connection. I plan on using a mongodb and because I might switch between different devices for development, I decided to sign up for a free account on a shared cluster at mongodb.com.

This is a longer description of my journey with this. If you are only interested in the results, click here.

First steps with the db

Since I’m new to NestJS (and developing with Node as a backend in general) I started this, by following the frameworks documentation. Although there is support for mongoose(link: https://mongoosejs.com), I decided to go with TypeORM. TypeORM is an Object Relational Mapper, that is written in TypeScript and since NestJS offers an integration, I felt like this would be a good choice. Another reason is, that I have no experience with mongodb, so if, for some reason, I decide, I won’t be using it for this project, it seems like it will be much easier changing databases with TypeORM, than if I would have used mongoose.

So following the documentation, I started by adding the database configuration to the app.module.ts like this:

@Module({
  imports: [
    TypeOrmModule.forRoot({
      type: 'mongodb',
      url: 'mongodb+srv://<username>:<password>@some.subdomains.of.mongodb.net/<databasename>'
    })
  ],
  controllers: [AppController],
  providers: [AppService],
})

When I then started the server, the connection worked. Nice, this was easy!
Sadly at that point I stopped reading the documentation and started looking at the code. I realised, that I have to commit this to a public repository and I maybe shouldn’t commit login data to a publicly hosted database (actually, I think, no database connection data, publicly hosted or not, should go into a git repository).

Looking for options to configure

So instead of reading the next few paragraphs in the database part of the NestJS documentation, I started to search for options on how to do this configuration outside of the code, in a configuration- or environment-file, that I simply don’t have to commit.
So instead of just reading about the ormconfig.json file, that is a way to configure TypeORM I looked for different options. And I found one, the config module of NestJS.
It internally uses dotenv, which is used to load the content of a file called .env into the Node process.env. This is a common approach in Node.js and is useful to quickly swap the applications environment. The .env-file holds key-value pairs to represent the environment.
The configuration module of NestJS offers different ways to access and to store environment variables, ranging from different locations for the .env-file, to ignore the file and only use the runtime environment variables or even use a custom environment file, that allows for a more complex structure such as yaml or json. In this case, the values are parsed into key-value pairs for use in the process.env.
There is a lot more advanced options for more complex situations, but these are the basic options, that the module offers for a simpler use case.

So I again, opened the documentation of NestJS regarding the config module and ended up including it in the app.module.ts like this:

@Module({
  imports: [
    ConfigModule.forRoot()
    })
  ],
  controllers: [AppController],
  providers: [AppService],
})

For a first test, my .env file only included one value:

testvalue=testvalue

As I wanted to see this in action, I changed the bootstrapped app.service.ts to look like this:

@Injectable()
export class AppService {

  constructor(private configService: ConfigService) {}

  getHello(): string {
    console.log(this.configService.get<string>('testvalue'));
    return 'Hello World!' + this.configService.get<string>('testvalue');
  }
}

This loads the value of the key ‘testvalue’ from the .env file and it works as expected. Nice!

Now one could be so naive to just think, well let’s use this in my app.module.ts. That was exactly what I did, but I didn’t get it to work (and I found some examples on the internet, so I think this is more a me problem). So I look into alternatives and found the ormconfig.json (yes, the one I could have found immediately …).

Configuration using the ormconfig.json

So I actually found this in the documentation of TypeORM and because, when I tried to configure TypeORM in the app.module.ts with the config module, I got this error:

Error: No connection options were found in any orm configuration files.

Searching for this pointed me in the right direction. Configuring the database connection with the ormconfig.json file (or a lot of other file formats) is done by creating the file in the root of the project. In my case, I created an ormconfig.json and it looks like this:

{
    "type": "mongodb",
    "url": "mongodb+srv://<username>:<password>@some.subdomains.of.mongodb.net/<databasename>"
}

If this configuration file is used, it is important to not put any configuration in the app.module.ts. One problem of this is, that it is an additional configuration file and since I don’t want too many different config files, I’d prefer not to create one only for the database. Another point is that I maybe want to run this in a container and then configure it from Environment variables for example. So this solution might not fit my needs.

I also discovered the option to configure TypeORM from the environment variables directly, by using the keys TypeORM provides. This would work for me, but there is one thing, that is not possible with this configuration (and most likely I won’t need it for this project, but better having it, than needing it, right?). I can not have more than one database configured.

Configuration using a configuration service in NestJS

So what I ended up doing was inspired here. This describes an injectable service, that implements the TypeOrmOptionsFactory interface, that can be used to asynchronously configure TypeORM. I combined this with the ConfigModule of NestJS (and I guess, it would be possible to use other config options as well). The service that configures my mongodb looks like this:

@Injectable()
export class MongoDBConfigService implements TypeOrmOptionsFactory {

    constructor(private configService: ConfigService) {}

    createTypeOrmOptions(): TypeOrmModuleOptions {
        return {
            type: 'mongodb',
            url: this.configService.get<string>('MONGODB_URL'),
            username: this.configService.get<string>('MONGODB_USER'),
            password: this.configService.get<string>('MONGODB_PASSWORD'),
    };
  }
}

In this service I can add all the configuration parameters I need and I can just create one service per database.

Conclusion

This was an unnecessary long journey. I think I could have avoided part of it, by just reading the next few instructions or doing a little more research.  On the other hand, I figured out a solution myself. From my point of view it works good and is really flexible, but I also don’t know much about NestJS and TypeORM so this is only a feeling I can’t prove.

NestJS and TypeORM database configuration

Raphael Häcker — Tue, 04 May 2021 21:47:47 +0000

After some initial problems I had with this topic, I decided to spend some time to look into different options. Here is an overview of the results:

Static configuration in the code

This is the most obvious way to configure the database. Just write the parameters (url, username, …) in the code, where you import the TypeORM.

@Module({
  imports: [
    TypeOrmModule.forRoot({
      type: 'mysql',
      host: 'localhost',
      port: 3306,
      username: 'root',
      password: 'root',
      database: 'test',
      entities: [],
      synchronize: true,
    }),
  ],
})
export class AppModule {}

There are a lot of problems with this. You can’t easily change this in a deployed application, it clutters your app.module.ts file and if you commit this to a public repository, you make your database access data public. Also if you always hard code configurations like this, you might end up having them all over the place in your code.
So while this is fine for a quick test in your development environment, I wouldn’t recommend using this for much more than that.

Configuring TypeORM using ormconfig.json

TypeORM offers the possibility to pass configuration parameters for one or more databases using an ormconfig.json file.

This file includes all the relevant configuration options. In my case it was not much:

{
    "type": "mongodb",
    "url": "mongodb+srv://<username>:<password>@some.subdomains.of.mongodb.net/<databasename>"
}

When using the ormconfig.json, you should not put any options in the app.module.ts, where you import TypeORM.

Configuring TypeORM this way is a clean solution, you have one file for the database connection(s) and you can configure more than one database in this file. The downside is, that you have to have this additional file. So in some environments, where you may not be able to access all the files of the application, changing that database might be difficult.

Configuring TypeORM with environment variables

As described here it is possible to configure TypeORM using environment variables. TypeORM offers a list of different variables to use in this case. This uses either a .env-file or the actual environment variables.

A limitation of this method is, that it is only possible to configure one database. But if you only need one database in your application, this is a good way to quickly configure TypeORM depending on the environment.

Configuring TypeORM with a configuration service

This combines the asynchronous configuration that is described in the NestJS documentation with the config module from NestJS, that supports in getting values from the environment variables.

Basically you have to create a service, that can be injected into the TypeORM import and that implements the TypeOrmOptionsFactory interface. The values this service provides to TypeORM are retrieved from the NestJS config module.

For my mongodb this looks like this:

import { Injectable } from "@nestjs/common";
import { ConfigService } from "@nestjs/config";
import { TypeOrmModuleOptions, TypeOrmOptionsFactory } from "@nestjs/typeorm";

@Injectable()
export class MongoDBConfigService implements TypeOrmOptionsFactory {

    constructor(private configService: ConfigService) {}

    createTypeOrmOptions(): TypeOrmModuleOptions {
        return {
            type: 'mongodb',
            url: this.configService.get<string>('MONGODB_URL'),
            username: this.configService.get<string>('MONGODB_USER'),
            password: this.configService.get<string>('MONGODB_PASSWORD'),
    };
  }
}

You can create such a service per database you need. Using it to configure TypeORM in your app.module.ts would look like this:

@Module({
  imports: [
    ConfigModule.forRoot({
      isGlobal: true
    }),
    TypeOrmModule.forRootAsync({
      useClass: MongoDBConfigService,
      inject: [MongoDBConfigService]
    })
  ],
  controllers: [AppController],
  providers: [AppService, MongoDBConfigService],
})

export class AppModule {}

The advantage of this method, is that you can configure multiple databases from the environment of the app and keep your app.module.ts more or less clean. The downside is, it takes a few more steps to set up.

Since I am new to NestJS and TypeORM, I’m writing down what I’m learning. So if I missed a method to do this or anything I wrote is incorrect, please let me know!

My problem with starting out

Raphael Häcker — Mon, 26 Apr 2021 14:55:29 +0000

A lot of people know this struggle. You have something in mind, that you want to do, like starting to work out, build that cool side project or start a new challenge. But you can’t really force your self to commit to it. There is always either something else to do or you feel like you’re missing something, that you need to get started. So you just keep pushing the thing you actually want to do.

The excuses keep coming

“I can start, after this is done”
“I promised to help with this other project, so I can’t really do this now”
“Starting now means, I will be interrupted again soon”
“My old laptop lacks the storage to do this in a comfortable way”

These are just a few of the things that came to my mind, whenever I thought about starting something like a side project or the 100 days of code challenge. And the worst part?
I got a new laptop, but I didn’t start. The old one was never really the reason I didn’t start.
The other project I should have done? Not really worked on this either, just felt bad about it.

It turns out, all the reasons my mind came up with were just excuses. Excuses that allowed me to keep not starting something and just keep procrastinating.

So what can I do?

A few weeks ago, I scrolled through twitter and came across a tweet by Nader Dabit. He was talking about how he creates technical content and learns something new. That somehow resonated with me. So I asked him for a tip, hoping for something like a hint on tooling or techniques. But he said, just do it and that he has the tendency to procrastinate. Oh? I absolutely have this tendency. I make plans and keep thinking about how to do the things I’d like to do, but don’t actually start to do them.

From another source (and I sadly can’t remember where) I also got the advice, that if you have a problem motivating yourself, just starting the activity can help. And I know that feeling. Just opening the editor and writing a line to two of code or even just reading it, got me motivated enough to keep going. After a while there are also some results I can look on to keep me motivated.

So for me it all comes down to just doing, just starting with something.

So I’m starting today

With this post I’m starting with the 100 days of code challenge.
What I want to do in the 100 days:

I want to build an app, that works like a buzzer. You get a button you can press for example if you know the answer in a quizz. The host of the game then sees who pressed the button first.
I want to use technologies or frameworks that I am not used to. The current planned stack is react for the frontend (I want to try react native!), NestJS for the Backend and MongoDB for storage.
I want to write about what I do, maybe produce some videos or stream my coding sessions.

I haven’t everything ready. For example I didn’t set anything up to do the live streams or produce the videos. I also haven’t everything planned out for the app. So I will do this as I go with the challenge. I hope planning and set up sessions can also count towards the daily goal.

And now?

Now I’m starting with my 100 days of code. Today I will

Set up all the repositories
Write a description of the app I want to build
Tweet about it

I know, none of that is coding, but if I keep planning for this, I will never start. In the next days I will definitely code. And if something interesting happens, I will try to write about it.
So if you’re interested in my journey through this challenge follow me here or on Twitter.