DEV Community: Alessandro Pecorilla

How to manage multiple environments with dotenv and Databases config in NestJS

Alessandro Pecorilla — Thu, 07 Nov 2024 19:43:36 +0000

Prerequisites

This guide is an evolution of what I have already described in the guide that follows: https://dev.to/jardiin/connecting-a-serverless-postgresql-database-neon-to-nestjs-using-the-config-service-221l

Introduction

The main purpose of this project is to show you how to define multiple instances of configuration files in NestJS, using the cross-env library and creating the necessary number of .env files. This allows you to:

Use different database instances for development, staging, or production environments.
Clean up and maintain .env files.
Establish a solid structure for the config/ folder.

Project Structure

This is the structure of my project. Essentially, we will be touching files located within the config folder, the app.module.ts file, and the main.ts file. The structure of the other folders (such as controller, entities, modules, services) may vary compared to yours because all my files (e.g., controllers) are saved within the controllers/ folder and exported from the index file.

├── src
│   ├── config
│   │   ├── development.config.ts
│   │   ├── production.config.ts
│   │   ├── app.config.ts
│   │   ├── typeorm.config.ts
│   │   └── index.ts
│   │   └── other config file...
│   ├── controllers
│   │   └── index.ts
│   │   └── users.controller.ts
│   ├── entities
│   │   └── index.ts
│   │   └── users.entity.ts
│   ├── modules
│   │   └── index.ts
│   │   └── users.module.ts
│   ├── services
│   │   └── index.ts
│   │   └── users.service.ts
│   ├── app.controller.ts
│   ├── app.module.ts
│   ├── app.service.ts
│   └── main.ts
├── package.json
├── tsconfig.json
├── other nestjs files..
├── .env.production
└── .env.development

tsconfig.json

This is the file that contains the compilerOptions of the project in Nest, to which I have applied the modifications that follow for the custom exporters I mentioned above (of various index.ts files).

{
  "compilerOptions": {
    "module": "commonjs",
    "declaration": true,
    "removeComments": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "allowSyntheticDefaultImports": true,
    "target": "ES2021",
    "sourceMap": true,
    "outDir": "./dist",
    "baseUrl": "src",
    "incremental": true,
    "skipLibCheck": true,
    "strictNullChecks": false,
    "noImplicitAny": false,
    "strictBindCallApply": false,
    "forceConsistentCasingInFileNames": false,
    "noFallthroughCasesInSwitch": false,
    "paths": {
      "@app/*": ["app/*"],
      "@controllers/*": ["controllers/*"],
      "@entities/*": ["entities/*"],
      "@modules/*": ["modules/*"],
      "@services/*": ["services/*"],
      "@config/*": ["config/*"],
      ... _OTHER_CUSTOM_PATH_HERE_ ...
    }
  }
}

(1) Package.json - Scripts Update

At this point, it becomes necessary to update the various scripts within the package. In this way, thanks to the cross-env library, we will be able to define the development/prod environment of NODE_ENV.

"scripts": {
    "build": "nest build",
    "start:production": "cross-env NODE_ENV=production nest start --watch",
    "start:development": "cross-env NODE_ENV=development nest start --watch",
    ... OTHER_SCRIPTS_HERE ...
  },

It is recommended to define the NODE_ENV within the scripts, giving the same naming convention as the .env files!

(2) Environment Files

In these files the environment variables for linking to the various databases will be defined. In my case the connection is made to Neon PostgreSQL databases. The .env files should look similar:

.env.development / .env.production

# ====== NEON POSTGRES ====== #

DATABASE_HOST='NEON PGHOST'
DATABASE_USER='NEON PGUSER'
DATABASE_PASSWORD='NEON PGPASSWORD'
DATABASE_PORT= 5432 #(usually it's 5432)
DATABASE_ENDPOINT_ID='NEON ENDPOINT_ID'
DATABASE_NAME='NEON PGDATABASE'

(3) Config Files

Considering that point (1) has been completed with the installation of the necessary dependencies (I remind you again that additional dependencies were installed in the guide mentioned in the prerequisites), it will now be possible to define the various configuration files.

The following files configure the connection to PostgreSQL databases, designed for development and production, using the credentials defined in the respective .env files.

(3.1) development.config.ts

import * as dotenv from 'dotenv';
dotenv.config({
  path: `.env.development`,
});

import { registerAs } from '@nestjs/config';

export default registerAs('development', () => ({
    type: 'postgres', // Neon PostgreSQL database type
    host: process.env.DATABASE_HOST || 'localhost', 
    database: process.env.DATABASE_NAME, 
    port: process.env.DATABASE_PORT, 
    username: process.env.DATABASE_USER, 
    password: process.env.DATABASE_PASSWORD, 
    entities: [`${__dirname}/../**/*.entity{.ts,.js}`], // TypeORM Entities to be stored in the database
    subscribers: [`${__dirname}/../**/*.subscriber{.ts,.js}`], // OPTIONAL
    synchronize: process.env.NODE_ENV === 'development', // Set `true` to synchronize the database schema with the entities
    logging: true,
    ssl: true, 
    connection: {
        options: `project=${process.env.DATABASE_ENDPOINT_ID}`,
    },
    migrations: [`${__dirname}/../database/migrations/*{.ts,.js}`], // Migrations 
    migrationsTableName: 'typeorm-migrations', // Set the name of the migrations table
}))

(3.2) production.config.ts

import * as dotenv from 'dotenv';
dotenv.config({
  path: `.env.production`,
});

import { registerAs } from '@nestjs/config';

export default registerAs('production', () => ({
    type: 'postgres', // Neon PostgreSQL database type
    host: process.env.DATABASE_HOST || 'localhost', 
    database: process.env.DATABASE_NAME, 
    port: process.env.DATABASE_PORT, 
    username: process.env.DATABASE_USER, 
    password: process.env.DATABASE_PASSWORD, 
    entities: [`${__dirname}/../**/*.entity{.ts,.js}`], // TypeORM Entities to be stored in the database
    subscribers: [`${__dirname}/../**/*.subscriber{.ts,.js}`], // OPTIONAL
    synchronize: process.env.NODE_ENV === 'development', // Set `true` to synchronize the database schema with the entities
    logging: false,
    ssl: true, 
    connection: {
        options: `project=${process.env.DATABASE_ENDPOINT_ID}`,
    },
    migrations: [`${__dirname}/../database/migrations/*{.ts,.js}`], // Migrations 
    migrationsTableName: 'typeorm-migrations', // Set the name of the migrations table
}))

(3.3) app.config.ts

This setup register the application configuration settings under the name 'config' with @nestjs/config. Here we will define the port and node environment.

import { registerAs } from '@nestjs/config';

export default registerAs('config', () => ({
    port: 5575,
    nodenv: process.env.NODE_ENV,
}));

(3.4) typeorm.config.ts

Inside this folder we configure a TypeORM DataSource for Neon database using settings from environment variables.

// ====== IMPORTS =========
import { DataSource } from 'typeorm';
import { ConfigService } from '@nestjs/config';
import { config } from 'dotenv';
config();

// 1. Define a configuration for TypeORM
const configService = new ConfigService();

export default new DataSource({
    type: 'postgres', // Neon PostgreSQL database type
    host: configService.get<string>('DATABASE_HOST'), 
    database: configService.get<string>('DATABASE_NAME'), 
    port: configService.get<number>('DATABASE_PORT'), 
    username: configService.get<string>('DATABASE_USER'), 
    password: configService.get<string>('DATABASE_PASSWORD'), 
    entities: [`${__dirname}/../src/**/*.entity{.ts,.js}`], 
    subscribers: [`${__dirname}/../**/*.subscriber{.ts,.js}`] // OPTIONAL,
    synchronize: process.env.NODE_ENV === 'development', 
    logging: process.env.NODE_ENV === 'development', 
    ssl: true, 
    migrations: [`${__dirname}/../database/migrations/*{.ts,.js}`], 
    migrationsTableName: 'typeorm-migrations', // Set the same name as you did on database.config
})

(3.5) config/index.ts

Now it is finally possible to export, within the index file the files you just created

export { default as DevelopmentConfig } from './development.config';
export { default as ProductionConfig } from './production.config';
export { default as AppConfig } from './app.config';

(4) App Files

The app.module.ts orchestrates the main structure and dependencies of the NestJS application. It imports the necessary modules to configures TypeORM for database access using environment variables. In questo caso eseguiremo un aggiornamento del ConfigModule e del TypeORM module come segue :

// ======== MODULES =========
import { Module, forwardRef } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { ConfigModule, ConfigService } from '@nestjs/config';
// ======== CONTROLLERS =========
import { AppController } from './app.controller';
// ======== SERVICES =========
import { AppService } from './app.service';
// ======== CONFIG =========
import { AppConfig, DevelopmentConfig, ProductionConfig  } from 'config/index';

@Module({
  imports: [
    // Load environment variables
    ConfigModule.forRoot({
      isGlobal: true, // Set `true` for global configuration
      cache: true,
      load: [AppConfig, DevelopmentConfig, ProductionConfig],
    }),
    // TypeORM configuration
    TypeOrmModule.forRootAsync({
      imports: [ConfigModule], // Import the ConfigModule 
      useFactory: (configService: ConfigService) => ({
        ...configService.get(
          process.env.NODE_ENV === 'production' ? 'production' : 'development',
        ),
      }),
      inject: [ConfigService], // Inject the ConfigService
    }),
    // Other Project Modules
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule { }

The condition process.env.NODE_ENV === 'production' ? 'production' : 'development' is used to determine which configuration options to use based on the value of the NODE_ENV environment variable.

Here's what it does:

If NODE_ENV is set to 'production', it returns the configuration options for the production environment.
If NODE_ENV is set to anything else (including undefined), it returns the configuration options for the development environment.

Conclusion

That's it! You will now be able to run via terminal the commands necessary to connect to the database of your choice, whether it is intended for production or development, via the following commands :

yarn start:production

yarn start:development

If all goes well, you will have from the terminal a message like this for your Database.

Additional Resources

Managing Multiple environments in NestJS: https://dev.to/pitops/managing-multiple-environments-in-nestjs-71l
Setting Up Environment Variables and Configurations in NestJS: https://bhargavacharyb.medium.com/setting-up-environment-variables-and-configurations-in-nestjs-a6372fb81f31

Connecting a Serverless PostgreSQL Database (Neon) to NestJS Using the Config Service

Alessandro Pecorilla — Sat, 13 Jul 2024 13:43:27 +0000

Introduction

The primary goal of this project is to illustrate the process of connecting a Neon database to a NestJS application through the Config Service, which helps manage configuration settings in a clean and organized manner. Neon PostgreSQL is designed to deliver a seamless database experience without the hassle of managing infrastructure.

In addition, I will demonstrate (in another guide) how to implement create and manage database entities with TypeORM, a popular Object-Relational Mapping (ORM) tool, to define and manage database entities.

Prerequisites

A NestJS and TypeScript project already initialized -> If you don't have a NestJS project set up yet, you can follow the official NestJS guide to initialize a new project with TypeScript. (Link)
An account on Neon PostgreSQL -> You can sign up for a Neon PostgreSQL account (Link)
For setting up a new project in Neon PostgreSQL, you can follow their official guide (official link). As of now, Neon PostgreSQL offers advantageous free solutions when setting up a new project

Packages

In addition to the packages provided by Nest on the initial setup, the following dependencies will help us to connect to Neon database.



"dependencies": {
    "@nestjs/common": "^10.0.0",
    "@nestjs/config": "^3.2.2",
    "@nestjs/core": "^10.0.0",
    "@nestjs/platform-express": "^10.3.10",
    "@nestjs/typeorm": "^10.0.2",
    "class-transformer": "^0.5.1",
    "class-validator": "^0.14.1",
    "dotenv": "^16.4.5",
    "express": "^4.19.2",
    "pg": "^8.12.0",
    "typeorm": "^0.3.20",
  }

Configuring Environment Variables

Create a .env file in the root directory of your NestJS project. This file will contain all the necessary environment variables for your application. Below there is an example of a ".env" file for our project



# ====== NEON POSTGRES ====== #
DATABASE_HOST='NEON PGHOST'
DATABASE_USER='NEON PGUSER'
DATABASE_PASSWORD='NEON PGPASSWORD'
DATABASE_PORT= 5432 #(usually it's 5432)
DATABASE_ENDPOINT_ID='NEON ENDPOINT_ID'
DATABASE_NAME='NEON PGDATABASE'
# ====== PORTS ====== #
NEST_PORT= YOUR_NEST_PORT #(usually it's 3000)

Given that the prerequisite N°3 has been met and you have successfully created a project on Neon PostgreSQL, you will find the necessary environment variables directly from the Neon dashboard. Here is an image illustrating the relevant variables you will need to configure

(1) Configuring the Nest.JS Project -> config/ folder

We'll start by organizing our configuration files in the config folder. Here, we'll create two essential files: app.config.ts and database.config.ts

app.config.ts

This setup register the application configuration settings under the name 'config' with @nestjs/config. Here we will define the port and node environment.



import { registerAs } from '@nestjs/config';

export default registerAs('config', () => ({
    port: parseInt(process.env.NEST_PORT, 10) || 5575,
    nodenv: process.env.NODE_ENV,
}));

database.config.ts

This setup configures the connection options for Neon PostgreSQL database connections using the environment variables defined earlier. It utilizes @nestjs/config to manage environment variables



import { registerAs } from '@nestjs/config';

export default registerAs('database', () => ({
    type: 'postgres', // Neon PostgreSQL database type
    host: process.env.DATABASE_HOST || 'localhost', 
    database: process.env.DATABASE_NAME, 
    port: process.env.DATABASE_PORT, 
    username: process.env.DATABASE_USER, 
    password: process.env.DATABASE_PASSWORD, 
    entities: [`${__dirname}/../**/*.entity{.ts,.js}`], // TypeORM Entities to be stored in the database
    subscribers: [`${__dirname}/../**/*.subscriber{.ts,.js}`], // OPTIONAL
    synchronize: process.env.NODE_ENV === 'development', // Set `true` to synchronize the database schema with the entities
    logging: process.env.NODE_ENV === 'development',
    ssl: true, 
    connection: {
        options: `project=${process.env.DATABASE_ENDPOINT_ID}`,
    },
    migrations: [`${__dirname}/../database/migrations/*{.ts,.js}`], // Migrations 
    migrationsTableName: 'typeorm-migrations', // Set the name of the migrations table
}))

index.ts

This index.ts file simplifies exporting configurations by re-exporting AppConfig and DatabaseConfig from their respective files



export { default as AppConfig } from './app.config';
export { default as DatabaseConfig } from './database.config';

(2) Configuring the Nest.JS Project -> database/ folder

Inside this folder we configure a TypeORM DataSource for Neon database using settings from environment variables.

typeorm.config.ts



// ====== IMPORTS =========
import { DataSource } from 'typeorm';
import { ConfigService } from '@nestjs/config';
import { config } from 'dotenv';
config();

// 1. Define a configuration for TypeORM
const configService = new ConfigService();

export default new DataSource({
    type: 'postgres', // Neon PostgreSQL database type
    host: configService.get<string>('DATABASE_HOST'), 
    database: configService.get<string>('DATABASE_NAME'), 
    port: configService.get<number>('DATABASE_PORT'), 
    username: configService.get<string>('DATABASE_USER'), 
    password: configService.get<string>('DATABASE_PASSWORD'), 
    entities: [`${__dirname}/../src/**/*.entity{.ts,.js}`], 
    subscribers: [`${__dirname}/../**/*.subscriber{.ts,.js}`] // OPTIONAL,
    synchronize: process.env.NODE_ENV === 'development', 
    logging: process.env.NODE_ENV === 'development', 
    ssl: true, 
    migrations: [`${__dirname}/../database/migrations/*{.ts,.js}`], 
    migrationsTableName: 'typeorm-migrations', // Set the same name as you did on database.config
})

From now on, this file will manage the entities you create in your project. Additionally, if you need to make modifications to the database schema, the setup includes support for using migrations.

(3) Configuring the Nest.JS Project -> app.module.ts

The app.module.ts orchestrates the main structure and dependencies of the NestJS application. It imports the necessary modules to configures TypeORM for database access using environment variables.



// ======== MODULES =========
import { Module, forwardRef } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { ConfigModule, ConfigService } from '@nestjs/config';
// ======== CONTROLLERS =========
import { AppController } from './app.controller';
// ======== SERVICES =========
import { AppService } from './app.service';
// ======== CONFIG =========
import { AppConfig, DatabaseConfig } from 'config/index';

@Module({
  imports: [
    // Load environment variables
    ConfigModule.forRoot({
      isGlobal: true, // Set `true` for global configuration
      cache: true,
      load: [AppConfig, DatabaseConfig],
    }),
    // TypeORM configuration
    TypeOrmModule.forRootAsync({
      imports: [ConfigModule], // Import the ConfigModule 
      useFactory: (configService: ConfigService) => ({
        ...configService.get('database'),
      }),
      inject: [ConfigService], // Inject the ConfigService
    }),
    // Other Project Modules
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule { }

(4) Configuring the Nest.JS Project -> main.ts

This main.ts file initializes the NestJS application, sets up middleware, establishes a database connection using configuration from ConfigService, and starts the server to handle incoming requests.



// ====== IMPORTS ======
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import * as dotenv from 'dotenv';
import { static as expose } from 'express';
import { ConfigService } from '@nestjs/config';

dotenv.config();

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  // Configuration Service
  const configService = app.get(ConfigService);

  const port = configService.get('NEST_PORT');

  await app
    .listen(port || 3375)
    .then(() => {
      console.log('Current Database Name: ', configService.get('DATABASE_NAME'));
      console.log(`Application is running on port`, configService.get('NEST_PORT'));
    })
    .catch((err) => {
      console.error('Error while starting application: ', err.message);
    });
}
bootstrap();

Conclusion

The configuration steps are now complete. To start the application and connect to the configured database, simply run the following command:



yarn start:dev OR yarn start

You should see the database connection details in the terminal, indicating that the connection has been successfully established like the picture below