Things you must have in every Repo for Javascript

Best way to baseline nestjs Microservice

Originally Published here https://tkssharma.com/nestjs-microservice-baseline-setup

There is no specific way or standard way of doing it, but we can maintain minimum standards of writing code or service like

• Moduel structure
• shared and feature Modules
• proper test setup for e2e and unit tests
• proper test environment
• proper setup for docker for local development providing (dev and test DB both)
• proper eslint setup with prettierc
• commit lint setup for proper commit messages
• docker and docker compose for spinning local environments
• jest configurations for E2E and unit tests
• compiler configuration for test and src application code
• husky hook to enforce commitlint
• CI pipeline configurations

Lets start a basic app using nestjs CLI

i hope you already know hoe to create basic nestjs app using CLI
Nest.js Application
Let’s continue with NestJS! We are going to install the NestJS CLI, so open the terminal of your choice and type:

$ npm i -g @nestjs/cli
$ nest new nest-env

We initialize a new NestJS project with its CLI. That might take up to a minute. The “-p npm” flag means, that we going to choose NPM as our package manager. If you want to choose another package manager, just get rid of this flag.
After this command is done you can open your project in your code editor. Since I use Visual Studio Code, I gonna open the project by typing:

$ cd nest-env
$ code .

Now we want to have our file structure Look like this, its our final Goal, lets see how we can achieve this

Now lets follow what all things we need to have this setup for our application

Step-1 commitlint and Husky Git Hooks

we need husky Hooks to enforce commit lint for our code with commitlint
Lets first install https://github.com/conventional-changelog/commitlint module

What is commitlint

commitlint checks if your commit messages meet the conventional commit format.

In general the pattern mostly looks like this:

• type(scope?): subject #scope is optional; multiple scopes are supported (current delimiter options: "/", "\" and ",")
  Real world examples can look like this:

• chore: run tests on travis ci

• fix(server): send cors headers

• feat(blog): add comment section

  "devDependencies": {
    "@commitlint/cli": "15.0.0",
    "@commitlint/config-conventional": "15.0.0",
    "commitizen": "^4.2.4",
    "cz-conventional-changelog": "^3.3.0",

  },
  "config": {
    "commitizen": {
      "path": "./node_modules/cz-conventional-changelog"
  }

commitlint.config.js

module.exports = { extends: ["@commitlint/config-conventional"] };

With the help of Git Hooks, you can run scripts automatically every time a particular event occurs in a Git repository. With this amazing feature of Git and with the help of Husky, You can lint your commit messages, prettify the whole project’s code, run tests, lint code, and … when you commit.
So here we can have a meeting between commitlint and husky git hooks, we want to run commit lint to check commit messages using hooks which Husly is managing
so Husky and commitlint will work for us

example like a simple git Hook, it is using commitlint Module to tun this script

#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

if [ "$NO_VERIFY" ]; then exit 0; fi
exec < /dev/tty && node_modules/.bin/cz --hook || true

npm install husky --save-dev

Enable Git hooks

npx husky install

To automatically have Git hooks enabled after install, edit package.json
npm set-script prepare "husky install"
You should have:

{
  "scripts": {
    "prepare": "husky install"
  }
}

# commit-msg
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"
npx --no-install commitlint --edit $1

# pre-commit
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"
npm run lint && npm run prettier

#  prepare-commit-msg
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

if [ "$NO_VERIFY" ]; then exit 0; fi
exec < /dev/tty && node_modules/.bin/cz --hook || true

From above configuration, make sure we have all these 3 script in .husky folder in the root
Husky Git hooks are using commit lint to provide proper commit message syntex and also running lint with prettier command to clean the linting
of the code, These all three Hooks works to place best standards for our code.

Step-2 Docker setup for local and test env

we need docker setup with docker-compsoe files, our container depends on what we are doing in service like

• Node JS container
• Postgres container
• redis container
• rabbit MQ container etc etc

docker-compose.yml

version: "3.6"
services:
  node:
    build: .
    volumes:
      - .:/app
      - ~/.npmrc/:/root/.npmrc
  postgres:
    image: postgres
    restart: unless-stopped  

docker file

FROM node:12-buster-slim

WORKDIR /app

COPY package.json package-lock.json /app/

RUN npm install && \
    rm -rf /tmp/* /var/tmp/*

COPY ./docker-utils/entrypoint/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh

COPY . /app

RUN npm run build

EXPOSE 3000

USER node

ENV TYPEORM_MIGRATION=ENABLE
ENV NPM_INSTALL=DISABLE
CMD npm run start:prod

docker-compose.override.yml

version: "3"
services:
   node:
     container_name: document_node
     command: npm run start
     environment:
       NPM_INSTALL: ENABLE
       TYPEORM_MIGRATION: ENABLE
     ports:
       - 3000:3000
  postgres:
    environment:
      - POSTGRES_USER=api
      - POSTGRES_PASSWORD=development_pass
      - POSTGRES_MULTIPLE_DATABASES="example-api","example-api-testing"
    volumes:
      - ./docker-utils:/docker-entrypoint-initdb.d
      - api_data:/data/postgres
    ports:
      - 5434:5432
volumes:
  api_data: {}

With all this we also wants to bootstrap postgres container with init database so we don't have to create manually

    environment:
      - POSTGRES_USER=api
      - POSTGRES_PASSWORD=development_pass
      - POSTGRES_MULTIPLE_DATABASES="example-api","example-api-testing"
    volumes:
      - ./docker-utils:/docker-entrypoint-initdb.d

This we can do with script and same script we can mount to volume as init script
we can craete script inside docker-utils folder

#!/bin/bash
set -e
set -u

function create_user_and_database() {
    local database=$1
    echo "  Creating user and database '$database'"
    psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
        CREATE USER $database;
        CREATE DATABASE $database;
        GRANT ALL PRIVILEGES ON DATABASE $database TO $database;
EOSQL
}

if [ -n "$POSTGRES_MULTIPLE_DATABASES" ]; then
    echo "Multiple database creation requested: $POSTGRES_MULTIPLE_DATABASES"
    for db in $(echo $POSTGRES_MULTIPLE_DATABASES | tr ',' ' '); do
        create_user_and_database $db
    done
    echo "Multiple databases created"
fi

Step-3 setting up eslint and prettier

Prettier can be run as a plugin for ESLint, which allows you to lint and format your code with a single command. Anything you can do to simplify your dev process is a win in my book. Prettier + ESLint is a match made in developer heaven.
If you’ve ever tried to run Prettier and ESLint together, you may have struggled with conflicting rules. Don’t worry! You’re not on your own. Plug in eslint-config-prettier, and all ESLint’s conflicting style rules will be disabled for you automatically.

    "@typescript-eslint/eslint-plugin": "4.33.0",
    "@typescript-eslint/parser": "4.33.0",
    "eslint": "7.32.0",
    "eslint-config-prettier": "8.3.0",
    "eslint-plugin-prettier": "4.0.0",
    "eslint-plugin-unused-imports": "2.0.0",

.eslintrc

module.exports = {
  "env": {
    "browser": true,
    "es2021": true
  },
  "extends": [
    "eslint:recommended",
    "plugin:@typescript-eslint/recommended"
  ],
  "parser": "@typescript-eslint/parser",
  "parserOptions": {
    "ecmaVersion": 12,
    "sourceType": "module"
  },
  "plugins": [
    "@typescript-eslint"
  ],
  "rules": {
    // strict https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/docs/rules/explicit-module-boundary-types.md
    // strict which we may not be able to adopt
    "@typescript-eslint/explicit-module-boundary-types": 0,
    // allow common js imports as some deps are still with common-js 
    "@typescript-eslint/no-var-requires": 0,
    "no-useless-escape": 0,
    // this is for regex, it will throw for regex characters
    "no-useless-catch": 0,
    // this i have to add as we mostly use rollbar to catch error 
    "@typescript-eslint/no-explicit-any": 0,
    // this is needed as we assign things from process.env which may be null | undefined | string 
    // and we have explicitly this.configService.get().azure.fileUpload.containerName!
    "@typescript-eslint/no-non-null-assertion": 0,
    "no-async-promise-executor": 0
  }
};

prettierc

{
  "bracketSpacing": true,
  "printWidth": 80,
  "proseWrap": "preserve",
  "semi": true,
  "singleQuote": true,
  "trailingComma": "all",
  "tabWidth": 4,
  "useTabs": true,
  "parser": "typescript",
  "arrowParens": "always",
  "requirePragma": true,
  "insertPragma": true,
  "endOfLine": "lf",
  "overrides": [
    {
      "files": "*.json",
      "options": {
        "singleQuote": false
      }
    },
    {
      "files": ".*rc",
      "options": {
        "singleQuote": false,
        "parser": "json"
      }
    }
  ]
}

And we can add required scripts to have linters enabled

    "lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix --quiet",
    "prettier": "./node_modules/.bin/prettier --check \"**/*.{js,json,ts,yml,yaml}\"",
    "prettier:write": "./node_modules/.bin/prettier --write \"**/*.{js,json,ts,yml,yaml}\"",
    "format": "prettier --write \"src/**/*.ts\" \"test/**/*.ts\"",
    "lint:fix": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",

Step-4 Build Configurations using tsconfigs

The presence of a tsconfig.json file in a directory indicates that the directory is the root of a TypeScript project. The tsconfig.json file specifies the root files and the compiler options required to compile the project.

JavaScript projects can use a jsconfig.json file instead, which acts almost the same but has some JavaScript-related compiler flags enabled by default.
Its good to have one global tsconfig and rest we can also create tsocnfig for build and test like tsconfig.build.json and using this config for build

{
  "compilerOptions": {
    "module": "commonjs",
    "declaration": true,
    "removeComments": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "target": "es2017",
    "allowJs": true,
    "sourceMap": true,
    "outDir": "./dist",
    "baseUrl": "./",
    "incremental": true,
    "strict": true,
    "skipLibCheck": true,
    "paths": {
      "@app/*": ["src/app/*"],
      "@auth/*": ["src/app/auth/*"]
    }
  },
}

tsocnfig Build configuration, we can override anything we want, we are extending base config and changing things if needed

{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "rootDir": "./",
    "declaration": false,
    "removeComments": true,
    "sourceMap": false,
    "incremental": false
  },
  "exclude": ["node_modules", "coverage", "test", "build","dist", "**/*spec.ts", "**/*mocks.ts"]
}

step-5 test related configurations

In most of the Projects we are using jest now
In test Folder we can have setEnvVars which will populate test config in process.env

const dotenv = require('dotenv');
dotenv.config({ path: './env.test' });

module.exports = {
  setupFiles: ['<rootDir>/test/setEnvVars.js'],
  silent: false,
  moduleFileExtensions: ['js', 'ts'],
  rootDir: '.',
  testRegex: '[.](spec|test).ts$',
  transform: {
    '^.+\\.(t|j)s$': 'ts-jest',
  },
  coverageDirectory: './coverage',
  testEnvironment: 'node',
  roots: ['<rootDir>/'],
  moduleNameMapper: {
    "^@app(.*)$": "<rootDir>/src/app/$1",
    "^@auth(.*)$": "<rootDir>/src/app/auth/$1"
  },
};

Rest all the configurations are project related like ormconfig.ts or knex.ts

• ormconfig.ts and knex.ts ORM related config
• nodemon.json or nest-cli.json nestjs cli configuration
• env and env.test files for local and test env
• CI configuration files
• Any other APM related file like newrelic.js
• deployment related file like procfile for Heroku

Conclusion

As we are building and working in everyday changing environemnt, its better to have a base template which we can keep evolving day by day and easy to start for any new project
from ground Zero, I hope above example can help you to build your template for different services

References

• https://www.npmjs.com/package/@commitlint/cli
• https://nestjs.com/