DEV Community: Kevin Merckx

JavaScript is not meant for backend

Kevin Merckx — Sat, 08 Feb 2025 12:10:12 +0000

I recently started learning Go. It has been advertised to me as a great programming language to create web services with a very good built-in concurrency library.

While going through the tutorials, I noticed the way Go makes developers handle errors and how different it is compared to Javascript. In this post, I want to go through this difference and will conclude by stating my preference.

Javascript is not typed. Functions can return whatever they want: numbers, booleans, functions, strings, etc. It's up to developers to document functions properly and/or use JSDoc, Typescript and alike. Let's look at a very simple example together.

function main() {
  const stringified = JSON.stringify({ aBigInt: 2n });
  console.log(stringified);
}

main();

The code above will result in an exception that makes the program crash:

  const stringified = JSON.stringify({ aBigInt: 2n });
                           ^

TypeError: Do not know how to serialize a BigInt
    at JSON.stringify (<anonymous>)
    at main (/home/kmerckx/dev/blog/index.js:2:28)
    at Object.<anonymous> (/home/kmerckx/dev/blog/index.js:6:1)
    at Module._compile (node:internal/modules/cjs/loader:1565:14)
    at Object..js (node:internal/modules/cjs/loader:1708:10)
    at Module.load (node:internal/modules/cjs/loader:1318:32)
    at Function._load (node:internal/modules/cjs/loader:1128:12)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:219:24)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:170:5)

If that piece of code that serializes an object would be running on a HTTP server, it would likely crash it. To get around that issue, developers (or the framework they use) have to wrap the call to JSON.stringify in a try...catch. A modified version of the script could be:

function main() {
  try {
    const stringified = JSON.stringify({ aBigInt: 2n });
    console.log(stringified);
  } catch (error) {
    console.error('We got an error here, can not print the stringified object', error);
  }
}

main();

The program doesn't crash and exits with code 0. What we are experiencing here is a consequence of JavaScript error handling: errors can be thrown and it's up to you to catch them. Within a simple program, it's not a big deal: you call standard ECMAScript functions, put some try catch around them, done. As soon as your program is more complex, those standard functions end up being called pretty deep in the call stack. Every level of this stack can receive errors that bubble up from the levels below, making the error handling very hard to figure out.

You would think that Typescript could help in this endeavor but it doesn't. The error that you catch is of type unknown.

A language like Go has a built-in error type and lets developers handle errors as values. It forces you to handle errors. You can not ignore them. In the following example, you can see that in action:

f, err := os.Open("filename.ext")
if err != nil {
    log.Fatal(err)
}
// do something with the open *File f

Example from https://go.dev/blog/error-handling-and-go

I like the simplicity of JavaScript but I came to realize that the error handling was one of the major flaws of its design: a simple scripting language that lets you walk very fast on the happy path. Unfortunately, web services are rarely made of happy paths. You might have to deal with malformed data, network issues, etc.

I work a lot with JavaScript and while I think it's not meant for backend development, I also think there is a lot of value in using it with Typescript in a full-stack team to rapidly create products. In the second part of this post, I will present an attempt to make JavaScript error handling a lot nicer. It does have its limits and I will also present them.

I really like Go's approach and the solution I implemented is inspired from it. It is especially powerful when used with Typescript.

import { makeSuccess, makeFailure, SafeResult } from '@km/saferjs'; // this is not published on NPM!!

function someSafeFunction(): SafeResult<number, SomeCustomError | TypeError> {
  try {
    return makeSuccess(
      ... // your logic here
    );
    // you can also return a makeFailure(...)
  } catch(error) {
    // maybe it's a TypeError: error instanceof TypeError
    return makeFailure(error);
    // error handling goes here
    return makeFailure(new SomeCustomErrorClass());
  }
}

const { result, error } = someFunction();
// at this point Typescript knows that result is either null or a number
// and error is either null or one of the type SomeCustomError, TypeError

if (error) {
  // handle the error
  // at this point, you can early-throw or -return
  // Typescript will understand that after this if-block, result is defined.
} else {
  // result is a number
}

The type inference is perfectly done by Typescript, giving your a lot more safety at transpile time. I also came up with two helper functions isFailure and isSuccessthat can be used as follows:

const someResult = someFunction();

if (isFailure(someResut)) {
  // handle error
}

if (isSuccess(someResult)) {
  // happy days!
}

After implementing that approach and adopting it in the code I write, I realized it suffers on major drawback: none of the JS standard functions follow this patter and they all throw errors. Making the code in my projects be perfectly safe to use would mean that I have to wrap all standard function calls. Tedious.

I can make Nodejs backend very safe with such an approach but I came to realize that, if it's not built-in the foundation of the language, it's never going to be a good developer experience. JavaScript is just not meant for backend.

TL8 goes 100% open source

Kevin Merckx — Sat, 04 Jan 2025 18:51:05 +0000

I opened the source code of TL8. It is available on Github under two separate repositories: https://github.com/kevinmerckx/tl8 and https://github.com/kevinmerckx/tl8-app.

I started TL8 as a side project almost exactly three years ago. It proved to be very useful in a few projects of mine, which was already a good achievement in itself. Furthermore, the application was downloaded by 50 different users in the past 9 months (I lost the data before that...). TL8 is an application that helps development teams translate their web application. I was expecting to get users from all over the world and that is the case! TL8 has been used in Ireland, Germany, Brazil, Poland, USA, Spain, France, Romania, Canada, Belgium, Italy, Egypt, Argentina, Syria, etc.

I based my decision of making TL8 open source on 4 key arguments:

many developers mentioned that not being open-source was a concern to them
some components in TL8 could be useful to the community
it could give a second life to the project and motivate me and others to pursue its development
my strategy to generate some revenues through donations failed (I might have been naive)

Overall, it was a positive experience: a great technical achievement and important lessons learned on the way. I will explain that further in an upcoming post.

TL8 goes React (Beta)

Kevin Merckx — Fri, 29 Mar 2024 12:03:07 +0000

TL8 now supports React. I assumed it would not require any change in the TL8 application code and should only require me to write the React TL8 counterpart (the one that developers have to put in their React application). And I was happily surprised to be right!

TL8 is based on a home-made protocol that makes it talk to the Angular and React plugins. This protocol is framework agnostic. That is why making TL8 compatible with React was very straight-forward.

I have also made the TL8 React library 100% open-source and am planning to do the same for the Angular library. Several people asked me whether TL8 was open-source. It is partially open-source now.

tl8-react provides a translation system and the necessary pieces to be used with the TL8 application. It is available in Beta on NPM: https://www.npmjs.com/package/tl8-react

The latest TL8 application is available on https://tl8.io/ and works out of the box with your React application.

Enjoy.

The power of End-to-End tests

Kevin Merckx — Tue, 26 Dec 2023 21:05:16 +0000

In this article, I will advocate for using End-to-End (E2E) tests. I will start by defining what E2E tests are and what they are not. After this, I will try to convince you to use them as one of the essential bricks of your future and current projects. The last section is providing answers to the most common criticisms addressed to E2E tests.

Definition

E2E tests ensure that an entire system verifies a set of requirements. There are two implications to that definition:

A set of requirements is defined.
We have the ability to run an entire system in our testing setup.

E2E tests are written in a way that we can verify that the requirements are fulfilled by the system. They can be as simple as "Users can sign into the application using the Login form" and as complex as describing a sequence of actions that users will likely go through while using the system.

What a good E2E testing setup should not do:

Mock parts of the application, especially the critical ones, like the database and the ORM (Object Relational Mapping). Ideally, a E2E testing setup doesn't mock anything.
Tests should not describe how the system works, rather focus on how users interact with the system.

Why use E2E tests?

I see two main reasons why every project should have a strong emphasis on E2E tests:

They guarantee that requirements are fulfilled. They verify that, given an input (given by the user), some output is returned or that the system reached a certain state. We are making sure that the system is doing what is expected. Not how it's done. Therefore, they highly focus on making sure the value is delivered.
They speed up development. Yes, you read me correctly. Let's look at two examples: backend and frontend development.

Backend development

Imagine that you are working on implementing an API delivered through HTTP. Whether you want it or not, you are going to end up calling your API endpoints and checking that they behave like you expect. Calling your API endpoints manually requires you to use Postman, or Chrome, or any other client. With a good E2E test setup, you can:

write your tests,
change your code,
and see the results of the tests as frequently as hitting the save button of your IDE.

That way, you don't have to trigger the tests manually. You code, you see the results. Imagine your screen being split in two windows: one is your IDE, one is the testing framework constant running the tests for you.

Frontend development

Testing your frontend from end to end brings even more value. Manually interacting with a user interface is slow and tedious. Being able to automate those interactions saves enormous amounts of time. You can have your testing environment set up in a way that saving files automatically triggers the tests you pre-selected and verifies that they do what is required. You can also use those tests to reach a certain state of your application and work from there.

The tests you are writing to automate interactions can then be used and should be meant to last. They become part of the requirements that must be verified. You achieved two goals: your software is tested and an excellent developer experience (DevX). You save time and deliver quality. It's all about the value.

Common criticisms

E2E tests are slow: E2E tests, especially for frontend, are slower to run that integration or unit tests. That is true but also, what do you care? On a developer's machine, one can always select the tests they want to run. As part of a continuous integration pipeline, a few minutes more won't make a huge difference on your budget. It will take more time to complete until delivery but the value they bring seems to completely outweigh those longer jobs. Furthermore, in a Continuous Delivery setup, releasing software to production regularly is better done when we have the guarantee that the software is doing what it is supposed to do. And there is nothing better than E2E tests to achieve that. You can then release your software more often, as it requires less manual testing.
Writing tests takes time: as most skills, you get better at it by practicing. The first time you write a test, you must learn the testing framework, formulate exactly what you want to test, put it in code. And like many skills, you will be slow at the beginning. The good news is that you won't get any slower! You will get better at it pretty fast. Invest in yourself, learn this skill and save your future self from ever having to test things manually. Furthermore, we now have access to AI based code generation tools that can write the tests for you in a matter of seconds.
Maintaining tests take time: yes, just like any code you write. The advantage of E2E tests is that they should resist changes pretty well, assuming that the requirements of the system don't change. E2E tests are very useful to guarantee that requirements are fulfilled. Therefore, E2E tests most likely break due to a change of requirements. When it happens, they are a reminder that they must be adjusted (replaced or modified) as well.

Thanks for reading this post. I hope you are convinced now that writing E2E tests should be part of your workflow. I was skeptical at first (cf. https://blog.merckx.fr/integration-testing-setup-with-nestjs-and-mongodb/). Writing E2E tests take practice. Once it's there, you will never want to go back.

How to protect endpoints of a Nestjs application - Revisited

Kevin Merckx — Sat, 23 Dec 2023 09:33:38 +0000

Two years ago, I wrote a post about protecting API endpoints of a Nestjs application. To summarize it:

use the declarative approach of Nestjs and protect the whole application by default behind a guard.
provide your project with a decorator that you can add on controllers and controllers' methods to opt out of the default guard protection.

I realized there was a nicer syntax than @SetMetadata(AUTH_GUARD_CONFIG, { disabled: true } as AuthGuardConfig). Simply create a function that does it, just with a much better name:

export const AllowUnauthorized = () => SetMetadata(AUTH_GUARD_CONFIG, { disabled: true } as AuthGuardConfig);

That's it!

tl8 3 is out

Kevin Merckx — Mon, 30 Jan 2023 11:21:24 +0000

TL8 1 was a great proof of concept and proved to be useful. TL8 2 was a step to improve the user experience. TL8 3 is now available on tl8.io and is bringing three new features:

the Markdown editor
the Full Editor
import & export of work in progress

Markdown Editor

The Markdown editor is feature that is coming handy for all Angular applications that use markdown to format the content of translation keys.

The Markdown Editor in action

Full Editor

The Full Editor enables users to go through translation keys of the entire application without navigating to the according views. Users can search for translations and use the tree view to easily find and edit translations.

Import & Export

The Import & Export of work in progress is a first step to support collaboration workflows. Users can now work on the same application in parallel and integrate each other's work into theirs. The workflow of import-export is made through "work-in-progress" TL8 files. One user can export their work, hand the .tl8 file to someone else who can import it and integrate the changes into theirs. The application lets you know about conflicts and gives the opportunity to users to pick which version they wish to keep.

The import workflow here letting the user resolving conflicts.

Design adjustments

Along with those 3 features, TL8 has been slightly redesigned: the sidebar now displays the language switch, the sidebar toggle and the switch from Live to Full Editor at the bottom.

What's coming next?

TL8 will soon move to the next level. A live collaboration is currently in Alpha. This should help users work on bigger projects as a team.
Getting a first version of an application translated to a new language should not be hard. TL8 will help achieve that.

Many thanks to Asset Ismagambetov for the help and support to pull that new version.

tl8 1.0.0 is out

Kevin Merckx — Fri, 08 Apr 2022 21:38:40 +0000

I released tl8 last year. It came with a few integration issues. The main one was that I built the application with Electron and embedded a web view to render the application to be translated. Problem: some web apps have security settings or workflows that don't allow for such a rendering.

With tl8 1.0.0, all Angular apps can be translated.

It took some time but it's here. The application is available for Mac, Windows and Linux. You can check the home page for more information: https://tl8.io/. The web application no longer exists as it would be too much work to maintain it and would need iframes to run and would therefore suffer from the limitations I mentionned above. The journey to get around the limitations of Electron involved several BrowerViews. One of them is dedicated to run the target application and doesn't have any limitation regarding the content it can run!

1.0.0 provides a solid user experience that enables you to easily change translations and content in an Angular application. It also comes with some bug fixes.

There are features and improvements planned for the next iteration that will be described in a further blog post.

An Angular technique that might surprise you

Kevin Merckx — Fri, 11 Feb 2022 14:11:33 +0000

Special thanks to https://twitter.com/ngfelixl for helping me structure this work.

I was recently working on an Angular application that had a specific requirement. The application shell loads feature modules under specific routes. Each module should have the ability to expose an item in the application shell toolbar. With one colleague we discussed how we could achieve that.

If you can't wait to see it in action, here is the link to the repo: https://github.com/kevinmerckx/ng-feature-extension

First naive attempt

We considered using the Angular CDK and its portal API. The limitation appeared pretty quickly: the menu item declaration, from within the template of the root component of the feature module will only be evaluated when the feature is loaded by the router. Therefore, this approach is not suitable.

We need an "offline" way to declare this toolbar item, without the entire feature module to be loaded.

Solution

The solution I suggest is based on three pillars:

injection tokens
extension modules
component outlets

Let's describe each pillar first.

Injection tokens

Injection tokens are an important part of Angular. They give developers the opportunity to augment the application. For instance, to declare a directive as validator you use NG_VALIDATORS. When you want to declare a custom control value accessor (cf. https://dev.to/kevinmerckx_47/create-a-custom-angular-form-control-3one), you use NG_VALUE_ACCESSOR. When you use them, Angular gives you the ability to extend its API.

Extension modules

When you create a feature module, you usually do so by exporting one Angular module. You then load it in the main module, lazily or not. Keep in mind that you are allowed to split your feature module into several smaller modules. You can provide the shell of your feature through one module and export another one that provides a smaller set of features. Let's call the later kind Extension Modules.

Component Outlets

This API is provided by Angular and gives developers the ability to inject components in a template.

<ng-container *ngComponentOutlet="theComponentToLoad"></ng-container

With those 3 pillars, we can create a mechanism that enables a feature module to use offline an extension API provided by the shell application.

First, you should declare an interface that extensions of feature modules must implement.

For instance, if you want a module to be able to add an item in your application toolbar, your interface could look like that:

import { Type } from '@angular/core';
import { Observable } from 'rxjs';

export interface Extension {
  toolbarItem: Type<any>;
  route: Observable<string>; // here we also provide the route to load when the item is clicked
}

Then, you must declare the injection token that each feature module can provide. Let's call it FEATURE_EXTENSION.

import { InjectionToken } from '@angular/core';

export const FEATURE_EXTENSION = new InjectionToken('FEATURE_EXTENSION');

It is now possible for our toolbar component to use this token at runtime:

import { Component, Inject } from '@angular/core';
import { Extension, FEATURE_EXTENSION } from '../shared';

@Component({
  selector: 'toolbar',
  templateUrl: './toolbar.component.html',
  styleUrls: ['./toolbar.component.css'],
})
export class ToolbarComponent {
  constructor(@Inject(FEATURE_EXTENSION) public extensions: Extension[]) {}
}

It is now time to use the ngComponentOutlet directive from the toolbar template:

<div
  *ngFor="let extension of extensions"
  tabIndex="0"
  [routerLink]="extension.route | async"
  [routerLinkActive]="'active'"
  [routerLinkActiveOptions]="{ exact: true }"
>
  <ng-container *ngComponentOutlet="extension.toolbarItem"></ng-container>
</div>

Our application shell and toolbar are now ready to receive feature module extensions!

Let's move on to a feature module that we call the "Planning" module. This module consists of two things:

a classic feature shell module that loads components depending on the route: PlanningShellModule
a lightweight extension module: PlanningExtensionModule

The PlanningShellModule has nothing particular and is loaded by the router (optionnaly lazily). The PlanningExtensionModule is declared as follows:

import { CommonModule } from '@angular/common';
import { NgModule } from '@angular/core';
import { of } from 'rxjs';
import { Extension, FEATURE_EXTENSION, ToolbarItemModule } from 'path/to/some/shared/folder';
import { PlanningToolbarItemComponent } from './planning-toolbar-item.component';

@NgModule({
  imports: [CommonModule, ToolbarItemModule],
  providers: [
    {
      provide: FEATURE_EXTENSION,
      useValue: {
        toolbarItem: PlanningToolbarItemComponent,
        route: of('planning'),
      } as Extension,
      multi: true
    },
  ],
})
export class PlanningExtensionModule {}

The most important piece is in within the providers property where we provide a FEATURE_EXTENSION value with our toolbar item component PlanningToolbarItemComponent to load and the route to navigate to when clicked. Note the use of multi: true that makes sure we can declare several times this provider from other feature modules!

The PlanningToolbarItemComponent can make use of all the components, directives and pipes that are declared in the ToolbarItemModule.

You can now display custom content in the toolbar of the application shell from a feature extension module.

Feel free to check this repository https://github.com/kevinmerckx/ng-feature-extension for the full code of this fully functioning proof of concept. Here is a screenshot:

At the top the toolbar, with 3 items. Plan and Code both use a customized toolbar item, provided by their respective extension module.

To summarize, by combining InjectionToken with multi: true, ngComponentOutlet and by splitting feature modules into a shell and an extension modules, we managed to provide a nice way for feature modules to customize the application shell through a nice API defined by an Extension interface.

Photo by Lance Anderson on Unsplash

How to protect endpoints of a Nestjs application

Kevin Merckx — Thu, 30 Dec 2021 00:04:05 +0000

Security is a major concern when writing a web service. Nestjs provides awesome features that enables developers to structure code in a very declarative way. For instance, one can protect applications against unauthorized access by using Guards. Guards are injectable blocks that can be added to protect a specific route, an entire controller or an entire application.

By default, I prefer to restrict all routes and refine the route that requires special handling. This is why I apply guards at the top-most level possible. Let's imagine for a moment that your service is protected by an authentication layer, let's say cookie based. Putting the whole API behind an authentication guard is the easiest way to secure it against unauthorized access. You can do that in Nestjs with:

import { NestFactory, Reflector } from '@nestjs/core';
import { AppModule } from './app.module';
import { AuthGuard } from './guards/auth-guard';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  // here we guard the whole application with one Guard
  app.useGlobalGuards(new AuthGuard());

  await app.listen(3000);
}

bootstrap();

Any access to the routes provided by the application is going through this guard, enabling you to protect them. Here is the example of a very dummy guard, where nobody can access anything, as we return false every time:

import { CanActivate, ExecutionContext, Injectable } from '@nestjs/common';
import { Observable } from 'rxjs';


@Injectable()
export class AuthGuard implements CanActivate {
  canActivate(
    context: ExecutionContext,
  ): boolean | Promise<boolean> | Observable<boolean> {
    return false;
  }
}

However, there are several use cases where you need to disable the guard from checking for authorization. For instance, if you have a /login endpoint. In that case, you need to specifically tell the guard to not do anything for this route. You preferably should do it the Nestjs way. Here is one.

Guards can access the execution context. The execution context contains information about the request and can be used to create guards, filters and interceptors. Nestjs provides a decorator SetMetadata that you can add to endpoints to add metadata to the execution context. Therefore, if we add information on a specific route or controller, we can retrieve it from the guard.

Let's consider the following controller:

import { Controller, Get, SetMetadata } from '@nestjs/common';
import { AuthGuardConfig, AUTH_GUARD_CONFIG } from './guards/auth-guard';

@Controller()
export class AppController {
  @Get('guarded')
  guarded(): string {
    return 'This is open.';
  }

  @Get('open')
  @SetMetadata(AUTH_GUARD_CONFIG, { disabled: true } as AuthGuardConfig)
  open(): string {
    return 'This is open.';
  }
}

The first endpoint /guarded is guarded by the guard by default. The second one is not, as specified by the @SetMetadata(AUTH_GUARD_CONFIG, { disabled: true }).
Here is the code of the guard:

import { CanActivate, ExecutionContext, Injectable } from '@nestjs/common';
import { Reflector } from '@nestjs/core';
import { Observable } from 'rxjs';

export interface AuthGuardConfig {
  disabled?: boolean;
}

export const AUTH_GUARD_CONFIG = Symbol('AUTH_GUARD_CONFIG');

@Injectable()
export class AuthGuard implements CanActivate {
  constructor(private reflector: Reflector) {}

  canActivate(
    context: ExecutionContext,
  ): boolean | Promise<boolean> | Observable<boolean> {
    const handlerConfig = this.reflector.get<AuthGuardConfig>(
      AUTH_GUARD_CONFIG,
      context.getHandler(),
    );
    const controllerConfig = this.reflector.get<AuthGuardConfig>(
      AUTH_GUARD_CONFIG,
      context.getClass(),
    );
    if (controllerConfig?.disabled || handlerConfig?.disabled) {
      return true;
    }
    return false;
  }
}

The guard uses the reflector to get ahold of the metadata symbol AUTH_GUARD_CONFIG. It does it at the controller level with context.getClass() and at the method handler level context.getHandler() If a route specifies this metadata and puts the disabled parameter to true, the guard will read it and return true and therefore authorize the access. If a controller does it, all the routes of this controller will be authorized.
The last remaining piece to adjust is that your guard, when declared globally for your app, needs to access the reflector. This is how you can achieve it:

import { NestFactory, Reflector } from '@nestjs/core';
import { AppModule } from './app.module';
import { AuthGuard } from './guards/auth-guard';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  app.useGlobalGuards(new AuthGuard(app.get(Reflector)));
  await app.listen(3000);
}

bootstrap();

Here is the link to a Github repository for easier reference and hope it helps: https://github.com/kevinmerckx/app-guard-nestjs

KM
Photo by Rowan Heuvel on Unsplash

Integration Testing Setup with Nestjs and MongoDB

Kevin Merckx — Tue, 31 Aug 2021 14:01:01 +0000

Originall posted on my blog: https://blog.merckx.fr/integration-testing-setup-with-nestjs-and-mongodb/

Automated testing provides tremendous value when creating software. It enables you to follow a Test Driven Development approach: you can write your tests and design the API and its requirements upfront, then write the code to make the tests pass.

Very often comes the usual questions:

do I create unit tests, integration tests or E2E tests?
what should be mocked?
what am I testing?

While many debates still go on, there are pretty clear rules of thumbs one should follow:

E2E tests provide a very good answer to the question Can we release? but a very poor one to Why do the tests fail?. They are expensive to write.
Unit tests do the exact opposite. They are very good to locate the origin of the failure but are bad to know if you can actually release if they all pass. They are effortless to write.
Integration tests provide a good balance between the two. They are testing the integration points of the components and you can assume that, from one release to the other, if the interfaces between your components didn't change and your integration tests pass, the release can be deployed. They are cheap to write. Not as cheap as unit tests, but far cheaper than E2E tests.

This is why I recommend to focus on integration tests. When writing integration tests, you should always keep in mind that they should be easy to write. Mocking functionnality requires writing code and therefore should be avoided. You should aim at integrating as much as you can from the code that will actually run into your tests.

I use Nestjs in 90% of the projects I work on. In 90% of those projects, I use MongoDB as a database. Writing integration tests for a module should be as easy as writing unit tests without having to mock all the functionnalities around the tested module. Therefore, we must find a way to integrate a real Mongo database when running our integration tests. This is what is going to be presented in the next lines.

Do yourself a favor and create a friendly developer environment

We are going to use Docker to set our development environment up. You might not be friend with Docker but you better become. Docker is a friend that will enable the following:

once set up, you can work offline. You don't need to host development databases in the cloud.
you work with containers that you can later on use in production
you can create many workflows that suit your needs

So go ahead and install Docker on your machine. You need five minutes to do that https://docs.docker.com/get-docker/ and then you can come back.

Containerize your application

At this point, you have Docker running on your machine and a Nestjs project that uses Mongo as a database. We are going to use Docker Compose to create several containers that run different services. The following docker-compose.yml file describes an environment with two services: api and mongo. mongo creates a container that runs a MongoDB server and api is a container that runs NodeJS 14 that exposes ports 3333 and 9229. That way, you can reach your Nest application and the debugger.

version: "3.9"
services:
  api:
    image: node:14
    volumes:
      - .:/app
    ports:
      - 3333:3333
      - 9229:9229
    depends_on:
      - mongo
    environment:
      NODE_ENV: development
  mongo:
    image: mongo:4.4.8
    volumes:
      - ./.docker/mongodb/data/db/:/data/db/
      - ./.docker/mongodb/data/log/:/var/log/mongodb/

You are almost fully set up. Let's describe a few useful workflows.

Start the things you need

We have configured containers. It's time to start and test our NestJS application. Save the following command (as a script in your package.json or as a shell script in your repository:

docker-compose run --service-ports -w /app api bash

This will start the services api and mongo and start a bash shell. In this shell, you can use NPM and Node the same way you would do it without Docker! Try your usual commands npm install and npm start and npm test

I usually save this command in my package.json under the start:env script:

{
  ...
  "scripts": {
    "start:env": "docker-compose run -w /app api bash",
    ...
  }
}

Run the tests once

If you want to run tests in your CI, use the following command:

docker-compose run -w /app api npm run test

where the test script is for instance:

npm install && jest

Write integration tests

Imagine you have a NestJS application with a PostModule that has a controller PostController that has two API endpoints:

GET /post: returns all the posts stored in the database
POST /post: creates a new post and stores it in the database

This module also has a service PostService that PostController instantiates and uses. The service accesses the database. To test your module, one can create tests for the PostService and the PostController. The controller doesn't directly access the database and needs the service to do that. When writing tests for the controller, you have the following choices:

you can mock the service completely
you can mock the database access
you can use the real implementation of the whole module, therefore test the integration of its components: the controller, the service, the database access

We are going for the solution 3. For each integration test, we are going to create a completely new database and delete it when we are finished. Have a look at the following spec file:

import { getConnectionToken, MongooseModule } from '@nestjs/mongoose';
import { NestExpressApplication } from '@nestjs/platform-express';
import { Test } from '@nestjs/testing';
import { Connection } from 'mongoose';
import * as supertest from 'supertest';
import { IPost } from '../model/post';
import { PostModule } from '../post.module';

describe('PostController', () => {
  let app: NestExpressApplication;

  const apiClient = () => {
    return supertest(app.getHttpServer());
  };

  beforeEach(async () => {
    const moduleRef = await Test.createTestingModule({
      imports: [
        MongooseModule.forRoot('mongodb://mongo:27017', { dbName: 'test' }), // we use Mongoose here, but you can also use TypeORM
        PostModule,
      ],
    }).compile();

    app = moduleRef.createNestApplication<NestExpressApplication>();
    await app.listen(3333);
  });

  afterEach(async () => {
    await (app.get(getConnectionToken()) as Connection).db.dropDatabase();
    await app.close();
  });

  it('creates a post', async () => {
    await apiClient()
      .post('/post')
      .send({
        content:
          'I am all setup with Nestjs and Mongo for more integration testing. TDD rocks!',
      })
      .expect(201);
    const posts: IPost[] = (await apiClient().get('/post')).body;
    expect(posts[0].content).toBe(
      'I am all setup with Nestjs and Mongo for more integration testing. TDD rocks!',
    );
    expect(posts[0].likes.length).toBe(0);
  });
});

A few notes:

we do not mock anything
we test the whole PostModule
beforeEach and afterEach are our hooks to create and delete databases. We specify to Mongoose which database name to use and we drop the database with (app.get(getConnectionToken()) as Connection).db.dropDatabase()
we use supertest to create a consumer for our API
we only add our module PostModule to the app
if you would to start the whole application, you would be very close to an actual End-to-end test

I believe that you are set up right now. You have at your disposal an environment where you can write as many integration tests as you can. You can now adopt a TDD approach and deploy a fully tested API.

Note: you can find an example of the setup we described at https://github.com/kevinmerckx/nesjts-with-mongo.

Photo by Glen Carrie on Unsplash

Resources

Impersonation with Nestjs

Kevin Merckx — Wed, 07 Jul 2021 20:29:41 +0000

Originally posted on my blog: https://blog.merckx.fr/impersonation-with-nestjs/

Impersonation is the ability of users to act as other users. Usually it consists in administrators of a system having the ability to interact as standard users on the same system. Gitlab, for instance, provides this feature. It is also part of the feature to provide a way to revert the impersonation.

In this post, we present one approach to implement this feature and we illustrate principles with code from Nestjs.

Assumptions

We assume you already have some sort of user session implementation: a cookie that either directly contains information (client-side cookie, signed with a secret) about the current user or an identifier that links to such information in a store.

We also assume that a REST API that filters in data that belong to the user.

With Nestjs, we will assume that the request object contains a property called user that contains the current information about the user. We will also assume that this property is filled in by a guard, based on the session. We will also assume that the session is implement with a client-side cookie that contains the current user identifier. We will also assume that TypeOrm is used to provide CRUD endpoints and that resources are filtered by user.

interface CustomRequest extends express.Request {
  user: User;
}


interface CustomSession {
  user {
    id: string;
  }
}


@Crud({
  model: {
    type: ...,
  },
})
@CrudAuth({
  property: 'user',
  filter: (user: User) => {
    return {
      userId: user.id,
    };
  },
})
@Controller('entity')
export class MyEntityController implements CrudController<...> {
 …
}

The last assumption is that the Nestjs application has a guard that checks whether the user is authenticated and that adds a user property to the request object.

@Injectable()
export class IsAuthenticatedGuard implements CanActivate {
  constructor(
    @InjectRepository(User) private users: Repository<User>,
  ) {}

  async canActivate(
    context: ExecutionContext,
  ): Promise<boolean> {
     const request = context.switchToHttp().getRequest();
    const session = request.session as CustomSession;
    if (!session.user) {
      return false;
    }
    request.user = await this.users.findOne({ where: { id: session.user.id }});
    return true;
  }
}

Principles

We are going to change as less code as possible but still make pretty explicit what happens. The main idea is to store two pieces of information in the user session:

the actual logged in user
the impersonated user

Therefore, we change the session structure to the following:

interface CustomSession {
  loggedInUser: {
    id: string;
  }
  impersonatedUser: {
    id: string
  }
}

We must know provide two API endpoints that enable the user to impersonate and to stop the impersonation.

The impersonation endpoints

  @Get('impersonate')
  removeImpersonation(@Session() session: CustomSession) {
    session.impersonatedUser = session.loggedInUser;
  }

  @Get('users/:id/impersonate')
  async impersonate(@Param('id') id: number, @Session() session: CustomSession) {
    const user = await this.users.findOne({ where: { id }});
    if (!user) {
      throw new NotFoundException();
    }
    session.impersonatedUser = user;
  }

Now, when logging in, we must fill in the two properties session.impersonatedUser and session.loggedInUser. Here is an example:

The modified login route

  @Post('/login')
  async login(@Session() session: CustomSession, @Body('username') username: string, @Body('password') password: string) {
    session.impersonatedUser = null;
    session.loggedInUser = null;
    const user = await this.authService.validateUser(username, password);
    if (!user) {
      throw new UnauthorizedException();
    }
    session.impersonatedUser = session.loggedInUser = this.serializeUser(user);
    return session.loggedInUser;
  }

Our authentication guard must fill in the request.user with the impersonated user. Here is an example:

The modified authentication guard

  async canActivate(
    context: ExecutionContext,
  ): Promise<boolean> {
    const request = context.switchToHttp().getRequest();
    const session = request.session as CustomSession;
    if (!session.impersonatedUser) {
      return false;
    }
    request.user = await this.users.findOne({ where: { id: session.impersonatedUser.id }});
    return true;
  }

Our authentication fills in the session with the logged in user and the impersonated user information. Our guard fills in the user property with the impersonated user from the user session. Our CRUD controllers can now filter data by user.

Let's walk through the mechanism.

The user logs in. A POST request is sent to the login endpoint. The session is filled in with impersonatedUser and loggedInUser properties.
The user starts impersonating by reaching the /admin/users/1234/impersonate endpoint. Once reached, the session is modified: the impersonatedUserId is filled in with information about user 1234.
The user reaches a CRUD endpoint, let's say /entity. The authentication guard gets the impersonatedUserIdproperty from the session and adds a user property to the request object based on it.
TypeOrm filters the entity by the information stored in the userproperty of the request object.
The user stops the impersonation by reaching /admin/impersonate endpoint. The session is modified: the impersonatedUserId is filled in with the loggedInUserId.

Do not forget to protect the impersonation endpoint!

That's it! With this approach, we make minimal changes to the code base and we enable a great feature: Impersonation.

Photo by Felicia Buitenwerf on Unsplash

Avoid global styles in your Angular projects

Kevin Merckx — Thu, 27 May 2021 16:09:56 +0000

Originally posted on my blog: https://blog.merckx.fr/avoid-using-global-styles-in-your-angular-projects/

CSS is the language used to style HTML pages. It has a simple set of rules to compute values of properties of every DOM element on a page. Angular provides view encapsulation to make sure component's styles only apply to the given component. However, the style of the DOM elements generated by those components are still affected by global styles. This makes debugging CSS hard.

In every project I worked on, custom CSS code is always at least 10% of total amount of code (in number of lines of code). And we always tend to underestimate the effort needed to maintain it. I never actually measured the time spent working on CSS files but I wouldn't be surprised if it would be around 10% of my time as well. CSS deserves a better place in our all web projects.

In a good article, Ben Lorantfy compared global styles in CSS with global variables in JS. To summarize, they both share the following issues:

break encapsulation
poor readability
break information hiding
namespace pollution

Unfortunately, it is very rare to use a UI library that doesn't come with global styles. Bootstrap, Material Design, Prime NG and NG Zorro all come with global styles. However, it doesn't mean that you should add more mess to the mess.

Global styles and especially classes are handy because they enable developers to apply styles to elements by labelling (hopefully, those labels are semantically meaningful but that's another debate). That is something we want to keep in a new approach. The attribute we want to get rid of is that global styles are globally applied. Instead, we want global styles that are locally applied.

In this post, I'm going to present an approach based on SASS mixins. I quote the SASS language's website:

Mixins allow you to define styles that can be re-used throughout your stylesheet.

File structure

The first thing to this approach is to organize your SASS code the same way you would organize your JS/HTML code. Don't neglect it. I put all the global styles under a folder called styles. I suggest putting all the mixins in under the styles/mixins. Modularize mixins in meaningful packages: containers, lists, tables, inputs, buttons, typography etc. At this point, I would suggest something that would look like:

styles/
|__ mixins/
    |__ typography.sass
    |__ containers.sass
    |__ quote.sass
    |__ tables/
        |__ compact-table.sass
        |__ data-table.sass

Mixins in actions

Let's have a look at the quote mixin. This mixin can add some quote-like style to an element: a grey background, some padding, some font style and color.

quote.sass

@mixin quote
  font-style: italic
  background: rgb(240,240,240)
  border-left: 2px solid rgb(200,200,200)
  color: rgb(100,100,100)
  padding: 10px

Here is how we can use our mixin.

app.component.html

<p class="main-quote">Lorem ipsum dolor sit amet, consectetur adipisicing elit.</p>
<p class="second-quote">Molestiae tempore officiis animi sint, corporis officia, vel eaque in, quaerat exercitationem ut.</p>

app.component.sass

@use 'mixins/quote'

.main-quote
  @include quote.quote
  font-size: 20px

.second-quote
  @include quote.quote

At this point, there are several things to note.

Styles stay local

We don't have any style applied globally. We apply global mixins (available to the SASS pre-processor) to local styles.

It keeps the template clean

We assign one class to each <p>, the classes are then defined in the SASS file of the component. We can then apply several mixins to each class.

While assigning one class to each <p> seems to be a tedious process, it actually makes sure our template stays clean: we don't overload class attributes. Style definitions are specified in the SASS file.

"Class check" for free

The SASS pre-processor will complain if a mixin doesn't exist, while CSS won't complain if you have a typo in a class.

I will try this approach along with the one described in An elegant way to enable CSS-customizable Angular components in the next big project I have to lead. I'm curious about the results. The intermediate results on a small project were already promising.

Photo by Ricardo Viana on Unsplash