DEV Community: Arnaud Cortisse

Trying out NestJS part 4: Generate Typescript clients from OpenAPI documents

Arnaud Cortisse — Sun, 28 Mar 2021 13:55:27 +0000

This article was originally published on my personal blog.

Introduction

In my last blog post, we saw how easy it is to get started with OpenAPI using NestJS.

In this blog post, I'd like to show you how you can leverage the generated OpenAPI document in order to generate a typescript client that's going to be used in the React App.

Why would I do that? I like to have statically-typed endpoints, rather than having to do the typing myself. Besides, the fact that it's automatically generated means that we can automate the generation in a CI and make sure everything is OK at compile-time.

Getting Started

The source code for this part of the project is available here: https://github.com/arnaud-cortisse/trying-out-nestjs-part-4.

OpenAPI Generator

There are plenty of tools that we can use in order to generate OpenAPI clients.

The one I'm going to use is the following: typescript-axios.

The OpenAPI document

In the last blog post I only told you about http://localhost:3001/api/, which hosts the Swagger UI.

But there is another key endpoint: http://localhost:3001/api-json. This endpoint hosts the generated OpenAPI document that we'll refer to in order to generate the client.

Setting up the environment for the OpenAPI generator

The OpenAPI generator tool requires that we install several dependencies on our machine, but I don't like to bloat my machine with project-specific dependencies.

Let's try to make use of Docker again!

Preparing the files

In the root folder, execute the following:

mkdir -p tools/openapi-generator
cd tools/openapi-generator
touch Dockerfile
touch openapitools.json
touch generate.sh
touch .gitignore

tools/openapi-generator/Dockerfile

This docker image is going to be used for generating the OpenAPI document by reaching out to NestJS's /api-json endpoint.

FROM timbru31/java-node:jdk-14
RUN npm install @openapitools/openapi-generator-cli -g
RUN mkdir /local
WORKDIR /local
COPY . .
CMD ["sh", "generate.sh"]

We make use of a docker image with preinstalled JDK (because openapi-generator-cli needs it).
We install the openapi-generator-cli.
We create a folder /local and copy everything that's in /tools/openapi-generator into it.
When starting the image we launch the script generate.sh (we still need to fill it).

tools/openapi-generator/openapitools.json

The OpenAPI generator configuration file. See Configuration for more info.

{
  "$schema": "node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "version": "5.0.0"
  }
}

tools/openapi-generator/generate.sh

The script that's executed when starting the newly defined Dockerfile.

openapi-generator-cli generate \
    -i http://nestjs:3001/api-json \
    --generator-name typescript-axios \
    -o /local/out \
    --additional-properties=useSingleRequestParameter=true

-i param indicates where the OpenAPI document is located. Here I decided to use http://nestjs:3001/api-json instead of http://localhost:3001/api-json (both work, but I prefer the former). You won't be able to access http://nestjs:3001/api-json in your browser, since it's not a name you are able to resolve on your machine (but that is resolvable within docker-compose, since both images are going to run in the same network).
--generator-name to indicate that generator we wanna use.
-o to indicate where we want to ouput the generated files.
--additional-properties is used to provide additional parameters to the generator (see this page).

tools/openapi-generator/.gitignore

We don't want to version the file that are output by generator in this folder (but we will version the generated files in the React App).

.build

Modifying docker-compose.yml

Let's make it possible to start openapi_generator from the existing docker-compose file.

  openapi_generator:
    build:
      context: ./tools/openapi-generator
      dockerfile: Dockerfile
    depends_on:
      - nestjs
    volumes:
      - ./tools/openapi-generator/.build:/local/out

We make the service depend on nestjs. That way, nestjs is going to be started if it hadn't been already before. Indeed, it is mandatory for nestjs to be running in order for the openapi_generator to be able to generate the client API.
We mount the folder ./tools/openapi-generator/.build inside the service, where the client is going to be generated (we configured that path ourselves just above). That way, we get access to the generated files on the host machine.

Modifying the root package.json

In the root package.json, add the following script:

"scripts": {
  ...
    "generate-api-client": "docker-compose up --build openapi_generator"
  ...
}

Trying out the OpenAPI Generator

In the root folder, type the following:

npm run generate-api-client.

If everything went fine, you should have files in this folder: tools/openapi-generator/.build.

If you don't have any files, it might be because the nestjs service wasn't ready yet when the generator tried to reach it. Just try to relaunch npm run generate-api-client and everything should be OK.

Delivering the client to the React App.

In the root folder, execute the following:

mkdir scripts
touch scripts/update-api.sh

update-api.sh

#!/bin/bash
cd "$(dirname "$0")"

SOURCE_FOLDER="../tools/openapi-generator/.build"
DEST_FOLDER="../packages/react-app/src/api/generated"

rm -rf $DEST_FOLDER
mkdir -p $DEST_FOLDER
cp $SOURCE_FOLDER/**.ts $DEST_FOLDER

With this script, we essentially are delivering the files generated automatically by the service openapi_generator to the React App.

Modifying the root package.json

In the root package.json, add the following scripts:

"scripts": {
  ...
    "update-api-client": "sh ./scripts/update-api.sh",
    "generate-and-update-api-client": "npm run generate-api-client && npm run update-api-client"
  ...
}

Trying out the delivery mechanism

In the root folder, type the following:

npm run generate-and-update-api-client.

If everything went well, you should have files in packages/react-app/src/api/generated.

Make use of the client in the React App

Installing new dependencies

In the packages/react-app/src directory, execute the following:

npm install axios react-query

Deleting some files

cd packages/react-app/src
rm App.css App.test.tsx App.tsx

Creating new files

cd packages/react-app/src
mkdir axios
mkdir api (but it should already exist)
mkdir components
touch axios/axios-client.ts
touch api/api.ts
touch components/App.tsx
touch components/Example.tsx

packages/react-app/src/axios/axios-client.ts

Used to configure an axios instance so that it is preconfigured to reach to NestJS.

import axios, { AxiosRequestConfig } from "axios";

export const axiosBaseUrl = `${process.env.REACT_APP_BACKEND_SCHEMA}://${process.env.REACT_APP_BACKEND_HOSTNAME}:${process.env.REACT_APP_BACKEND_PORT}`;

export const axiosConfig: AxiosRequestConfig = {
  baseURL: axiosBaseUrl,
};

const axiosBackendClient = axios.create(axiosConfig);

export default axiosBackendClient;

packages/react-app/src/api/api.ts

Configuration of an instance of TasksApi (a class automatically generated by the generator) that we'll use to communicate with our backend.

import axiosBackendClient, { axiosBaseUrl } from "../axios/axios-client";
import { TasksApi } from "./generated";

export const tasksApi = new TasksApi(
  {
    basePath: axiosBaseUrl,
    isJsonMime: () => false,
  },
  undefined,
  axiosBackendClient
);

packages/react-app/src/components/App.tsx

import React from "react";

import { QueryClient, QueryClientProvider } from "react-query";
import Example from "./Example";

const queryClient = new QueryClient();

export default function App() {
  return (
    <QueryClientProvider client={queryClient}>
      <Example />
    </QueryClientProvider>
  );
}

We configure the react-query provider.
We render the Example component (yet to be defined).

packages/react-app/src/components/Example.tsx

import { useQuery } from "react-query";
import { tasksApi } from "../api/api";

export default function Example() {
  const id = "fake id";

  const { isLoading, error, data } = useQuery(`tasks_find_one_${id}`, () =>
    tasksApi.tasksControllerFindOne({
      id,
    })
  );

  if (isLoading) return <div>Loading...</div>;

  if (error as Error) return <div>An error has occurred</div>;

  return <div>{data?.data.title}</div>;
}

Have a look at the query. This is where the magic happens: we make use of the automatically-generated client and, as a result, have all the benefits of static types.

Modifying existing files

packages/react-app/src/index.tsx

I just removed some useless lines (in the context of this blog) and imported the App component from the appropriate path.

import React from "react";
import ReactDOM from "react-dom";
import "./index.css";
import App from "./components/App";

ReactDOM.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
  document.getElementById("root")
);

Trying out the client

In the root folder, perform the following:

docker-compose up --build (might take a while since there are new dependencies in the React App to be installed).

Go over http://localhost:3000/ in your browser.

You should have the following message at some point: An error has occurred.

Open your developer tools: you should see a CORS error. We can fix that by updating the Nest app.

Enable CORS

In packages/nestjs/src/main.ts, add the following

...
  app.enableCors();
...

Mind you, you should definitely configure the CORS rules appropriately in a production environment.

Testing everything out

Now, if you go on http://localhost:3000/ in your browser, you should see the message fake title.

It means we are indeed able to communicate with our API using an automatically-generated client.

Final Words

Setting everything up wasn't straightforward. Nevertheless, we now have a nice way of communicating with our API: we have a typed client that's going to greatly improve the development experience inside React. What's more, it basically costs nothing to regenerate that client so that it matches the latest API. Lastly, we are now able to detect any desynchronization between the React App and the NestJS app at compile time.

Trying out NestJS part 3: Creating an OpenAPI document

Arnaud Cortisse — Sat, 06 Mar 2021 15:31:03 +0000

This article was originally published on my personal blog.

Introduction

In my last blog post, I showed you how easy it is to set up REST API routes using NestJS.

In this blog post, I'd like to try out the OpenAPI module NestJS provides. I'm doing it at this stage because I want to be able to test my routes easily without resorting to any external tools (like Postman).

Getting Started

The source code for this part of the project is available here: https://github.com/arnaud-cortisse/trying-out-nestjs-part-3.

The official documentation is here: OpenAPI.

Stop the running Docker containers, if any.
Go in packages/nestjs and run npm install --save @nestjs/swagger swagger-ui-express.
Go in the root directory and type in docker-compose up --build.

Now you're good to go with the OpenAPI module.

Creating the OpenAPI document

The bootstrap function (in main.ts) is where you're going to instantiate the OpenAPI module.

import { NestFactory } from '@nestjs/core';
import { DocumentBuilder, SwaggerModule } from '@nestjs/swagger';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  const config = new DocumentBuilder()
    .setTitle('Tasks')
    .setDescription('The tasks API description')
    .setVersion('1.0')
    .addTag('tasks')
    .build();

  const document = SwaggerModule.createDocument(app, config);
  SwaggerModule.setup('api', app, document);

  await app.listen(process.env.NEST_PORT);
}
bootstrap();

The code speaks for itself: we're essentially setting up the module that's going to deal with the OpenAPI documentation. SwaggerModule.setup('api', app, document); makes the documentation available on the route /api.

Testing Current Setup

Once main.ts has been modified, just go over http://localhost:3001/api/ in the browser. You should have the following:

As you can see, the OpenAPI module in NestJS is pretty cool: it automatically picks up the existing routes and DTOs and makes them available.

However, you can see that every route is under the tab default, while I would have liked the routes related to the Tasks to be under the tab Tasks. In order to fix that, modify the tasks.controller.ts file and add the @ApiTags('tasks') decorator:

...
import { ApiTags } from '@nestjs/swagger';

@ApiTags('tasks')
@Controller('tasks')
export class TasksController {
...

and reload http://localhost:3001/api/.

Creating the Task DTO

TaskDto is going to be a class containing an exhaustive list of all the properties a Task contains.

Let's create a file task.dto.ts inside ./packages/nestjs/tasks/dto/.

import { ApiProperty } from '@nestjs/swagger';

export class TaskDto {
  id: number;
  title: string;
  description: string;
}

Modifying CreateTaskDto

CreateTaskDto will be used to create new Tasks. Right now it's empty (both in code and in the OpenAPI document). Let's change that:

import { PickType } from '@nestjs/swagger';
import { TaskDto } from './task.dto';

export class CreateTaskDto extends PickType(TaskDto, [
  'description',
  'title',
] as const) {}

You can see that I used the utility class PickType. That way, we can keep the code as DRY as possible.

When creating a task, we want to provide a title and a description, while the ID should somehow be generated internally (we don't have any DB yet).

When I refresh http://localhost:3001/api/, I can't see any of my changes reflected in the OpenAPI document:

It's because I didn't explicitly tell the OpenAPI module that I wanted the new properties to be exposed. To do so, just add the decorator @ApiProperty:

export class TaskDto {
  @ApiProperty()
  id: number;

  @ApiProperty()
  title: string;

  @ApiProperty()
  description: string;
}

Now you can see those properties if you go over http://localhost:3001/api/:

But what's cool about it is that I can now play with the OpenAPI UI more easily.

Say that you want to test the route POST /tasks. Just use the OpenAPI UI (by clicking on Try it out):

The request body will be prefilled with some default values and you'll be able to visualize what is your API replying very easily:

Modifying UpdateTaskDto

UpdateTaskDto will be used to update our tasks. It currently looks like this:

import { PartialType } from '@nestjs/mapped-types';
import { CreateTaskDto } from './create-task.dto';

export class UpdateTaskDto extends PartialType(CreateTaskDto) {}

We want to be able to update both the title and the description, so we don't need the utility class PartialType:

import { CreateTaskDto } from './create-task.dto';

export class UpdateTaskDto extends CreateTaskDto {}

That change is now reflected in the OpenAPI endpoint:

Add more information about the input parameters

At present the only pieces of information we have about the title and the description is that there are strings. We could also decide to provide further information, such as the minimum and the maximum length of those strings:

export class TaskDto {
  @ApiProperty({
    description: 'The ID of the task',
    default: 'This is a fake ID',
  })
  id: number;

  @ApiProperty({
    description: 'The title of the task',
    minLength: 3,
    maxLength: 30,
    default: 'This is a fake title',
  })
  title: string;
  @ApiProperty({
    description: 'The description of the task',
    minLength: 0,
    maxLength: 200,
    default: 'This is a fake description',
  })
  description: string;
}

which translates like this in the OpenAPI endpoint:

Be aware that those decorators don't perform any data validation operations. By the way, NestJS recommends the use of the class-validator package for data validation (which I'll try out later).

Add further information about the responses

Currently our API don't expose what any of its responses look like. The default is status code 200 and doesn't specify the shape of the data. Let's improve that.

In tasks.controller.ts, add these lines:

...
  @Get(':id')
  @ApiOkResponse({
    description: 'The task has been successfully found.',
    type: TaskDto,
  })
  findOne(@Param('id') id: string) {
    return this.tasksService.findOne(+id);
  }
...

The @ApiOkResponse decorator allows us to specify what our API responds code-wise (200 in this case, but we could have used the @ApiResponse decorator and specified the response code ourselves) as well as the shape of the data returned. It yields the following:

You can also specify what the response looks like when you can't find the requested task:

...
  @ApiNotFoundResponse({
    description: "Couldn't find the task",
  })
...

Final Words

As you can see, getting started with OpenAPI inside NestJS is very easy. I would strongly encourage people to document their APIs using such tools: It's directly integrated inside the source code and it's very easy to maintain. The benefits of documenting your APIs are multiples. Among other things, that makes it very easy to understand and test them.

Trying out NestJS part 2: Creating REST endpoints

Arnaud Cortisse — Wed, 27 Jan 2021 11:40:03 +0000

This article was originally published on my personal blog.

Introduction

In my last blog post, I talked about why I wanted to give NestJS a try and what project I would build in order to evaluate it.

In this blog post, I'd like to set up the CRUD endpoints for the Tasks concept.

Before going on

One of the main benefits of using NestJS is that it's very opinionated about the way you should structure your application. As a result, you'll end up with an application that's loosely coupled and highly testable without even thinking about it.

However, NestJS can be hard to apprehend (especially if you are a junior developer). In my opinion, it's very important that you understand what Dependency Injection (and broadly speaking, the SOLID principles) and DI containers are since NestJS relies heavily on those concepts.

Setting up new endpoints

The source code for this part of the project is available here: https://github.com/arnaud-cortisse/trying-out-nestjs-part-2.

Creating CRUD endpoints is not very exciting. It's actually very tedious because the code always look the same.
Fortunately, NestJS provides a sweet tool to generate all the boilerplate required to setup REST endpoints.

Using the CRUD generator

Before using the CRUD generator, please install the following package:

npm install @nestjs/mapped-types

In the root of your project, just type in

nest g resource tasks

You will be prompted with several choices. In this case, we're interested in developing a REST API.

? What transport layer do you use? (Use arrow keys)
❯ REST API
  GraphQL (code first)
  GraphQL (schema first)
  Microservice (non-HTTP)
  WebSockets

Then, choose YES.

? Would you like to generate CRUD entry points? (Y/n)

You should have the following output:

CREATE src/tasks/tasks.controller.spec.ts (566 bytes)
CREATE src/tasks/tasks.controller.ts (890 bytes)
CREATE src/tasks/tasks.module.ts (247 bytes)
CREATE src/tasks/tasks.service.spec.ts (453 bytes)
CREATE src/tasks/tasks.service.ts (609 bytes)
CREATE src/tasks/dto/create-task.dto.ts (30 bytes)
CREATE src/tasks/dto/update-task.dto.ts (169 bytes)
CREATE src/tasks/entities/task.entity.ts (21 bytes)
UPDATE src/app.module.ts (312 bytes)

The NestJS CLI has just generated a lot of boilerplate code for you.

Let's have a closer look into some of the generated files.

tasks.service.ts

It's a service that's going to deal with the data storage and retrieval.

import { Injectable } from '@nestjs/common';
import { CreateTaskDto } from './dto/create-task.dto';
import { UpdateTaskDto } from './dto/update-task.dto';

@Injectable()
export class TasksService {
  create(createTaskDto: CreateTaskDto) {
    return 'This action adds a new task';
  }

  findAll() {
    return `This action returns all tasks`;
  }

  findOne(id: number) {
    return `This action returns a #${id} task`;
  }

  update(id: number, updateTaskDto: UpdateTaskDto) {
    return `This action updates a #${id} task`;
  }

  remove(id: number) {
    return `This action removes a #${id} task`;
  }
}

@Injectable() is a TypeScript decorator.
Decorating classes with @Injectable allows them to act as providers, that is, instances of classes that can be injected in other instances of classes (at runtime) via Dependency Injection.

tasks.controller.ts

It's a controller that's going to handle incoming requests related to Tasks.

import { Controller, Get, Post, Body, Put, Param, Delete } from '@nestjs/common';
import { TasksService } from './tasks.service';
import { CreateTaskDto } from './dto/create-task.dto';
import { UpdateTaskDto } from './dto/update-task.dto';

@Controller('tasks')
export class TasksController {
  constructor(private readonly tasksService: TasksService) {}

  @Post()
  create(@Body() createTaskDto: CreateTaskDto) {
    return this.tasksService.create(createTaskDto);
  }

  @Get()
  findAll() {
    return this.tasksService.findAll();
  }

  @Get(':id')
  findOne(@Param('id') id: string) {
    return this.tasksService.findOne(+id);
  }

  @Put(':id')
  update(@Param('id') id: string, @Body() updateTaskDto: UpdateTaskDto) {
    return this.tasksService.update(+id, updateTaskDto);
  }

  @Delete(':id')
  remove(@Param('id') id: string) {
    return this.tasksService.remove(+id);
  }
}

@Controller() is a TypeScript decorator.
Decorating classes with @Controller, along with methods decorated with HTTP verbs (@Get(), @Post() etc.), allows you to declare the routing of your app.

You can also notice that the constructor expects an instance of TaskService (the class we talked about earlier). Behind the scene, an instance of the TaskService is going to be injected by the DI container (built in the NestJS runtime, see custom-providers).

tasks.module.ts

It's a module that's going to hold all the instances of classes related to the Tasks.

import { Module } from '@nestjs/common';
import { TasksService } from './tasks.service';
import { TasksController } from './tasks.controller';

@Module({
  controllers: [TasksController],
  providers: [TasksService]
})
export class TasksModule {}

@Module() is a TypeScript decorator.
Decorating classes with @Module allows for regrouping and encapsulating code that's closely related.

It's also going to affect how the DI container behaves.

app.module.ts

The app module is the root module (see modules). It's created inside main.ts, at startup.

import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { TasksModule } from './tasks/tasks.module';

@Module({
  imports: [TasksModule],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

You can see the CRUD generator updated imports and now references TasksModule, which is required for our app to expose all the routes declared inside TasksController. TasksModule is going to be directly connected to the AppModule in the application graph.

Not putting TasksModule inside the imports array would cause your tasks routes not to be available.

Test current setup

Go in the root of your project and type in

docker-compose up --build

Once everything is up and running, go to http://localhost:3001/tasks and verify that your get the message "This action returns all tasks".

Final words

As you can see, setting up new REST endpoints with NestJS is extremely easy.

However, you might be thinking there are way too many files and concepts involved. While I think that's quite true for such a small project, keep in mind that NestJS shines best when dealing with medium to large size projects.

Trying out NestJS part 1: Setting up a dev environment for your React / NestJS applications that rocks

Arnaud Cortisse — Sun, 03 Jan 2021 11:15:11 +0000

This article was originally published on my personal blog.

Introduction

In the context of my current job, I wanted to evaluate the various existing backend frameworks based on NodeJS. Why is that? The only Node.js backend framework I had ever used so far was express, which is an awesome light-weight framework, but that doesn't have any opinion whatsover on how you should structure your app.

During my investigation, I came across NestJS several times. The most appealing thing to me was its thorough documentation and its large ecosystem. I was especially interested in the OpenAPI integration, which I knew could greatly improve the frontend development experience when coupled with a code generator.
In the end, I decided to create a small POC to see whether it would be a fit.

Specifications of the project

Functional requirements

The POC is going to be a minimal, hideous "TODO list" app (styling is not in the scope of this endeavor).
In this POC, I'll be able to:

Add tasks,
Remove tasks,
List all tasks

Technical requirements

Use Typescript everywhere.
NestJS for the backend.
React for the frontend.
Tasks are saved in a Postgres DB.
Redis is used for caching responses.
API endpoints are documented using OpenAPI.
API endpoints' parameters are validated in the backend.
Frontend code related to the API endpoints is auto-generated.
The development environment is set up in docker.
Monorepo containing both the backend and the frontend.

Building the project

The source code for this part of the project is available here: https://github.com/arnaud-cortisse/trying-out-nestjs-part-1.

Setting up Docker dev environment

Since I took Docker and Kubernetes: The Complete Guide and Microservices with Node JS and React courses, I've been a huge fan of setting up my dev environment inside docker instances instead of setting it up directly on my machine. I love the fact that I can have everything up and running with a single command, without having to worry about dependency conflicts (is my current version of NPM compatible with that project?, etc.).

A few commands to execute

Install the Nest CLI: npm i -g @nestjs/cli (you might need to prefix it with sudo)
Create an empty folder: mkdir todo-list-app
Go inside the folder: cd todo-list-app
Init npm package: npm init -y
Init git, if you want to save your work: git init
Create frontend folder: mkdir -p packages/react-app
Create backend folder: mkdir -p packages/nestjs
Create the React app: npx create-react-app packages/react-app --template typescript
Create the NestJS app: nest new packages/nestjs
Delete the .git folder automatically created by NestJS: rm -rf packages/nestjs/.git
Create frontend env variables file: touch packages/react-app/.env.dev
Create backend env variables file: touch packages/nestjs/.env.dev
Create frontend Dockerfile for dev environment: touch packages/react-app/Dockerfile.dev
Create backend Dockerfile for dev environment: touch packages/nestjs/Dockerfile.dev
Create docker-compose file for dev environment: touch docker-compose.yml
Create frontend .dockerignore file: touch packages/react-app/.dockerignore
Create backend .dockerignore file: touch packages/nestjs/.dockerignore

A few files to fill / change

packages/react-app/Dockerfile.dev

FROM node:alpine
WORKDIR /app

COPY package.json .
RUN npm install --legacy-peer-deps
COPY . .

CMD ["npm", "run", "start"]

--legacy-peer-deps is just a temporary fix for https://github.com/facebook/create-react-app/issues/9515.

packages/nestjs/Dockerfile.dev

FROM node:alpine

WORKDIR /app
RUN npm install -g @nestjs/cli
COPY package.json .
RUN npm install
COPY . .

CMD ["npm", "run", "start:dev"]

Nothing crazy here, but we just make sure we install the NestJS CLI globally before doing anything else.

packages/react-app/.env.dev

REACT_APP_BACKEND_SCHEMA=http
REACT_APP_BACKEND_HOSTNAME=localhost
REACT_APP_BACKEND_PORT=3001
CHOKIDAR_USEPOLLING=true

CHOKIDAR_USEPOLLING is required when developing inside a docker container and using create-react-app (https://github.com/facebook/create-react-app/issues/1049#issuecomment-261731734).
The other variables are defined so that we can communicate with the NestJS API.

packages/nestjs/.env.dev

NEST_PORT=3001
PGHOST=postgres
PGPORT=5432
PGUSER=postgres
PGPASSWORD=example
PGDATABASE=postgres
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_PASSWORD=password
REDIS_TTL=10

We define the port on which NestJS will run as well as the Postgres and Redis configs.

packages/react-app/.dockerignore

node_modules

We don't want the local node_modules folder to be copied over the instance.

packages/nestjs/.dockerignore

node_modules

docker-compose.yml

version: "3.5"

services:
  nestjs:
    build:
      context: ./packages/nestjs
      dockerfile: Dockerfile.dev
    env_file:
     - ./packages/nestjs/.env.dev
    ports:
      - 3001:3001
    volumes:
      - ./packages/nestjs/:/app
      - /app/node_modules

  react_app:
    build:
      context: ./packages/react-app
      dockerfile: Dockerfile.dev
    env_file:
     - ./packages/react-app/.env.dev
    ports:
      - 3000:3000
    volumes:
      - ./packages/react-app/:/app
      - /app/node_modules

  postgres:
    image: postgres:13.1
    environment:
      POSTGRES_PASSWORD: example
    ports:
     - 5432:5432

  redis:
    image: redis:6.2-rc1
    environment:
      REDIS_PASSWORD: password

  redis_commander:
    image: rediscommander/redis-commander:latest
    restart: always
    environment:
      - REDIS_HOSTS=local:redis:6379
    ports:
      - 8081:8081
    depends_on:
      - redis

nestjs is our backend.
react_app is our frontend.
postgres is going to be used to store the tasks.
redis is going to be used by NestJS to cache responses.
redis_commander is just a tool that allows us to examine the Redis DB quickly.
volumes under react_app and nestjs is key to get auto-reload whenever you modify files inside your editor. The only annoying thing with this setup is that you'll need to rebuild your docker images whenever you add a new dependency inside your node_modules (see https://github.com/BretFisher/node-docker-good-defaults for fixes).

packages/nestjs/src/main.ts

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(process.env.NEST_PORT);
}
bootstrap();

Modify the port the app is listening to using the process.env.NEST_POST environment variable (defined in packages/nestjs/.env.dev).

Test current setup

You should now be able to start your dev environment typing docker-compose up in the root directory.

You can then go to the following addresses:

localhost:3000 --> React app.
localhost:3001 --> NestJS app.
localhost:8081 --> Redis Commander (which should be connected to your Redis instance).

Final words

With the current state, I've a working dev environment inside dev containers. All I have to do to get started is docker-compose up (sometimes, I have to do a docker-compose up --build, depending on whether or not new npm packages have been installed).
Whenever I update any .ts files in my code editor, the apps are reloaded accordingly, making it a very pleasant dev experience for the task at hand: asserting whether or not NestJS is going to be a good fit for me by developing a POC.

Part 2 is available here.

My Journey Creating My First Solo Project (part 3): Cloud Architecture

Arnaud Cortisse — Sat, 29 Aug 2020 14:22:56 +0000

This article was originally published on my personal blog.

Introduction

In my previous blog post, I talked about the technologies and tools I used to create DoNotSkip.

In this blog post, I'd like to describe the overall cloud architecture of DoNotSkip.

Cloud Architecture

Below, the big picture of the cloud architecture.
Many things are not represented in the schema, but it gives you an idea.

Firebase
- Hosting: Serves static files such as HTML files, CSS files, JavaScript files, images, etc. with a global CDN.
- Storage: Stores coaches' assets such as images. These assets are only available to those uploading them.
- Authentication: Allows authentication via identity providers such as Google, Apple, etc.
- Cloud Functions: Hosts the backend logic. Examples of what is done in those functions: crop and resize images, transfer images from one bucket to another, etc.
Heroku
- Hasura: GraphQL API over postgreSQL, event system (enabling the [3factor app] architecture), etc.
- PostgreSQL.
Google Cloud
- Storage: Host workout programs' assets for the public.
- App Engine: Host Next.js app.

In reality, things are a bit more complex than they are in the schema:

There are several instances of Hasura and postgreSQL.
There are two different Firebase projects: one for the coaches and one for clients.
Everything is duplicated between staging and production environments.

A few examples

To give you an idea of how everything is put together, I want to show some practical examples of how every piece communicates with one another in some user use cases.

What happens when logging in?

Below is an overview of how the login process takes place. This logic is the same across the different parts of the platform.

Using the Firebase SDK, sends a request to the Google Authentication service.
In the client, an observer is listening to all Firebase auth state changes. When the previous request succeeds, the observer gets information about the user such as their unique User ID and a JWT token. The User ID allows to uniquely identify in the whole platform while the JWT token is used to confirm the identity of the user making requests.
When the user is identified successfully, the client sends a request to the backend (in this case, Hasura) to notify that the user has actually logged in.
The Hasura GraphQL Engine automatically checks the validity of the JWT token.
Communicates with the Authentication Service if JWK has expired to get a new one.
When the JWT token is valid, the database is updated.
The backend replies with a successful response.

How are coaches' assets uploaded?

Below is an overview of how images are uploaded in DoNotSkip Coaches.

Using the Firebase SDK, an upload task is started client-side.
The file is uploaded successfully, the client receives a successful response.
Metadata about the file is added to the SQL database via Hasura. The JWT token is sent automatically with every request.
The Hasura GraphQL Engine automatically checks the validity of the JWT token.
If the JWK has expired, get a new one.
The token is valid, Hasura proceeds with the update.
Sends a successful response to the client.

How are workout programs published?

Below is an overview of how program publications work inside DoNotSkip Coaches. The architecture used here is called 3factor app.

A GraphQL mutation is sent to Hasura. This GraphQL mutation inserts a row in a table called Publish Program.
The Hasura GraphQL Engine automatically checks the validity of the JWT token.
If the JWK has expired, get a new one.
The token is valid and a row is added to Publish Program.
A response is sent to the client saying that the row has been added successfully.
Because the response is successful, the client creates a GraphQL subscription to get real-time updates on the publication of the program (using WebSocket).
An event trigger is set up in the database so that whenever a row is added in Publish Program, a request is sent to a custom webhook (in this case, it's a cloud function hosted on Firebase).
The cloud function does everything it needs to do in order to make the program available publicly (compresses images, sanitizes text, etc.). Once everything is done, it updates the SQL database to notify that the program has been successfully uploaded. The client is in-turn notified.

The publication process itself is explained in more details below.

The webhook "Publish Program" is called. A cloud function is started. This function has two purposes: 1) the creation of the landing page for the program and 2) the creation of the program that's going to be used in the app.
All information regarding the program to publish is retrieved in Hasura.
Various stuff is done in the function:
1. Images are cropped and optimized (if required).
2. Pieces of text are sanitized.
3. Images are moved in a public bucket.
4. Various JSON documents are created.
The JSON documents are published to an Hasura instance hosting the published programs.

Final words

As you can see, Hasura plays a crucial role in this architecture (3factor app): most of the time, whenever the frontend app wants to perform a backend action that requires authorization, it does so by inserting a new row in the SQL database via the GraphQL API and then subscribes to some database row changes. The cloud functions are invoked behind the scene via a clever event system provided by Hasura.

This cloud architecture would be able to accept a high volume of traffic:

It's going to be a long time before Hasura becomes a bottleneck (see Scaling to 1 million active GraphQL subscriptions with Hasura). Furthermore, I have set up three different Hasura instances that can scale vertically independently of each other (one for Coaches, one for Publications and another one for User savings). It wouldn't be difficult to put a load-balancer in front of the Hasura instances in order to scale horizontally.
Cloud functions automatically scale.

I could go on and on and explain other use cases, but I think you get the gist of DoNotSkip's cloud architecture.
In the next blog post, I'd like to talk about the code architecture of DoNotSkip App, the mobile app of the platform built with React Native.

My Journey Creating My First Solo Project (part 2): Technologies and tools

Arnaud Cortisse — Mon, 17 Aug 2020 15:10:38 +0000

This article was originally published on my personal blog.

Introduction

In my previous blog post, I described the motivations behind building DoNotSkip, what problems it would try to solve and the big picture of how I would go about building it.

In this blog post, I'd like to describe what are the main technologies / tools I used to create it and why I chose them.

Main parts

As described in my previous blog post, the project consists of 3 main parts:

A platform for coaches to create and publish workout programs: a single-page application.
A platform for the workout programs' landing pages: a server-side rendered application.
A mobile app to use the workout programs

Technologies / tools

I'm going to list the most important libraries and tools I used in this project. Just so you know, I didn't have to pay anything for the entire development period (and I think that's pretty amazing).

Common

Most of the technologies and tools are common to the three main parts of the platform, and that's one of the reasons JavaScript is so awesome!

The community is very strong and a lot of empowering tools have been built for you.

Even though I had to build 3 quite different products (a SPA, a SSR app and mobile app), I was able to leverage stuff from each app and apply it to the others. No need to relearn everything from scratch as soon as the requirements change!

Front-end

Typescript as the programming language. I actually got started with JavaScript, but I soon realized it didn't scale well (more on that in another blog post). I do not regret migrating to Typescript whatsoever.
React for the UI. I could have chosen Vue or Angular, but React was the only modern web UI framework I'd used so far.
Redux for local state management. In my opinion (and I know it's a moot point), Redux is the go-to state management library when developing complex React apps. In retrospect, I shouldn't have used it in DoNotSkip Programs (because the state isn't complex in that app). Otherwise, I think it's an awesome library that scales very well! If I was to start a new React project with Redux, I would definitely have a look at Redux Toolkit, which seems to reduce the amount of boilerplate code to write.
Reselect to access the Redux state. Not only can you centralize and test the way you access your state, but you can also compute derived state and memoize the result.
Redux Typed Saga for side-effects. It's just a library providing Typescript over Redux Saga. I basically had to choose between Redux Saga and Redux Thunk. Redux Saga documentation looked very clear and I liked how the code read (I find it way easier to read asynchronous code than nested callbacks).
react-18next for internationalization. I actually don't remember why I chose it over react-intl but all I can say is it's very easy to use.
Apollo Client to perform client-side GraphQL requests. There exists plenty of other libraries, but this is one of the most famous. Furthermore, it supports subscriptions which I knew I would need. I just think I haven't really used it properly because I don't leverage its built-in cache system.
Immer makes updating states in Redux reducers way easier than in plain JavaScript. I started out with plain JavaScript and felt the code of my reducers was becoming too difficult to read.

Back-end

Typescript as the programming language. Like I said before, I started out with JavaScript but didn't like where my codebase was going.
Node.JS as the backend framework. Very popular backend framework in the web development world since the programming language is JavaScript. Deno might be its replacement, but not anytime soon.
Firebase for:
- Authentication: you get OAuth authentication for free (with all major sign-in providers).
- Serverless functions: to me, serverless is the future of backend. Why would I even bother managing my own servers when I can have them managed by experts? You only pay when the functions are up (and they go back to sleep after a while) and they scale automatically as demand grows. Of course, they have drawbacks (such as cold-starts), but the advantages far outweighed them in my case.
- Storage: I didn't really compare it with anything else, since I was already all in on Firebase.
Google Cloud is the cloud provider I chose. First of all, it's what Firebase uses behind the scene. Secondly, they offer a generous 300€ credit (only for the first year). Finally, after having tested Azure and AWS, I found it way easier to use.
Hasura here is the official description of the product: "Hasura GraphQL Engine is an opensource product that connects to your databases & services and gives you a realtime GraphQL API, instantly." I won't go into the details in this blog post but what I can already tell you is that it's really awesome. You basically have fine-grained access to your database via an automatically generated GraphQL API.
PostgreSQL for the database. I didn't really choose this one, since this is the only database compatible with Hasura.
Heroku to host Hasura engines and PostgreSQL databases. Their free tier is also very generous and you can get up and running in a few minutes.

Miscellaneous

Axios as the HTTP clients for both the frontend and the backend. That way, I can use the same syntax everywhere.
Yup to parse and validate JavaScript objects.
Jest for unit-testing. I picked up this framework because it was the only one I was familiar with in the JavaScript ecosystem.
Github.
Lerna makes dealing with monorepos easier.
Husky to launch linting + testing automatically whenever I commit changes to Github.
GraphQL Code Generator to automatically generate Typescript types for my GraphQL requests.
Google Analytics for analytics. The free version of Google Analytics provides more than enough insight.
Sentry for application monitoring and error tracking. I had heard it advertised many times in some of the podcasts I listen to and decided to give it a go.
Github Actions for CI/CD. Pretty easy to use, integrated with Github and generous free tier.

Project-specific

I needed to use the right tool for the right job. The 3 projects are each of different nature.

DoNotSkip Coaches

Create React App to set up the project. You get everything configured and you are good to go in no time.
Material-UI for the UI components and design system. I've always loved the simplicity of Google products and since I'm not good at designing, I thought it would be a wise decision to have strict guidelines and predefined components.

DoNotSkip Programs

Next.JS to render server-side pages. It's the go-to framework when it comes to building SSR apps with React. It's also the framework I had the most difficulty getting started with because I didn't immediatly understand the core concepts. A few major releases came out since the first time I used it that have made it easier to get started with.
App Engine to host the Next.JS app.
Material-UI for the UI components and design system.

DoNotSkip App

React Native to build hybrid apps. It was a no-brainer since I already had used React so much.
Expo to have even more tools and services around React Native.
NativeBase for the UI components and design system.

DoNotSkip Landing Page

Gatsby to create static pages with React.
Material-UI for the UI components and design system.

Conclusion

As you can see, I've used a lot of third-party libraries in order to build DoNotSkip (and I didn't mention all of them...). It's pretty common to do that in the JavaScript ecosystem instead of reinventing the wheel. When picking a library, I just made sure it was still maintained.

Now that you know the tech stack I used, I'm going to describe the cloud architecture of DoNotSkip in the next blog post.

My Journey Creating My First Solo Project (part 1): Defining the project

Arnaud Cortisse — Mon, 10 Aug 2020 14:30:20 +0000

This article was originally published on my personal blog.

Introduction

In my previous blog post, I talked about how I got into web development and that at some point, I felt that I needed to create a project on my own in order to consolidate all my newly-acquired knowledge.

In this article, I'm going to talk about DoNotSkip, the project I decided to create.

Coming up with an idea

I wanted to find a project idea meeting the following requirements:

make use of my previously acquired knowledge,
is complex enough that I still need to learn new stuff in the process,
is unique and solve a real world "problem",
can potentially make a little money.

If all of these requirements were met, I knew the project would keep me interested and motivated, which meant I would go through to the end of it, whatever it takes.

The project idea

It's a fair statement to say that it's easier to solve problems in domains you already know about.

Therefore, I decided to try and find an idea related to fitness since I've been working out for about 8 years.
When I'm working out, I like to take note of my performance and see whether I'm improving. Maybe I could develop a mobile app that would allow me to record my performance? Let's be honest, that's not original! I wanted to go a bit further...

And then, I came up with something that would make it more original. You see, I like to watch fitness Youtubers every once in a while. I've noticed that some of them sell or share workout programs in PDF files. PDF files... Really? Maybe that's something I should fix with my project.

What if I created a platform that would allow coaches to create their own workout program (with their own images, pieces of text, etc.) and to share it with their community, which in turn would use it in a mobile application? I found similar platforms existed, but I couldn't find any that offered the possibility to distribute workout programs at scale (because their pricing model is not thought out that way).

Requirements

The product would consist of three main parts:

A platform coaches can use to create highly customizable workout programs. That platform must also allow coaches to publish their workout programs.
A platform used to host the workout program landing pages. When coaches are done with creating their workout program, they need to be able to publish it and share it with their community.
An app members of the communities can use with their coach's workout program.

Initially, I wanted coaches to be able to sell their programs on the platform, but I decided against it in the end (I'll explain why in another blog post).

Roadmap

My first roadmap was pretty straightforward:

Get an idea of which technologies I would use for each part of the product.
Create a prototype of:
1. the coach platform.
2. the user app.
3. the platform hosting the workout landing pages.
4. the landing page of the project.

I couldn't plan too much in advance at that point because there were too many unknowns.

Picking up technologies

It was time for me to choose which technologies I would use in each part of DoNotSkip.

In the next blog post, I'll tell you how I picked them and what my thought process was.

My Journey Learning Web Development

Arnaud Cortisse — Thu, 06 Aug 2020 05:14:05 +0000

This article was originally published on my personal blog.

Introduction

My name is Arnaud Cortisse and I've been a developer for 5 years. The first 4 years of my professional career were mostly spent using low-level languages. At present, I'm exclusively using web technologies.

In this article, I want to explain my journey getting into web development: Why I got into it, how I went about learning it, the resources I used, etc.

What got me into web development

In February 2019, I decided to resign from the company I was working for to go work for a startup.

Only about 6 months after I joined, me and half of my coworkers got laid off because they decided to cut costs.

I saw there an opportunity to learn stuff I'd been eager to learn for a long time, that is, web development. Why web development? I'm a huge podcast listener and a big part of what I listen to includes web technologies. Being exposed to those talks over and over made me want to dig more into that topic.

I decided that I would learn it full-time, before taking another job.

Learning web development

At that time, even though I had been a professional developer for several years, I hadn't done any serious web-related projects. I knew I would basically have to start from the beginning but that I would be able to leverage my previous work experiences and move (way) faster than somebody who has never been exposed to programming before.

The methodology

From my own experience, I know what works best for me when it comes to learning new stuff is watching video courses while coding along.
Back then, I had already used Udemy a couple of times and I had enjoyed it, so I decided to stick with it.

What I also do when I feel like one concept is poorly explained in a course is search for other resources and delve deeper on my own (e.g. Medium, dev.to, Youtube, etc.).

The journey

HTML and CSS (2 weeks)

The first thing I set out to do was learning the web foundations, that is, HTML and CSS. To do so, I took "Modern HTML & CSS From The Beginning". From what I remember, this course was OK.

Javascript (2 weeks)

Once I was done with it, I immediately started learning about JavaScript. Once again, I had to choose a resource that would guide me and decided to go with "Modern JavaScript From The Beginning". I don't recommend this one because the instructor makes plenty of mistakes and I remember feeling quite frustrated at some point (which is why I only went through 60% of it). Still, I had gained some insights into JavaScript, especially in a browser context and I felt like it was enough for now.

General knowledge (4 weeks)

Now that I had the basics of web development, I wanted to expand my general knowledge of the web world. It just so happens that "The Complete Web Developer in 2020" is designed to give you an overview of web development. Granted, this course contains some stuff I already knew about, but repetition is good!

After I finished this course, that wasn't enough. I still wanted to have more insights into web development.
I bought yet another course: The Complete Junior to Senior Web Developer Roadmap. It is kind of the follow-up of the previous one.

Overall these two courses were, in my opinion, really great.

Front-end framework (4 weeks)

OK... I had the basics covered and knew a bit more as to what it takes to create a full-fledged web application.
But I didn't feel like I was ready to create my own application just yet. I had dabbled in React and Redux in the previous course, but it was clearly not enough to get started and create a project on my own (besides, I didn't know what to develop).

I decided to take one last course exclusively focusing on the React ecosystem: Complete React Developer in 2020.

Why React?

I had already been in touch with it in one of the previous courses I took and I had enjoyed it.
I did some research and it appeared to be one of the most (if not the most) popular front-end frameworks.
I would be able to use it for a lot of stuff: web applications, static websites, server-side rendered websites, mobile applications, desktop applications, etc.

This last course gave me a good grounding in React. In hindsight, I do not regret having chosen the React framework.

Creating my own project

At that point, I was already 3 months into my learning journey. I remember thinking to myself "now, you ought to create something on your own and get out of this tutorial hell". I'd learned enough that I felt like I could create my own project to consolidate all my newly-acquired knowledge. I wanted to create something from scratch.

And then, I got an idea. In the next blog post, I'll give the big picture of my journey developing DoNotSkip, the project I chose to develop.