DEV Community: Xavier Canchal

Automatically generate Typescript types for your GraphQL schema

Xavier Canchal — Sat, 30 Oct 2021 10:27:50 +0000

Introduction

In this post, I will show you how to automatically generate types for your GraphQL APIs written in Typescript using GraphQL codegen.

Prerequisites

Some GraphQL and Typescript knowledge is expected. NodeJS and Typescript must be installed on your machine.

Context

Typescript

Typescript is a static type checker for JavaScript. It is used as a development tool and helps writing better code and catching potential errors while developing instead that on runtime.

GraphQL

GraphQL is a query language for writing HTTP APIs. It is very flexible and can help optimizing network load as well as the number of endpoints that you would need in a typical REST API.

Apollo GraphQL

Apollo GraphQL is a framework/toolset for building GraphQL APIs. It provides solutions both for server and client.

GraphQL code generator (graphql-codegen)

graphql-codegen is a tool that automatically generates Typescript types from GraphQL types and resolvers definition.

What are we going to build

We will build a simple GraphQL API that will manage painters and it's paintings. We'll use Apollo server and graphql-codegen for generating the Typescript types automatically, which will be available to use across the codebase.

If you feel lost at any point or simply want to speed up things, here you can find the final code: https://github.com/xcanchal/apollo-server-typescript

Hands-on

First of all, create a new folder for the project and initialize the npm project:



$ mkdir {project-name}
$ cd {project-name}
$ npm init --yes

Install the following dependencies and devDependencies:



$ npm install --save apollo-server graphql

$ npm install --save-dev typescript @tsconfig/recommended graphql-codegen @graphql-codegen/cli @graphql-codegen/typescript nodemon ts-node

Create the tsconfig.json, the configuration file for Typescript . We'll use the recommended example but we'll add an extra property outDir, because we want the generated files to be put all inside the 'dist/' folder instead of next to each original .ts file:



{
  "compilerOptions": {
    "outDir": "dist",
    "target": "ES2015",
    "module": "commonjs",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true
  },
  "$schema": "https://json.schemastore.org/tsconfig",
  "display": "Recommended"
}

To finish with the basic initial setup, add the following dev command under the package.json scripts. This command will be used for running the server in development mode (building the JS files and restarting it on every change):



"scripts": {
  "dev": "nodemon --exec ts-node ./server.ts --watch"
}

Now, let's write the code for our GraphQL server. Create a new server.ts file and ignore editor errors, if any, for now:



import { ApolloServer } from 'apollo-server';

import typeDefs from './type-defs';
import resolvers from './resolvers';

(async () => {
  const server = new ApolloServer({ typeDefs, resolvers });
  const { url } = await server.listen();
  console.log(`🚀 Server ready at ${url}`);
})();

We will use a couple of arrays to work as a database. Create a new file named database.ts and paste the following content. Temporarily, we will use any for the entities types (don't judge me, we'll fix that later!)



export const painters: any[] = [];
export const paintings: any[] = [];

Great! so now we can start defining the schema for our API. Create a new file named type-defs.ts and add the types for the Painter and Painting entities:



import { gql } from 'apollo-server';

export default gql`
  type Painter {
    name: String!
    country: String!
    techniques: [String]!
  }

  type Painting {
    author: String!
    title: String!
    technique: String!
    date: String!
  }
`

We need a way to insert new painters and paintings into our database. Let's define our first mutation in the type-defs.ts file, which will allow us to create painters:



# [...]

input PainterInput {
  name: String!
  country: String!
  techniques: [String]!
}

type Mutation {
  createPainter(input: PainterInput!): Painter!
}

After that, let's add a similar mutation for creating paintings:



# [...]

input PaintingInput {
  author: String!
  title: String!
  technique: String!
  date: String!
}

type Mutation {
  # [...]
  createPainting(input: PaintingInput!): Painting!
}

The next step will be creating the resolvers, which will tell GraphQL how to query or mutate the data associated with the previously defined types.

Create a file named resolvers.ts with the following content (again, we'll use any until we generate the typescript types):



import { painters, paintings } from './database';

const resolvers = {
  Mutation: {
    createPainter(_: any, { input: painter }: any): any {
      painters.push(painter);
      return painter;
    },
    createPainting(_: any, { input: painting }: any): any {
      paintings.push(painting);
      return painting;
    }
  }
};

export default resolvers;

Up to this point, we can insert painters and paintings. The next step is to implement a few queries to retrieve the data from the database. Add the following queries to the type-defs.ts file.



# [...]

type Query {
  painters: [Painter]! # get all painters
  paintings: [Painting]! # get all paintings
  painter(name: String): Painter # get a painter by name
  painting(title: String): Painting # get a painting by title
}

And also add the respective resolvers to the resolvers.ts file.



// [...]

const resolvers = {
  // [...]
  Query: {
    painters: (): any => painters,
    paintings: (): any => paintings,
    painter(_: any, { name }: any): any {
      console.log(name);
      return painters.find((painter) => painter.name === name);
    },
    painting(_: any, { title }: any): any {
      return paintings.find((painting) => painting.title === title);
    },
  },
// [...]
};

Your type-defs.ts file should look like this:



import { gql } from 'apollo-server';

export default gql`
  type Painter {
    name: String!
    country: String!
    techniques: [String]!
  }

  type Painting {
    author: String!
    title: String!
    technique: String!
    date: String!
  }

  input PainterInput {
    name: String!
    country: String!
    techniques: [String]!
  }

  input PaintingInput {
    author: String!
    title: String!
    technique: String!
    date: String!
  }

  type Query {
    painters: [Painter]!
    paintings: [Painting]!
    painter(name: String): Painter
    painting(title: String): Painting
  }

  type Mutation {
    createPainter(input: PainterInput!): Painter!
    createPainting(input: PaintingInput!): Painting!
  }
`

And the resolvers.ts file should look like:



import { painters, paintings } from './database';

const resolvers = {
  Query: {
    painters: (): any => painters,
    paintings: (): any => paintings,
    painter(_: any, { name }: any): any {
      console.log(name);
      return painters.find((painter) => painter.name === name);
    },
    painting(_: any, { title }: any): any {
      return paintings.find((painting) => painting.title === title);
    },
    },
  },
  Mutation: {
    createPainter(_: any, { input: painter }: any): any {
      painters.push(painter);
      return painter;
    },
    createPainting(_: any, { input: painting }: any): any {
      paintings.push(painting);
      return painting;
    }
  }
};

export default resolvers;

Now that we have defined defining the types and resolvers for our API, let's run the server in development mode and see how it looks inside Apollo Studio, which is a playground for testing it.

Execute npm run dev, open a new browser navigate to it:



$ npm run dev

// -> 🚀 Server ready at http://localhost:4000/

After clicking on the "Query your server" button you'll land at the Apollo Studio, where you'll be able to explore the schema definition as well as trying to execute the mutations and queries that we have implemented.

The last thing to do, and the cherry on top of this article, is to generate the Types to be used in our typescript files that match our GraphQL schema.

Let's return to the codebase to configure graphql-codegen. Create a new file named codegen.yaml and paste the following basic configuration (see the complete list of available options here):



schema: "./type-defs.ts" # GraphQL types (input file)
generates:
  ./gql-types.d.ts: # Typescript types (output generated file)
    plugins: # List of needed plugins (installed as devDeps)
      - typescript

Finally, add a new script in the package.json for convenience:



"scripts": {
  "generate-gql-types": "graphql-codegen"
}

Execute it (npm run generate-gql-types) and see how a new file with the name we defined in the codegen.yaml(gql-types.d.ts) gets generated. This file contains all the Typescript types:



export type Maybe<T> = T | null;
export type Exact<T extends { [key: string]: unknown }> = { [K in keyof T]: T[K] };
export type MakeOptional<T, K extends keyof T> = Omit<T, K> & { [SubKey in K]?: Maybe<T[SubKey]> };
export type MakeMaybe<T, K extends keyof T> = Omit<T, K> & { [SubKey in K]: Maybe<T[SubKey]> };
/** All built-in and custom scalars, mapped to their actual values */
export type Scalars = {
  ID: string;
  String: string;
  Boolean: boolean;
  Int: number;
  Float: number;
};

export type Mutation = {
  __typename?: 'Mutation';
  createPainter: Painter;
  createPainting: Painting;
};


export type MutationCreatePainterArgs = {
  input: PainterInput;
};


export type MutationCreatePaintingArgs = {
  input: PaintingInput;
};

export type Painter = {
  __typename?: 'Painter';
  country: Scalars['String'];
  name: Scalars['String'];
  techniques: Array<Maybe<Scalars['String']>>;
};

export type PainterInput = {
  country: Scalars['String'];
  name: Scalars['String'];
  techniques: Array<Maybe<Scalars['String']>>;
};

export type Painting = {
  __typename?: 'Painting';
  author: Scalars['String'];
  date: Scalars['String'];
  technique: Scalars['String'];
  title: Scalars['String'];
};

export type PaintingInput = {
  author: Scalars['String'];
  date: Scalars['String'];
  technique: Scalars['String'];
  title: Scalars['String'];
};

export type Query = {
  __typename?: 'Query';
  painter?: Maybe<Painter>;
  painters: Array<Maybe<Painter>>;
  painting?: Maybe<Painting>;
  paintings: Array<Maybe<Painting>>;
};


export type QueryPainterArgs = {
  name?: Maybe<Scalars['String']>;
};


export type QueryPaintingArgs = {
  title?: Maybe<Scalars['String']>;
};

Pretty cool, right? Then you'll love it when you see how they looks when we actually use them in the code and we really benefit from the type checking:

First of all, let's update the database.ts file:



import { Painter, Painting } from './gql-types';

export const painters: Painter[] = [];
export const paintings: Painting[] =[];

After that, do the same in the resolvers.ts file:



import { painters, paintings } from './database';

import {

  Painter,

  Painting,

  MutationCreatePainterArgs,

  MutationCreatePaintingArgs,

  QueryPainterArgs,

  QueryPaintingArgs,

} from './gql-types';

const resolvers = {

  Query: {

    painters: (): Painter[] => painters,

    paintings: (): Painting[] => paintings,

    painter(: any, { name }: QueryPainterArgs): Painter | undefined {

      console.log(name);

      return painters.find((painter) => painter.name === name);

    },

    painting(: any, { title }: QueryPaintingArgs): Painting | undefined {

      return paintings.find((painting) => painting.title === title);

    },

  },

  Mutation: {

    createPainter(: any, { input: painter }: MutationCreatePainterArgs): Painter {

      painters.push(painter);

      return painter;

    },

    createPainting(: any, { input: painting }: MutationCreatePaintingArgs): Painting {

      paintings.push(painting);

      return painting;

    }

  }

};

export default resolvers;

Conclusion

Awesome! you have completed this tutorial. By following this approach, there's no need to define the same entities twice (one for GraphQL and one for Typescript) and we can focus on what really matters when designing, implementing and maintaining a GraphQL API: its schema types, queries, and mutations.

With graphql-codegen, we get the Typescript types automatically generated and our code is consistent with the GraphQL schema without much effort, apart from any configuration tweaks that may be needed as the application evolves.

This is one of many ways to work with Typescript and GraphQL, so if you follow a different approach, don't doubt to share it so we can learn something new!

Follow me on Twitter for more content @xcanchal

Buy me a coffee:

Automatic versioning in a Lerna monorepo using Github actions

Xavier Canchal — Mon, 04 Oct 2021 15:00:41 +0000

Prerequisites

This is a continuation of my previous article: Monorepo using Lerna, Conventional commits, and Github packages. The prerequisites of that previous article are assumed to understand this one, so you may want to read it first for extra context.

If you feel stuck at any point during the article or you simply want to get the code right now, it can be found in the repository.

Context

Continuous Integration (CI)

Continuous Integration is a practice in software development that consists in integrating the code as frequently as possible. Before integrating the code, it's common to execute a series of checks such as running tests or compiling/building the project, aiming for detecting errors the earlier the better.

A common practice is to automatically execute these checks when opening a new Pull Request or even pushing code to the repository to force that all these checks pass before we can safely integrate the changes into the codebase.

Github actions

Github actions are a Github feature that allows developers to execute workflows when certain events happen in our repositories, such as pushing code or closing a Pull Request(often used in Continuous Integration scenarios). Github actions are free of charge for public repositories.

These workflows are organized in jobs, steps and actions in a nested fashion, and are triggered by one or more events. Each workflow is a single file written in the YAML language.

What are we going to build?

We are going to automate the versioning and publication of the packages in our monorepo using Lerna (with conventional commits) and Github actions.

We are going to implement two different Github workflows:

1 - Checks workflow: When we open a new Pull Request or push changes to a pull request that is open, it will run a set of checks that we consider essential for integrating the changes into our codebase.

2 - Publish workflow: Whenever a Pull Request is merged, we'll execute a workflow that will version and publish our packages. It will behave slightly different depending on the destination branch:

When merged against the development branch, it will publish beta versions of the changed packages (suitable for QA or testing).
When merged against the main branch, it will publish final versions (ready for production).

We will start from an existing monorepo that already contains two javascript packages that I created for this previous article.

The following picture illustrates the workflows that we will implement in Github actions terminology:

Hands-on

Part 1 - Checks workflow on PR open/modified

Github expects workflows to be located under the ${projectFolder}/.github/workflows, so let's create a new Github branch and add our first workflow checks.yaml inside that directory (you can create workflows from the Github UI too):

The project structure looks like this:



/
  .github/
    workflows/
      checks.yaml
  [...]

Now, let's start working on the workflow. Open the checks.yaml file in an editor and add the following attributes:



name: Checks # Workflow name

on:
  pull_request:
    types: [opened, synchronize] # Workflow triggering events

name: The name of the workflow.
on: The listener of the event(s) that will trigger this workflow. In our case, it will be triggered every time that a Pull Request gets opened or modified.

Next, we will add a job to the workflow and configure the type of instance that Github will spin up for running it with the runs-on attribute:



name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs: # A workflow can have multiple jobs
  checks: # Name of the job
    runs-on: ubuntu-latest # Instance type where job will run

This job will contain several steps:

Checkout: Get the code from the repository where the workflow is defined.
Setup NodeJS: Setup NodeJS with a specific version.
Setup npm: Since we will install dependencies from our private registry (in Github packages), we have to add it to the npm config.
Install dependencies: Install the needed npm packages.
Run tests: Execute tests, if any.

In a real-world project it's likely that we run other steps such as checking syntax using a linter, building the project or running any other check/process that we consider essential to mark the changes as valid before integrating them into the codebase.

Custom vs public actions

For some of the mentioned steps we will write the commands from scratch but for others, we will take advantage of existing public actions that have been created by the community and are available in the Github marketplace.

The public actions use the uses keyword and the custom commands (single or multiple lines) use the run one.

Let's implement the first two steps of the build job:



name: Checks
on:
  pull_request:
    types: [opened, synchronize]
jobs:
  check:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout" # Download code from the repository
      uses: actions/checkout@v2 # Public action
      with:
        fetch-depth: 0 # Checkout all branches and tags

    - name: "Use NodeJS 14" # Setup node using version 14
      uses: actions/setup-node@v2 # Public action
      with: 
        node-version: '14'

The Checkout step will download the code from the repository. We have to add the depth: 0 option so Lerna can properly track the tags of the published packages versions and propose new versions when it detects changes.
In the Use NodeJS 14 step we are configuring NodeJS to use the version 14 but we could even execute it for multiple versions at once using a matrix.

Let's commit and push this version of the workflow to Github and open a Pull Request afterward (if you don't have a development branch already created, create one from main because we'll open the pull request against it).

Once the Pull Request has been opened our workflow will be executed. Open a browser and navigate to the "Actions" section of the repository to see the execution result:

If we click on it, we can see the execution details, and by clicking on any of the jobs (in our case, the checks job) we will be able to see the status and outputs of each of its steps:

Let's add the next step: Setup npm. In this step, we'll add our Github packages registry to the .npmrc file so npm can find the packages published in our Github packages registry.

One or multiple commands can be executed in every step action. In this case, we'll run a couple of npm set commands in the same action:



name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with: 
        node-version: '14'

    - name: "Setup npm" # Add our registry to npm config
      run: | # Custom action
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

Workflow environment variables

In the previous snippet, you'll have noticed the secrets.GITHUB_TOKEN. This environment variable is added by Github and can be used to authenticate in our workflow when installing or publishing packages (know more).

A part from that one, Github adds other variables such as the branch name or the commit hash, which can be used for different purposes. The complete list is available here.

Next, we'll add another step: Install dependencies. In this step action we'll install the root dependencies in production mode (see npm ci command) as well as running lerna bootstrap for installing the dependencies for each of our packages and create links between them.



name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Setup npm"
      run: |
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

    - name: Install dependencies
      run: | # Install and link dependencies
        npm ci
        npx lerna bootstrap

Commit and push the changes and see how the "Pull Request synchronized" event triggers our workflow, which now contains the last steps that we added:

Before adding our last step Running tests we need to make a change in our date-logic and date-renderer packages, modifying the npm test script. Since we haven't implemented any actual test yet, we'll simple echo "TESTS PASSED" when that command is executed.

Modify the test script in the package.json of the date-logic package and push the changes to the repo. Then, repeat the same process for the date-renderer.



# package.json
"scripts": {
  "test": "echo TESTS PASSED"
}



# commit and push
$ git add .
$ git commit -m "feat(date-logic): echo tests"
$ git push

After pushing the new test command to our packages we can add the Running tests step to our workflow.



name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Setup npm"
      run: |
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

    - name: Install dependencies
      run: |
        npm ci
        npx lerna bootstrap

    - name: Run tests # Run tests of all packages
      run: npx lerna exec npm run test

Push the changes to the repository and see the execution results in the Github actions section:

Congrats! we completed our first job and half of this tutorial.

Part 2 - Publish workflow on PR merged

Create a publish.yaml file under the workflows repository with the following content. You'll notice that we added a new branches attribute to the event listeners. With this configuration, we're telling Github that only executes this workflow when a Pull Request is merged either against development or main branch.



name: Publish

on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

Now, we'll add a job named publish to this workflow, the runs-on attribute and a new one that we haven't used yet: if. This attribute is used to evaluate an expression to conditionally trigger the job if it evaluates to true or false (it can be used in steps too).

According to the on attribute that we configured, this workflow will trigger on every "Pull Request closed" event against development or main, but what we actually want is to execute it ONLY when the Pull Request has been merged (not discarded). Therefore, we have to add the github.event.pull_request.merged == true condition to the job:



name: Publish
on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

jobs:
  publish:
    if: github.event.pull_request.merged == true # Condition
    runs-on: ubuntu-latest

Now, let's replicate the same first three steps that we added in the checks workflow (Checkout, Use NodeJS 14 and Setup npm)



name: Publish

on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

jobs:
  publish:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Setup npm"
      run: |
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

Finally, we will add the final (and interesting) step: Publish and version. Let's analyze in detail the step attributes and the commands inside the action:

Since Lerna will be in charge of publishing new versions of the packages, we have to set the GH_TOKEN environment variable with our Personal Access Token as the value, so Lerna has the required permissions.
We have to add a couple of Github configuration lines to specify the username and email credentials, so Lerna can make commits and create tags for the new versions in the repository. For that, we'll take advantage of the github.actor variable available in the environment.
In the if/else statement we're checking the ${{ github.base_ref }} variable to see if the destination branch of the PR is development. In that case, we will send the --conventional-prerelease and the --preid flags to the Lerna version command to generate beta versions. Otherwise (it only can be main because we restricted at the workflow level that it must be one of these two branches), we will use the --conventional-graduate argument to generate final versions. Last but not least, the --yes flag autoconfirms the version and publish operations (otherwise Lerna would prompt for manual confirmation and the CI would fail).



name: Publish
on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

jobs:
  publish:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Version and publish" # Interesting step
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: |
        git config user.name "${{ github.actor }}"
        git config user.email "${{ github.actor}}@users.noreply.github.com"

        if [ ${{ github.base_ref }} = development ]; then
          npx lerna version --conventional-commits --conventional-prerelease --preid beta --yes
        else
          npx lerna version --conventional-commits --conventional-graduate --yes
        fi

        npx lerna publish from-git --yes

Let's commit the new workflow to the repository and merge the Pull Request afterward, so it gets triggered. If we inspect the output of the Version and publish step we can see a lot of information about the two steps that Lerna executed:

1) When running the lerna version command, it detected the changes in the packages and proposed new beta versions (notice the -beta.1 prefix) that were auto-accepted. After that, it pushed the version tags to the Github repo:

2) When running the lerna publish from-git command, it analyzed the latest Github tags to determine the versions that had to be published and published the packages to the Github package registry.

So now we have some testing versions in our Github packages registry:

We'll assume that they have been through testing and that are marked as ready for production. Let's create a new Pull Request from development against master, merge it and see how the same Publish job is executed, but this time Lerna will publish the final versions:

Conclusion

We have seen how powerful a couple of Lerna commands can be (in conjunction with a proper conventional commits history) for the Continuous Integration workflows of our monorepos.

By automating these steps we can forget about having to manually decide the versions for all of our packages and thus, avoiding human errors. In this case, we used Github actions for doing it but any other tool such as Jenkins or CircleCI would work too.

Next steps

Configure Github branch protection rules to block the Pull Request merge button if the checks workflow failed.
Set up a commit syntax checker (e.g. commitlint) to avoid human mistakes that could impact the versioning due to an inconsistent commit history.

Follow me on Twitter for more content @xcanchal

Buy me a coffee:

Monorepo using Lerna, Conventional commits, and Github packages

Xavier Canchal — Thu, 23 Sep 2021 20:22:35 +0000

Prerequisites

Some Javascript and Git knowledge and a Github account. Also, NodeJS has to be installed on your computer. If you don’t have it installed already I recommend doing it using a version manager such as nvm.

Context

Monorepo

A monorepo (mono = single, repo = repository) is an approach for managing multiple software projects inside the same repository, often called packages.

Lerna

Lerna is a tool for managing JavaScript projects with multiple packages.

Conventional commits

Conventional commits are a convention built on top of commits that consist of a set of rules to follow when writing commit messages. To specify the nature of the changed code, a set of instructions that conform to the SemVer (Semantic Versioning) specification must be followed.

Github packages

Github packages is the package registry of Github. It allows developers to store software packages for some of the most used package registries (Npm, Docker, Maven…). In our case, we'll use the npm one.

What are we going to build?

We will create a monorepo that will contain two projects (packages). After making changes to any of the projects we will commit them following the conventional commits specification.

After finishing making changes, we'll use Lerna in conjunction with conventional commits for analyzing the commit history and detecting which packages have changed, the level of affectation of these changes, and determining the versions that have to be bumped and published to the registry.

Hands-on

Setting up the monorepo

The very first thing to do is to create a new Github repository. I will call it monorepo.

Clone the repository, navigate to the root folder, and execute the following command to initialize the npm project.



$ npm init

After that, install Lerna as a dependency and execute the command to initialize the Lerna project:



$ npm install --save lerna

$ lerna init --independent

The following lerna.json file will be generated. This file is used to configure the different options supported by Lerna. The --independent flag is important because we want that each package in the repo is versioned independently instead of having a single version for all the packages.



{
  "packages": [
    "packages/*" <-- folder where the packages will be located
  ],
  "version": "independent" <-- versioning strategy
}

In order to avoid publishing the node_modules folder to the repository, create a .gitignore file with the following content:



node_modules

Our project structure should look like this:



/
  .gitignore <-- avoid publish certain files to the repository
  package.json <-- Lerna installed in the root dependencies
  lerna.json <-- Lerna configuration file
  packages/ <-- folder where the packages will be located

Now, let's publish these initial changes to the repository following the conventional commits specification (notice that we're using feat as the commit type and root as the scope). Later, in the scope section of the commit, we'll set the name of the affected package but since the current changes are global we'll just pick a name like root or any other one that you prefer:



$ git add .
$ git commit -m "feat(root): adds npm, lerna and packages"
$ git push

Creating the packages

We will create the following two packages:

date-logic: It will export a function that returns the current date.
date-renderer: It will use the date-logic to print the current date to the console.

Package 1 (date-logic)

Create a new folder named date-logic inside the packages folder, navigate to it, and execute npm i to generate its own package.json file. After that, apply the following changes:

Add an npm scope to the name attribute to indicate who's the owner of the package. In my case, @xcanchal.
Add the repository attribute, with the URL to the Github repository.
Add the publishConfig.registry attribute pointing to the Github Packages registry. This specifies the npm registry where the packages will be published.

The package.json should look like the following:



{
  "name": "@xcanchal/date-logic", <-- @{scope}/{package-name}
  "version": "1.0.0",
  "description": "A package that returns the current date",
  "main": "index.js",
  "repository": "https://github.com/xcanchal/monorepo", <-- repo
  "publishConfig": { <-- publish config
     "@xcanchal:registry": "https://npm.pkg.github.com/xcanchal"
  }
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Xavier Canchal",
  "license": "ISC"
}

Now, we'll implement a very simple script for the date-logic package. Create a new index.js file with the following content:



module.exports = function getDate() {
  return new Date();
};

Let's push the changes to the repo (remember that we have to follow the conventional commits specification). Because the changes are about adding a new feature to the date-logicpackage, we will use the feat type of commit and the date-logic scope:



$ git add .
$ git commit -m "feat(date-logic): creates package"
$ git push

We will now publish the very first version of the package to the Github Packages npm registry, so we can install it from the second package that we'll implement later (the date-renderer).

Authentication in Github Packages and npm

Before being able to publish packages we have to set up a Github Personal Access Token and modify the .npmrc config file to be able to authenticate when executing publish or install commands.

Go to your "Github > Settings > Developer settings > Personal access tokens" and click "Generate new token". Once in the form, set a descriptive name and check the write:packages,(read:packages implicit) and delete:packages permissions:

You can learn more about Github packages authentication in the docs.

Add the following lines to the .npmrc file, which is an configuration file for npm:



@xcanchal:registry=https://npm.pkg.github.com/xcanchal
always-auth=true
//npm.pkg.github.com/:_authToken={YOUR_GITHUB_TOKEN}

Finally, we can publish our date-logic package. To do so, execute the following command from the package folder:



$ npm publish

We’ll see the following output (notice that the version 1.0.0 has been published):



npm notice 
npm notice 📦  @xcanchal/date-logic@1.0.0
npm notice === Tarball Contents === 
npm notice 61B  index.js    
npm notice 400B package.json
npm notice === Tarball Details === 
npm notice name:          @xcanchal/date-logic                    
npm notice version:       1.0.0                                   
npm notice filename:      @xcanchal/date-logic-1.0.0.tgz          
npm notice package size:  397 B                                   
npm notice unpacked size: 461 B                                   
npm notice shasum:        4e48d9d684539e0125bf41a44ae90d6c6fc4b7df
npm notice integrity:     sha512-DowuECiLPHd55[...]/LV5T/2pFqucQ==
npm notice total files:   2                                       
npm notice 
+ @xcanchal/date-logic@1.0.0

Let’s check how this looks in Github. Open a browser and navigate to your Github repository. There, you can see published packages on the bottom-right of the page:

By clicking the package name you will be redirected to the details page. There, some information such as the installation instructions, the versions published, or the download activity is available.

Package 2 (date-renderer)

Now, let’s implement our second package: the date-renderer. Create a new date-renderer folder under packages and repeat the same steps that we did for the date-logic package.

Then, install the date-logic package as a dependency (remember, the date-renderer will use the date-logic to print the value to the console).



$ npm install --save @xcanchal/date-logic

Great, we have installed a package of our Github packages registry! After that, we will create a new index.js file and add the following code, which is a simple script that imports the date-logic package and executes the function exported there to print the date to the console.



const getDate = require('@xcanchal/date-logic');

(() => {
  console.log(`Date: ${getDate()}`);
})();

We can test it to check that it works correctly:



$ node index.js

// -> Date: Wed Sep 22 2021 22:50:51 GMT+0200 (Central European Summer Time)

Our project structure now should look like this (this is how a typical Lerna project looks like):



/
  package.json
  lerna.json
  packages/
    date-logic/
      index.js
      package.json
    date-renderer/
      index.js
      package.json <-- date-logic installed as a dependency

Let’s publish the date-renderer package to the Github Packages registry too by running npm publish from the package folder.

Modifying packages

Let’s make some changes to our packages. Modify the code in the index.js file of the date-logic package to render the date formatted according to a given a locale and some options:



module.exports = function getDate(
  locale = 'en-US',
  options = { weekday: 'long', year: 'numeric', month: 'long', day: 'numeric' }
) {
  return new Date().toLocaleDateString(locale, options);
};

Before pushing these changes we have to determine the commit type since it will impact the consumers that use our package. Since we have changed the return type of the function from a Date object to a String we may consider this as a breaking change. In order to specify it using conventional commits, the body of the footer has to be multi-line and the footer line must start with “BREAKING CHANGE:”



$ git add .

$ git commit -m "feat(date-logic): returns localized date string
BREAKING CHANGE: changes the return type of the getDate function"

$ git push

Leveraging the power of Lerna

Execute git log to see the three different commits that we have made up until now (from newest to oldest):



commit 7decbab3aab121c2235e3fa8fd79fe30ad4350c4 (HEAD -> main, origin/main, origin/HEAD)
Author: Xavier Canchal <xaviercanchal@userzoom.com>
Date:   Thu Sep 23 13:45:02 2021 +0200

  feat(date-logic): returns localized date string

  BREAKING CHANGE: changes the return type of the getDate function

commit d2497bbb357d41b0f4ed81e9a5f1af45b38e5fce
Author: Xavier Canchal <xaviercanchal@userzoom.com>
Date:   Thu Sep 23 12:48:59 2021 +0200

  feat(date-renderer): creates package

commit 857efc7057941c254f97d7cf2d49b4f8eae3b196
Author: Xavier Canchal <xaviercanchal@userzoom.com>
Date:   Thu Sep 23 09:48:02 2021 +0200

  feat(date-logic): creates package

Now, we will use Lerna to analyze the conventional commits history to detect which packages have changed and the level of affectation of those changes to determine the appropriate version to be bumped.

Execute the following command from the root folder of the monorepo (notice the --conventional-commits flag).



$ lerna version --conventional-commits

Some logs will appear and Lerna will list the packages that will be versioned and will ask for confirmation:



[...]

Changes:
 - @xcanchal/date-logic: 1.0.0 => 2.0.0

? Are you sure you want to create these versions? (ynH)

If we confirm by pressing the y key, Lerna will update the version attribute in the date-logic‘s package.json and will push a tag to Github. See the output:



lerna info execute Skipping releases
lerna info git Pushing tags...
lerna success version finished

If we visit the tags page of our Github repo, we can see the created tag:

But there's more! Lerna also generated a particular CHANGELOG.md for the date-logic package with all the changes history. Pretty neat, right?

We still haven’t published this new version 2.0.0. To do it we’ll use another Lerna command: lerna publish with the from-git argument. This argument tells Lerna to decide which versions have to be published by looking at the Git tags, which are used as the source of truth.

But first, we have to extend the Lerna configuration by adding the registry URL under the commands.publish.registry attribute in our lerna.json file, which now looks like this:



{
  "packages": [
    "packages/*"
  ],
  "version": "independent",
  "command": {
    "publish": {
      "registry": "https://npm.pkg.github.com/xcanchal"
    }
  }
}

Commit and publish the Lerna configuration change:



$ git add .
$ git commit -m "feat(root): adds publish registry to lerna config"
$ git push

And execute the Lerna publish command:



$ lerna publish from-git

Which will ask for confirmation too, like in the version stage (add a --yes flag if you want to autoconfirm):



[...]

Found 1 package to publish:
 - @xcanchal/date-logic => 2.0.0

? Are you sure you want to publish these packages? (ynH)

We confirm and we get the following output:



[...]

Successfully published:
 - @xcanchal/date-logic@2.0.0
lerna success published 1 package

Let’s visit our repository packages page and see how our package now has two different versions published:

Now we can use the new version of the date-logic package in the date-renderer. Update the date-renderer's package.json to target from the version 2.0.0 and up and execute npm install.



{
...
  "dependencies": {
    "@xcanchal/date-logic": "^2.0.0"
  }
...
}

Navigate to the date-renderer package folder and execute node index.js to see the updated result:



$ node index.js
// -> Date: Thursday, September 23, 2021

And that’s it!

Conclusion

What have we covered in this article?

Conventional commits specification.
Using Github packages as an npm registry.
Configuring authentication in Github packages and npm.
Using Lerna in conjunction with conventional commits to version and publish packages, and get a nice CHANGELOG.md file as a bonus.

Next steps

Setting up a commit syntax checker (e.g. commitlint) to avoid human mistakes that could impact the versioning due to wrong commit history.
Automate the package versioning and publication workflow when pushing new code to the repository using Github actions.
Publish different types of versions: beta versions when pushing to development and final versions when pushing to master as part of the previous Github action. See Lerna’s --conventional-prerelease and --conventional-graduate flags.

The last two steps are covered in this following article.

Have you ever used a monorepo for managing packages? Did you use Lerna or any other tool? Don’t hesitate to leave some feedback!

Follow me on Twitter for more content @xcanchal

Buy me a coffee: