DEV Community: Javier Brea

Testing localStorage exceptions with Cypress

Javier Brea — Wed, 31 Aug 2022 10:20:58 +0000

How to test localStorage exceptions handling using Cypress.

Web Storage error handling

When developing a web that implies using Web Storage, possible exceptions should be handled. From MDN docs:

"Developers should make sure to always catch possible exceptions from [localStorage.]setItem()"

Some of the reasons that may produce localStorage exceptions are:

The user may have their browser configured to deny permission to persist data for the specified origin.
The storage is full.

So, the application should take this into account and it should handle these possible errors and act in consequence.

Error handling should be tested

In this article, we are going to suppose that our web application handles the Web Storage exceptions properly, and it displays a notification when the localStorage is disabled or full. That's great, but, as this could be considered as another feature of our application, we should also test it, right?

No problem, in this post we are going to figure out how to use the cypress-localstorage-commands plugin to simulate that the localStorage is disabled, and then write a test checking that our notification is displayed properly.

The cypress-localstorage-commands plugin

The cypress-localstorage-commands plugin extends Cypress' cy commands with localStorage methods. It allows to:

Preserve localStorage between tests and spec files
Disable localStorage to check error handling 👍
Get, set and remove values from localStorage

Installation

The plugin is distributed via NPM and should be installed as one of your project's devDependencies:

npm i --save-dev cypress-localstorage-commands

Then, import the plugin at the top of your Cypress' support file (usually cypress/support/e2e.js for e2e testing type):

import "cypress-localstorage-commands";

This will extend Cypress' cy commands, adding the plugin's ones.

Note: For some features, the plugin also requires to install its Node.js events, but that is not needed for disabling localStorage, so, we are going to skip that step in this tutorial.

Simulating disabled localStorage

Now that the plugin is installed, we have available all its commands in the Cypress cy object. Let's use the disableLocalStorage one for simulating that localStorage is disabled:

describe("when localStorage is disabled", () => {
  beforeEach(() => {
    cy.disableLocalStorage();
    cy.visit("/");
  });
});

This would deactivate localStorage and visit the page.

Testing the error handling

Now we only need to write some tests checking the expected behavior of the web when localStorage is disabled. For example:

describe("when localStorage is disabled", () => {
  beforeEach(() => {
    cy.disableLocalStorage();
    cy.visit("/");
  });

  it("should display localStorage warning", () => {
    cy.get("#localstorage-warning").should("be.visible");
  });

  it("should display accept-cookies button disabled", () => {
    cy.get("#accept-cookies").should("be.disabled");
  });
});

Testing different errors

If the application handles different localStorage exceptions, we can also use the cy.disableLocalStorage command to simulate all of them. It accepts passing a custom error as a parameter. So, we could simulate different errors and test the expected behavior for each case. For example:

describe("when localStorage throws X error", () => {
  beforeEach(() => {
    cy.disableLocalStorage({
      withError: new Error("X"),
    });
    cy.visit("/");
  });

  it("should display X error message", () => {
    cy.get("#localstorage-error").should("have.text", "X");
  });
});

Conclusion

Web Storage possible exceptions should be caught.
If our application handles localStorage exceptions, we should test it, as any other feature
We can use the cypress-localstorage-commands Cypress plugin to simulate localStorage exceptions, and write corresponding tests.

Using a mock server with Cypress

Javier Brea — Tue, 16 Aug 2022 05:26:35 +0000

Why to use a mock server with Cypress?

When testing front-end applications using Cypress, we usually need to mock API responses in certain testing phases. Talking about testing phases deserves another post, but in this one we are going to suppose that we are in a testing phase in which we are testing the front-end application in the browser, but without the dependency of the real API services. So, it could be like testing unitary the whole application.

Cypress provides itself some mechanisms allowing to mock API responses in the website under its control, as the cy.intercept command. It is a great Cypress feature that, apart from stubbing the API responses, it also allows to spy the requests and write assertions related to them.

So, if Cypress already provides a mechanism to simulate API responses, why should we need a mock server? Well, the Cypress features are limited to its usage on Cypress, and, using a separated mock server allows us to reuse the same API mock for other purposes, such as simulating the API while we are developing the application, execute contract tests between the API mock and the OpenApi definition to ensure that we are accomplishing the API contract, etc.

We could even use a combination of both techniques, and write assertions about the API requests that our app is performing using the cy.intercept command while the responses are sent by our mock server.

Workflow

So, we can start the development by making an agreement about the API contract before it is developed. Then, using a mock server will allow us to create an API mock based on the API contract, and control the responses of the API during the development and the "unitary" testing with Cypress.

Controlling the responses of the API will improve the development workflow, avoiding early dependencies with the team developing the API. It also improves the testing of error cases or another cases that are commonly hard to reproduce with a real API.

Defining the API responses during the earliest phases of development will improve the communication among team members and align their expectations.

Mock server

In this tutorial we are going to use Mocks Server. It is a Node.js mock server running live, interactive mocks in place of real APIs. It makes able to define many different responses for a same route, so, we can change the whole mocked API behavior by simply changing the response of one or many routes while the server is running.

It also allows to create collections defining the specific responses to be sent by each different route, and the user can choose which collection has to be used on each particular moment. This makes able to store multiple collections and change the whole API behavior by simply changing the current one. So, suppose that we are testing a books store application, then we could store collections as "books-with-long-name", "get-book-error", "empty-category", etc. Each collection can change at a time many API responses, so it can simulate a specific API state useful for executing some specific tests.

And, more interesting even (given that we are talking about Cypress), it provides Cypress commands to change the current routes collection or the responses of specific routes while the server is running.

Installation

Mocks Server is essentially a set of NPM packages. For using it with Cypress, we should install @mocks-server/main and @mocks-server/cypress-commands:

npm i -D @mocks-server/main @mocks-server/cypress-commands

Then, you can add a NPM script that will allows to start the mock server using the command line interface:

{
  "scripts": {
    "mocks" : "mocks-server"
  }
}

This enables to start the mock server by simply running a command in the project folder:

npm run mocks

About the Mocks Server Cypress commands, we need to register them on Cypress. At the top of your Cypress' support file (usually cypress/support/e2e.js for e2e testing type):

import { register } from "@mocks-server/cypress-commands";
register();

Basic usage

When Mocks Server is started, the interactive CLI is displayed by default. It allows you to see some details about the current configuration. Using the arrow keys and the Return key you can choose menu options in order to perform some actions, like changing the current collection, setting a delay time for the server responses, etc.

This interactive CLI is a great tool for controlling the mock server while you are in the development phase, because it allows to change the server responses in real time using it without modifying any code.

When the server is started for the first time, it creates a configuration file and a scaffold folder containing some examples of routes and collections.

project-root/
├── mocks/
│   ├── routes/ <- DEFINE YOUR ROUTES HERE
│   │   ├── common.js
│   │   └── users.js
│   └── collections.json <- DEFINE YOUR COLLECTIONS HERE
└── mocks.config.js <- DEFINE YOUR CONFIGURATION HERE

The server loads all files in the mocks/routes folder, which must contain the route definitions.
The mocks/collections.json file is used to define collections of route variants.
The server watches for changes in all files in the mocks folder, so changing a file will immediately update the responses of the mocked API.

Note: In this section we are not going to learn deeper how to use Mocks Server, because we want to focus on its usage with Cypress, so, you can refer to its own documentation for further info.

Starting the app, the mock server and Cypress

Now we already have our mock server installed and we can start it using a command. This allows to start the mock server manually when starting the application and opening Cypress in headed mode by simply running three separated processes in three terminals.

Supposing that we have next commands in the package.json file:

{
  "scripts": {
    "start:app": "react-app-rewired start",
    "cypress:open": "cypress open",
    "mocks" : "mocks-server"
  }
}

We could start the processes separately and control each one on its own terminal.

Start the web application:

npm run start:app

Start the mock server:

npm run mocks

Open Cypress in headed mode:

npm run cypress:open

For the moment, this is all we need to have a look at the Mocks Server Cypress commands. In another section of this post we will figure out how to start all processes using a single command to be able to run the tests on a CI pipeline.

Changing the responses of the API mock

Now that we have the mock server running, our web application started (and, of course, configured to use the mock server as its API), and Cypress opened, we can start using the @mocks-server/cypress-commands package to change the responses of the API and test different scenarios in the app.

Suppose that we are testing a books store app. In Mocks Server we have one collection of routes simulating that the books catalogue contains two elements, and another one simulating that it is empty. We have named those collections "two-books" and "no-books". Then, we could write two different tests, one for each case:

describe("books page", () => {
  describe("when there are two books", () => {
    before(() => {
      cy.mocksSetCollection("two-books"); // Use "two-books" collection
      cy.visit("/");
    });

    it("should display two books", () => {
      cy.get("#books li").should("have.length", 2);
    });
  });

  describe("when there are no books", () => {
    before(() => {
      cy.mocksSetCollection("no-books"); // Use "no-books" collection
      cy.visit("/");
    });

    it("should display no books", () => {
      cy.get("#books li").should("have.length", 0);
    });
  });
});

We could also simulate that the API is slow using the cy.mocksSetDelay command, and test that our application is displaying a loader properly. For example:

describe("when the books API is slow", () => {
  before(() => {
    cy.mocksSetDelay(3000); // Set a delay of 3 seconds in API responses
    cy.visit("/");
  });

  after(() => {
    cy.mocksSetDelay(0); // Restore the delay to 0
  });

  it("should display loading", () => {
    cy.get("#books .loading").should("exist");
  });

  it("should display two books when finish loading", () => {
    cy.get("#books li").should("have.length", 2);
    cy.get("#books .loading").should("not.exist");
  });
});

Other Cypress commands

Mocks Server Cypress commands also enable to change only the response of an specific route using the cy.mocksUseRouteVariant() command, for example:

cy.mocksUseRouteVariant("get-authors:error");

And it also allow us to change any other Mocks Server configuration property:

cy.mocksSetConfig({
  files: {
    watch: false,
  },
  server: {
    port: 3500
  },
});

Note: For further info about all available commands, you can checkout the Mocks Server Cypress integration docs.

Starting all using a single command

The instructions for starting the processes described in a previous section are useful when developing the Cypress tests, because we can check the logs of all processes, and changing the code of any of them would produce a hot reload and we could execute the tests again until they are ready. But, what about starting all processes and executing the tests in a single command, which is what we would like to do on a CI pipeline, for example?

In this case, we can use start-server-and-test to start all processes in a single command.

Supposing that our web application is started in the port 3000, and the mock server is started in the port 3100, we could add the next scripts to the package.json file:

cypress:run: It will run Cypress in headless mode.
mocks:no-cli: It will start the mock server without the interactive CLI. Logs will be printed instead, which is ideal for a CI pipeline.
mocks:no-cli_and_start:app: It will start and wait for the mock server to be ready, and then it will start the web application.
test: It will start and wait for the mock server to be ready, then start and wait for the web application to be ready, and then run Cypress in headed mode.

{
  "scripts": {
    "start:app": "react-app-rewired start",
    "cypress:run": "cypress run",
    "cypress:open": "cypress open",
    "mocks" : "mocks-server",
    "mocks:no-cli" : "mocks-server --no-plugins.inquirerCli.enabled",
    "mocks:no-cli_and_start:app": "start-server-and-test mocks:no-cli tcp:3100 start:app",
    "test": "start-server-and-test mocks:no-cli_and_start:app tcp:3000 cypress:run",
  }
}

Now, when we run the next command, all processes will be started waiting for the others, and then the Cypress tests will be executed in headless mode:

npm run test

Conclusion

While Cypress provides us tools for intercepting the API requests and simulate responses, using a separated mock server enables us to reuse the same mock for different phases of the development and testing workflow.

As we have seen, the Mocks Server project provides a library enabling to use Cypress commands to change the responses of the API mock, which is great in the testing phase. And it also provide other integration tools enabling to use it easily during the development phase, for example.

For further information, you can checkout:

A Node.js configuration provider reading config from files, environment and arguments

Javier Brea — Wed, 25 May 2022 12:33:40 +0000

When creating a Node.js application, one usual task is to read configuration somehow in order to let the user define some settings for it. There are lots of awesome configuration libraries at charge of making easy this task, but each one is specialized in reading config from one single source, such as files, arguments or environment variables. I usually use these libraries to read configuration from arguments or configuration files:

cosmiconfig - Reads configuration from a file. It searches for many file types and file names, and even supports defining config in the package.json file. Very customizable, it's an awesome library very flexible both for the app developer and for the app user.
commander - A great library that allows to read configuration from process arguments, among other things.

But I personally like to be even more flexible with the users, and let them choose the source to define the configuration, because each one may have different requirements that can make easier to define the configuration using one than the others. So, I used to repeat the task of defining, reading and merging configuration of each different source in a lot of my projects. And that's why I have created the configuration library that I'm going to talk about in this post:

@mocks-server/config - It allows to define configuration options, and it reads environment variables and uses cosmiconfig and commander under the hood to provide values to them.

Note: The library was developed as a part of the @mocks-server project ecosystem, but it is not coupled to it and it can be used anywhere else, because it is fully configurable.

As a summary, it reads, merge and validates configuration from:

Default option values
Configuration received programmatically
Configuration files (using cosmiconfig internally)
Environment variables
Command line arguments (using commander internally)

It also provides:

Parsing objects from command line arguments or environment vars
Isolated configuration namespaces
Objects to get/set options values internally at any moment
Events when any option value changes

Note: Nconf is another popular library reading configuration from different sources, you can read about the differences with @mocks-server/config in the alternatives section of this post.

Quick start

In this example we are going to create a simple option of type string, and we are going to see how to read its value:

Add the library @mocks-server/config to your package dependencies:

npm i --save @mocks-server/config

Import the library and create a configuration instance. You must provide a moduleName option. It will determine the name of the configuration files that will be searched for, and the prefix of the environment variables:

import Config from "@mocks-server/config";

const config = new Config({ moduleName: "myApp" });

Now that we have created the config instance, we can start adding options to it. In this case, we are going to create an option named myOption, of type string, with a fooValue default value:

const myOption = config.addOption({
  name: "myOption",
  type: "string",
  default: "fooValue",
});

Now, we only have to load the configuration. Note that it is an async process, so we have to wait for it to finish before reading the options values:

config.load().then(() => {
  console.log(myOption.value);
});

At this point, supposing that our file was named app.js, we can define the value for our option simply defining an environment variable named MY_APP_MY_OPTION (Environment variables must be prefixed with the value of the moduleName option, and they must be defined using "screaming snake case"):

MY_APP_MY_OPTION=anotherValue node app.js

Or we can define it using a command line argument:

node app.js --myOption=anotherValue

We can also create a .myApprc.json file at the same folder, and simply run node app.js:

{
  "myOption": "anotherValue"
}

Or a myApp.config.js file:

module.exports = {
  myOption: "anotherValue"
};

Or even a .myApprc.yml file. You can check the whole list of supported file formats at the @mocks-server/config docs.

myOption: anotherValue

Sources priority

When reading sources, the library will try to search for the value of each option in every source (unless it is explicitly configured for skipping some sources). So, the values for different options, or even for the same option, can be defined in different sources at a time. In that case, it applies a priority to the sources, which is, from lower to higher:

Option default value
Configuration file
Environment variable
Process argument

This is very useful, because you can have a configuration file in your app with some values, but override some of them defining environment variables when you start the application, or even using command line arguments, which will override even the values of environment variables.

Option types

It does not only read values from different sources, but it also parses the values to each correspondent option type.

Options can be of one of next types: boolean, number, string, object or array. The array type also allows to define the type of items contained in it.

For example, if an option is of type boolean and it is defined in a environment variable, its value will be converted from false, true, 1 or 0 strings to a boolean type:

MY_APP_MY_BOOLEAN_OPTION=1 node app.js
# value -> true

If the option is of type number, it will be parsed to a numeric value:

node app.js --myNumberOption=2
# value -> 2 as a number

And it parses even options of type object from command line arguments and environment variables:

MY_APP_MY_OBJECT_OPTION='{"foo":"var"}'
# value -> {foo: "var"}

Changing settings in runtime. Events

Apart from reading the configuration, the library can also be used to modify options in runtime. Suppose that your application provides an API for changing settings while it is running (which is the case of Mocks Server, for example). If that is the case, you can modify the values of the options from the API component, and the other components can listen to changes in the options and act in consequence whenever it is needed, because the library also emits events whenever an option changes its value.

Use the onChange method to add event listeners to value changes:

const myOption = config.addOption({
  name: "myOption",
  type: "string",
});

myOption.onChange((newValue) => {
  console.log(`myOption value has changed to ${newValue}!`);
});

Use the value setter to change the value of an option:

myOption.value = "anotherValue";
// console -> myOption value has changed to anotherValue!

Modularity: namespaces

For sure that you have noticed the word "modular" in the title of this post. But, for the moment, what makes this library to be "modular"? It seems to be a simple configuration provider reading values from some different sources. Well, here is where the "namespaces" concept enter.

But, first of all, why should a configuration provider be modular?

Well, we are going to suppose that we have an app that is very well designed. It has a very clean architecture in which each internal component is responsible to do one single thing (that's what we all want in our applications, don't we?). And some of that components need some user configuration to do their job. Suppose also that our configuration contains some logic about the options, such as validation, parsing, etc. In my experience, the configuration is usually something that is provided by a specific component in the application, and it is usually placed very next to the application higher levels. One of the first things that we usually do is reading the configuration in some place in our app, and then we pass that configuration to the other internal components (of course that this wouldn't be always the case, but I have seen it lots of times, and I usually did it also in the same way).

"Global" configuration provider. It contains the logic for reading config for all components

If that is the case, then it may become a problem, because every time we need to modify or to add an option to any of the internal components, we must modify also our "global" configuration provider. And, in an ideal world, we should modify only the involved component, am I right?

Any change in the configuration of any component impacts in the "global" config component

Using the @mocks-server/config library, you could avoid this problem simply passing the config instance to each component, and let them add their options. It is a good solution, but, depending on the scale of the project and the amount of options, it may result in conflicts between the names of the options from different components.

Namespaces to the rescue

In order to avoid that problem, the @mocks-server/config library provides the "namespaces" concept, so each component can be the owner of its own configuration namespace, and it can modify its options whenever it is needed without the risk of conflicts with the other components.

Following with the previous example, we can use the addNamespace config method for creating a namespace. We must pass the name for the namespace as first argument:

const myNamespace = config.addNamespace("myNamespace");

And now that we have our namespace created, we can add options to it as we did in the config instance in the previous example:

myNamespace.addOption({
  name: "myOption",
  type: "string",
  default: "fooSecondValue",
});

Then, when defining environment variables, we must add the namespace name as a prefix to the option name:

MY_APP_MY_NAMESPACE_MY_OPTION=anotherValue node app.js

When using command line arguments, we must add the prefix separated by a dot:

node app.js --myNamespace.myOption=anotherValue

And when using configuration files, each namespace corresponds to an object key:

{
  "myNamespace": {
    "myOption": "anotherValue"
  }
}

Keeping components isolated

So, you can keep your components configuration isolated creating and passing a different namespace for each one of them. The next example shows a theoretical app creating different namespaces for some components:

const dbConnector = new DBConnector({
  config: config.addNamespace("db"), 
});

const api = new Api({
  config: config.addNamespace("api"), 
});

await config.load();
await dbConnector.start();
await api.start();

Each component contains the logic for defining, parsing and validating its own options

As complex or simple as you may need

Even when namespaces is a great feature, it may not be useful to you if your app only needs few configuration options, or if there is no risk of conflicts between the component options, or even if you want to keep the configuration as much simple as possible for the user. In that case, you could simply pass the config instance to each component and let them to add their own options to the root namespace.

Or maybe you need even a more complex structure for your configuration, because some of your components depends on many other internal components. In that case, nested namespaces are supported also:

const myNestedNamespace = config.addNamespace("first")
  .addNamespace("second")
  .addNamespace("third")
  .addOption({
    name: "foo",
    type: "number",
  });

Which, for example, would result in a yaml configuration file like this:

first:
  second:
    third:
      foo: 3

Or in an argument like this:

node app.js --first.second.third.foo=3

Breaking the modularity rules

Even when the library was designed to provide modularity, it is flexible enough to allow breaking the rules whenever it is needed. For example, in a previous example I talked about an API changing the configuration. Supposing it is able to change the configuration of any component, then that API is breaking the modularity rules.

If this is needed, you can use some library methods to access to any namespace configuration options, or even provide a whole configuration object that will set every namespaces at a time. For example:

config.set({
  first: {
    second: {
      third: {
        foo: "foo"
      }
    }
  }
});

This would set options for all provided namespaces in the object. It is not desirable to do things like this when we are talking about modular solutions, but it can be used if there is no other better alternative. You can read the library API docs to know more about config available methods.

Alternatives

Another library able to read configuration from files, arguments and environment is Nconf. It is a great and very popular library. The main difference with @mocks-server/config is that it is more focused on the options to be loaded and its types in order to execute validations and parse the data, while Nconf leaves the door more opened to get any value unless you explicitly configure restrictions separately for each different source.

More in detail, the main differences between both libraries are:

mocks-server/config uses Cosmiconfig under the hood, so it supports more file formats out of the box, such as yaml, cjs, etc.
Nconf allows to get any key from the sources, unless you use each source options separately to set restrictions. On the contrary, mocks-server/config requires to specifically define the details of each option to be loaded, as its type, etc. Then, it parses the data and executes validations for all sources automatically.
Nconf requires to initiate separately each different source, while mocks-server/config initiates all sources using only the load method, unless the user specifies another behavior using the config options. On the other hand, mocks-server/config uses exactly the same hierarchy described in the Nconf docs as a good practice.
mocks-server/config always executes validations and parses data based on the option types using Ajv under the hood. This is something that seems to be defined separately for each different source in Nconf.
mocks-server/config supports nested namespaces, so keys like foo.var.foo2.var2=x are supported.

Further information

This post tried to be only an introduction to the main features of the library, so there are many other interesting things that were not mentioned here, like:

Configuring the library itself. Deactivating sources, using custom file names, etc.
How to define values for each different option type on each different source
Library lifecycle. Handling complex use cases

For further information, you can read the whole technical docs of the library here.

CI/CD for Pnpm and Nx monorepo using Github actions

Javier Brea — Tue, 26 Apr 2022 11:23:25 +0000

Preface

In the previous chapters of this series of posts, we saw the reasons for using a monorepo to maintain many Node.js dependent packages, and the requirements to have a good development and continuous integration workflow. Then we saw how to build a Pnpm and Nx monorepo.

Now we are going to see how to create a continous integration workflow for the Nx and Pnpm monorepo using Github actions.

We are going to use the repository created in the previous post for the examples of this one. This is a step-by-step guide, but you can directly check the repository code while reading for a faster approach.

Workflow requirements

First of all, let's see what we want to build.

As a quick recap, in the previous post we finished with a monorepo containing two Node.js packages and one E2E testing project. This is what we are going to use for checking the pipelines that we are going to build, but our continuous integration workflow should be valid for any quantity of projects built with Node.js and E2E testing projects.

The workflow will consist of four basic tasks: Build, unit testing, E2E testing, and deployment. Adding more tasks, such as linting the code, running integration test, etc., could be also added easily, but we will create only these ones for keeping the examples as simple as possible.

Let's make a list with all of the requirements:

Scalable - It must be possible to add more projects without changing the workflow. Adding new tasks should be done easily too.
Incremental - It must build, test and deploy only the affected packages.
Fast - It must run tasks in parallel.
Multi version - It must run tests using different Node.js versions in order to check backward compatibility.
Multi platform - It must run tests on Windows and Linux operating systems.
Automatic - Obviously, the pipelines must be triggered automatically when some actions are performed in the code repository, such as opening a PR or tagging a specific branch.
Blocker - It must run every needed verification before allowing to promote code. It must block code promotion when any check fails.

Note that some of these are specific requirements for projects containing Node.js packages, such as the multi version testing. This is something that we wouldn't need when talking about front-end projects, for example. This article is focused on Node.js packages, but most of the contents could be valid also for other type of projects with minimal changes.

Branching model

Our branching model will consist only on two long-living branches: main and release. In this model we would create a feature branch which is short lived and merged often to release. The code would remain in the release branch until the main project maintainer considers that it is ready for declaring a formal release (ideally often too). At that moment, a PR would be opened to main, and once it passed the checks and it is merged, a new release would be created by tagging the main branch.

The workflow

Now that we have defined our branching model and we know how the code is going to be promoted, let's see which tasks should the workflow run on each "event" of that model.

As we can see in the schema, there are two different pipelines that we should create for our workflow. One for testing and another one for the deployment. The testing pipeline will be able to run incremental testing or to test all depending on the event that triggered it:

Incremental testing - Only testing of affected projects is executed. It is triggered whenever a PR is opened to the main or release branches, or when code is pushed to the release branch. We could also run it on every push to any branch, or only a part of it, but for simplicity of the examples we are going to run it only in these cases.
Test all - When code is pushed to the main branch, then we execute tests on all projects. This is made in order to add extra verifications, and to be able to know the build status of the branch (in our branching model we don’t have a branch to compare with the main branch. We could compare to the latest tag, but this is something we are not going to see in this post)
Deploy - Triggered every time a new release is created in Github.

Creating the first pipeline

So, now that we have the requirements and the design of the workflow, we can create the first pipeline. It will build and test only the affected projects whenever a PR is opened or the code is pushed to release or main branches.

To create the first Github action, add a file named .github/workflows/build.yml:

├── .github/
│   └── workflows/
│       └── build.yml
├── ...

For the moment we will add to it the configuration defining the events that will trigger it:

name: build
# Events configuration
on:
  # Execute it on pushing to next branches
  push:
    branches: 
      - main
      - release
  # Execute it on opening any pull request
  pull_request:

Information about the branches

We will need to know which is the base branch in many steps of our workflow in order to configure Nx properly and to be able to calculate the affected projects, for example. So, we will add a first job, getting the information and exporting it as a Github Action output. We will use it afterwards in other jobs. Note that, depending whether the workflow is triggered by a pull request or not, we will get the information using different Github actions variables.

# ...events configuration
jobs:
  # Get branch info
  branch-info:
    runs-on: ubuntu-latest
    steps:
      # Get current branch name
      - name: Get branch name
        id: branch-name
        uses: tj-actions/branch-names@v5.2
      # Get base branch name to compare with. Base branch on a PR, "main" branch on pushing.
      - name: Get base branch name
        id: get-base-branch-name
        run: |
            if [[ "${{github.event.pull_request.base.ref}}" != "" ]]; then
              echo "::set-output name=branch::${{github.event.pull_request.base.ref}}"
            else
              echo "::set-output name=branch::main"
            fi
    outputs:
      # Export the branch names as output to be able to use it in other jobs
      base-branch-name: ${{ steps.get-base-branch-name.outputs.branch }}
      branch-name: ${{ steps.branch-name.outputs.current_branch }}

Now every time we open a PR or push to main or release branches the workflow will be executed. As a tip, you can add your own branch name to the list of branches in the workflow for testing it without opening a PR.

Affected projects

The first thing that we are going to do in the pipeline is to get the affected projects. But, if Nx is able by itself to calculate the affected projects and run the desired tasks only on them, why do we want to know which are? Well, we need to know the affected projects because we are going to create one different job in the workflow for each different task to be executed in an affected project. Even when Nx is able to run tasks in parallel, all of them would be executed in the same step, so it could end in performance issues depending on the heavy that our tasks are. That's why we are going to use Github matrixes and jobs to distribute the tasks. This way we can configure the number of max parallel tasks to be executed also in the workflow configuration, and, depending on the number of projects, and the heavy the tasks are, we can adjust the parameters to get the workflow finished as fast as possible.

Let's add the first job steps. Note that here we are configuring Nx using the output from the previous job, so we have to declare the dependency using the needs property:

# ...events configuration
jobs:
  # ...previous jobs
  get-affected:
    needs: [branch-info]
    runs-on: ubuntu-latest
    steps:
      # Check out the repository
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      # Configure Nx to be able to detect changes between branches when we are in a PR
      - name: Derive appropriate SHAs for base and head for `nx affected` commands
        uses: nrwl/nx-set-shas@v2
        with:
          main-branch-name: ${{needs.branch-info.outputs.base-branch-name}}
      # Install Pnpm
      - name: Install pnpm
        uses: pnpm/action-setup@v2.2.1
        with:
          version: "7.x"
      # Install Node.js
      - name: Use Node.js
        uses: actions/setup-node@v3
        with:
          node-version: "16.x"
          cache: 'pnpm'
      # Install workspace dependencies
      - name: Install dependencies
        run: pnpm install

Nx provides a command for listing the affected projects. We will use it to get the list of projects in which each specific task should be executed. But in order to create dynamic steps in the Github workflow, it would be better if we get the list as a stringified array, because this way we can use the Github action's fromJson expression.

So, the things that we have to do to get the list of tasks to be executed for each affected project in an optimal format is:

Call Nx command to get the list of affected projects for each specific task. For example: pnpm nx print-affected --target=[task] --base [base branch] --select=tasks.target.project.
When we want to test all projects, passing as modified the package.json file will do the trick. Modifying that file affects all projects in the workspace. For example: pnpm nx print-affected --target=[task] --files package.json --select=tasks.target.project.
Get the output, clean the traces from it, and convert it from something like sum-one, sum-two into something like ["sum-one", "sum-two"].

NOTE: If you are using a Pnpm version lower than 7.0, you should add an extra -- to the Pnpm commands before the Nx arguments: pnpm nx print-affected -- --target=[task] --files package.json --select=tasks.target.project

This can be achieved using a bash script in the workflow itself, or using any other type of script at your convenience, so for brevity I will omit the code in this post. I have created a scripts/print-affected-array.js script in the sample repository that gets the needed output, and that's what we are going to call from the pipeline, passing two arguments to it: the task to be executed and the base branch. If you want to use my script to get the affected projects, follow the next steps:

Install the cross-spawn development dependency:

pnpm add -wD cross-spawn

Copy this file into scripts/print-affected-array.js.
Add the next step to the get-affected job:

# ...events configuration
jobs:
  # ...previous jobs
  get-affected:
    steps:
      # ...previous get-affected steps
      - name: Get affected
        id: get-projects-arrays
        # When not in a PR and the current branch is main, pass --all flag. Otherwise pass the base branch
        run: |
            if [[ "${{github.event.pull_request.base.ref}}" == "" && "${{needs.branch-info.outputs.branch-name}}" == "main" ]]; then
              echo "::set-output name=test-unit::$(node scripts/print-affected-array.js test:unit --all)"
              echo "::set-output name=test-e2e::$(node scripts/print-affected-array.js test:e2e --all)"
            else
              echo "::set-output name=test-unit::$(node scripts/print-affected-array.js test:unit origin/${{needs.branch-info.outputs.base-branch-name}})"
              echo "::set-output name=test-e2e::$(node scripts/print-affected-array.js test:e2e origin/${{needs.branch-info.outputs.base-branch-name}})"
            fi
    outputs:
      test-unit: ${{ steps.get-projects-arrays.outputs.test-unit }}
      test-e2e: ${{ steps.get-projects-arrays.outputs.test-e2e }}

Note that we call to the script twice, one time for the test:unit task and another one for the test:e2e task. In the next steps we will use the two different outputs for creating different matrixes.

Unit testing

At this point, we have a workflow that is able to detect which tasks should we execute depending on the affected projects. We have that information available in different Github Actions outputs, so we can create jobs dynamically in order to run each different task in parallel.

So, now we are going to use that information to run the unit tests of the affected projects in different jobs in parallel. Let's add the test-unit job:

# ...events configuration
jobs:
  # ...previous jobs
  test-unit:
    runs-on: ubuntu-latest
    needs: [get-affected]
    # Skip the job if there are not affected projects containing unit tests
    if:  ${{ fromJson(needs.get-affected.outputs.test-unit)[0] }}
    strategy:
      # Run in parallel
      max-parallel: 4
      # One job for each different project and node version
      matrix:
        node: ["16.14.2"]
        projectName: ${{fromJson(needs.get-affected.outputs.test-unit)}}
    env:
      NODE: ${{ matrix.node }}
    steps:
      # Checkout and setup environment
      - name: Checkout
        uses: actions/checkout@v3
      - name: Install pnpm
        uses: pnpm/action-setup@v2.2.1
        with:
          version: "7.x"
      - name: Use Node.js
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node }}
          cache: 'pnpm'
      - name: Install dependencies
        run: pnpm install
      # Run test:unit script in the affected project
      - name: Test unit
        run: pnpm nx test:unit ${{ matrix.projectName }}

Here is a screenshot showing how the test-unit matrix contains one different job for each project containing the test:unit script (supposing that both projects are affected):

Coverage

I have not configured in the example any tool to collect the project's unit testing coverage, this is something that I will explain in another post. Currently there are tools that support incremental coverage analysis with easy configuration, such as Codecov.

Multi version

As we are already creating our unit testing jobs from a dynamic matrix, so now it is easy to add more Node.js versions to it. Modify the next line in the workflow, adding all currently active Node.js versions:

      # One job for each different project and node version
      matrix:
        node: ["14.19.0", "16.14.2", "17.2.0", "18.0.0"]

As we can see in the next screenshot, now eight jobs are created, one for each different combination of project and Node.js version:

E2E testing

Now we can add another jobs matrix for running the E2E tests. It is very similar to the one running the unit tests, but now we are going to execute it using only the two latest Node.js versions:

# ...events configuration
jobs:
  # ...previous jobs
  test-e2e:
    runs-on: ubuntu-latest
    needs: [get-affected]
    # Skip the job if there are not affected projects containing unit tests
    if:  ${{ fromJson(needs.get-affected.outputs.test-e2e)[0] }}
    strategy:
      # Run in parallel
      max-parallel: 4
      # One job for each different project and node version
      matrix:
        node: ["17.2.0", "18.0.0"]
        projectName: ${{fromJson(needs.get-affected.outputs.test-e2e)}}
    env:
      NODE: ${{ matrix.node }}
    steps:
      # Checkout and setup environment
      - name: Checkout
        uses: actions/checkout@v3
      - name: Install pnpm
        uses: pnpm/action-setup@v2.2.1
        with:
          version: "7.x"
      - name: Use Node.js
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node }}
          cache: 'pnpm'
      - name: Install dependencies
        run: pnpm install
      # Run test:e2e script in the affected project
      - name: Test E2E
        run: pnpm nx test:e2e ${{ matrix.projectName }}

Multi platform

At this point, the workflow is almost ready. It is calculating the affected projects and it is creating two matrixes of jobs for the affected projects, one for the unit testing and another one for E2E testing. Both of them are executed using multiple Node.js versions.

But remember that we defined as a requirement that the tests should be executed on different platforms also. So, let's add another matrix for running the E2E tests on Windows OS:

# ...events configuration
jobs:
  # ...previous jobs
  test-e2e-windows:
    runs-on: windows-2019
    needs: [get-affected]
    if: ${{ fromJson(needs.get-affected.outputs.test-e2e)[0] }}
    strategy:
      max-parallel: 2
      matrix:
        node: ["17.2.0", "18.0.0"]
        projectName: ${{fromJson(needs.get-affected.outputs.test-e2e)}}
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Install pnpm
        uses: pnpm/action-setup@v2.2.1
        with:
          version: "7.x"
      - name: Use Node.js
        uses: actions/setup-node@v2
        with:
          node-version: ${{ matrix.node }}
          cache: 'pnpm'
      - name: Install dependencies
        run: pnpm install
        env:
          HUSKY_SKIP_INSTALL: 1
      - name: Test E2E
        run: pnpm nx test:e2e ${{ matrix.projectName }}

Configuring Github checks

Now we should check that every job is finished properly before allowing to merge any pull request. For doing that, we are going to use the Github repository settings. We can add branch protection rules in order to require status checks to pass before merging. But the problem is that we don't know which status checks will be executed on each different pull request, because not every job will be executed always. Remember that we are creating jobs only for the affected projects.

The solution is to create a last job that will be executed whenever all of the others have finished. It will ignore the skipped matrixes, because sometimes no unit-test matrix will be created, or no test-e2e will be executed, etc. It will depend on the affected projects.

# ...events configuration
jobs:
  # ...previous jobs
  build-finished:
    runs-on: ubuntu-latest
    needs: [test-unit, test-e2e, test-e2e-windows]
    if: |
      always() &&
      (needs.test-unit.result == 'success' || needs.test-unit.result == 'skipped') &&
      (needs.test-e2e.result == 'success' || needs.test-e2e.result == 'skipped') &&
      (needs.test-e2e-windows.result == 'success' || needs.test-e2e-windows.result == 'skipped')
    steps:
      - name: Trace
        run: echo "All jobs finished"

This is the job that we are going to use to require the status check in order to let all of the other jobs finishing before allowing to merge the pull request.

Go to the Settings tab of the repository. Then select Branches -> Add rule. Check the Require status checks to pass before merging option and add a new status check for the build-finished job:

Now, every time a pull request is created it will require the build-finished job to be finished before allowing to merge it. As this job is waiting for all other dynamic jobs to have finished, if any of them fails, then the pull request will be blocked.

Creating the second pipeline. Deploy

At this point, we are checking that every test of all affected projects is passing before allowing to merge a pull request, and we are running all tests in the main branch as an extra check. But there is still something important to do: We should automatically publish the modified packages whenever a release is created.

For this, we are going to create another workflow file: .github/workflows/deploy.yml. Let's start configuring the environment and installing the dependencies:

name: deploy
on:
  release:
    types: [created]
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Install pnpm
      uses: pnpm/action-setup@v2.2.1
      with:
        version: "7.x"
    - uses: actions/setup-node@v3
      with:
        node-version: '16.x'
        registry-url: 'https://registry.npmjs.org/'
        cache: 'pnpm'
    - run: pnpm install

Then, as some of our projects may require to run a build script before publishing them, we'll add a build:all script to the workspace's package.json file. In this case, we can't make it incrementally because we have no branch to compare with, we are already in the main branch, so we will call to the Pnpm publish command and it will determine which packages are already published and will skip them. So, as we don't know which packages are going to be published, we will build all of them:

{
  "scripts": {
    "build:all": "nx run-many --target=build --all"
  }
}

Then, we can add the build and publish steps to the publish job:

# ...events configuration
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
    #... previous steps
    # Run `build` script in every projects
    - run: pnpm run build:all
    # Publish all packages in the workspace which version still does not exist
    - run: pnpm -r publish --no-git-checks
      env:
        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

The --no-git-checks option is used because otherwise the publish command may produce errors when executed on Github actions due to the checkout method.

Note also that you must configure your own NPM_TOKEN Github action secret in the repository settings. The example is valid if you are going to publish to the NPM registry, otherwise, you should also set up your own registry url in the Pnpm configuration before publishing.

From now, every time a formal release is declared in Github, the workflow will be triggered:

And the packages will be built and published automatically:

Conclusion

In this post we have seen how to build a continuous integration workflow for a Pnpm and Nx monorepo using Github actions. Its branching model and pipeline stages are oriented to a repository containing NPM packages, but it can be easily adapted to any other type of projects.

We defined a set of requirements for the workflow. Let's make a short recap in order to check if all of them have been met:

Scalable - ✅ Any number of packages can be added without modifying the workflow.
Incremental - ✅ Only the affected packages are tested when a pull request is opened. Only the packages that are upgraded are published on the deploy step.
Fast - ✅ Parallel jobs are created dynamically for each task to be executed. The parallelization parameters can be easily modified to adjust performance.
Multi version - ✅ Tests are executed using multiple versions of Node.js
Multi platform - ✅ E2E tests are executed also on Windows OS
Automatic - ✅ Tests are executed whenever a pull request is opened. The deployment is executed when creating a new release on Github.
Blocker - ✅ Github actions status checks are required before allowing to merge a pull request.

There are still some points that could be improved. I have omitted them in order to keep the examples as simple as possible, but let's see some of them. I hope to explain them in coming posts:

Cache of Pnpm dependencies between jobs.
Check if the versions of publish affected packages have been upgraded properly when opening a pull request.
Report coverage of unit testing.
Do not test all of the projects in the main branch. Instead of that, the branch could be compared to the latest tag.

Note: The examples of this post are available at this Github repository.

Building a Node.js monorepo with Pnpm and Nx

Javier Brea — Tue, 26 Apr 2022 11:11:51 +0000

Preface

In the previous chapter of this series of posts, we analyzed the reasons for using a monorepo to maintain many Node.js dependent packages, and the requirements to have a good development and continuous integration workflow. In this post we are going to see how to build a monorepo using Pnpm and Nx, that, used together, will cover all the described requirements.

We are going to create a repository containing two simple projects that theoretically would be published to NPM, and a project containing E2E tests of both packages integrated. This is a step-by-step tutorial, but if you want to have all the code available, all of the examples are in this Github repository.

Repository structure

In this tutorial, we will store the packages being published to NPM into a packages folder, and E2E tests into a test folder. The repository will contain also a package.json file in the root folder.

├── packages/
├── test/
├── .gitignore
└── package.json

The name of the folders is something that can be changed at your convenience. You could store your packages depending of their type in different folders, for example. Nx recommends as best practices to have an app folder and a libs folder for reusable libraries, but in my opinion that structure is more recommendable if you are going to include also front-end projects or other type of not versioned and distributable applications. In this case, all packages could be distributable and dependencies of other ones, that's why the folder is named packages. I also like to keep integration or E2E tests in a different folder.

Installing Pnpm

As seen in the previous post, we are going to use Pnpm mainly to be able to link packages locally using its workspace feature.

First of all you have to install Pnpm globally in order to use it as Npm client and to run the package.json scripts that we will create in next steps. There are many methods for installing it, you can check them all in the Pnpm installation docs. Here we are going to use npm to install it:

npm install -g pnpm@next-7

Now we can fill the package.json file with some basic info, and we can use Pnpm to install the dependencies.

{
  "private": true,
  "description": "Pnpm and Nx monorepo example",
  "scripts": {
    "prepare": "echo \"installed!\""
  }
}

pnpm i

Configuring Pnpm

Now that we have installed Pnpm, we have to configure it so it can know which is the root of the workspace, and where are the packages. Create a pnpm-workspace.yaml file in the root folder containing:

packages:
  # all packages in subdirs of packages/ and test/
  - 'packages/**'
  - 'test/**'

You can read Pnpm workspace docs for further info about how to include/exclude subfolders.

Creating a project

Let's create our first project in the packages folder. We will name it sum-one. It will contain its own package.json file, and an index.js file exporting a method which sums one to a given number:

├── packages/
│   └── sum-one/
│       ├── index.js
│       └── package.json
├── test/
├── .gitignore
├── package.json
└── pnpm-workspace.yaml

For the moment, the packages/sum-one/package.json file only has to contain the basic package information:

{
  "name": "sum-one",
  "version": "1.0.0-alpha.1",
  "description": "Sums one to a given number"
}

And the packages/sum-one/index.js file exports a very simple method. We will import this package and use it from the other one afterwards in order to see how we can define dependencies between projects in the monorepo workspace.

function sumOne(number) {
  return number + 1;
}

module.exports = sumOne;

Project and workspace dependencies

Now we have two package.json files in the repository. The one in the root folder should contain shared devDependencies across many projects, such as test, build or lint libraries, etc. but note that the dependencies of each package (those required to work once they are published) must remain on their own package.json files. Otherwise, they will be missing when the packages are published.

Note that the dependencies of all packages in the repository are installed when pnpm i is executed in the root folder. And we can also use executables of the dependencies installed in the workspace from the npm scripts of the packages! 🥳

Project and workspace scripts

The scripts of each project can be executed independently from each package folder, and we can also define scripts for the workspace. So, let's add one unit test to our recently created package in order to see an example.

We will install Jest in the workspace (as mentioned above, it is a devDependency, so we don't have to install it on every single package):

pnpm add -wD jest

The -wD in the example means "Install it in the workspace, as a devDependency"

Now we can use the jest executable in the packages/sum-one/package.json file, so let's create one basic test in that package. Create a packages/sum-one/index.spec.js containing:

const sumOne = require("./index");

describe("sumOne", () => {
  it("should sum one to the given number", () => {
    expect(sumOne(2)).toEqual(3);
  });
});

And then add the script for running the test:

{
  "name": "sum-one",
  "version": "1.0.0-alpha.1",
  "description": "Sums one to a given number",
  "scripts": {
    "test:unit": "jest"
  }
}

Now, you can run the package unit tests script from the package folder:

cd packages/sum-one && pnpm test:unit

But we don't want to be moving from one folder to another to run our development scripts, right? Be patient, in the next steps we will add workspace scripts to run any script from any package in the workspace from the root folder. 😉

Creating a dependent project

Now we are going to add another project in order to see how we can define dependencies between them, so we don't have to publish one package to be able to use it from another one. For that, we will use the Pnpm Workspace feature, that allows to reference workspace packages through aliases. When packages are published, it automatically changes the references to the real versions or semver ranges, so we don't have to care about that.

We will name sum-two to our new package, and it will depend on sum-one:

├── packages/
│   ├── sum-one/
│   │   ├── index.js
│   │   ├── index.spec.js
│   │   └── package.json
│   └── sum-two/
│       ├── index.js
│       ├── index.spec.js
│       └── package.json
├── test/
├── .gitignore
├── package.json
└── pnpm-workspace.yaml

Here is the content of the packages/sum-two/package.json file. Note the usage of workspace:* for creating a local reference from one package to another, that will be replaced by the real version of the other package (1.0.0-alpha.1) when this one is published.

{
  "name": "sum-two",
  "version": "1.0.0-beta.1",
  "description": "Sums two to a given number",
  "scripts": {
    "test:unit": "jest"
  },
  "dependencies": {
    "sum-one": "workspace:*"
  }
}

Now we can add the content of the packages/sum-two/index.js file, which will require the sum-one package:

const sumOne = require("sum-one");

function sumTwo(number) {
  return sumOne(sumOne(number));
}

module.exports = sumTwo;

And we are ready also to add the unit test of this package to the packages/sum-two/index.spec.js file:

const sumTwo = require("./index");

describe("sumTwo", () => {
  it("should sum two to the given number", () => {
    expect(sumTwo(3)).toEqual(5);
  });
});

Now we have to reinstall the workspace dependencies, and then we can run the unit tests of the sum-two package, which proves that our local reference works properly:

pnpm i
cd packages/sum-two && pnpm test:unit

Installing Nx

As a fast recap, at this point we have two dependent packages in the same repository, linked locally using the Pnpm Workspace feature. Both packages have unit tests and they share the jest dependency, but we can run them only from each different package folder.

Now we are going to install Nx because it provides the other requirements that we saw in the previous post:

Dependencies analysis
Detection of affected projects
Tasks orchestration

Nx can also be installed globally, but I prefer to install it as a project dependency in order to define a specific version, so the same version is always used in all different local and remote environments. Afterwards we will expose some of the Nx commands that we need for our development workflow using our own package.json scripts.

Install as devDependencies the required nx packages and typescript, which is a peer dependency:

pnpm add -wD nx @nrwl/workspace typescript

Configuring Nx

Nx has a lot of configuration options, but for the moment we are going to see only the basic configuration that allows to execute any package script from the root workspace folder, see the dependencies graph, etc. In next posts of this series we'll see more advanced options.

Create a nx.json file in the root folder. It will contain some Nx basic configuration:

{
  "extends": "@nrwl/workspace/presets/npm.json",
  "affected": {
    "defaultBase": "main"
  },
  "npmScope": "@my-scope",
  "tasksRunnerOptions": {
    "default": {
      "runner": "@nrwl/workspace/tasks-runners/default"
    }
  },
  "implicitDependencies": {
    "pnpm-workspace.yaml": "*",
    "pnpm-lock.yaml": "*",
    "package.json": {
      "dependencies": "*",
      "devDependencies": "*"
    },
    "nx.json": "*"
  }
}

Using the Nx CLI

As we have not installed Nx globally, we need to define a script in our workspace package.json file to allow running its commands. For the moment we will only add a "shortcut" or "alias" to the nx command, but later we will add specific scripts for running usual development tasks. For simplicity of the examples, here we will show only the scripts property of the package.json file:

{
  "scripts": {
    "nx": "nx"
  }
}

Now we can use nx to run any script of any project in the workspace from the root folder. For example, we can run the unit tests of sum-one:

pnpm nx test:unit sum-one

Note that we use pnpm nx to run any nx command through our workspace nx installed dependency instead of having it installed globally. nx supports passing a npm script name as first argument and the package name as second argument: pnpm nx [script] [project]

So, for running the unit:test script of the sum-two package we can do:

pnpm nx test:unit sum-two

Adding workspace scripts

Now we are able to run any package script from the root folder of our workspace using Nx. But we can add more scripts for running usual tasks instead of using directly the nx interface. For example, let's add one script for running all unit tests of all packages in the workspace. (As we will see later, in reality, this script shouldn't be used very usual, because we should run only the unit tests of the affected packages, but it is added only to illustrate how to add custom scripts for making our workflow easier).

{
  "scripts": {
    "nx": "nx",
    "test:unit:all": "nx run-many --target=test:unit --all"
  }
}

Now, we can run the unit tests of every package in our workspace with a single command, and it will be done in parallel! 🥳

pnpm test:unit:all

You can learn more about the nx interface in its API reference page. It provides lots of options for filtering projects, enabling/disabling parallel executions, etc.

Dependencies graph

An awesome feature of Nx is the ability of starting a web app drawing a graph with the dependencies between the projects in our workspace. Simply run:

pnpm nx graph

This will open automatically a browser tab with the Nx dependencies graph:

Affected projects

In a previous step, we added a script to run all unit tests of all packages in the workspace just to show how to add useful alias scripts to our workspace. But this is something that we normally wouldn't want to do. We usually want to run just the unit tests of the packages that have been modified, and also the unit tests of the packages that are dependent on them, and the affected integration or E2E tests.

Here is where the Nx affected command enters. Using it, we can compare our repository branch with the desired base branch in order to know which projects have been modified, and then calculate the affected ones, and run any script from all of them.

Let's add one script to the root package.json file in order to provide an easy alias for doing it:

{
  "scripts": {
    "nx": "nx",
    "test:unit:all": "nx run-many --target=test:unit --all",
    "test:unit:affected": "nx affected --target=test:unit"
  }
}

Note that this script has to be executed in a Git repository to be able to compare changes. The default base branch to compare with is defined in the nx.json file that we created previously, which in this case is main. So, supposing that you have a repository and you have all previous changes promoted to the main branch, if you modify the packages/sum-two/index.js file, running the next command would execute only the unit tests of the sum-two package, because no other package depends on it.

pnpm test:unit:affected

But, if you create another branch and modify the packages/sum-one/index.js file, running pnpm test:unit:affected would execute the unit tests of both packages, because modifying sum-one also affects to sum-two.

Dependent tasks

Now we can run the unit tests of the affected packages with a single command, but, what about if the unit:test script in any package needs any other script to be executed before? For example, suppose that the sum-two package has a build script that must be executed before running the unit tests. Should we execute it manually before running the affected command? If we have twenty packages in the workspace, some of them containing a build command, and some others not, then should we create a build:affected script and execute it always before running the unit tests? Well, we could, but Nx provides a mechanism to define dependent tasks, so it detects whether any package have dependent tasks that should be executed before others, and it executes them in that case.

Nx also provides a cache mechanism, so it won't execute a dependent task of a package that has not been modified since it was executed the last time, so you don't have to care about the performance of running every dependent tasks always. You can read about how to configure it in the Nx docs.

So, let's add a fake build script to the packages/sum-two/package.json file:

{ 
  "scripts": {
    "build": "echo \"Built!\"",
    "test:unit": "jest"
  }
}

You can check that the new script works properly by running:

pnpm nx build sum-two

If the command above does not work, try cleaning the nx cache running pnpm nx reset. Sometimes Nx needs a "little push" when adding scripts or modifying configurations.

Now that we have a build command that theoretically should be executed before running the unit tests, let's let Nx know about that dependency. We must edit the nx.json file, adding the targetDependencies property as in the next example:

{
  "targetDependencies": {
    "build": [
      {
        "target": "build",
        "projects": "dependencies"
      }
    ],
    "test:unit": [
      {
        "target": "build",
        "projects": "self"
      }
    ]
  }
}

With this configuration, we are saying to Nx: "Before running the test:unit script of any package, run the build script of the package itself, and before running the build script of any package, run the build script of its dependencies.". So, in our case, when we run:

pnpm nx test:unit sum-two

It will run first the build script of the sum-two package, and then the test:unit one. If there was a build script in the sum-one package, it would execute it as well, because it is a dependency of the sum-two one.

Adding integration or E2E tests

In order to have a full example before going to the next chapter, in which we will create a continuous integration pipeline using a Github action, we are going to add also a project containing E2E tests. In our case, the unit tests of the sum-two package should be enough, because unit tests are not mocking the dependency with sum-one. But note that the code of the projects here is not really relevant.

├── packages/
│   ├── sum-one/
│   │   ├── index.js
│   │   ├── index.spec.js
│   │   └── package.json
│   └── sum-two/
│       ├── index.js
│       ├── index.spec.js
│       └── package.json
├── test/
│   └── sum-e2e/
│       ├── index.spec.js
│       └── package.json
├── nx.json
├── package.json
└── pnpm-workspace.yaml

Here is the code of the test/workspace-e2e/package.json file. We will use the sum-two and sum-one packages to execute the test, so we define the dependencies, and note that we have also added a script named test:e2e. This is useful to differentiate the type of test that we want to execute, so we can run affected E2E tests and unit tests in a different stage of the pipeline, for example.

{
  "name": "sum-e2e",
  "private": "true",
  "description": "E2E tests of all sum packages",
  "scripts": {
    "test:e2e": "jest"
  },
  "dependencies": {
    "sum-one": "workspace:*",
    "sum-two": "workspace:*"
  }
}

Let's write the code of the test. We are going to use also the sum-one package just to add another dependency. It may have no sense in a real project, but it is useful for this example:

const sumTwo = require("sum-two");
const sumOne = require("sum-one");

describe("When using sumOne and sumTwo", () => {
  it("should sum three to the given number", () => {
    expect(sumOne(sumTwo(4))).toEqual(7);
  });
});

Let's add also a new task dependency, so before running E2E tests it will also run the build script of all dependent projects. Modify the targetDependencies property in the nx.json file as in:

{
  "targetDependencies": {
    "build": [
      {
        "target": "build",
        "projects": "dependencies"
      }
    ],
    "test:unit": [
      {
        "target": "build",
        "projects": "self"
      }
    ],
    "test:e2e": [
      {
        "target": "build",
        "projects": "dependencies"
      }
    ]
  }
}

Now, we are ready to install the dependencies of the new project, reset the nx cache (this is recommended when adding a new project) and run the E2E tests:

pnpm i
pnpm nx reset
pnpm nx test:e2e sum-e2e

We can also run the Nx graph command to see how our dependencies graph has changed after adding this new project:

pnpm nx graph

You may have noticed that Nx does not recognize the sum-e2e project as of e2e type. For allowing Nx to recognize it, we should add a project.json file to the project defining its type as "application", but this produces Nx not detecting the dependencies automatically in our simple example if we don't add more configuration such as TypeScript aliases. So, I have omitted that extra configuration in order to keep the examples of this post as simple as possible.

Incremental testing

Based on the previous dependencies graph, and in the mentioned "affected" Nx feature, note this: If we execute pnpm nx affected --target=test:e2e, our E2E tests project would be always executed because it is always affected by any modification, but the unit tests of each project (pnpm nx affected --target=test:unit) would be executed only when it has sense depending on the modification made.

NOTE: If you are using a Pnpm version lower than 7.0, you should add an extra -- to the Pnpm commands before the Nx arguments: pnpm nx affected -- --target=test:unit

Conclusion

In this post we have seen how to create a monorepo using Pnpm and Nx. Let's see the list of requirements that we set in in the previous chapter of this series of posts in order to check whether it meets all of them or not:

Configuration for linking dependencies locally - ✅ It host many packages linked locally. The local references between packages are changed automatically to pinned versions when they are published.
Dependencies analysis - ✅ It is able to detect dependencies between projects, and even provides a graph.
Detecting affected packages/projects - ✅ It determines which projects might be affected by a change.
Task orchestration - ✅ It is able to run dependent tasks in the correct order with some minimal configuration.

Effectively, our monorepo meets all of the requirements that we defined in the previous chapter, with minimal configuration. The configuration could be improved in order to let Nx know the type of each project for adding more features like architecture boundaries using eslint, etc., but this, among other lot of things that obviously can be improved, is out of the scope of this post.

In the next chapter, we will use this example to create a continuous integration workflow for the monorepo using a Github action.

Note: The examples of this post are available at this Github repository.

Why a Pnpm and Nx monorepo? Requirements for a good workflow

Javier Brea — Tue, 26 Apr 2022 11:02:05 +0000

Why a Node.js monorepo?

Those who have worked maintaining many dependent packages in different repositories would answer to this question with lots of reasons very quickly. I've been dealing with this scenario for many years, and I can tell you that I have suffered it in my own skin. In fact, in the past I have suffered it even when trying to maintain many packages in the same repository.

I don't want to have a monorepo just because it is a trend, I want to make a list of the problems that I suffered in the past and that I want to solve, and really check if they would be solved using a monorepo. Let's the some problems:

Dealing with independent repositories

In theory, it is perfectly possible to work in one package independently, then publish it, and then we can go to the other package, upgrade the dependency and continue working on the integration until it is ready, then publish it, and so on... They are different packages because they are independently testable and deployable/distributable, and that's how they must be. But, in the real world, unfortunately, this workflow is not so easy. In the real world we make mistakes, we don't define the details of the implementation needed in one package at the sufficient low level, and when we are integrating the next package we realize that we have to make a step back and make a change in the package that we have just published, and then return to the next package again. Suppose that you are maintaining a project consisting on about ten packages, you are running the E2E tests of the last package, and then you realize that you have to modify again the first one... well, I can say that it's very frustrating, to say the least. Even if everything goes well, the process of release becomes slow and tedious.

And this is only one of the main problems that I have found working with different repositories. Let's see some more:

Continuous integration workflow - The workflow for releasing a new version of the package who depends on the others becomes slow and tedious. The mere fact of upgrading a dependency in a "core" package can result in opening lots of pull request in different repositories until we have released new versions of all of the dependencies in series. As mentioned, it may result in a never ending history if we find problems that make us to give steps back again and again.
Duplicity of dependencies, configuration and tooling - The configuration for the continuous integration workflow is duplicated. The same happens with the development dependencies and some configuration, such as linter configuration, NPM commands, and even documentation. In the practice, we see ourselves using copy-paste more times than we would like.
Duplicity of testing - Normally, each package should be testing the integration with its dependencies, and this usually ends in duplicating some integration tests in different repositories. Not to mention end to end tests, which usually are heaviest to run and require more tooling, and it could also become duplicated in some of the repositories, making the continuous integration workflow even harder.
POCs or debugging - Developing fast prototypes for introducing a major change in the system becomes hard when you can only modify one package at a time. This is not necessarily bad, normally we want to introduce changes progressively and in a very controlled way, but the truth is that sometimes we need to probe some concept in a fast way, or we want to debug many packages at a time in an easy way.

Mitigating these problems in the past

Let me make some memory about how we tried to solve some problems in the past, because it may be useful to check if all of these problems are really solved using the tools that we have now. During years we all have found different ways for mitigating one or another problem from the list above using different "hacks", but the truth is that usually, when you mitigate one, then the other gets worse.

Let's see some examples. I have named some of them as "pseudo workspace using x", so, first of all, what I mean by "pseudo workspace"? Well, by "workspace" I refer to the feature of linking dependencies between the packages in a repository, so it does not install the dependency from the remote repository, but from the code of the other package in the repository at that moment. That's what today is named a workspace in NPM and Pnpm, for example. It allows to make changes in one package and see them in the other packages in the repository without the need to release a new version. As years ago I didn't know about a real workspace tool, I have tried by myself all of the next approaches, usually with bad results:

Pseudo workspace using npm link - Npm provides a link command since many years ago for linking packages locally, but it required to execute it locally for each package that you wanted to link or unlink, and there was no way to define those dependencies in a configuration file. The symlinks were created globally, and in my experience, it produced many issues. (I don't know if has been improved by today).
Pseudo workspace using local paths - As of NPM version 2.0.0 you can provide a path to a local directory that contains a package. The approach was very similar to the previous one, but with the improvement of defining the local dependency in the package.json file. It also produced undesired changes in the package-lock.json files and lots of other issues that also made hard to work with it. I even published my own tool providing a command line interface to link packages easier using local paths under the hood (now the package is pending to be deprecated).
Overwriting code in node_modules - Really? Well, not as a part of the usual workflow, but to be honest, sometimes I found myself overwriting a package in the node_modules folder in order to run some integration tests prior to publish it. 😞
Publishing to a private registry - Maybe you know the Verdaccio local NPM registry project (or even the deprecated Sinopia, if you are as old as I am 😉). Well, these projects provide a local NPM registry that allows us to publish one package locally, then we can install it in the next one, and so on, so it solves partially the local development and integration workflow problems mentioned above. That if we set up a local registry, we take care of the order in which packages have to be published, etc. In my experience, the local development becomes a crazy thing when you have to deal with many packages, publish/unpublish or republish different versions, clean caches, etc. Not to mention the complexity of the pipelines if we keep the code in different repositories, then you'll probably have to define which repository pipelines trigger other ones where you'll probably have the integration tests, etc. That, supposing that the NPM private registry is ephemeral, because I worked in a project in which it was not ephemeral, it was allowed to unpublish packages, and continuous integration pipelines were continuously overwriting packages versions. I don't get nostalgic when I remember those days.
Using Babel aliases - Using Babel or TypeScript aliases allows to change the import references, so when you import one package from another, it loads the one from the local workspace instead of the one downloaded by the NPM client. This is the approach that worked better for me, but it still requires some manual configuration for each package, and I don't like to always have to build my code just for these types of requirements. In fact, we'll see that this approach can still be used with some tools as Nx if you don't use Pnpm or other workspace tool.

All of these methods and "hacks" talk about solving the problem of linking dependencies locally to avoid having to publish packages before building the others or running the integration tests, but none of them talk about solving another "problem" created by a monorepo:

If we have everything in a repository, we'll have to test everything every time we try to promote the code, until we have a method for detecting which packages have been modified and which ones depends on them.

Should we give up?

At this point someone could say, "Well, if having different packages produces such problems, why not to have all of the code in one single package in one repository?". The answer is simple: because we don't want to have a monolithic package, we want to solve all of the mentioned problems in the workflow, but keeping isolated each package to be able to distribute it independently, and continue testing it independently, apart of running the appropriate integration tests when correspond.

In the next chapters we will see that most of the problems have been already solved if we choose the appropriate tools. But first, let's recapitulate a list of requirements to avoid the inconvenience mentioned above.

Requirements for a monorepo

Based on my ugly experiences described above, we can see that a monorepo is not just about "code colocation". It should solve the real underlying problems that we find when working with many dependent packages or projects. So, here is my list of "must have" requirements for a good workflow:

Configuration for linking dependencies locally - The monorepo tooling should provide a way to link dependencies. Locally the code of the other packages in the repository should be loaded from the repository itself, but it should still be versioned and downloaded from a binary repository when packages are published.
Dependencies analysis - It should be able to understand the dependencies between the projects in the repository without extra configuration.
Detecting affected packages/projects - It should determine which projects might be affected by a change in the repository, to build or test only those. By "affected" projects I mean modified projects, or projects that depend on those modified at any level.
Task orchestration - Based on the repository dependencies, it should be able to run tasks in the correct order. Integration tests can't be executed before building all needed projects first, for example.

Choosing the appropriate tools

Now that we have the list of minimum requirements, we can look for the appropriate tools to satisfy them all. At this moment there are many tools for working with monorepos, and the list is increasing quickly.

At this point, you can check the monorepo.tools webpage, which provides a good explanation about what a monorepo should be, and a larger list of requirements. Here I have included only those which I consider strictly necessary, and I have omitted those which I consider only "nice to have". The site also includes a list of monorepo tools with a comparison of their features.

I'm not going to make a comparison between tools here, so I will focus in two tools that I am currently using with good results. We will see how these tools meet the mentioned requirements.

Pnpm

Using Pnpm we will meet the first one criteria: "Configuration for linking dependencies locally". It includes a workspace feature which is able to link packages locally, and it does the job pretty well. It also changes automatically the linked dependencies to the desired version when the package is published, with support for pinned dependencies or semver ranges. As a brief summary, it is able to:

Link packages locally
Publish all packages with a single command. It skips those which version already exists. (This is very useful in continuous integration, as we will see in the next chapters)
Save disk space and boost dependencies installation speed. (A great point when talking about NPM dependencies)

Nx

Nx provides to us the other needed features. It is a monorepo tool that is able to make a dependencies analysis, detect affected projects, and orchestrate tasks. As an extra, it is plugabble, and it provides boilerplates to create monorepos for some specific libraries or frameworks, such as React, Angular, etc. But I personally prefer to use only the core features in order to avoid coupling my projects too much to a specific technology or plugin. Among other things, it provides:

Dependencies analysis
Detection of affected projects
Task orchestration
Dependency graph visualization
Parallelization of tasks
Local computation caching
Pluggable

Conclusion

In this post we have seen some problems that were common in the past when working with many dependent packages and some methods or "hacks" that tried to mitigate them. We also have seen how using a monorepo with the appropriate tools might solve those problems. After choosing the tools based on the requirements, in the next chapters I will provide examples about how to configure them to create a complete continuous integration workflow using Github Actions that will make our life as developers easier (at least it did in my case 🙂).

Spell checking Markdown documents using a Github action

Javier Brea — Tue, 26 Apr 2022 10:50:13 +0000

How to spell check your docs in a continuous integration pipeline

As software engineers, we write a lot of documentation (or we should, at least). It is usual to write these docs in Markdown format in the code repository for many reasons, but mainly:

Docs are kept in plain text, isolated from the format.
Docs are under version control.
Docs modifications can be reviewed in the same pull request than the code modifications.

While it should not be necessary to mention the importance of avoiding spelling mistakes in our docs, it is very common to see comments in code reviews about typos in them. If the errors are not detected on time, they would impact on our perceived trustworthiness. Tolerance for typos is low. It makes the writer look careless.

You can avoid typos by checking your spelling locally, using a plugin for the IDE, for example, but it would depend only on you. I personally prefer to add automatic check mechanisms to the continuous integration pipeline that don't depend on recommendations for local environments.

So, in this short post we'll see how to configure a Github action to check the spelling of Markdown documents whenever a pull request is opened on our repository.

Creating the Github workflow

If you already have a Github workflow configured in your repository and you want to add the spell check to it as another job or step, you can skip this chapter and read the next one directly.

We are going to create a new Github workflow to execute the spell check. Create a .github/workflows/spellcheck.yml file in the repository.

├── .github/
│   └── workflows/
│       └── spellcheck.yml

We will name the workflow as spellcheck. It will be executed each time a new pull request is opened, and it will contain only one job named check-spelling. For the moment we will only add a step for checking out the repository.

name: spellcheck
on:
  pull_request:
jobs:
  check-spelling:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

Adding the spell check step

For the spellchecking, we will use the rojopolis/spellcheck-github-actions Github action, which is able to check Python, Markdown, HTML and Text files, and it supports following languages:

English
German
Spanish

Add the step to the workflow where correspond. Based on the above example, we will add it right after checking out the repository code:

name: spellcheck
on:
  pull_request:
jobs:
  check-spelling:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Check Spelling
        uses: rojopolis/spellcheck-github-actions@0.23.0
        with:
          config_path: .spellcheck.yml
          task_name: Markdown

Configuration

Now we have to add a configuration file for the spelling checker. It uses PySpelling under the hood. When checking Markdown files, it first converts a Markdown text file's buffer using Python Markdown and returns a single SourceText object containing the text as HTML. Then it captures the HTML content, comments, and even attributes and performs the check. It has a lot of configuration options, but here we are going to see only an example with some basics. For further info you can read the docs of the rojopolis/spellcheck-github-actions Github action.

Create a file named .spellcheck.yml in the root folder of the repository, and paste the contents of the example below.
Change the sources property depending on your repository structure. Provide patterns for every folder containing the files that you want to automatically check.

matrix:
- name: Markdown
  expect_match: false
  apsell:
    mode: en
  dictionary:
    wordlists:
    - .wordlist.txt
    output: wordlist.dic
    encoding: utf-8
  pipeline:
  - pyspelling.filters.markdown:
      markdown_extensions:
      - markdown.extensions.extra:
  - pyspelling.filters.html:
      comments: false
      attributes:
      - alt
      ignores:
      - ':matches(code, pre)'
      - 'code'
      - 'pre'
      - 'blockquote'
  sources:
  - '*.md'
  - 'content/blog/**/*.md'

The above configuration will check the spelling of your repository's README.md and other Markdown files in the content/blog folder against an English dictionary. This is the configuration used to check the contents of this blog.

Checking for bad spelling

Now, whenever a new pull request is opened, the Github action will be executed and it will help you make sure most spelling errors do not make it into your repository. If the check detects any misspelling, the action will fail and you'll see a trace in the Github action logs like:

Misspelled words:
<htmlcontent> README.md (1)
--------------------------------------------------------------------------------
Github

!!!Spelling check failed!!!

Now you can fix the error and push the docs again, or you can also add the word to your own dictionary in case it is a false positive, as described in the next chapter.

Adding custom words

Sometimes you'll probably have to use some words that are not contained in the default Aspell dictionary used by PySpelling. This is very usual when talking about terms used in technical docs. Look again the configuration example above, and you'll see that we have added a wordlists property to the dictionary one. It makes reference to a .wordlist.txt file, so you can create that file and add your own words to it, and PySpelling will ignore them.

Github
Markdown

Note also that the configuration includes some filters to ignore some patterns. So, if the word is surrounded by a code tag in an HTML file it will be ignored, for example. Then, you could format your text in a more convenient way instead of adding the word to your custom dictionary. It would depend on the case, but sometimes it may be a good practice to give a different format to some kinds of terms.

In a Markdown file, Github will be detected as a misspelling, but `Github` won't.

Finally

Then, after fixing all errors or adding the necessary custom words, at some point we get:

Spelling check passed :)

And this is how our repository will finally like after adding all the needed configuration files:

├── .github/
│   └── workflows/
│       └── spellcheck.yml
├── .spellcheck.yml
└── .wordlist.txt

Conclusion

Adding an automatic spellchecking step to your continuous integration pipeline will prevent your docs containing ugly typos, and it will save you lots of hours of code reviews. It won't prevent other types of orthographic errors, like semantic or style related ones, so I you'll still have to reread carefully your docs before publishing them, but it will be a great help.

This blog is being checked with the examples provided in this post, because it is generated from Markdown files, so you can get the code of the examples directly from https://github.com/javierbrea/personal-website.

Resources and references

How to preserve localStorage between Cypress tests

Javier Brea — Tue, 20 Apr 2021 04:58:00 +0000

Cypress by default clears localStorage between tests, which may be a problem when you are trying to test features related to it. But there is a Cypress plugin that allows preserving localStorage between tests and disabling localStorage.

The problems

You want to preserve localStorage between Cypress tests.
You want to preserve localStorage between Cypress spec files.
You want to disable localStorage to check error handling.

The solution

The cypress-localstorage-commands plugin allows you to use all browser localStorage methods through Cypress commands, and preserve it between tests and spec files. It also allows to simulate that localStorage is disabled in the browser.

Alternatives

As from Cypress 12, you can use cy.session and Cypress Test Isolation in order to persist localStorage between tests. Anyway, the plugin can be still used for an easier manipulation of the localStorage, writing localStorage assertions and even disabling it for checking the error handling.

Installation

The module is distributed via npm which is bundled with node and should be installed as one of your project's devDependencies:

npm i --save-dev cypress-localstorage-commands

Installing commands

cypress-localstorage-commands extends Cypress' cy commands.

At the top of your Cypress' support file (usually cypress/support/e2e.js for e2e testing type):

import "cypress-localstorage-commands";

Read Cypress configuration docs for further info.

Installing Node events

⚠ In order to support preserving localStorage across Cypress spec files, the plugin's Node events must be installed also. Otherwise, localStorage will be preserved only across tests in the same spec file.

In the cypress.config.js file:

module.exports = {
  e2e: {
    setupNodeEvents(on, config) {
      require("cypress-localstorage-commands/plugin")(on, config);
      return config;
    },
  },
};

Usage

Commands

`cy.saveLocalStorage()`

Saves current localStorage values into an internal "snapshot".

`cy.restoreLocalStorage()`

Restores localStorage to previously "snapshot" saved values.

`cy.clearLocalStorageSnapshot()`

Clears localStorage "snapshot" values, so previously saved values are cleaned.

`cy.getLocalStorage(item)`

Gets localStorage item. Equivalent to localStorage.getItem in browser.

item (String): Item to get from localStorage.

`cy.setLocalStorage(item, value)`

Sets localStorage item. Equivalent to localStorage.setItem in browser.

item (String): Item to set value.
value (String): Value to be set.

`cy.removeLocalStorage(item)`

Removes localStorage item. Equivalent to localStorage.removeItem in browser.

item (String): Item to be removed.

`cy.disableLocalStorage(options)`

Disables localStorage. It produces localStorage methods to throw errors.

options (Object): Options to use when disabling localStorage.
- withError (Error): If provided, invocations to localStorage methods will throw this error.

Preserving local storage between tests

Use cy.saveLocalStorage() to save a snapshot of current localStorage at the end of one test, and use the cy.restoreLocalStorage() command to restore it at the beginning of another one. The usage of beforeEach and afterEach is recommended for this purpose.

⚠ When the plugin's Node events are installed, the cy.restoreLocalStorage() command will be able to restore the localStorage snapshots saved in other spec files. Otherwise, snapshots are completely cleared between spec files.

Examples

Cookies button example

Next example shows how the plugin can be used to test a "cookies button" (which theorically sets a flag into localStorage and can be clicked only once)

describe("Accept cookies button", () => {
  const COOKIES_BUTTON = "#accept-cookies";

  before(() => {
    cy.clearLocalStorageSnapshot();
  });

  beforeEach(() => {
    cy.restoreLocalStorage();
    cy.visit("/");
  });

  afterEach(() => {
    cy.saveLocalStorage();
  });

  it("should be visible", () => {
    cy.get(COOKIES_BUTTON).should("be.visible");
  });

  it("should not be visible after clicked", () => {
    cy.get(COOKIES_BUTTON).click();
    cy.get(COOKIES_BUTTON).should("not.be.visible");
  });

  it("should not be visible after reloading", () => {
    cy.get(COOKIES_BUTTON).should("not.be.visible");
  });
});

Note the usage of beforeEach and afterEach for preserving localStorage between all tests. Also cy.clearLocalStorageSnapshot is used in the before statement to avoid possible conflicts with other spec files preserving localStorage.

localStorage assertions

Based on the previous example, assertions could be added to check values of localStorage:

describe("localStorage cookies-accepted item", () => {
  beforeEach(() => {
    cy.restoreLocalStorage();
    cy.visit("/");
  });

  afterEach(() => {
    cy.saveLocalStorage();
  });

  it("should be null first time page is visited", () => {
    cy.getLocalStorage("cookies-accepted").should("equal", null);
  });

  it("should be true after clicking cookies button", () => {
    cy.get("#accept-cookies").click();
    cy.getLocalStorage("cookies-accepted").should("equal", "true");
  });

  it("should be true after reloading", () => {
    cy.getLocalStorage("cookies-accepted").then(cookiesAccepted => {
      expect(cookiesAccepted).to.equal("true");
    });
  });
});

Disabling localStorage

Use cy.disableLocalStorage() to simulate that localStorage is disabled, producing that any invocation to localStorage.setItem, localStorage.getItem, localStorage.removeItem or localStorage.clear will throw an error. As MDN docs recommend, "developers should make sure to always catch possible exceptions from setItem()". This command allows to test that possible exceptions are handled correctly.

Note that:

Only pages loaded after calling this command will have localStorage disabled, so always use cy.reload or cy.visit after executing it.
The localStorage only remains disabled for all pages loaded during the current test. If you want to disable it for multiple tests, execute it in all of them, or in a beforeEach statement.
If any of the other plugin commands (except clearLocalStorageSnapshot) is executed while localStorage is disabled, it will do nothing but producing a Cypress log as: "localStorage.setItem is disabled"

Examples

Disabling localStorage in a single test

Based on previous "Accept cookies button" example, next tests could be added:

//...
const LOCALSTORAGE_DISABLED_WARNING = "#localstorage-disabled-warning";
const LOCALSTORAGE_ERROR = "#localstorage-error";

//... should not be visible after clicked

it("should still be visible when reloading if localStorage is disabled", () => {
  cy.disableLocalStorage();
  cy.reload();
  cy.get(COOKIES_BUTTON).should("be.visible");
});

it("should display warning if localStorage is disabled", () => {
  cy.disableLocalStorage();
  cy.reload();
  cy.get(LOCALSTORAGE_DISABLED_WARNING).should("be.visible");
});

it("should display localStorage error message", () => {
  cy.disableLocalStorage();
  cy.reload();
  cy.get(LOCALSTORAGE_ERROR).should("have.text", "Error");
});

// ...should not be visible after reloading

Disabling localStorage in multiple tests

describe("when localStorage is disabled", () => {
  beforeEach(() => {
    cy.disableLocalStorage({
      withError: new Error("Disabled by cypress-localstorage-commands"),
    });
    cy.visit("/");
  });

  it("should display localStorage warning", () => {
    cy.get("#localstorage-disabled-warning").should("be.visible");
  });

  it("should display localStorage error message", () => {
    cy.get("#localstorage-error").should("have.text", "Disabled by cypress-localstorage-commands");
  });

  it("should display accept-cookies button disabled", () => {
    cy.get("#accept-cookies").should("be.disabled");
  });
});