Building a Monorepo with Yarn 2

In true JavaScript fashion, there was no shortage of releases in the JavaScript ecosystem this year. This includes the Yarn project’s release of Yarn 2 with a compressed cache of JavaScript dependencies, including a Yarn binary to reference, that can be used for a zero-install deployment.

Yarn is a package manager that also provides developers a project management toolset. Now, Yarn 2 is now officially supported by Heroku, and Heroku developers are able to take advantage of leveraging zero-installs during their Node.js builds. We’ll go over a popular use case for Yarn that is enhanced by Yarn 2: using workspaces to manage dependencies for your monorepo.

We will cover taking advantage of Yarn 2’s cache to manage monorepo dependencies. Prerequisites for this include a development environment with Node installed. To follow these guides, set up an existing Node project that makes use of a package.json too. If you don’t have one, use the Heroku Getting Started with Node.js Project.

Workspaces

First off, what are workspaces? Workspaces is Yarn’s solution to a monorepo structure for a JavaScript app or Node.js project. A monorepo refers to a project, in this case, a JavaScript project, that has more than one section of the code base. For example, you may have the following set up:

/app
 - package.json
 - /server
   - package.json
 - /ui
   - package.json

Your JavaScript server has source code, but there’s an additional front end application that will be built and made available to users separately. This is a popular pattern for setting up a separation of concerns with a custom API client, a build or testing tool, or something else that may not have a place in the application logic. Each of the subdirectory’s package.json will have their own dependencies. How can we manage them? How do we optimize caching? This is where Yarn workspaces comes in.

In the root package.json, set up the subdirectories under the workspaces key. You should add this to your package.json:

"workspaces": [
    "server",
    "ui"
]

For more on workspaces, visit here: https://yarnpkg.com/features/workspaces

Additionally, add the workspaces-tools plugin. This will be useful when running workspace scripts that you’ll use later. You can do this by running:

yarn plugin import workspace-tools

Setting up Yarn

If you’re already using Yarn, you have a yarn.lock file already checked into your code base’s git repository. There’s other files and directories that you’ll need up to set up the cache. If you aren’t already using Yarn, install it globally.

npm install -g yarn

Note: If you don’t have Yarn >=1.22.10 installed on your computer, update it with the same install command.

Next, set up your Yarn version for this code base. One of the benefits of using Yarn 2 is that you’ll have a checked in Yarn binary that will be used by anyone that works on this code base and eliminates version conflicts between environments.

yarn set version berry

A .yarn directory and .yarnrc.yml file will both be created that need to be checked into git. These are the files that will set up your project’s local Yarn instance.

Setting Up the Dependency Cache

Once Yarn is set up, you can set up your cache. Run yarn install:

yarn

Before anything else, make sure to add the following to the .gitignore:

# Yarn
.yarn/*
!.yarn/cache
!.yarn/releases
!.yarn/plugins
!.yarn/sdks
!.yarn/versions

The files that are ignored will be machine specific, and the remaining files you’ll want to check in. If you run git status, you’ll see the following:

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    .gitignore
    .pnp.js
    .yarn/cache/
    yarn.lock

You’ve created new files that will speed up your install process:

• .pnp.js - This is the Plug’n’Play (PnP) file. The PnP file tells your Node app or build how to find the dependencies that are stored in .yarn/cache.
• .yarn/cache - This directory will have the dependencies that are needed to run and build your app.
• yarn.lock - The lock file still is used to lock the versions that are resolved from the package.json.

Check all of this in to git, and you’re set. For more information about Yarn 2’s zero-install philosophy, read here: https://yarnpkg.com/features/zero-installs

Adding Dependencies to Subdirectories

Now that Yarn and the cache are set up, we can start adding dependencies. As initially shown, we have a server directory and a ui directory. We can assume that each of these will be built and hosted differently. For example, my server is written in TypeScript, using Express.js for routing, and running on a Heroku web dyno. For the front end app, it is using Next.js. The build will be run during the app’s build process.

Add express to the server dependencies.

yarn workspace server add express

Additionally, add @types/express and typescript to the devDependencies. You can use the -D flag to indicate that you’re adding devDependencies.

yarn workspace server add @types/express typescript -D

We now have our dependencies in our server workspace. We just need to create our ui workspace. Next, build a Next.js app with the yarn create command.

yarn create next-app ui

Finally, run yarn again to update the cache and check these changes into git.

Running Scripts with Workspaces

The last piece is to run scripts within the workspaces. If you look through your source code, you’ll see that there’s one global cache for all dependencies under your app’s root directory. Run the following to see all the compressed dependencies:

ls .yarn/cache

Now, lets run build scripts with workspaces. First, set up the workspace. For server, use tsc to build the TypeScript app. You’ll need to set up a TypeScript config and a .ts file first:

cd server
yarn dlx --package typescript tsc --init
touch index.ts

yarn dlx will run a command from a package so that it doesn’t need to be installed globally. It’s useful for one-off initializing commands, like initializing a TypeScript app.

Next, add the build step to the server/package.json.

"scripts": {
    "build": "tsc",
    "start": "node index.js"
},

Change directories back to the application level, and run the build.

cd ..
yarn workspace server build

You’ll see that a server/index.js file is created. Add server/*.js to the .gitignore.

Since we already have build and start scripts in our Next.js app (created by the yarn create command), add a build script at the root level package.json.

"scripts": {
    "build": "yarn workspaces foreach run build"
},

This is when the workspaces-tool plugin is used. Run yarn build from your app’s root, and both of your workspaces will build. Open a second terminal, and you’ll be able to run yarn workspace server start and yarn workspace ui start in each terminal and run the Express and Next servers in parallel.

Deploy to Heroku

Finally, we can deploy our code to Heroku. Since Heroku will run the script is in the package.json under start, add a script to the package.json.

"scripts": {
    "build": "yarn workspaces foreach run build",
    "start": "yarn workspaces server start"
},

Heroku will use the start script from the package.json to start the web process on your app.

Conclusion

There are plenty more features that Yarn, and specifically Yarn 2, offers that are useful for Heroku developers. Check out the Yarn docs to see if there are additional workspace features that may work nicely with Heroku integration. As always, if you have any feedback or issues, please open an Issue on GitHub.

Comments

[rxliuli]

It is still a toy at present, too many changes to the surrounding ecological chain can not be supported (including IDE)