Lerna What is a mono-repository and why you should try Lerna

Learn how to set up a Gatsby Theme Project with Lerna: this guide will help you understand what is a mono-repo and how to manage it with Lerna using NPM instead of Yarn Workspaces.

Index

• What is Lerna and why it's worth a try?
• Setting Lerna with NPM from the shell
• Scaffolding folders and files with shell
• Using Lerna
  • Choose your approach
• Orchestrate everything with Lerna bootstrap
• Run commands and scripts
• Update Wizard

What is Lerna and why it's worth a try?

I started to follow the Gatsby Theme Authoring course from Jason Lengstorf (@jlengstorf ) and soon realized that I don't know nothing about 'Yarn Workspaces'. It's a pretty way to manage what's called a mono-repo: a repository that has more than one project inside and could share dependencies.

By doppelmutzi from doppelmutzi.github.io:

Such projects are called workspaces or packages. In contrast, using multiple repositories with each repository housing only one project is called a Multi-Repo approach. [...]

One thing that amazed me with the mono-repo approach was that my Packages/Workspaces can locally share the same node_modules dependencies. This saves me time and physical space.

lerna and yarn workspaces give us the ability to build libraries and apps in a single repo without forcing us to publish to npm or other registries. The beauty behind these technologies is that they can find package dependencies by analyzing package.json files located at each project’s root folder. Thereby, these tools make it obsolete to manually create symlinks or use "low-level" npm link directly.

Now, even if you are already using yarn workspaces I think Lerna worth the try because it provides a series of vantages. Besides you could use Lerna with Yarn Workspaces:

with one lerna command you can iterate through all or particular packages, running a series of operations (such as linting, testing, and building) on each package. Thereby, it compliments yarn workspaces that takes over the dependency management process.

So finally I decided to learn Lerna - sorry for the tongue twister - but I didn't want to switch to Yarn (since I'm just comfortable with npm). Here's how I managed to make it work:

Setting Lerna with NPM from the shell

Create a directory for your project and enter on it with this command:

mkdir try-lerna
cd try-lerna

You can install Lerna globally (npm i -g lerna) or use it with npx.

• Init lerna

  npx lerna init --independent
  

This will:

• init a git repository
• create a package.json
• create a lerna.json
• create a packages folder (here you can place your workspaces).

Packages is how lerna call workspaces.

You can arrange the Packages as you want, if you prefer you can put all the packages in the root folder and then change the array in order to match (like a "yarn workspaces" approach).

My lerna.json file has stream: true (in order to see gatsby output when run develop with lerna) and hoist: true (in order to hoist node_modules also with the command lerna add; I'll explain this last feature forward):

{
    "packages":["packages/*"],
    "version":"independent",
    "stream":true,
    "command":{
        "bootstrap":{
            "hoist":true
        }
    }
}

You can add this manually or use this simple shell command:

echo "{\r\t\"packages\":[\"packages/*\"],\r\t\"version\":\"independent\",\r\t\"stream\":true,\r\t\"command\":{\r\t\t\"bootstrap\":{\r\t\t\t\"hoist\":true\r\t\t}\r\t}\r}" > lerna.json       

Scaffolding folders and files with shell

This section can be done by hand following the first lesson from Jason Lengstorf's course linked in the introduction.
I just collect all the basic passages in order to accomplish them with few commands from the shell.

In case you want to use my approach remember to replace "NAME" with the name of your theme and then just copy and paste this commands:

mkdir -p packages/{gatsby-theme-NAME,site}

touch packages/{gatsby-theme-NAME,site}/{index.js,package.json}

echo "//loop" >> packages/{gatsby-theme-NAME,site}/index.js

Initial content for package.json on the site folder:

echo "{\r\t\"private\":true,\r\t\"name\":\"site\",\r\t\"version\":\"1.0.0\",\r\t\"license\":\"MIT\",\r\t\"scripts\":{\r\t\t\"builds\":\"gatsby build\",\r\t\t\"develop\":\"gatsby develop\",\r\t\t\"clean\":\"gaysby clean\"\r\t}\r}" >> packages/site/package.json    

package.json on the folder theme (replace NAME again and also insert YOUR_NPM_USER):

echo "{\r\t\"name\": \"@YOUR_NPM_USER/gatsby-theme-NAME\",\r\t\"version\": \"1.0.1\",\r\t\"license\": \"MIT\",\r\t\"main\": \"index.js\",\r\t\"scripts\": {\r\t\t\"builds\": \"gatsby build\",\r\t\t\"develop\": \"gatsby develop\",\r\t\t\"clean\": \"gaysby clean\"\r\t}\r}" >> packages/gatsby-theme-NAME/package.json    

Here's the result:

├── packages
│  ├── site
│  └── gatsby-theme-NAME
├── .gitignore
├── package.json
└── lerna.json

PS: I advice you to create a .gitignore and add node_modules right away:

echo "node_modules" > .gitignore

Using Lerna

One problem with Lerna is that you can't install multiple dependencies per Package with one single command (like you do with yarn workspace name_workspace add package1 package2 etc). You need to do it one by one, like this (if you didn't install it globally you'll have to precede every command with npx):

lerna add gatsby --scope site 

lerna add react --scope site

lerna add react-dom --scope site

Here --scope site is used to tell Lerna to install only in the Package "site".

However, you can install a dependency, let's say gatsby, for every Package with just one command:

lerna add gatsby

This command will:

• Install Gatsby for every Package (so in this case for Site and also for the Theme Package).
• If you enabled hoist on the lerna.json it will create a node_modules folder in your root (see 'Lerna Bootstrap forward'):

Choose your approach

If you like this approach you can continue installing all the common dependencies (react, react-dom) on the Packages. If you need to install some dependencies only for one Package use --scope name_of_the_package (remember it's the name specified on the package.json not the name of the folder).

You could also use npm by installing every dependency you want in the respective Package. But remember this approach will install a node_modules dependencies for every respective folder. This could be clean and then hoisted afterwards. Although this option is valid it will take you more time.

Whatever option you choose remember to add/link the local Theme Package as a dependency for the Site Package (use the name you set on package.json):

lerna add @YOUR_NPM_USER/gatsby-theme-NAME --scope=site

Orchestrate everything with Lerna bootstrap

In the end you can safely do:

lerna bootstrap 

This command is really helpful, specially if you already have your package.json filled with their respective dependencies.

With hoist: true option (on lerna.json) it will place a shared node_modules folder located in the root, then it will link local dependencies in their respective package.json.

I really like --hoist feature it's a process that avoid duplicated dependencies on the node_modules, so it saves physical space and installation time.

If for whatever reason you want to clean anything first you could do:

lerna clean && lerna link && lerna bootstrap 

Run commands and scripts

To run scripts (placed on the package.json) you need to specify (or not) the target Package:

lerna run develop --scope Package_name

Lerna also let you run any commands from the root to every Package (use dash). For example:

lerna exec -- rm -rf node_modules 

This will remove node modules on every Package folder.

Update Wizard

In the future, when you'll need to update your packages you can use the update-wizard:

npm install --save-dev lerna-update-wizard

then you'll need to run only this command and the wizard will help you:

lernaupdate

Comments

[intrnl]

The one common pitfall with hoisting dependencies is issues regarding typings, some types depends on Node's typings and it can wreck havoc thanks to differences between Node and browsers (setTimeout and clearTimeout being the notable example)

[Denny George]

Hey, thanks for writing this. This was super helpful. I quickly overgrew the official website documentation and found this to add more value to my understanding of lerna.

Even though I find lerna to be very useful and promising, I am unable to master it. How did you learn about the update-wizard package and the hoist command? I'm trying to understand if there's a place where these conversations are happening or was it by reading the source code?

thank you :)

[Santiago]

Hi, thanks I glad you appreciate :)
I learned this just googling and reading a lot online.
My main information sources are linked in this same guide though. The most helpful was the linked article from doppelmutzi.
Indeed the difficult topic,for me, was the --hoist concept. Once I understand what the heck was going on I got the rest :D