Why we stopped using Lerna for monorepos

And what’s the new fashion?

Intro

I need to be honest: Lerna is my mentor to the concept of monorepo, and it opened a new door for me to manage big JavaScript projects. It also allows for a shared repo within the sole ecosystem (JavaScript and TypeScript, specifically) for both the frontend and backend.

Lerna is great. In the previous article, TypeScript all-in-one: Monorepo with its pains and gains, I mentioned that we were using Lerna for executing commands and publishing workspace packages. But now, it’s time to say goodbye.

The context

ℹ️ TL;DR Our project Logto used PNPM + Lerna for dependency and monorepo management.

Why did we need Lerna?

When Lerna was born (v1.0.1, Dec 2015), the NPM official support for monorepo (or workspaces) was still far away (until Oct 2020 with NPM v7). There were several pain points for managing multiple JS packages in one repo:

• Import another package in the same repo (workspace) without publishing it to NPM
• Run scripts for multiple workspace packages in the dependency order
• Version and publish all workspace packages by a reasonable strategy

None of them sounds easy, and Lerna can cover all.

The monkey-patch situation

However, Lerna does not include package manager features; thus, you still need to use NPM or something else. As you can imagine, this leads Lerna to have the “all-in-one” monorepo support to eliminate the gap across different package managers. In other words, redundancy.

I started to scaffold the project with Lerna + PNPM (not NPM) since I’m a newbie to monorepo, and using Lerna was the standard approach in my mind. But some discord popped up over time.

💡 When it has 20% redundancy, it is acceptable; when it turns to 90%, I have a solid feeling to replace it with something more specific.

🔗 Workspace dependencies

PNPM has a built-in command to add workspace dependencies:

pnpm add @logto/some-package --workspace

You can use it in ANY workspace package. It will also add a dependency with the special protocol workspace: to the package.json.

Unfortunately, Lerna cannot recognize it. So we need to fall back to the command:

lerna add @logto/package-a --scope=@logto/package-b

This will add package-a as a dependency of package-b. But you must run it in the workspace root and specify the scope.

Plus, PNPM uses hard links to reuse and link dependencies, which makes lerna bootstrap unused (this is a core feature of Lerna).

🏃 Run scripts

As I mentioned in the previous article:

Now pnpm has a -w option to run commands in the workspace root and --filter for filtering. Thus you can probably replace lerna with a more dedicated package publishing CLI.

Besides, PNPM supports the -r option that allows you to run a script recursively, with dependency order support. Then we don’t need lerna run and lerna exec anymore, but Lerna has to keep the commands because not everyone uses PNPM.

P.S. I also love that PNPM doesn’t require adding additional -- to pass down the options!

🚀 Version and publish

Lerna’s opinionated and automatic versioning and publishing strategy is a significant advantage. It is extremely useful when you have like ten packages in the workspace. It will also help you tag and push to the remote git, even creating GitHub releases.

We’ll hold this part for now.

The trigger

In fact, none of the things above triggered us to remove Lerna since our development was moving fast, and we thought the removal would naturally happen one day. When Lerna v6 was published, it finally pushed us to make the change.

The Nx team called the version a “reborn” for Lerna, and yes, Lerna was declared “dead” in April 2022, and the team took over stewardship. This update also adds official support for PNPM.

Sounds really awesome, right? Soon we upgraded to v6 and found an interesting issue with lerna version:

It will automatically update the quote marks (from single to double) and the indent style of pnpm-lock.yaml.

❔ So, no test for the lockfile changes and no bug report during the beta?

We think it’s time to say goodbye.

Removing Lerna

Thanks to the powerful workspace features in PNPM, the process was easier than we expected.

🔗 Workspace dependencies

Everything still works without change, but updating all workspace dependency version numbers in package.json to the workspace protocol is highly recommended since it’s smarter and PNPM uses it for workspace dependencies by default. Don’t forget to run pnpm i to fix the lockfile.

From now, we can use pnpm add @logto/some-package --workspace right in the directory for which we want to add the workspace dependency.

🏃 Run scripts

Replace lerna run foo with pnpm -r foo.

That’s it. And for filtering? Not a problem for the filtering option.

🚀 Version and publish

PNPM can take care of publishing but not versioning. Here are several options:

Using Lerna

⚠️ Lerna (and NPM) doesn’t recognize the workspace protocol, so you cannot use pnpm add --workspace in this approach. Instead, you need to update package.json files for workspace dependency changes manually, then run pnpm i to fix the lockfile.

• Details about the workspace protocol issue

  Lerna will keep the workspace: protocol during versioning, which is fine. When publishing, Lerna invokes npm publish with a non-negotiable attitude. Unlike pnpm publish, npm publish won’t update the workspace: protocol to the proper version number when packing files, which will cause failed installations for your package users.

Yes, you can still version with Lerna without listing it in the dev dependencies. We used it as a transition plan in our workflow:

   run: |
-    pnpm lerna publish \
+    pnpm \
+    --package=conventional-changelog-conventionalcommits \
+    --package=lerna@^5.0.0 \
+    dlx lerna publish \

We are using dlx since our changelog was generated by the rules of Conventional Commits. If you don't need it, do this:

   run: |
-    pnpm lerna publish \
+    pnpx lerna@^5.0.0 publish \

Version with Changesets

I found a tool named Changesets for versioning and publishing monorepo in PNPM’s docs. After some study on its philosophy, I believe it’ll be a good fit for monorepos:

• Package-level version bumping with flexible strategy configuration
• Customizable linked and fixed packages for versioning (easily to achieve the same effect as Lerna)
• Shipped with a GitHub bot and workflow, and a workable publishing practice

It’s not perfect yet. For example, we still need to manually define the “publish group” in our repo and configure the GitHub workflow for creating releases. But it’s still a good tool that enables much more possibilities and another level of flexibility for monorepo publishing.

We are still exploring this approach, and I can write another article once we figure things out.

Write a version script

We didn’t try this, while it shouldn’t be too hard for the simplest implementation. Write a script for versioning all necessary packages and run it before pnpm -r publish and it will be ok. E.g., use the “since” filter to version all changed packages since the last version.

Closing notes

Our lockfile size decreased dramatically after removing Lerna, which didn’t affect our daily development. Actually, it’s hard to tell the difference, and I’m happy we could achieve the same dev experience with much fewer dependencies.

For the past year, our codebase has increased a lot, and we still feel manageable. I think monorepo deserves a big credit. If you are interested in more articles about monorepo, feel free to drop a comment to let us know!

👉 Our project Logto is an OSS that helps you build the sign-in, auth, and user identity within minutes.

Comments

[Ghislain B.]

You should seriously look at Lerna-Lite, it's more modular, thus much smaller (only version & publish are in the core, all other commands are optional and must be installed separately). Also, all the issues you mentioned about Lerna 6 aren't a problem in Lerna-Lite. You can read About Lerna-Lite, I created the fork in March 2022, that is couple months before Nrwl took over Lerna and Lerna-Lite also has nearly 300 stars already. The main difference is its modularity (much smaller) and also the fact that it's not becoming another Nx/Nrwl product (Lerna 6+ requires Nx and even TypeScript behind the scene, Lerna-Lite does not). Lerna-Lite also support, and actually uses it too, yarn/pnpm workspace: protocol which was a long time feature request in Lerna.

So anyway, I just wanted to give you and the readers another possible alternative (I personally prefer Lerna's version/publish process with conventional changelog, I find it much better than changesets and with Lerna-Lite you now have the option to only install version/publish commands and nothing else, sweet!)

[gao-sun]

Wow, from what you describe I think lerna-lite is EXACTLY what I was looking for! Definitely will give it a try. Thanks!