DEV Community

Exploring the Monorepo #2: Workspaces (npm, pnpm)

Jon Lauridsen on June 27, 2021

Table Of Contents npm pnpm Conclusion Okay so attempt #1 didn't quite work, but all the package managers have a feature called Worksp...

Read full post

Ruy Adorno • Jun 28 '21

hi @jonlauridsen, did you looked up docs.npmjs.com/cli/v7/using-npm/wo... ? Was that the page that you found to be really terse?

From the community side, I've seen some nice tutorials that you might find useful in order to give npm workspaces a try:

Jon Lauridsen • Jun 28 '21 • Edited

Hi Ruy,

Yes that's the documentation I read. Maybe terse is the wrong word, but I followed it to set up my project but didn't get much working, and I didn't find a way to read my way to clarity. It was a couple different things, like commands were failing until I upgraded npm because what came with my npm 16.1 wasn't workspace-aware (despite being 7.x, but I think I was running install --workspaces which didn't land until 7.14 (?), but none of that is in the docs so it was quite an initial confusion).

Anyway, then, with npm 7.19.0 installed, I got into some weird state when I did this:

$ cd apps/web
$ npm install
$ npm start
[tsc] ../../libs/types/src/index.ts(1,23): error TS2307: Cannot find module 'type-fest' or its corresponding type declarations.

I think in this case workspaces get totally ignored, and it installed "types" from the registry, which is so very far from what I wanted.

But installing from the root folder also fails:

$ npm install
$ cd apps/web
$ npm start
[tsc] src/pages/Home.tsx(3,29): error TS2307: Cannot find module 'types' or its corresponding type declarations.

I'm probably doing something silly and the tool itself works fine, but I didn't get enough feedback from the docs and npm to make progress, and it took a long time to install the dependencies, so I gave up. Happy to try again though, I have an in-progress branch at github.com/gaggle/exploring-the-mo... that I'm happy to give another go on if you or someone else sees what I do wrong.

And thanks for the links, I do remember reading them both when they came out but I didn't come across them in my attempts this time around.

Ruy Adorno • Jun 29 '21

I see! thanks a lot for the feedback! 😄

It seems to me you were hitting one of the biggest DX hiccups (IMO) we're still yet to solve, which is the fact that in order to work with workspaces you can NOT cd into that folder (e.g: cd apps/web) - instead what you want to do is to run the command from the project root level and set a workspace config value, like: npm start -w ./apps/web

It's the same for adding new dependencies to a workspace, let's say you want to add a new dependency named path-complete-extname to a child workspace at ./apps/web, then the syntax to do so is to run from the root: npm install path-complete-extname -w ./apps/web (notice you can also just use the "name" value of a child workspace, so if the name property of ./apps/web/package.json is web then you can simply run: npm install path-complete-extname -w web

Again, thanks for the feedback! This type of real life UX report helps us a lot!

Zoltan Kochan • Jun 28 '21

In the pnpm monorepo we also use project references.

I created this helper package to generate the tsconfig files: github.com/pnpm/meta-updater

Here's how we use it: github.com/pnpm/pnpm/blob/main/uti...

Jon Lauridsen • Jul 4 '21 • Edited

That looks good Zoltan, thanks for the links!

Very interesting to browse the pnpm monorepo, I'm left wondering if pnpm decided on using project references knowing things like "strict" doesn't work across projects, or if it was a case of "good enough" and you don't actually hit that problem…🤔

At this point I'm more confused about how Typescript intends their References feature should be used than when I began these articles, I'm not at all sure how tsconfig options get merged, if they do at all. Am I the only person who was expecting projects to be truly separate?

I see all your packages make use of "outDir" and "rootDir" and I think those settings only get used when running tsc -b or equivalent to build to source, which I assume you run before publishing the packages. My only concern is the runtime though and in that context I'm hitting the strict-failure, and when I can't clearly read why from the documentation I start dreaming about a world without Typescript 😄.

I've noted meta-updater for later research, I think it's exactly what I'll be looking for to help script away the downsides I ended up on in attempt 3.

Rahul Tarak • Jun 27 '21

Happy that my article helped! Super cool to see you explore monorepos like this.

panta82 • Jun 28 '21

Since all code is supposed to live and work together, why don't you standardize typescript options accross the entire monorepo?

Jon Lauridsen • Jun 28 '21

That's a good question! I try to cover that in Finding the middle ground, specifically with:

Maybe that code is deep inside the product, maybe it's some hardcoded functions, maybe it's a concept that's been copy-pasted across multiple systems, maybe it lacks tests, whatever the case it's a shared pattern that just needs to be extracted without too much ceremony. It can be improved later, but right now we just want to put a box around it.

What I mean is: I have code I want to improve, but I don't want to or can't prioritize that work right now. The code isn't great, but it's by extracting it I start the process of improving it. To that end I feel it should be possible to run that code with different settings. In this case analytics isn't Typescript strict compatible to represent that kind of code-quality issues that I've seen in the codebases I've worked on.

panta82 • Jun 28 '21

Understood.

AFAIK when you're compiling api project and you import files from analytics project, what happens is:

ts compiler picks up settings from api project
picks files from wherever you references them
smashes them into one program

If files from analytics aren't compatible with the typescript version and options from api, it will break. This is because they are treated as loose files, not independent codebases. And you can't have two different rulesets apply to different files (AFAIK - if someone knows different, I'd be happy to learn).

In this use case, I'd pull analytics out of the monorepo and put it into its own project. Publish as a private npm module. Then reference compiled code from monorepo.

Jon Lauridsen • Jun 28 '21

That's really good information, thanks!

So "can't have two different rulesets apply to different files" applies to all settings? Wow, yikes. I got some work ahead of me then. I very much want to avoid pulling anything out of the monorepo.

I've feared from the start that I have to explore building all the code and only rely on js references, so api would use analytics's compiled js output. I'm confident that'll work, but I don't know a way to make that a good coding experience because changes won't be reflected live… 😕

panta82 • Jun 28 '21

Once again, AFAIK.

There is one thing you can experiment with that I use in one project. You can declare a local npm module, that lives in the same monorepo with the rest of the code.

The declaration would look like this:

"dependencies": {
  "analytics": "file:libs/analytics",
}

Then in lib/analytics define a package.json and tsconfig with the settings you like. Build.

Then when you do import x from 'analytics'; is should find the compiled code from libs/analytics/dist. Then add npm hooks to run compilation on post npm-install or whenever you need depending on your workflow.

I haven't tried this, but maybe it's worth a shot.

Jon Lauridsen • Jun 28 '21 • Edited

Yeah that makes sense. It's the "npm hooks to run compilation" that bothers me, because it sounds a lot like I'll be fighting the tools to do things they weren't meant for. But it might be the only way 😬

Thanks for the insights.

Angra974 • Oct 20 '23

Hi. I'm facing this current problem.
I'm trying to access some app/web functions ( like react hooks ) code from libs/analystics packages.
Do you think this is possible with the workspace ?
I know i can move the hooks in the packages folder but I will need to do lot of rewrite in an app i'm trying to put in a monorepo.
Thanks.

Jon Lauridsen • Jan 3

Somehow libs/analytics will have to pull them in as dependencies. And it can't be for app/web if app/web already pulls libs/analytics in as a dependency. If they're custom hooks perhaps that's a new package to extract, and set it to be consumed by both libs/analytics and app/web?