Exploring the Monorepo #2: Workspaces (npm, pnpm)

Table Of Contents

• npm
• pnpm
• Conclusion

Okay so attempt #1 didn't quite work, but all the package managers have a feature called Workspaces, which npm describes like this:

[Workspaces] provides support to managing multiple packages from your local files system from within a singular top-level, root package.

That sure sounds relevant, so let's give it a try!

npm

The npm documentation is so terse I've honestly no clue how to get anything working 🤷‍♀️. If you know your way around npm workspaces I'm happy to swap stories, but for now I'm giving up on this.

pnpm

Documentation here is definitely a step up, with more examples to draw inspiration from. Still a bit hard to grasp though, but I also came across this nice "actual complete guide to typescript monorepos" article by @cryogenicplanet that put some of the details together in an understandable way. Thanks Rahul!

The end-result of workspacifying the product comes out like this:

webby
├── apps
│  ├── api/
│  └── web/
├── libs
│  ├── analytics/
│  ├── logging/
│  └── types/
├── package.json
├── pnpm-lock.yaml
├── pnpm-workspace.yaml
└── tsconfig.json

ℹ️ This project is prepared on GitHub1s if you'd like to familiarize yourself with the code.

Each app and lib's package.json is scoped to just that piece of code, so just like before we have a great immediate overview. The real question is: Does it work?

Well… apps/web runs fine:

$ cd apps/web
$ pnpm install
Scope: all 6 workspace projects
└─ Done in 3.2s
$ pnpm start
[razzle] > Started on port 3000

So just running pnpm install in apps/web actually resolved all dependencies for the whole repository, which is very nice. And all it takes to configure it are a few lines in the ppm-workspace.yaml file, so it's all very easy to get working.

But apps/api fails just as it did in the previous article:

$ cd ../api
$ pnpm start
[api] TSError: ⨯ Unable to compile TypeScript:
[api] ../../libs/analytics/src/index.ts(8,3): error TS2564: Property 'uninitializedProperty' has no initializer and is not definitely assigned in the constructor.

With help from @cryogenicplanet's article I finally looked into Typescript Project References, which is a way to tell Typescript how to deal with multiple code pieces. But the same error occurs with or without references so I’m not sure if they’re the answer and I’ve just done it wrong or if there’s a deeper problem.

Conclusion

I'm not sure how to proceed from here. The pnpm tool itself seems to work great, what I need to figure out is how to get Typescript to use each package's own tsconfig file.

At this point I think my best bet is to focus on Typescript configuration, rather than dive further into alternative dependency managers like yarn. If you have ideas of how to get Typescript configured to respect a package's tsconfig settings then please leave a comment.

Comments

[Ruy Adorno]

hi @jonlauridsen, did you looked up docs.npmjs.com/cli/v7/using-npm/wo... ? Was that the page that you found to be really terse?

From the community side, I've seen some nice tutorials that you might find useful in order to give npm workspaces a try:

• ruanmartinelli.com/posts/npm-7-wor...
• dev.to/limal/simplify-your-monorep...

[Jon Lauridsen]

Hi Ruy,

Yes that's the documentation I read. Maybe terse is the wrong word, but I followed it to set up my project but didn't get much working, and I didn't find a way to read my way to clarity. It was a couple different things, like commands were failing until I upgraded npm because what came with my npm 16.1 wasn't workspace-aware (despite being 7.x, but I think I was running install --workspaces which didn't land until 7.14 (?), but none of that is in the docs so it was quite an initial confusion).

Anyway, then, with npm 7.19.0 installed, I got into some weird state when I did this:

$ cd apps/web
$ npm install
$ npm start
[tsc] ../../libs/types/src/index.ts(1,23): error TS2307: Cannot find module 'type-fest' or its corresponding type declarations.

I think in this case workspaces get totally ignored, and it installed "types" from the registry, which is so very far from what I wanted.

But installing from the root folder also fails:

$ npm install
$ cd apps/web
$ npm start
[tsc] src/pages/Home.tsx(3,29): error TS2307: Cannot find module 'types' or its corresponding type declarations.

I'm probably doing something silly and the tool itself works fine, but I didn't get enough feedback from the docs and npm to make progress, and it took a long time to install the dependencies, so I gave up. Happy to try again though, I have an in-progress branch at github.com/gaggle/exploring-the-mo... that I'm happy to give another go on if you or someone else sees what I do wrong.

And thanks for the links, I do remember reading them both when they came out but I didn't come across them in my attempts this time around.

[Ruy Adorno]

I see! thanks a lot for the feedback! 😄

It seems to me you were hitting one of the biggest DX hiccups (IMO) we're still yet to solve, which is the fact that in order to work with workspaces you can NOT cd into that folder (e.g: cd apps/web) - instead what you want to do is to run the command from the project root level and set a workspace config value, like: npm start -w ./apps/web

It's the same for adding new dependencies to a workspace, let's say you want to add a new dependency named path-complete-extname to a child workspace at ./apps/web, then the syntax to do so is to run from the root: npm install path-complete-extname -w ./apps/web (notice you can also just use the "name" value of a child workspace, so if the name property of ./apps/web/package.json is web then you can simply run: npm install path-complete-extname -w web

Again, thanks for the feedback! This type of real life UX report helps us a lot!

[Zoltan Kochan]

In the pnpm monorepo we also use project references.

I created this helper package to generate the tsconfig files: github.com/pnpm/meta-updater

Here's how we use it: github.com/pnpm/pnpm/blob/main/uti...

[Jon Lauridsen]

That looks good Zoltan, thanks for the links!

Very interesting to browse the pnpm monorepo, I'm left wondering if pnpm decided on using project references knowing things like "strict" doesn't work across projects, or if it was a case of "good enough" and you don't actually hit that problem…🤔

At this point I'm more confused about how Typescript intends their References feature should be used than when I began these articles, I'm not at all sure how tsconfig options get merged, if they do at all. Am I the only person who was expecting projects to be truly separate?

I see all your packages make use of "outDir" and "rootDir" and I think those settings only get used when running tsc -b or equivalent to build to source, which I assume you run before publishing the packages. My only concern is the runtime though and in that context I'm hitting the strict-failure, and when I can't clearly read why from the documentation I start dreaming about a world without Typescript 😄.

I've noted meta-updater for later research, I think it's exactly what I'll be looking for to help script away the downsides I ended up on in attempt 3.

[Rahul Tarak]

Happy that my article helped! Super cool to see you explore monorepos like this.

[panta82]

Since all code is supposed to live and work together, why don't you standardize typescript options accross the entire monorepo?

[Jon Lauridsen]

That's a good question! I try to cover that in Finding the middle ground, specifically with:

Maybe that code is deep inside the product, maybe it's some hardcoded functions, maybe it's a concept that's been copy-pasted across multiple systems, maybe it lacks tests, whatever the case it's a shared pattern that just needs to be extracted without too much ceremony. It can be improved later, but right now we just want to put a box around it.

What I mean is: I have code I want to improve, but I don't want to or can't prioritize that work right now. The code isn't great, but it's by extracting it I start the process of improving it. To that end I feel it should be possible to run that code with different settings. In this case analytics isn't Typescript strict compatible to represent that kind of code-quality issues that I've seen in the codebases I've worked on.

[panta82]

Understood.

AFAIK when you're compiling api project and you import files from analytics project, what happens is:

• ts compiler picks up settings from api project
• picks files from wherever you references them
• smashes them into one program

If files from analytics aren't compatible with the typescript version and options from api, it will break. This is because they are treated as loose files, not independent codebases. And you can't have two different rulesets apply to different files (AFAIK - if someone knows different, I'd be happy to learn).

In this use case, I'd pull analytics out of the monorepo and put it into its own project. Publish as a private npm module. Then reference compiled code from monorepo.

[Jon Lauridsen]

That's really good information, thanks!

So "can't have two different rulesets apply to different files" applies to all settings? Wow, yikes. I got some work ahead of me then. I very much want to avoid pulling anything out of the monorepo.

I've feared from the start that I have to explore building all the code and only rely on js references, so api would use analytics's compiled js output. I'm confident that'll work, but I don't know a way to make that a good coding experience because changes won't be reflected live… 😕

[panta82]

Once again, AFAIK.

There is one thing you can experiment with that I use in one project. You can declare a local npm module, that lives in the same monorepo with the rest of the code.

The declaration would look like this:

"dependencies": {
  "analytics": "file:libs/analytics",
}

Then in lib/analytics define a package.json and tsconfig with the settings you like. Build.

Then when you do import x from 'analytics'; is should find the compiled code from libs/analytics/dist. Then add npm hooks to run compilation on post npm-install or whenever you need depending on your workflow.

I haven't tried this, but maybe it's worth a shot.

[Jon Lauridsen]

Yeah that makes sense. It's the "npm hooks to run compilation" that bothers me, because it sounds a lot like I'll be fighting the tools to do things they weren't meant for. But it might be the only way 😬

Thanks for the insights.

[Angra974]

Hi. I'm facing this current problem.
I'm trying to access some app/web functions ( like react hooks ) code from libs/analystics packages.
Do you think this is possible with the workspace ?
I know i can move the hooks in the packages folder but I will need to do lot of rewrite in an app i'm trying to put in a monorepo.
Thanks.

[Jon Lauridsen]

Somehow libs/analytics will have to pull them in as dependencies. And it can't be for app/web if app/web already pulls libs/analytics in as a dependency. If they're custom hooks perhaps that's a new package to extract, and set it to be consumed by both libs/analytics and app/web?