DEV Community: Zuri Klaschka (they/them)

pnpm and Parcel based monorepo

Zuri Klaschka (they/them) — Sun, 13 Mar 2022 15:57:16 +0000

The problem

I have tried several ways of managing JavaScript/TypeScript Library Monorepos in the past: lerna, yarn workspaces, etc.

Now don't get me wrong: These are great tools and I very much appreciate the effort their authors have put into them.

But they always felt a little bit like gambling. It never felt like I was really in control of what was happening (with a lot of black magic), and I found that they felt a bit ... fragile (I was alsways worried about breaking some symlinks or stuff like that when running any commands).

A solution?

I wanted to try both pnpm and Parcel. I had heard good things about both tools and have recently gotten more and more frustrated with their more established competitors.

When I looked at their respective documentation pages, it looked like both had great monorepo support. Since I was also still on a long-lasting search for some "building an npm library"-compatible monorepo solution with a better developer experience than what I had seen so far, I decided to give it a shot.

The repository

So, I built (and documented) a test repository to try this new monorepo setup:

pklaschka / pnpm-parcel-monorepo-test

A repository for trying a JS/TS npm library monorepo setup using pnpm as package manager and parcel as a build tool.

`pnpm` & `parcel` monorepo test

A repository for trying a JS/TS npm library monorepo setup using pnpm as package manager and parcel as a build tool.

Development prerequisites:

Tech Stack

This is an overview of the most important components of this monorepo's tech stack

pnpm -- package and monorepo workspace manager
Parcel -- build tool
Jest / ts-jest -- Unit testing framework
TypeScript / tsc -- TypeScript type checking
TypeScript ESLint -- linting
Prettier -- code formatter
fliegdoc -- documentation generator
GitHub Actions -- CI pipeline
Renovate -- Dependency Update Management

Usage

Setting up the development environment

Install the dependencies:

pnpm install

automatically runs recursively for the workspace, cf. https://pnpm.io/cli/install
Alias: pnpm i
npm ci-equivalent: pnpm i --frozen-lockfile (automatically true in CI environment)

You can also run pnpm install when anything about your dependencies becomes out of date to fix it back up.

Dependency Management

Installing

…

View on GitHub

The repository contains a test setup with a more or less full tech stack consisting of, among others:

TypeScript
ESLint
Prettier
fliegdoc (a self-built documentation generator)
jest / ts-jest
GitHub Actions

I described most things in the README.md, but I also created an additional public Notion page describing more details.

Results

I'm really happy with how it works and will definitely use this approach in the future. I'll also probably migrate existing monorepos to this approach, in the future.

Advantages

🟢 you feel like you're in control with pnpm, it's pretty straight-forward to understand how their workspace system works, so you feel like you're in control and don't have to guess about fixes to your problems 🎉. E.g., pnpm install sets up everything. Before, I always was unsure if I should now run npm install, lerna bootstrap, or something else.
🟢 quick build times since parcel builds all the packages at once (instead of running build scripts one package at a time), build times (especially with the build cache in place) are incredibly fast
🟢 development experience with parcel watch, it's possible to very quickly develop
🟢 "native" workspaces working with workspaces / multiple packages feels "native" to pnpm (compared to its competitors, where I unfortunately found that it feels more like a "hacky side feature"). Commands work how I would expect them to work, "internal" dependencies betweeen packages automatically get hydrated with version numbers on publish, and so on.

Drawbacks

Of course, every approach comes with a few drawbacks. Here are the ones I've found so far:

🟡 less ecosystem support while pnpm and parcel work great in 99 % of cases, tools often don't test their support for these as much as, for example, for yarn and webpack
🟢 (no Dependabot support) at the time of writing this, GitHub's Dependabot doesn't support pnpm. Thankfully, Renovate seems to work well.
🟢 (no included "release automation" tooling) lerna came with great Changelog / Conventional Commit / ... based release automation tooling. Unfortunately, I haven't yet found a great solution for doing something similar with pnpm. Do you have any recommendations?

A quick fix for a Parcel bug that almost made me dismiss it

When I initially tested Parcel, it felt unstable. It wouldn't shut down, would from time to time just overwrite my package.json, and just overall not work very well at all.

I was almost ready to give up when I found this issue on GitHub. It turns out that I had a package-lock.json somewhere higher up the file tree (probably some backup I had created before). This lead to Parcel showing all kinds of weird behavior (not only the one described in the issue). So if you decide to try this approach and feel like Parcel is "freaking out" in a weird way, it might be worth checking for package.json, packaage-lock.json or similar files higher up in the file tree.

So overall, this is easy to fix. But this almost made me (which would have been a shame!) turn away from Parcel, so I wanted to include this note here.

Even more details

Furthermore, I've documented everything I learned about the process / how the repo is structured in a Notion Page. If you decide to try this monorepo configuration, this could be useful to you as it includes all the commands you need to know, links to various important resources, and so on.

Author

Zuri Klaschka (they/them)Follow

Working Student, Creative Cloud Platform and Ecosystem Team @ Adobe; I do many things. Among them developing websites and plugins for Adobe products, primarily Adobe XD. Viva la Open-Source!

And so, I wrote my own Typescript Documentation Generator...

Zuri Klaschka (they/them) — Fri, 08 Jan 2021 00:57:24 +0000

TL;DR: For a project fussel178 and I (pklaschka) were working on, we needed a documentation generator for a Typescript Monorepo.

The "Status Quo"

While the existing solution, TypeDoc, is a great tool, it didn't really fit our needs, and documentation on how to customize the output isn't easy to come by. Don't get me wrong: It's a great project that does a lot more than my own solution, but it simply didn't fit our project's needs. Still, for many projects, it works great, and I can highly recommend checking it out:

TypeStrong / typedoc

Documentation generator for TypeScript projects.

So... let's build it ourselves 😜

After spending a day searching for alternatives (unsuccessfully), I decided to "just" build my own solution. And thus, fliegdoc found its beginning 😉

fliegwerk / fliegdoc

A documentation generator for Typescript-based libraries with good support for monorepos

Welcome to fliegdoc 👋

A documentation generator for Typescript-based libraries with good support for monorepos

🏠 Homepage, Demo & Docs

Prerequisites

node >12.0.0

Install

npm install --global fliegdoc

Usage

CLI

$ fliegdoc --help
Usage: fliegdoc [command] [options]

Commands:
  fliegdoc build [options]  Build the documentation       [default] [aliases: b]
  fliegdoc serve [options]  Preview the documentation in the browser[aliases: s]
  fliegdoc completion       generate completion script

Options:
      --help     Show help                                             [boolean]
  -s, --serve    Serve the static files after build   [boolean] [default: false]
  -p, --port     The port on which the documentation gets hosted        [number]
  -v, --version  Show version number                                   [boolean]

Get help for individual commands by running fliegdoc <command> --help

The CLI searches for a fliegdoc.config.js file and applies its options on top of the default options.

Example `fliegdoc.config.js` with default options

// fliegdoc.config.js
const { HTMLTheme } = require('fliegdoc');
module.exports = {
    baseUrl: '/'

…

View on GitHub

What fliegdoc is

fliegdoc documents only the exported members (!) of a TypeScript library, i.e., one or more (in the case of a monorepo) npm packages.

It first converts the source file's AST to a kind of "documentation-ready" AST, which then gets passed to a theme, converting it to HTML (but, if the theme gets adjusted, also any other format).

The generated web page then includes the project's root README.md, and the API References for the project's modules, i.e., packages.

Therefore, it can very easily be used to document mono-repo structures, including more than one package.

Demo

Since the project's exposed APIs themselves are documented using the project itself, you can visit the project documentation to see the generated results.

How to use it

Let's consider a Lerna-based monorepo with (for simplicity) two packages, resulting in a folder structure that looks something like this:

.
├── packages/
│   ├── package-1/
│   │   ├── src/
│   │   │   └── main.ts
│   │   ├── package.json
│   │   └── tsconfig.json
│   └── package-2/
│       ├── src/
│       │   └── main.ts
│       ├── package.json
│       └── tsconfig.json
├── package.json
├── lerna.json
├── README.md
└── tsconfig.json

Since fliegdoc generates the documentation for the entire repository, we add fliegdoc as a devDependency of the root package.json:

$ npm i -D fliegdoc

Next, we add two scripts in our root package.json:

{
  "scripts": {
    "docs:build": "fliegdoc build",
    "docs:serve": "fliegdoc serve"
  }
}

If our repository followed an easy structure, with just one root tsconfig.json, package.json, README.md, and a main.ts in a root src folder, that all we'd need to do. However, we are within a monorepo that ... makes things a bit more complicated.

"Things should be made as simple as possible but not any simpler" (Albert Einstein)

To configure fliegdoc to work with this repo structure, we create a fliegdoc.config.js:

module.exports = {
  modules: [
    {
      package: "./packages/package-1/package.json",
      tsconfig: "./packages/package-1/tsconfig.json",
      mainFile: "main.ts",
    },
    {
      package: "./packages/package-2/package.json",
      tsconfig: "./packages/package-2/tsconfig.json",
      mainFile: "main.ts",
    },
  ]
};

While we could configure many other parameters, we can already successfully generate the documentation for the repo with this config.

Further configuration options
Among others, see https://fliegwerk.github.io/fliegdoc/fliegdoc/#FliegdocConfig for a full reference:

readme = './README.md': The path to the repo's README.md
outDir = './docs': The directory into which the docs get generated
baseUrl = '/': the root URL of the documentation. Useful when you want to deploy your projects to subdirectories on your server, e.g., on GitHub pages (where /[repo-name]/ gets prefixed to the URL, compared to /).

We can now run

$ npm run docs:serve

fliegdoc, when running fliegdoc serve, does a few things:

Read the config and apply its overrides to the default configuration
Go through the modules specified in the config and parse the exported members of their mainFile
Build a "documentation-ready syntax tree", consisting only of abstracted elements interesting for the docs, from the parsed AST
Serve a site that consists of pages for
- the project's README.md
- API references for the modules

When instead of docs:serve, we run the docs:build script, the only difference will be that in step (4.), the site won't just be served on an HTTP server, but instead, the HTML files get built and outputted to a folder ./docs, which then can get deployed to, e.g., GitHub Pages, Netlify, or a "classic" web server.

What fliegdoc isn't

It isn't an "all-around" documentation generator for Typescript. It specializes heavily (!) in documenting package modules and has a focus on being easy to understand (the "basic" project was developed in only one day).

Conclusion

While the project is still in a rather early stage of development, and documentation is, right now, WIP (the irony 🤣), it already serves its purpose in multiple of my own projects, without problems.

Perhaps, its use-case also fits your project, in which case, please feel free to share any feature requests should you decide to use it.

It was certainly interesting to dive a bit deeper into the ASTs and other "lower-level" Typescript things apart from everything else.

Author

Zuri Klaschka (they/them)Follow

Working Student, Creative Cloud Platform and Ecosystem Team @ Adobe; I do many things. Among them developing websites and plugins for Adobe products, primarily Adobe XD. Viva la Open-Source!

Pablo Klaschka is the first chairman and co-founder of fliegwerk, a group of students developing Open-Source projects.

DEV Community: Zuri Klaschka (they/them)

pnpm and Parcel based monorepo

The problem

A solution?

The repository

pklaschka / pnpm-parcel-monorepo-test

A repository for trying a JS/TS npm library monorepo setup using pnpm as package manager and parcel as a build tool.

pnpm & parcel monorepo test

Tech Stack

Usage

Setting up the development environment

Dependency Management

Installing

Results

Advantages

Drawbacks

A quick fix for a Parcel bug that almost made me dismiss it

Even more details

Author

Zuri Klaschka (they/them)Follow

And so, I wrote my own Typescript Documentation Generator...

The "Status Quo"

TypeStrong / typedoc

Documentation generator for TypeScript projects.

So... let's build it ourselves 😜

fliegwerk / fliegdoc

A documentation generator for Typescript-based libraries with good support for monorepos

Welcome to fliegdoc 👋

🏠 Homepage, Demo & Docs

Prerequisites

Install

Usage

CLI

Example fliegdoc.config.js with default options

What fliegdoc is

Demo

How to use it

What fliegdoc isn't

Conclusion

Author

Zuri Klaschka (they/them)Follow

`pnpm` & `parcel` monorepo test

Example `fliegdoc.config.js` with default options