No BS monorepo - Part 2

Join me in this post as I take my “Pedalboard” monorepo and automatically publish it to NPM using GitHub actions. I recommend you read the 1st part of this article to get more understanding on where this post picks off from.

This is the 2nd and final part of “No BS monorepo”. Since there was a lot to cover I decided to divide this topic into 2 parts:

• Part 1 (previous one) - where I build the monorepo using Yarn workspaces and Lerna to the point I can release a new version of my package
• Part 2 (this one) - where I automatically publish the monorepo's packages to NPM using GitHub actions

Since writing this article Lerna has announced that the project is no longer actively maintained, so I started followup series of articles on migrating from Lerna here.

So -

It is time to push what we have so far to GitHub. I can start by thinking about how to integrate GitHub actions in order to create a sort of CI process which will eventually deploy the new packages to NPM.

I’ve created the Pedalboard GitHub remote repository and uploaded the files as the initial commit.
Now it is time to create a GitHub action which will publish my packages to NPM. I’m creating a new action (by clicking on the “Actions” tab on my repository page), and from the suggested templates I select the “Publish Node.js Package”. This creates a .yml configuration file which describes the steps for the pipeline.

Currently, what I want it to do is to only run the tests of the packages. That means that I want my pipeline to install all the NPM dependencies and then run “yarn test” which launches the lerna run npm-script to run all the tests of all the packages. I know, It is not efficient at the moment, since it will run on all packages, even those which did not change, but it is good enough for now.

I’m also configuring my action to support manual run - that means that I will be able to launch this pipeline without needing a new commit to trigger it off. I do that by adding the “on: workflow_dispatch:” to the configuration.
I’m commenting out everything that relates to npm publishing (we will get to that later) and here is how my npm-publish.yml looks like now:

 # This workflow will run tests using node and then publish a package to GitHub Packages when a release is created
# For more information see: https://help.github.com/actions/language-and-framework-guides/publishing-nodejs-packages

name: Node.js Package

on:
 workflow_dispatch:
 release:
   types: [created]

jobs:
 build:
   runs-on: ubuntu-latest
   steps:
     - uses: actions/checkout@v2
     - uses: actions/setup-node@v2
       with:
         node-version: 16
     - run: yarn
     - run: yarn test

#   publish-npm:
#     needs: build
#     runs-on: ubuntu-latest
#     steps:
#       - uses: actions/checkout@v2
#       - uses: actions/setup-node@v2
#         with:
#           node-version: 16
#           registry-url: https://registry.npmjs.org/
#       - run: npm ci
#       - run: npm publish
#         env:
#           NODE_AUTH_TOKEN: ${{secrets.npm_token}}

Let’s test it - I’m going into the “actions” tab on GitHub and I see my newly created action:

Boom! It fails. Can you tell why?

Expected value to strictly be equal to:
      "Importing a namespace is not allowed for \"./forbidden/module\"."
    Received:
      "Importing a namespace is not allowed for \"./forbidden/module\". Please use a named import instead"

So if you remember part 1 of this article, I modified the phrasing of the error message the lint rule spits out when invalid, but I committed it without running the tests (😱 shame!) and now it fails. Did I do that on purpose? I guess we will never know, but one thing is for sure - we would like to fix it and then commit it so that GitHub action can run the tests again and hopefully it will pass.

Our GitHub action is currently set for trigger for the “workflow_dispatch” (manual) and ‘release” events, but I would like it to also trigger on “push” events. You can see the docs for all the available GitHub actions events here, and it has exactly what I’m looking for - “push”.
I will add that to my .github/workflows/npm-publish.yml:

# This workflow will run tests using node and then publish a package to GitHub Packages when a release is created
# For more information see: https://help.github.com/actions/language-and-framework-guides/publishing-nodejs-packages
name: Node.js Package
on:
push:
  branches:
    - master
workflow_dispatch:
release:
  types: [created]
jobs:
build:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-node@v2
      with:
        node-version: 16
    - run: yarn
    - run: yarn test
#   publish-npm:
#     needs: build
#     runs-on: ubuntu-latest
#     steps:
#       - uses: actions/checkout@v2
#       - uses: actions/setup-node@v2
#         with:
#           node-version: 16
#           registry-url: https://registry.npmjs.org/
#       - run: npm ci
#       - run: npm publish
#         env:
#           NODE_AUTH_TOKEN: ${{secrets.npm_token}}

This will make any push to the “master” branch trigger this action (merging a different branch to master also results in a “push”, so it should trigger the action as well). I’m good with that :)
Now I need to fix the test and push it. I give it the following conventional commit message:

fix: Failing tests due to eslint error message match

Since I don’t have any auto-version-bumping yet, I will do this manually by running npx lerna version, and then I will push all these changes, including the version bumping, to the remote repo.

In the very near future we won’t need to do that since GitHub action will do the bumping for us.

When the push reaches GitHub the action is triggered and the build now passes:

Nice.
Obviously the build phase can benefit from more steps (like eslinint for example) but for now having my action triggered as required is sufficient.

Now is the time to integrate the NPM publishing. Let’s take a look at the commented out part of the .yml file which takes care of that:

#   publish-npm:
#     needs: build
#     runs-on: ubuntu-latest
#     steps:
#       - uses: actions/checkout@v2
#       - uses: actions/setup-node@v2
#         with:
#           node-version: 16
#           registry-url: https://registry.npmjs.org/
#       - run: npm ci
#       - run: npm publish
#         env:
#           NODE_AUTH_TOKEN: ${{secrets.npm_token}}

Hmmm… I will need to change the npm commands to Yarn commands obviously, but there is something which will need more attention - As you can see there is an environment token that needs to exist on gitHub secrets in order for the action to authenticate with NPM. I don’t have such a token configured so let’s get at it -

publish-npm:
   needs: build
   runs-on: ubuntu-latest
   steps:
     - uses: actions/checkout@v2
     - uses: actions/setup-node@v2
       with:
         node-version: 16
         registry-url: https://registry.npmjs.org/
     - run: yarn
     - run: yarn publish:lerna
       env:
         NODE_AUTH_TOKEN: ${{secrets.npm_token}}

Following the instructions here I set the npm_token as a secret for my repository.

Your token should be of “automation” type if you have 2FA enabled (as you should) on your NPM account.

As you can see I’ve added a yarn publish script to my root project’s package.json file, let’s check how it looks like:

"scripts": {
       "test": "lerna run test",
       "publish:lerna": "lerna publish --yes"
   }

The name is “publish:lerna” and not “publish” since I can’t have it conflicting with Yarn's built-in “publish” command.
The “--yes” flag is for avoiding the prompt asking if I would like to publish this version.

If you have a namespace (scope) to your packages you need to make sure that you create an NPM “organization” for it otherwise you won’t be able to publish.

I will now push these changes to the repo and see what happens:

My publish fails 🤔

The reason is that in order to perform Lerna’s chore commit of the release (“chore(release): version v1.0.3”) git needs to know who is the author, and none is set, so let’s fix that -
I would like to set it to a bot, a GitHub bot user, which is a dummy user just for the sake of these commits. I’m doing that by adding these lines to the action .yml file:

- run: |
        git config --local user.name 'github-actions[bot]'
        git config --local user.email 'github-actions[bot]@users.noreply.github.com'

Let’s try it again now… and it fails again 😕

This time it’s because Lerna has trouble verifying the user when dealing with NPM automation tokens. This issue is well described here. I edit the root project’s package.json accordingly:

"scripts": {
       "test": "lerna run test",
       "publish:lerna": "lerna publish --yes --no-verify-access"
   },

And it fails again, what the... 😮
This time it is because the packages in the repo, unless configured differently, are set with the same access type of the root project. In my case it means private, so in order to solve that, I’m setting the nested package (the one I'd like to publish) to have a public access, in its package.json file like so:

"publishConfig": {
       "access": "public"
   }

Keep your fingers crossed, we’re trying again…
And we have it!

...
Successfully published:
lerna success published 1 package
- @pedalboard/eslint-plugin-craftsmanlint@1.0.7
Done in 5.29s.

Since NPM takes some time to update, I can try the following command on the terminal and sure enough I see my published package:

npm show @pedalboard/eslint-plugin-craftsmanlint

@pedalboard/eslint-plugin-craftsmanlint@1.0.7 | MIT | deps: none | versions: 1
A set of ESlint rules for your code craftsmanship
...

And that’s it :)

Phew… that was some bumpy ride but we got there eventually, right?
The goals set at the beginning were accomplished - we have a monorepo on GitHub which automatically publishes versions (according to conventional commits) when changes are pushed to it.
Obviously, there is still a lot more to do in order to improve this monorepo but these steps described here and in the 1st part of this article will get you well started.

The Pedalboard GitHub repo is public, so drop by and have a look inside ;)

As always, if you have any ideas on how to make this better or any other technique, be sure to share with the rest of us!

Hey! If you liked what you've just read check out @mattibarzeev on Twitter 🍻

Photo by Jacob Bentzinger on Unsplash