How to use Verdaccio with GitHub registry

I've been asked for this couple of times and I want to share how you can achieve a seamless integration GitHub with Verdaccio. Node.js package managers only allow using one registry when you are running an eg: npm install unless you modify the .npmrc and add some specific configuration, but frankly, we can do better using a proxy.

Generating the Token at GitHub

First of all, we need to understand the GitHub registry is not a conventional registry, it does not support all npm commands you are get used to (eg: npm token).

I'd recommend you read first the official documentation at GitHub how to use packages.

Once you have set up and created a personal token in their User Interface (remember you cannot use npm adduser). Copy the token from the website and proceed to log in to your terminal.

$ npm login --registry=https://npm.pkg.github.com
> Username: USERNAME
> Password: TOKEN

The last thing is recovering the token generated by the GitHub registry in the ~/.npmrc file and find the line to verify npm you can use npm commands against GitHub registry.

//npm.pkg.github.com/:_authToken=TOKEN`.

One optional step is to publish a package, I have already one published one for my example below.

This step is required if you have not published packages, otherwise, you don't need to log in, just copy the token.

Great, you have a token and that's all you need for Verdaccio.

Installing Verdaccio

Let's imagine you don't know anything about Verdaccio. So here is what it does.

Verdaccio is a lightweight private proxy registry build in Node.js

and with straightforward installation, with no dependencies aside to have installed Node.js.

npm install --global verdaccio

to run Verdaccio just run in your terminal,

➜ verdaccio
 warn --- config file  - /Users/user/.config/verdaccio/config.yaml
 warn --- Verdaccio started
 warn --- http address - http://localhost:4873/ - verdaccio/4.5.0

for further information I'd recommend read our documentation.

For this article, we will focus on the proxy, which is the most powerful and popular feature by far.

Hooking the GitHub registry

First of all, you need a published package in the registry, here is mine and as you can see GitHub only support scoped packages.

This example is about how to fetch packages from npmjs and GitHub registries at the same time without modify the .npmrc file.

Uplinks

Open the verdaccio configuration file (eg: /Users/user/.config/verdaccio/config.yaml) and update the uplinks section adding a new registry.

uplinks:
  npmjs:
    url: https://registry.npmjs.org/
  github:
    url: https://npm.pkg.github.com
    auth:
      type: bearer
      token: xxxx

For demonstration purposes let's copy the token in the example above, populate the config file with token is not the best approach, I recommend using environment variables with auth property, read more about it here.

Package Access

To install packages, we need the list of dependencies in your package.json file. Here is my example:

  "dependencies": {
    "@types/babel__parser": "7.1.1",
    "@juanpicado/registry_test": "*",
    "lodash": "*"
  }

If you recall, I've published a package in my GitHub profile named registry_test, but GitHub requires to access my public package scoped with my user name, that would be @juanpicado/registry_test.

To make it more interesting, I also added a random published public package published by another user named @types/babel__parser.

The next step is setting up the package access section:

packages:
  '@juanpicado/*':
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: github
  '@types/babel__parser':
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: github 
  '@*/*':
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: npmjs
  '**':
    access: $all
    publish: $authenticated
    proxy: npmjs

As we describe in the packages documentation, the order is important. Define the scoped packages you want to match on top of '@*/*' and define the proxy properties to the name used in the uplink section, for our example would be proxy: github.

With such configuration, Verdaccio will be able to route the request to the right remote.

 http --> 200, req: 'GET https://registry.npmjs.org/lodash' (streaming)
 http --> 200, req: 'GET https://registry.npmjs.org/lodash', bytes: 0/194928
 http <-- 200, user: null(127.0.0.1), req: 'GET /lodash', bytes: 0/17599
 http <-- 200, user: null(127.0.0.1), req: 'GET /lodash', bytes: 0/17599
 http --> 200, req: 'GET https://npm.pkg.github.com/@types%2Fbabel__parser' (streaming)
 http --> 200, req: 'GET https://npm.pkg.github.com/@types%2Fbabel__parser', bytes: 0/1113
 http --> 200, req: 'GET https://npm.pkg.github.com/@juanpicado%2Fregistry_test' (streaming)
 http --> 200, req: 'GET https://npm.pkg.github.com/@juanpicado%2Fregistry_test', bytes: 0/2140
 http <-- 200, user: null(127.0.0.1), req: 'GET /@types%2fbabel__parser', bytes: 0/708
 http <-- 200, user: null(127.0.0.1), req: 'GET /@types%2fbabel__parser', bytes: 0/708
 http <-- 200, user: null(127.0.0.1), req: 'GET /@juanpicado%2fregistry_test', bytes: 0/911
 http <-- 200, user: null(127.0.0.1), req: 'GET /@juanpicado%2fregistry_test', bytes: 0/911

As we can observe if we have a close look at the server output.

• lodash is routed through -> https://registry.npmjs.org/ .
• "@types/babel__parser": "7.1.1" is routed through -> https://npm.pkg.github.com/@types%2Fbabel__parser.
• "@juanpicado/registry_test": "*" is routed through https://npm.pkg.github.com/@juanpicado%2Fregistry_test'..

Verdaccio is able to handle as many remotes you need, furthermore, you can add two proxy values as a fallback in case the package is not being found in the first option.

packages:
  '@juanpicado/*':
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: npmjs github

Verdaccio will try to fetch from npmjs and if the package fails for any reason will retry on github. This scenario is useful if you are not 100% sure whether the package is available or not in a specific registry. As a downside, if you add multiple proxies will slow down the installations due to the multiple looks up have to perform.

http --> 404, req: 'GET https://registry.npmjs.org/@juanpicado%2Fregistry_test' (streaming)
 http --> 404, req: 'GET https://registry.npmjs.org/@juanpicado%2Fregistry_test', bytes: 0/21
 http --> 200, req: 'GET https://npm.pkg.github.com/@juanpicado%2Fregistry_test' (streaming)
 http --> 200, req: 'GET https://npm.pkg.github.com/@juanpicado%2Fregistry_test', bytes: 0/2140
 http <-- 200, user: null(127.0.0.1), req: 'GET /@juanpicado%2fregistry_test', bytes: 0/908
 http <-- 200, user: null(127.0.0.1), req: 'GET /@juanpicado%2fregistry_test', bytes: 0/908

One more thing

During writing this blog post, I've noticed all files retrieved from the GitHub registry are not tarballs like those that come from other registries which always finish with the suffix *.tgz.

Wrapping up

Verdaccio is a powerful lightweight registry that can be used in multiple ways, you can find more about it in our website. This project is run by voluntaries and you can also be part of it.

If you would like to donate, it can be done through OpenCollective, help us to reach more developers to have a sustainable Node.js registry.

Thanks for using Verdaccio and please, keep safe, stay at home and wash your hands regularly.