Recently, I've published my very first library as an NPM module and I'd like to share my learnings and struggles with you.
The module I'm talking about is ella-math, which is a library for vector and matrix calculations.
These are useful for transformations in graphics programming, like scaling, moving and rotating shapes, or projecting things into 3d space.
Why?
To be honest, there really is no need to build my own vector/matrix calculation library. There are many of these already out there. But I wanted to do it anyway, just in order to learn how all this works.
Getting started
The language I've chosen to work with is TypeScript. I initialized the project from scratch:
mkdir ella-math
cd ella-math
git init
npm init
npm i typescript -D
./node_modules/.bin/tsc --init
In order to continue, I had to learn about the basics of the module systems used in JavaScript and by node.js and NPM.
Module systems
In the first place, NPM (and node.js) uses a module format that is called CommonJS. CommonJS syntax looks like this:
// a vector class
class Vec { /* ... */ }
// a matrix class
class Mat { /* ... */ }
module.exports = {
Vec,
Mat
};
In node.js, you can use the module by using require():
const { Vec } = require('ella-math');
const a = new Vec(1, 2, 3);
const b = new Vec(4, 5, 6);
const c = a.add(b); // (5, 7, 9)
Unfortunately, if you want to try to use it directly in the browser by embedding the library via <script> tag, that does not work, because the browser doesn't know about module or require.
One early approach to get a module system into the browser without having additional build steps was RequireJS. RequireJS also provides a require() function, but as the scripts are loaded asynchronously, the CommonJS syntax could not be used. So, requireJS came up with a module definition format that looks different and wasn't comaptible with CommonJS. It is called Asynchronous Module Definition (AMD). RequireJS isn't pretty common anymore and you will only find it in legacy projects. One selling point for RequireJS was that it even runs on IE6.
So, there were 2 different module formats out there. And sometimes, you don't want to use a module system at all and just use the library via a <script> tag.
A fix for this is called Universial Module Definition (UMD). The UMD format provides the best user experience because it supports
CommonJS, AMD and also an export via global variable when no module system is available.
The catch is, an UMD module looks quite messy:
(function (global, factory) {
typeof exports === 'object' && typeof module !== 'undefined' ? factory(exports) :
typeof define === 'function' && define.amd ? define(['exports'], factory) :
(global = global || self, factory(global.Ella = {}));
}(this, (function (exports) {
exports.Mat = Mat;
exports.Vec = Vec;
Object.defineProperty(exports, '__esModule', { value: true });
})));
Because this is a mess, the ECMAScript specification came up with another module system: ES modules (ESM).
It introduced the import and export keywords into the JavaScript language, which looks a lot cleaner:
export class Vec { /* ... */}
export class Mat { /* ... */}
On the consumer side:
import { Vec, Mat } from './ella-math.esm.js';
<script type="module" src="index.esm.js"></script>
Because this specification introduced a keywords, older browsers cannot understand this flavor of JavaScript.
To address this, there is a type="module" attribute that tells the browser the JavaScript is using the modern module system. Old browsers don't execute it at all (like IE11).
So, in an ideal world, you want to just use the modern module specification. But NPM uses CommonJS in the first place. Additionally, you may want to provide a way to directly use your library in the browser.
So, I ended up still using the UMD module format but also providing the modern ES module format.
Compiling to UMD and ESM
To compile my library to an UMD module, I am using Rollup alongside with the official TypeScript plugin. Rollup both supports the UMD format and the ES module specification out of the box, so I can provide both flavors.
The rollup configuration (rollup.config.js) looks like this:
import typescript from '@rollup/plugin-typescript';
export default {
input: 'src/ella.ts',
output: [
{
file: 'dist/ella.esm.js',
format: 'es',
},
{
file: 'dist/ella.umd.js',
format: 'umd',
name: 'Ella',
},
],
plugins: [typescript()],
};
My scripts section in the package.json looks like this:
{
"scripts": {
"test": "jest",
"docs": "typedoc && touch docs/.nojekyll",
"build:types": "tsc -t esnext --moduleResolution node -d --emitDeclarationOnly --outFile dist/ella.d.ts src/ella.ts",
"build:js": "rollup -c rollup.config.js",
"build:minjs": "terser dist/ella.umd.js --compress --mangle > dist/ella.umd.min.js",
"build": "npm run build:js -s && npm run build:minjs -s && npm run build:types -s"
}
}
Because the official rollup typescript plugin does not work with emitting type definitions, I'm using an additional build step to generate a .d.ts file containing all type definitions for TypeScript.
Currently, the declaration and sourceMap options in tsconfig throw an error when using @rollup/plugin-typescript.
There is a rewrite of the typescript plugin rollup-plugin-typescript2 that fixes the issue with type definitions and source maps, but I haven't tried it.
To also provide a minified bundle, I'm using terser which works quite well together with Rollup.
Additionally, I'm using typedoc to generate API documentation from code comments using JSDoc notation.
Specify files for publishing
To specify which files are going to be published, you specify all the things inside your package.json.
The entry file goes in the main field, the type definition goes into the types field. Finally, you define a files field with an array containing the files and folders to be included:
{
/* ... */
"main": "dist/ella.umd.js",
"types": "dist/ella.d.ts",
"files": ["src", "dist"]
/* ... */
}
Publishing on NPM
After you have created an account on NPM, you can log in to your account inside your command line shell:
npm login
Then, choose a package name and an initial version in your package.json:
{
"name": "ella-math",
"version": "1.0.0",
/* ... */
}
You will have to chose a package name that is unique. To finally
release your package on NPM, do
npm publish
Counting versions up
After you have made your changes, make sure everything is committed and you are on the main branch. npm provides a built in versioning count tool:
npm version major # v1.0.0 -> v2.0.0
npm version minor # v1.0.0 -> v1.1.0
npm version patch # v1.0.0 -> v1.0.1
# publish the new version
npm publish
Use major for breaking changes (every time your API changes).
The version command also creates a git tag for the specific version inside your repo. To push it to your remote repo, use:
git push
git push --tags
The released project
So, I must say, publishing a library on NPM that was created in TypeScript is not super straight forward.
You can check out the whole project at github:
https://github.com/terabaud/ella-math/
A demo using this library is on CodePen:
Top comments (14)
Good write up!
Is there a specific reason why you include the src folder in your package?
The way I do it is to only have dist-files in the npm package, and on the other hand gitignore those for the GitHub repo.
As in:
repo: sources and config only
package: dist only
Your consumers might enjoy the smaller package size if you leave out sources from the bundle :-)
Good point, I think excluding src from the package is the right way.
Initially, I thought of a scenario when using TypeScript, one could somehow import directly from the ts sources, and let the bundling be done by one's own transpiling toolchain, but that's out of scope of NPM, I guess. At least, there is no clean way to do that via NPM (yet).
cough deno.land/ cough :-)
Thanks for excellent article and background.
Do you think creating npm modules from Typescript is tedious?
Why was UMD the winner?
Are there easier ways if we just stick to esm2015?
Of course, I'd favorize ESM over UMD but I've had some issues using ESM. So, I stuck to UMD for now.
Main reason: Coding playgrounds like JSFiddle or CodePen. Although you can use ESM in Codepen, it does not work as soon as you select some sort of transpiler for your code (Babel, TypeScript, Coffeescript, ...). To get it work with TypeScript, you will need to include the script as an UMD module in the settings panel.
Also, the LTS version of node still displays the ExperimentalWarning.
I guess as soon it is safe to get rid of the UMD build, things may get easier.
Would you agree the whole JavaScript module system is a mess?
Angular has it's decorator class ngmodule which works but it seems difficult because error messages are very bad.
Great Work !
Just to say, I use to install my modules directly from GitHub with npm, in this case
Cool, didn't know about that :)
I have started a project in pure js and npm (not yarn) github.com/nazimboudeffa/vitaminx
Any chance for a tutorial on how to perform matrix operations ?
Best motivation ✌️
Epic
Good stuff! Perhaps I could use this for some machine learning tasks
I've just learnt about Documentation Generator today, but it seems you still have to write more TSDoc (or JSDoc).
Yes, it's not complete yet. Also, the jsdoc (or tsdoc) generates API documentation, which is nice for autocomplete and looking up functions in an API reference. But there should also be some kind of detailed documentation about how to get started using it.