What interesting things I can do with npm?

This thread made me think that I want to get a better grasp of what I can do through npm:

I knew about this feature and more or less know all the npm features, but I still feel a bit out of the loop about the ways people are using this tech/service. Care to fill me in on how you make use of the npm service and ecosystem?

Did you find this post useful? Show some love!
DISCUSSION (32)

My last uses of the npm were

  • lose 3 days to make a project work (because of x dependencies random errors)
  • spend hours and hours to "debug" errors like "undefined"
  • eat popcorn while browsing NPM's source and see how to not write code

Often this is the case when starting out. You may run into permissions issues too. I still think it is better than pip or gem any day.

Thereโ€™s a running joke that half the bandwidth on the internet is used by Netflix and the other half by npm. :D

I am a senior JS developer, I can find out why the script failed, but the code is so badly written that it takes tens of minutes to do the research. Also using a framework like Meteor (or other layer on top of node) adds a new layer of problems.

I found many issues on Github related to most of the problems I had, the problem is that many of the NPM developers (including the NPM contributors) only care about their local system, they do not use error handling and other common mistakes.

Most errors are "undefined at ..." because ... defensive programming, asserting and other techniques are obscure.

Of course, I cannot complain because they are for free :))

Sorry, I misunderstood earlier. What you said makes sense. Iโ€™ve had package specific issues a handful of times only. Maybe, Iโ€™m too conservative installing random packages. However, I tend to look at the popularity and star count before considering most packages. Anything less than popular, I start by looking at the issue tracker and code in some cases.

Are you still using Meteor? Is it phasing out?

Unfortunatelly the stars does not mean they work on windows ๐Ÿ˜€

I will not use Meteor ever again, is very proprietary, dependant on weird outdated packages and adds a thick layer of complexity that can be easily avoided.

It is very good for prototypes, proof of concepts and hackatons, after that it only gets worst.

Savage! But then, this truly reflects on the state of JavaScript: very popular, and very messy.

There is a new kid on the block. His name is Turbo. But I'm not familiar with neither npm nor with Turbo. I'm relying on you guys to check it out and tell me if it's stupid or not.

I'm on the same boat. 2 days in and still hasn't been successful resolving dependencies

I solved it by moving the project to Linux. Most of the issues were lower lvl libs.
Also copy pasting parts from dependencies into project and removing them.
You solve chaos with chaos ๐Ÿ˜€

I don't have the option to move it to Linux. I guess I have to go through this tedious process.

  1. NPM scripts - they automatically add "./node_modules/.bin" to your PATH when they are executed, so if you have mocha installed, for example, your NPM script can just be ..."test": "mocha ..."...
  2. NPX - use npx to run scripts in "./node_modules/.bin": npx mocha ... for one-off commands; if NPX cannot find the binary there, it will temporarily download it!
    • npx create-react-app is awesome
  3. npm ci - look it up it's pretty cool
  4. Use "devDependencies", and use it correctly!
  5. Use the "prepack" hook to run tests & build before your module is built!
  6. "npm pack" will build the *.tgz that NPM stores on their public registry. In many ways this is akin to Java WAR files, without all of the dependencies. You can then put this *.tgz on your server and do an "npm install my-package-0.0.1.tgz"
  7. "bundleDependencies" - worth knowing about (also a tool to use with it is: bundle-deps; run with "npx bundle-deps")

One of my pet-peeves is when a package is globally installed when it should be a devDependencies within a project. For example, if you use the TypeScript compiler in a project, "typescript" is a "devDependency"; do not make installing it globally a requirement. This lets different projects depend on different versions of the typescript compiler. It also makes it so that somebody can download your project and run a build without having to install additional dependencies. This goes for gulp, etc.

I also forgot a few:

  • npm link - useful if you depend on a development version of a package that only exists on your local machine
  • npm install /path/to/file - alternative way to accomplish the above point: recent versions of NPM just create a symlink! so you can edit the linked project live and have updates just like you would expect
  • npm audit - available in >=v6.x - runs a security audit on your dependencies
  • npm info - want to see what version of a package is the latest? Run npm info express dist-tags
  • npm install github.com/user/project#semver:^1.0.0
  • npm install some-package@next - install not the "latest" version, but the version tagged by "next"

NPM greater than version 5 is pretty amazing. Earlier versions, not so much. I would recommend yarn if you are stuck with earlier versions of NPM.

Oh, and don't forget that an .npmrc file local to your project overrides a global .npmrc file: useful for CI servers (store a .npmrc file with your project)!

Good point on the project specific .npmrc.

Another tip is that a separate repository (and credentials) can be configured per scope as well (credit Guillaume Martigny for mentioning scoped modules first below).

@<scope>:registry=http://host/repository/npm/private/
//host/repository/npm/private/:_password=<password>
//host/repository/npm/private/:username=<username>
//host/repository/npm/private/:email=<email
//host/repository/npm/private/:always-auth=true

This can be useful if you have some private modules in a private repository but do not wish to proxy all requests for public modules through it as well.

I also agree npm link is very useful if working on multiple modules and testing fixes.

One thing that comes to mind: you can use npm + unpkg to publish websites ๐Ÿ˜

Here's one not a lot of people know about.

npx comes bundled with npm 5.2.0+. It temporarily downloads then executes an npm package that's used as a script (such as create-react-app) without needing to install it anywhere on the machine.

A fun example if you want to try it out is cowsay, a little script that outputs an ASCII cow saying something:

> npx cowsay "Cows are the silent jury in the trial of mankind."
 __________________________________________
/ Cows are the silent jury in the trial of \
\  mankind.                                /
 ------------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Fantastic! Thanks for sharing Shawn!

I abuse the heck out of npm scripts. They aren't just for build, serve, and test!

For instance, as a SQA, I need to have packages like Jasmine and Protractor, and then use Protractor to kick off my tests and maintain my selenium package.

So in my package.json, I include in scripts:

  "scripts": {
    // snip
    "update-webdriver": "node node_modules/protractor/node_modules/webdriver-manager/bin/webdriver-manager update",
    "preprotractor": "npm run update-webdriver",
    "protractor": "node node_modules/protractor/bin/protractor path/to/conf/file",
    // snip
  }

So what happens when I run npm run protractor is:

  1. I get the latest selenium jar and chromedriver (and other drivers I don't care about like gecko but whatever)
  2. The local installation of protractor kicks off a test run with the default suite.

Things this buys me:

  1. Every single guide on protractor starts with npm i -g protractor. NO. Bad. I don't want every onboarding to be a laundry list of global packages to install. Just use project dev-dependencies and scripts that alias the unweildy bin paths. Now everyone's computers can be happy cookie cutters without needing local aliasing and global installs.
  2. Since scripts has a concept of pre and post runs, I can ensure everyone actually keeps chromedriver up to date with Chrome without having to keep tabs on that. It just works
  3. I can still override or include settings at runtime like npm run protractor -- --suite=api rather than always using the default.

I'm half tempted to put in a postprotractor for running npm run eslint -- --fix e2e-tests/** but I might hear complaints if I do that...

But I do this set up on every project I'm on to keep things nice and tidy.

Worth noting that you don't need to write node node_modules/.bin/protractor config - you can just use protractor config and npm will figure out the rest for you :)

Nifty! I'll try to remember that next time.

Don't want to be too much showy, but sub-packages are cool.

Basically, inside one repo, you have a main package that depend on smaller package. Check Pencil.js for a real-life example.
You don't have to use scoped packages, but I would recommend it.

Thanks to Lerna managing the whole thing is pretty easy.

I tried to like Lerna, but I canโ€™t seem to. The sub packages approach is a lot cleaner. Although, I donโ€™t recall what benefits lerna provides anymore given that npm supports linking and sub packages.

I used to use npm link, but when scope grows and dependency tree gets weirder, you can't continue to do everything by hand.
My favorite features from Lerna:

  • lerna bootstrap --hoist save space with hoist common dependencies.
  • lerna add whatever --scope=mySubPackage add a new dependency to a sub-package.
  • lerna publish Bump and publish all packages individually.

Thanks for the insights. I really appreciate it!

  1. Use npm pack <packagename> to download tgz file of what gets installed on your machine.
  2. Use npm view <packagename> to view the package.json of the package.

I use npm scripts to compile Pug and Sass, build my Hugo site, and optimize images. My Sam Hugo theme does something similar, without the images though.

Hey, lots of handy stuff in the comments but I noticed nobody has mentioned this yet. You can host your own NPM server and configure NPM to use your-registry.com in stead.

There are various packages on NPM itself (such as verdaccio) that offer a NPM server. Some are minimal, other are fully featured. You can use a custom NPM server not only to make your packages private (to your team) but also to serve as proxy, or have it fallback to NPM when your registry has no package by the name of what is being queried.

Checking if packages are up-to-date, I sometimes npm outdated to list packages' current version used, wanted version and latest version.

Best thing is that dev-dependencies are accessible like globals in npm scripts.

So you can run "npm i" and then simply run every script and use global packages without installing them globally.

Quick one: I recently made fastjs.link as a short linking service that links directly to the homepage of an npm package. Pretty useful for blogging on dev.to, actually!

Classic DEV Post from Sep 27

Who is hiring? (As of September 2018)

Members of the dev.to community sharing open job opportunities at their companies.

Ben Halpern
A Canadian software developer who thinks heโ€™s funny.

A blogging community of over 100,000 software developers Join dev.to