What interesting things I can do with npm?

This thread made me think that I want to get a better grasp of what I can do through npm:

Who uses private NPM publishing?

Joshua Johnson ・ Jun 8 '18

#discuss

I knew about this feature and more or less know all the npm features, but I still feel a bit out of the loop about the ways people are using this tech/service. Care to fill me in on how you make use of the npm service and ecosystem?

Comments

[Adrian B.G.]

My last uses of the npm were

• lose 3 days to make a project work (because of x dependencies random errors)
• spend hours and hours to "debug" errors like "undefined"
• eat popcorn while browsing NPM's source and see how to not write code

[Nick Karnik]

Often this is the case when starting out. You may run into permissions issues too. I still think it is better than pip or gem any day.

There’s a running joke that half the bandwidth on the internet is used by Netflix and the other half by npm. :D

[Adrian B.G.]

I am a senior JS developer, I can find out why the script failed, but the code is so badly written that it takes tens of minutes to do the research. Also using a framework like Meteor (or other layer on top of node) adds a new layer of problems.

I found many issues on Github related to most of the problems I had, the problem is that many of the NPM developers (including the NPM contributors) only care about their local system, they do not use error handling and other common mistakes.

Most errors are "undefined at ..." because ... defensive programming, asserting and other techniques are obscure.

Of course, I cannot complain because they are for free :))

[Nick Karnik]

Sorry, I misunderstood earlier. What you said makes sense. I’ve had package specific issues a handful of times only. Maybe, I’m too conservative installing random packages. However, I tend to look at the popularity and star count before considering most packages. Anything less than popular, I start by looking at the issue tracker and code in some cases.

Are you still using Meteor? Is it phasing out?

[Adrian B.G.]

Unfortunatelly the stars does not mean they work on windows 😀

I will not use Meteor ever again, is very proprietary, dependant on weird outdated packages and adds a thick layer of complexity that can be easily avoided.

It is very good for prototypes, proof of concepts and hackatons, after that it only gets worst.

[Jochem Stoel]

Anything less than popular, I start by looking at the issue tracker and code in some cases.

I used to do that but over time found that sometimes you need something very specific, hasn't been downloaded in weeks and only starred by the author that is a perfect unopinionated and applauseworthy documented.

Also the more people involved the more they pollute it with opinions and distractions. I don't mean that applies universally but look at what we did to JavaScript.

[Giorgos Kontopoulos 👀]

Also node_modules folders probably uses half of the GB of a developers hard drive ;-)

[Anton]

There is a new kid on the block. His name is Turbo. But I'm not familiar with neither npm nor with Turbo. I'm relying on you guys to check it out and tell me if it's stupid or not.

[kiranjd]

I'm on the same boat. 2 days in and still hasn't been successful resolving dependencies

[Adrian B.G.]

I solved it by moving the project to Linux. Most of the issues were lower lvl libs.
Also copy pasting parts from dependencies into project and removing them.
You solve chaos with chaos 😀

[kiranjd]

I don't have the option to move it to Linux. I guess I have to go through this tedious process.

[Nick Karnik]

Why don’t you try docker?

[Jonathan Apodaca]

1. NPM scripts - they automatically add "./node_modules/.bin" to your PATH when they are executed, so if you have mocha installed, for example, your NPM script can just be ..."test": "mocha ..."...
2. NPX - use npx to run scripts in "./node_modules/.bin": npx mocha ... for one-off commands; if NPX cannot find the binary there, it will temporarily download it!
   • npx create-react-app is awesome
3. npm ci - look it up it's pretty cool
4. Use "devDependencies", and use it correctly!
5. Use the "prepack" hook to run tests & build before your module is built!
6. "npm pack" will build the *.tgz that NPM stores on their public registry. In many ways this is akin to Java WAR files, without all of the dependencies. You can then put this *.tgz on your server and do an "npm install my-package-0.0.1.tgz"
7. "bundleDependencies" - worth knowing about (also a tool to use with it is: bundle-deps; run with "npx bundle-deps")

One of my pet-peeves is when a package is globally installed when it should be a devDependencies within a project. For example, if you use the TypeScript compiler in a project, "typescript" is a "devDependency"; do not make installing it globally a requirement. This lets different projects depend on different versions of the typescript compiler. It also makes it so that somebody can download your project and run a build without having to install additional dependencies. This goes for gulp, etc.

[Jonathan Apodaca]

I also forgot a few:

• npm link - useful if you depend on a development version of a package that only exists on your local machine
• npm install /path/to/file - alternative way to accomplish the above point: recent versions of NPM just create a symlink! so you can edit the linked project live and have updates just like you would expect
• npm audit - available in >=v6.x - runs a security audit on your dependencies
• npm info - want to see what version of a package is the latest? Run npm info express dist-tags
• npm install github.com/user/project#semver:^1.0.0
• npm install some-package@next - install not the "latest" version, but the version tagged by "next"

NPM greater than version 5 is pretty amazing. Earlier versions, not so much. I would recommend yarn if you are stuck with earlier versions of NPM.

[Jonathan Apodaca]

Oh, and don't forget that an .npmrc file local to your project overrides a global .npmrc file: useful for CI servers (store a .npmrc file with your project)!

[mbtts]

Good point on the project specific .npmrc.

Another tip is that a separate repository (and credentials) can be configured per scope as well (credit Guillaume Martigny for mentioning scoped modules first below).

@<scope>:registry=http://host/repository/npm/private/
//host/repository/npm/private/:_password=<password>
//host/repository/npm/private/:username=<username>
//host/repository/npm/private/:email=<email
//host/repository/npm/private/:always-auth=true

This can be useful if you have some private modules in a private repository but do not wish to proxy all requests for public modules through it as well.

I also agree npm link is very useful if working on multiple modules and testing fixes.

[Tim Ermilov]

One thing that comes to mind: you can use npm + unpkg to publish websites 😁

[Shawn Reisner]

Here's one not a lot of people know about.

npx comes bundled with npm 5.2.0+. It temporarily downloads then executes an npm package that's used as a script (such as create-react-app) without needing to install it anywhere on the machine.

A fun example if you want to try it out is cowsay, a little script that outputs an ASCII cow saying something:

> npx cowsay "Cows are the silent jury in the trial of mankind."

 __________________________________________
/ Cows are the silent jury in the trial of \
\  mankind.                                /
 ------------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

[Victoria Drake]

Fantastic! Thanks for sharing Shawn!

[ItsASine (Kayla)]

I abuse the heck out of npm scripts. They aren't just for build, serve, and test!

For instance, as a SQA, I need to have packages like Jasmine and Protractor, and then use Protractor to kick off my tests and maintain my selenium package.

So in my package.json, I include in scripts:

  "scripts": {
    // snip
    "update-webdriver": "node node_modules/protractor/node_modules/webdriver-manager/bin/webdriver-manager update",
    "preprotractor": "npm run update-webdriver",
    "protractor": "node node_modules/protractor/bin/protractor path/to/conf/file",
    // snip
  }

So what happens when I run npm run protractor is:

1. I get the latest selenium jar and chromedriver (and other drivers I don't care about like gecko but whatever)
2. The local installation of protractor kicks off a test run with the default suite.

Things this buys me:

1. Every single guide on protractor starts with npm i -g protractor. NO. Bad. I don't want every onboarding to be a laundry list of global packages to install. Just use project dev-dependencies and scripts that alias the unweildy bin paths. Now everyone's computers can be happy cookie cutters without needing local aliasing and global installs.
2. Since scripts has a concept of pre and post runs, I can ensure everyone actually keeps chromedriver up to date with Chrome without having to keep tabs on that. It just works
3. I can still override or include settings at runtime like npm run protractor -- --suite=api rather than always using the default.

I'm half tempted to put in a postprotractor for running npm run eslint -- --fix e2e-tests/** but I might hear complaints if I do that...

But I do this set up on every project I'm on to keep things nice and tidy.

[Tim Ermilov]

Worth noting that you don't need to write node node_modules/.bin/protractor config - you can just use protractor config and npm will figure out the rest for you :)

[ItsASine (Kayla)]

Nifty! I'll try to remember that next time.

[Guillaume Martigny]

Don't want to be too much showy, but sub-packages are cool.

Basically, inside one repo, you have a main package that depend on smaller package. Check Pencil.js for a real-life example.
You don't have to use scoped packages, but I would recommend it.

Thanks to Lerna managing the whole thing is pretty easy.

[Nick Karnik]

I tried to like Lerna, but I can’t seem to. The sub packages approach is a lot cleaner. Although, I don’t recall what benefits lerna provides anymore given that npm supports linking and sub packages.

[Guillaume Martigny]

I used to use npm link, but when scope grows and dependency tree gets weirder, you can't continue to do everything by hand.
My favorite features from Lerna:

• lerna bootstrap --hoist save space with hoist common dependencies.
• lerna add whatever --scope=mySubPackage add a new dependency to a sub-package.
• lerna publish Bump and publish all packages individually.

[Nick Karnik]

Thanks for the insights. I really appreciate it!

[Darryl Norris]

npm install -g yarn

[Aravind Balla]

1. Use npm pack <packagename> to download tgz file of what gets installed on your machine.
2. Use npm view <packagename> to view the package.json of the package.

[Guo-Guang]

Checking if packages are up-to-date, I sometimes npm outdated to list packages' current version used, wanted version and latest version.

[Victoria Drake]

I use npm scripts to compile Pug and Sass, build my Hugo site, and optimize images. My Sam Hugo theme does something similar, without the images though.

[Jochem Stoel]

It might be interesting to some that NPM is open. You can take it a step further than private packages and host your own NPM server that works out of the box with the npm command line tool we all know and love by simply setting a variable to a different host than npm.org somewhere.

[Mihail Malo]

My favorite feature of NPM (the repository) is that I can use pnpm instead of the default client.

[Jochem Stoel]

Hey, lots of handy stuff in the comments but I noticed nobody has mentioned this yet. You can host your own NPM server and configure NPM to use your-registry.com in stead.

There are various packages on NPM itself (such as verdaccio) that offer a NPM server. Some are minimal, other are fully featured. You can use a custom NPM server not only to make your packages private (to your team) but also to serve as proxy, or have it fallback to NPM when your registry has no package by the name of what is being queried.

[K]

Best thing is that dev-dependencies are accessible like globals in npm scripts.

So you can run "npm i" and then simply run every script and use global packages without installing them globally.

[Joshua Nelson ✨]

Quick one: I recently made fastjs.link as a short linking service that links directly to the homepage of an npm package. Pretty useful for blogging on dev.to, actually!