One of my biggest problems with JavaScript and Node.js dependency trees is that it's... never been super easy to understand what you've got and what you can do to remediate.
I've been exploring the npm ls API a bit more recently, and wanted to share some of the things I've found that I wish I'd known about over the last few years!
An Introduction to npm ls
If you're not familiar with npm ls, it's a command available with the npm CLI that will list dependencies that have been installed to node_modules. Additionally, it will return a non-zero exit code if the dependency tree that's resolved in node_modules is not what should be resolved from package.json.
Here's a quick example of npm ls from one of my projects, good-first-issue:
This command's output is 1339 lines long. See the full output in gist form here (it is too big to feel good about embedding it in this post!): https://gist.github.com/bnb/043d9d88820e3a5f31f0411e6ead141a
By just running npm install, I'll get 1337 modules in total. Yes, that's the real number with the module's current package.json – I'm just as surprised as you!
If you scroll through that list, you'll see a bunch of lines with deduped at the end. This means that npm was able to resolve a version of that module that met the requirements of multiple dependencies that require it to be installed. With my first-ever grep command (grep deduped npm-ls.txt -c), I was able to find the total number of modules that were deduped:
It turns out that of 1337 modules, 532 were successfully deduped. It's worth noting that every line with deduped is a module that didn't need to be installed because it was installed via another path that isn't marked as deduped. Given this context, we know that there were 805 modules installed in total.
Bare vs. --production vs. --development
Being able to understand our dependency tree better is awesome! That said, npm ls by itself will tell you the current state of the entire node_modules directory... if you care about what's going to production, it'd be nice to separate the dependencies that will be shipped to production from the devDependencies that are simply used to make your life as a developer easier.
From running npm ls --production on the same project, we get a... much smaller result:
This time, npm ls shows that we have only 110 modules. If we check for deduped with a slightly modified grep command, we'll see that 21 dependencies were deduped. Before deduping, npm ls --production in good-first-issue has a 12x reduction of modules when compared to the bare version of npm ls; after deduping, npm ls --production in good-first issue has a 9x reduction of modules when compared to the bare version of npm ls.
Understanding the modules introduced into production is fantastic and super valuable. But, what if you also want to understand your development dependencies?
Luckily, npm ls also provides a --development flag that allows you to understand the dependencies that are only used in development. There are a few more advanced use cases for this, most of which are geared toward helping developers like you and I understand what's being used locally and how it could be optimized.
Usage to find how a specific module is introduced into your project
One neat feature of npm ls is the ability to pass a package name as an argument to the command. For example, if I wanted to find all instances of graceful-fs in my dependency tree I can run npm ls graceful-fs which will spit out the following:
For me, this is a stand-out feature! I spend a lot of time exploring the security space within Node.js and the JavaScript module ecosystem. This particular feature is incredibly useful to me, as one of the most common ways that you'll introduce security vulnerabilities is through your dependency tree (rather than directly introducing them in package.json). Being able to surface all instances of a specific module you know is vulnerable is extremely useful, and this command makes that super simple.
Usage in CI/CD for JavaScript projects
In a recent discussion in the Node.js Package Maintenance team, usage of npm ls in CI/CD environments was raised as a possible best practice to ensure that the dependency tree that's being resolved by npm is entirely valid and will be able to run.
I'd not thought about this before, but it's an astoundingly good safeguard. Since npm ls will exit with a non-zero exit code if the dependency tree is invalid, the command effectively becomes a zero-effort safeguard in your CI/CD to make sure your dependency tree is resolving exactly how it should. Additionally, this idea can be combined with npm ls --production for production builds!
Fin
I've been exploring npm ls a bit over the past few days, so I wanted to share my knowledge with y'all. I'm totally sure there's still more utility I've not discovered in the command, and would absolutely love to hear about it if you've got any tips! Additionally, I'd love to know if you're going to start using npm ls more, and how you're planning on using it! 💖
Top comments (14)
Out of curiosity, if you are running
nmp installwithin your CI/CD pipeline, would inconsistencies betweenpackage.lockandpackage.jsonnot be picked up there? And if so, what’s the benefit of this?This is assuming that a
package-lock.jsonfile exists. Some projects (like most of mine) opt out of this because it adds maintainer burden for no tangible benefit, at least in the case of modules. It's definitely recommended for applications, but sincepackage-lock.jsondoesn't get published to the registry there's really very little point to keeping it around.That said, as far as I know (and I could totally be wrong!) you can still have unmet dependencies that wouldn't be caught between
package-lock.jsonandpackage.json.shrinkwrap.jsongets published :vIndeed it does, but it’s an antiquated approach that I try to keep out of my open-source packages.
IMO the cost of maintaining an
npm-shrinkwrap.jsonis higher than writing high-quality code that will be resilient enough to handle dynamic dependency resolution.If I am feeling especially picky about a certain module or set of modules, I’ll generally pin the versions in my projects’
package.jsonIt doesn't matter what kind of code you write if your dependencies introduce bugs or change published API with a patch version :D
You could not include those dependencies 😂
Be serious 😂
George, if you have inconsistencies between the package manifest and the package lock, an
npm installor ayarn installwill produce different install results. Meaning to say, the lockfile will not be used as the source of truth.Exactly for that you should actually use
npm ciin order to force the lockfile.I wrote about it in short here: dev.to/lirantal/so-you-think-youre...
Is there a way to
npm ls ...for below a version number? For example, if I donpm -ls lodashI get a huge tree, mostly all on the latest version. I just want to see where the non-latest versions of the package are.As far as I know, not currently. If you were absolutely dead-set on doing this, you could output the command text to a file and then grep that file (or use JSON output and then parse the JSON with
jq).I mostly live behind the corporate firewall and only pop out now and then. I have had similar issues when I want to discover what or how a module is used in an app. In the end I wrote mod-dep-mod. Which my friends and I find useful as you can run it locally and point it to github to search the dependency tree when you cannot actually install locally.
I appreciate the article, I often these pointers really helpful as we get comfortable with what we think the commands do and stop investigating, and I had not heard of dev.to either till this morning.
The gist below that line shows the same long list of dependencies. I guess you wanted to show a different gist?
I had to trim the first gist, since it was literally longer than the content of the post – added a link to the full gist at the bottom of the updated first gist. I'll update the post to include that in the content too!
Edit: Updated!
Great article. When I run
npm lsfrom the root project directory, I'm not getting the list of sub dependencies rather only the dependencies inpackage.json. Do you know why that's the case ?