loading...
Cover image for I reclaimed 10GB of disk space from node_modules

I reclaimed 10GB of disk space from node_modules

irreverentmike profile image Mike Bifulco Originally published at mike.biful.co ・4 min read

I really don't like my laptop.

I've been using a 2017 Macbook Pro for the past couple years. As I'm sure you've heard, it is a fiercely disappointing device largely due to its courageous lack of ports, flawed keyboard design, and astonishing sticker price. My personal machine is a 13" version, with 16GB of memory, no touchbar, and a 256GB SSD. When I bought it, upgrading to 16GB of memory seemed far more important splurging on processor speed or disk space. That's probably still true - but I've found myself endlessly frustrated by the tiny capacity of the SSD in my day to day work.

Roughly once every 6 weeks, my machine grinds to a halt as I run out of disk space, so I have to suffer through all the usual suspects for clearing up some room:

  • empty the trash
  • delete unused applications
  • make sure all my photos are moved to my NAS, cloud storage, and offsite archives
  • remove old virtualbox VM images

A new kind of purge

Last week, however, I came across a surprising new way to free up some disk space: clearing out node_modules directories.

Curiosity hit me, and I searched my mac for folders named node_modules. This is what I found:

  • My machine has 7,731 total node_modules folders. Wow!
  • Under ~/src/**, where I keep all of my dev projects, I had a grand total of 7,719 node_modules
  • These folders contain hundreds of thousands of files (which should be no surprise), and took up nearly 10GB of disk space!

Turtles all the way down 🐒🐒🐒

That last number was a bit of a revelation to me at first. If you're unfamiliar, node_modules contains a copy of each of the npm packages which you add to your project (with npm install or yarn add, etc) on projects which use node. Each of them are pinned to a specific version, which is typically specified in package.json.

Somewhat logically, each of the dependencies in your project's node_modules directory may also have a node_modules directory, containing their dependencies, each pinned to a specific version (again, defined in the package.json file of that dependency).

This is where file and disk space bloat happen - the libraries you add to your project may each contain dozens of dependencies, each of which have dozens more, etc etc, all the way down to super basic node and C libraries which make up the fundamental building blocks of node.

(Mind you - This is oversimplifying things a bit - depending on how you set up npm or yarn, there is some optimization done here. If one version of a given npm module is required more than once by your dependency tree, it may only appear once.)

The fix

Actually, this was pretty easy to fix. I did a search for all of the node_modules folders under ~/src/**, and deleted all of them, right from finder (sorry, terminal apologists). I then emptied the trash on my mac, and I was 10GB richer.

bye bye, birdie

What did it break?

It's been a week since I've done this, and so far the only inkling of a downside is that when I revisit projects I'm actively working on, I need to run yarn install to recreate the node_modules directory there. I'm good with that.

A week later, I'm back up to 1209 node_modules directories

Can we do better?

There is an alternative to npm and yarn called pnpm, which claims to be a "Fast, disk space efficient package manager". Its primary feature looks promising:

one version of each module sounds great

I haven't tried it yet, but will certainly be giving it a shot.

Epilogue

I think it's fair to say that this is a problem that the folks at npm and Facebook (who maintain yarn) are also looking closely at. My guess is that some healthy competition will improve this disk space nightmare... hopefully before I'm forced to buy a laptop with 1TB of storage. πŸ₯³

More reading

I'm certainly not the first one to write about this. Some more great reading for you to check out, if you're interested:


Originally published at mike.biful.co.

Discussion

pic
Editor guide
Collapse
tunaxor profile image
Angel D. Munoz

I have been using pnpm primarily for more than a year
and lately I've been using it for monorepo's and works quite nice

now something that's funny but you may think it's a pnpm problem sometimes are that you need to install more devDependencies for the project to actually run (not always but I've seen it)
and this is because if a dependency requires lodash for example, with the flat npm node_modules, you can require lodash without even having it on your package.json (you can inspect the node_modules and you'll find it there)
this is not true with pnpm, you need to actually put it on your package.json to have access to it.

I think pnpm is often underrated, but it is actually a nice tool.

Collapse
sqlrob profile image
Robert Myers

Honestly, this feels like the correct behavior to me. The dependencies of another package are an implementation detail and if they change, your project could break.

Collapse
irreverentmike profile image
Mike Bifulco Author

Not arguing that at all - I'm totally onboard with it. The whole family of dependencies needs to be there (and locked to a version) for things to work repeatably. I just needed some disk space back. As long as you don't mind running yarn or npm install when you come back to working on a given project, this seems to be a perfectly functional way of working to me.

Edit: Just realized you weren't replying directly to my post! Ha - well, seems we may be on the same page anyway. 🍳