What's taking so much storage on a dev computer ?(and how to fix it !)

Why this article

Hi everyone, today I want to talk about something that I just discovered after 10 years in the dev industry (as a software engineer). Until now, I wasn't the kind of person looking after the 'remaining disk space', but as I logged into my computer this morning, an alert popped up, telling me that I needed to free some disk space because I was running low. So here started my investigation !

The problem

So first things first, I needed something to visualize what was taking space on my computer.
As anyone would have done, I used the builtin storage visualizer available on my computer, but here started the problems. Builtin tools like that are limited, most of the time, they only try to give you an overview of your storage usage by grouping your files, applications, photos into bigger categories (like Documents, Apps...) and there is the problem, if a file doesn't fit the predefined category schema, then it falls into a global category called 'Other files' and you can't know which files are concerned by this 'filter'.
As a developer (and absolute console interface lover), I tried the good old way: du -h /path/to/folder, but as you can imagine this would have take a lot of time and an extensive knowledge of my computer folders and files architecture (which with all these years is starting to get a little bit messy)

The right tools bring great insights

So after long hours shelling my storage space, I decided to change my technique and I went online looking for a tool that could help in this time consuming task, few came up:

• CleanMyMac or CleanMyPC (for Windows' users). Fantastic interface, super simple to use and a panel of functionalities that exceeds by far all other known software in this category. But such capacities come for a price and for such a small task as finding what took so much space on my computer I wanted to find a free tool. I might be interested one day in paying a subscription for the kind of features that CleanMyMac proposes, but for now I need to find another tool.

• DaisyDisk: Even though it's not free, it was worth mentioning this software because of it's trial plan. It comes as a simple interface with on the left side an interactive colorful wheel that helps you navigate through your computer's folders and on the right side you can see the files located in the currently selected 'folder'. The free plan will for sure help you sweep away useless data but it's limited thus not interesting for our case.

• OmniDiskSweeper: Last arrow in my quiver, it is the one I was looking for: completely free, great user interface (that reminds me of the Finder), boosted on steroids when it comes to calculate the weight of your computer's folders. He is a hole in one for me.

My Discoveries

So if you are like me, you don't restart that often your computer and because of it caches' folders are getting bigger and bigger over time. As an example my computer Cache folder was taking 8 GB of storage. But that is nothing compare to my next discovery. When looking through OmniDiskSweeper, I've found out that the folder located at ~/.npm/_cacache was taking 20GB of storage. It's the result of years of npm install commands that kept building some secure backups of libraries that I used for my different projects as an 'integrity package validator'. I had no choice: I needed to clear that cache but I needed to be cautious, I didn't want to compromise my node and npm installation.

How to fix your npm consuming storage problem

Simple enough, when you look at npm's documentation there is an interesting command npm cache verify that is supposed to check and compress your npm cache data. After doing as told, the cache reduced to 17GB but it still was too much. I decided to use another command: npm cache clean. As you can understand, by executing this command it will delete the _cacache folder's content, thus claiming back our 17GB storage space. If you try to run this command as of npm@5 you should get the following error:

npm ERR! As of npm@5, the npm cache self-heals from corruption issues and data extracted from the cache is guaranteed to be valid. If you want to make sure everything is consistent, use 'npm cache verify' instead. On the other hand, if you're debugging an issue with the installer, you can use `npm install --cache /tmp/empty-cache` to use a temporary cache instead of nuking the actual one.
npm ERR! 
npm ERR! If you're sure you want to delete the entire cache, rerun this command with --force.

npm ERR! A complete log of this run can be found in:
npm ERR!     ~/.npm/_logs/2022-01-19T10_45_45_597Z-debug.log

No worries, as they tell you on npm's documentation website the reason for this error is:

npm stores cache data in an opaque directory within the configured cache, named _cacache. This directory is a cacache-based content-addressable cache that stores all http request data as well as other package-related data. This directory is primarily accessed through pacote, the library responsible for all package fetching as of npm@5.
All data that passes through the cache is fully verified for integrity on both insertion and extraction. Cache corruption will either trigger an error, or signal to pacote that the data must be refetched, which it will do automatically.

So what do we do ?
As they tell us: if we are looking for reclaiming storage space we can run the command npm cache clean with the parameter --force in order to overrule this new 'security' that came with npm@5. So the command becomes: npm cache clean --force.
After execution, you should see that all the storage eaten up by npm cache should be restored and that's how, by deleting my computer old caches's folders I claimed back more than 30GB of storage.

Conclusion

As you can see, most of the time we are missing crucial informations when it comes to managing storage on our computers. Even though the size of our HDD or SSD is getting bigger, the programs, applications and tools we use are also getting bigger and bigger. By getting the right storage analysis tool, we can definitely claim back a lot of storage without the hassle of figuring out what is taking so much space on our computers.
If you liked this (first) article, please consider letting a like or a comment, t will for sure help me making more and better content.
Hope you enjoyed !
See you next time

Useful links

• CleanMyMac
• CleanMyPC (to note that they are no longer maintaining nor updating this app)
• DaisyDisk
• OmniDiskSweeper
• OSXDaily article on useful storage manager tools

A little bit more about me:

I'm Lenny, ex software engineer, now CTO in a growing startup focused on Consulting for helping people to develop and grow their ideas into a reliable, rewarding and profitable business.
Here are some links to my networks:

• GitHub
• Linkedin
• Dev.to