DEV Community

Cover image for A Complete Guide to Removing Linux Disk Bloat
Zied Kharrat
Zied Kharrat

Posted on

A Complete Guide to Removing Linux Disk Bloat

#devops #linux #webdev #cloud

Introduction

If you're reading this, you probably use Linux pretty regularly. Who wouldn't? For anything software, Linux makes our lives much simpler. 

However, chances are that your Linux instance suffers from terrible bloat due to accumulated months (years?) of apt install like there's no tomorrow, chunky Docker images that you forgot about, and that one web development project from your college class that, while not very flashy, is still home to a 500 MB node_modules folder.

This will inevitably lead to slower performance, longer boot times, and overall system instability. Before you know it, everything is in the red and screaming at you for help.

Well, the good news is that it can be fixed.

This guide will feature cleaning tips for native Linux users as well as WSL2 users, specifically Ubuntu 24.04.

Before we begin, please keep in mind that most deletions here are irreversible, so it is wise to backup any important data and double-check before proceeding.

So without further ado, let's dive in!

What causes the bloat?

I've identified 5 main issues that cause disk space loss on Linux environments. Chances are you have at least 3 of these:

  1. Unused Docker images: By far the most common, they can span several GBs in size, and are very easy to forget about

  2. Old/orphaned apt packages: Remember that obscure library you used for that one lab? It's still very much alive and well in your system

  3. Old project repos: Especially JS-based projects, node_modules folders can get very heavy

  4. Accumulated log files: You'd be amazed to know that daemons like systemd and sshd log anything and everything

  5. Cache and temporary files: apt cache accumulates over time, so do temporary files, .tar archives, etc…

Fixing the problem 

In this section, we will fix each of the previously mentioned issues, one by one:

1. Unused Docker images

It's important to first distinguish between an unused image and a dangling image. 

A dangling image is untagged and is not referenced by any container. Unused images include dangling images, plus tagged images that no container is using (careful, a stopped container STILL uses its image!).

You can start by taking a look at all your images:

docker images
Enter fullscreen mode Exit fullscreen mode

Your output will look similar to this. You can see that I have several heavy images on my system.

Docker images

You can then run the following command to take a look at your containers (omitting the PORTS column for better readability):

docker ps -a --format "table {{.ID}}\t{{.Image}}\t{{.Command}}\t{{.CreatedAt}}\t{{.Status}}\t{{.Names}}"
Enter fullscreen mode Exit fullscreen mode

Docker containers

You can see that I have several stopped containers from weeks ago, leftover from labs, old builds, and more.

Now that you've taken a look, we can start cleaning! Start by running this:

docker container prune -a
Enter fullscreen mode Exit fullscreen mode

You will be prompted to confirm with [y/N]. Careful! This will delete all stopped containers; make sure there aren't any you're using!

Now let's list containers again; all except one are gone!

Docker containers

Now we can remove the unused images, run the following: (Note that the -a flag will remove all unused images, without it we only remove dangling images).

docker image prune -a
Enter fullscreen mode Exit fullscreen mode

You can see that all is gone except the kindest/node image (attentive readers will notice that while it's untagged, it's still being used by a container, so it's not dangling). Oh and we reclaimed 2 GB in space, woohoo!

Bonus: Unused Docker volumes!

Docker volumes are persistent storage areas that Docker manages outside a container's writable layer. Note that they remain even after a container is deleted.

Run the following to learn about volumes in your system:

docker volume ls
Enter fullscreen mode Exit fullscreen mode

You will see something like this:

Docker volumes

If you're unsure about deleting a volume, you can run the following to learn more about it:

docker volume inspect <volume_name>
Enter fullscreen mode Exit fullscreen mode

After you're done checking, run this command to remove all unused volumes (we can add the -f flag to skip the confirmation prompt, but be careful as
this will irreversibly delete unused volumes).

docker volume prune -af
Enter fullscreen mode Exit fullscreen mode

You can see that we reclaimed a solid 343 MB, nice!

Finally, let's look at our Docker filesystem's disk usage run this:

docker system df
Enter fullscreen mode Exit fullscreen mode

Looks like there's nothing left to reclaim here; we can move on to the next step!

2. Old/orphaned apt packages

It's so easy to install packages on Linux-based systems, which is why they accumulate so quickly over time. Let's take a look at our manually installed packages, run this:

apt-mark showmanual | while read pkg; do
  size=$(dpkg-query -W --showformat='${Installed-Size}\n' "$pkg" 2>/dev/null)
  if [ -n "$size" ]; then
    size_mb=$(echo "scale=2; $size/1024" | bc)
    echo -e "$pkg\t${size_mb} MB"
  fi
done | sort -k2 -nr
Enter fullscreen mode Exit fullscreen mode

This very neat script will retrieve your manually installed packages as well as their sizes in kilobytes, convert these sizes to megabytes, then display each package with its size in descending order. Here's a sample output:

Script output

It's up to you to decide which packages are worth deleting. When you do, just run this:

sudo apt purge pkg1 pkg2 pkg3
Enter fullscreen mode Exit fullscreen mode

Note that we can use remove instead of purge, which keeps the package's config files (useful if you think you'll need to reinstall it later).

Finally, we can run the following command:

sudo apt autoremove
Enter fullscreen mode Exit fullscreen mode

This will clean up dependencies installed for packages you just removed.

That's pretty much it for apt packages, we can move onto the next section!

3. Old project repos

This one is pretty straightforward, you've probably worked on some projects along the way (or else why are you here?).

Most of these projects will have dependencies or virtual environments that can get pretty heavy (especially JavaScript-based projects and Python virtual environments).
 
All you need to do is navigate to the directory where your projects are located. Then, you can use the command below to know a folder's size before deciding to delete it.

du -sh path/to/directory
Enter fullscreen mode Exit fullscreen mode

For example, I've contributed to Grafana, a big open-source project, you can see that the repo size is pretty significant (1.2 GB!).

Command output

All you have to do is use the following command to delete unused projects (but make sure to commit them to version control first!). Always double-check before using rm -rf, unless you want bad surprises!

rm -rf path/to/directory
Enter fullscreen mode Exit fullscreen mode

Obviously, this applies to all directories, not just projects, so make sure to check other folders as well. Anyway, next section!

4. Accumulated log files

Linux logs everything: System events, logins, cron jobs, application logs, and many more. It's useful for debugging, monitoring and auditing, but it can get pretty heavy. Most logs live under /var/log, so we'll focus on that directory, let's run the following command:

sudo du -sh /var/log
Enter fullscreen mode Exit fullscreen mode

As you can see below, the logs can get pretty heavy.

Logs size

Now, before you go ahead and rm -rf that directory to oblivion, note that many services depend on the files in that directory to exist. And while it will probably be recreated eventually, said recreation isn't guaranteed to be immediate. Plus, daemon behavior is variable, so permissions and ownership can change, which will cause further access and security issues.

What we can do instead is use the command below:

sudo truncate -s 0 /var/log/*.log
Enter fullscreen mode Exit fullscreen mode

What truncate does is essentially empty the file (its size becomes 0B), but keeps it so that processes can keep writing it without interruption.
You can also do:

sudo rm /var/log/**/*.gz
Enter fullscreen mode Exit fullscreen mode

This will remove old logs compressed as .gz files. Please double-check and make sure you don't actually need these archives (for auditing or archiving purposes, for example).

You can also run:

sudo journalctl --vacuum-time=2d
Enter fullscreen mode Exit fullscreen mode

This will remove all logs aged 2 days or older. Below is a variant that also deletes older logs until the journal size reaches a desired value (Be careful if you need the old logs for audits!).

sudo journalctl --vacuum-size=100M
Enter fullscreen mode Exit fullscreen mode

You can see that I ran it and saved upwards of 600 MB of space!

Command output

Finally, it is recommended to use the following command to force periodic rotation and cleanup:

sudo logrotate -f /etc/logrotate.conf
Enter fullscreen mode Exit fullscreen mode

5. Cache and temporary files

These are mainly .deb files from the package cache stored in /var/cache/apt/archives/. Very straightforward to clean, all we need to do is run the following:

sudo apt-get clean
Enter fullscreen mode Exit fullscreen mode

You can see the before and after below:

Command output

And don't forget to clean the ~/.cache directory from time to time too, it can get pretty heavy, especially if you work a lot with virtual environments in Python. Just see mine below:

Command output

You may experience some slower downloads after deleting cache (especially if you download the same packages over and over in different virtual environments), it's up to you to see if it's worth deleting!

6. Bonus: Optimizing VHDX (WSL2 only)

On WSL2, the .vhdx file is basically a Virtual Hard Disk that stores your Linux filesystem. It can grow, but doesn’t automatically shrink when you delete files.

To optimize it (reduce its size), we can compact the VHDX manually. Start by cleaning up inside WSL. The three commands below clear cached package files and metadata to free space, then trim unused blocks on the SSD to optimize performance.

sudo apt clean
sudo rm -rf /var/lib/apt/lists/*
sudo fstrim -v /
Enter fullscreen mode Exit fullscreen mode

You'll find that it says that 940 GB was trimmed. This might be surprising if, like me, you don't even have that much space on your physical hard drive.

Command output

In reality, WSL2’s VHDX has a 1 TB default virtual size (the maximum it can grow to), so Linux sees all that space when trimming, even if your Windows disk is smaller.

Now, let's run PowerShell as an Administrator and shut-down WSL:

wsl --shutdown
Enter fullscreen mode Exit fullscreen mode

Run the following to list your distros:

wsl -l -v
Enter fullscreen mode Exit fullscreen mode

Make sure that the WSL instance is stopped before proceeding any further. If it's not, double-check that you've closed all editors and killed all relevant processes.

Then, run this to get the exact path of your .vhdx file:

Get-ChildItem "$env:LOCALAPPDATA\Packages" -Recurse -Filter ext4.vhdx
Enter fullscreen mode Exit fullscreen mode

Your output should look like this, go ahead and copy that path:

Command output

Then, run the next command to enter diskpart, the command-line utility in Windows to manage disk partitions and volumes:

diskpart
Enter fullscreen mode Exit fullscreen mode

Next, select the .vhdx disk file using the path we previously retrieved:

select vdisk file="C:\Users\<you>\AppData\Local\Packages\<DistroPackage>\LocalState\ext4.vhdx"
Enter fullscreen mode Exit fullscreen mode

You should get confirmation that the virtual disk file was correctly selected:

Command output

You can then run the following command to get more info about the virtual disk:

detail vdisk
Enter fullscreen mode Exit fullscreen mode

You should see a similar output to this. What's relevant here is that the WSL2 disk can grow up to 1 TB maximum (virtual size) but only 72 GB are currently in use (physical size).

Command output

Now, let's mount the disk so that Windows can access it:

attach vdisk readonly
Enter fullscreen mode Exit fullscreen mode

Again, you will get confirmation:

Command output

Then, we can safely run this to start the compaction process:

compact vdisk
Enter fullscreen mode Exit fullscreen mode

We will get confirmation that the operation ran successfully:

Command output

We can run detail vdisk again to check how much space we reclaimed:

Command output

In my case, it's a very solid 4 GB!

Finally, detach the virtual disk:

detach vdisk
Enter fullscreen mode Exit fullscreen mode

And exit:

exit
Enter fullscreen mode Exit fullscreen mode

That's it for this final section, well done!

Wrapping it up

Well, that pretty much does it! I hope you cleaned up a few GBs along the way!

Remember, it's important to periodically clean-up your system to avoid heavy disk fragmentation and performance degradation.

I hope this article was helpful, good luck and until next time!

Top comments (0)