Last month, on the morning of April 28th, I woke up to a "Disk Space Critical" email from my own VPS. When I ran df -h, I saw that the / directory was 100% full. As I suspected, Docker was the culprit again. When managing over 13 containers on the same server, if even one gets out of control, it's only a matter of time before the entire system goes into swap and gets OOM-killed.
This situation is familiar to anyone who constantly struggles with disk storage issues, hosting multiple applications on their own server. In this post, I'll explain how I manage these "Docker Disk Storage Wars" on my own VPS, how I ensure data integrity, and how I optimize disk space. My goal is to guide you with practical solutions and experiences I've personally had.
Why Do Docker Disk Storage Wars Erupt?
My VPS's disk space suddenly filling up has almost become a routine for me. Most of the time, the root of the problem lies in unexpected container log growth, unnecessary images, or build caches. For example, at one point, the build caches of my Next.js applications reached 33 GB, and with unused images added on top, they consumed another 23 GB of disk space. That's when the disk hit 100%.
Such situations can lead to severe performance degradation and even service outages, especially if you're using a VPS with limited resources. A container's disk I/O maxing out can lead to kcompactd using 92% CPU and sshd being unable to accept new connections. That's why it's crucial to understand the root cause of the problem.
Common Disk Space Consumers
Knowing the biggest disk space consumers in the Docker ecosystem is the first step to solving the problem. In my experience, the main culprits are:
- Dangling Images and Volumes: Unused or disconnected images and volumes can unknowingly occupy gigabytes of space. This becomes inevitable, especially if you frequently rebuild images.
- Build Cache: Even if you don't use multi-stage builds, Docker creates intermediate layers at each build step. These caches can accumulate and reach enormous sizes. My 33 GB build cache issue was a prime example of this.
- Container Logs: Especially verbose applications or services running in debug mode can cause log files to grow uncontrollably. Gigabytes of logs can accumulate within a few days; I even witnessed critical services on an internal banking platform halt due to this.
- Ephemeral Data and Temporary Files: Temporary files created by applications during operation can take up permanent disk space if not properly cleaned.
The docker system df command is very useful for identifying these issues. This command provides a detailed summary of Docker's disk usage, helping me understand which component occupies how much space.
docker system df
The output of this command shows how much disk space each Docker component is using. For example, you can find detailed information under categories like "Images", "Containers", "Local Volumes", and "Build Cache". The "Reclaimable" area, in particular, shows the amount of disk space you can recover with manual intervention.
Data Integrity and Persistent Storage Strategies
One of the most crucial aspects when running applications in Docker is ensuring data persistence, even if containers die. Since I host my own sites and side products (like hesapciyiz.com, spamkalkani.com), data loss is critical. A wrong configuration or automatic cleanup can instantly wipe out your data.
Therefore, it's essential to correctly understand Docker's storage mechanisms and develop proactive strategies. I generally prefer using volumes because they are easier to manage and are a persistent storage solution designed within Docker itself.
Docker Volumes and Bind Mounts
Docker offers two primary methods for making data persistent: volumes and bind mounts. Both have their advantages and disadvantages, and the choice depends somewhat on the use case.
- Volumes: These are file systems managed by Docker. They are typically located under `/var/lib/
Top comments (0)