<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Igor</title>
    <description>The latest articles on DEV Community by Igor (@verin_is).</description>
    <link>https://dev.to/verin_is</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4234084%2F5d007bd8-3743-4a53-a504-02efb1df7505.png</url>
      <title>DEV Community: Igor</title>
      <link>https://dev.to/verin_is</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/verin_is"/>
    <language>en</language>
    <item>
      <title>Linux Says "No Space Left on Device" - But the Disk Isn't Full</title>
      <dc:creator>Igor</dc:creator>
      <pubDate>Sun, 08 Nov 2026 20:50:18 +0000</pubDate>
      <link>https://dev.to/verin_is/linux-says-no-space-left-on-device-but-the-disk-isnt-full-3h48</link>
      <guid>https://dev.to/verin_is/linux-says-no-space-left-on-device-but-the-disk-isnt-full-3h48</guid>
      <description>&lt;p&gt;I've been working on some Linux troubleshooting labs recently, and one of my favorite scenarios involves an error that almost every sysadmin has seen:&lt;/p&gt;

&lt;p&gt;No space left on device&lt;/p&gt;

&lt;p&gt;Normally, the first thing you'd check is disk usage.&lt;/p&gt;

&lt;p&gt;df -h&lt;/p&gt;

&lt;p&gt;And in most cases, that tells you exactly what you need to know.&lt;/p&gt;

&lt;p&gt;But sometimes the output looks completely normal:&lt;/p&gt;

&lt;p&gt;Filesystem      Size  Used Avail Use%&lt;br&gt;
/dev/vda1        40G   17G   21G  45%&lt;/p&gt;

&lt;p&gt;There's 21 GB of free space. Nothing particularly suspicious.&lt;/p&gt;

&lt;p&gt;Yet the application still can't create files. Even trying to create an empty file in the affected directory fails:&lt;/p&gt;

&lt;p&gt;touch /var/cache/myapp/test&lt;/p&gt;

&lt;p&gt;touch: cannot touch '/var/cache/myapp/test':&lt;br&gt;
No space left on device&lt;/p&gt;

&lt;p&gt;So what's going on?&lt;/p&gt;

&lt;p&gt;The problem isn't always disk space&lt;/p&gt;

&lt;p&gt;The important thing to remember is that filesystems have more limits than just available storage.&lt;/p&gt;

&lt;p&gt;On ext4, for example, files need inodes. An inode stores metadata about a file, and the filesystem has a finite number of them.&lt;/p&gt;

&lt;p&gt;You can have plenty of free disk space and still run out of inodes.&lt;/p&gt;

&lt;p&gt;That's why I usually check this next:&lt;/p&gt;

&lt;p&gt;df -i /var/cache/myapp&lt;/p&gt;

&lt;p&gt;Here's an example of what you might find:&lt;/p&gt;

&lt;p&gt;Filesystem     Inodes   IUsed  IFree IUse%&lt;br&gt;
/dev/vda1     2621440 2621440     0  100%&lt;/p&gt;

&lt;p&gt;There it is. No free inodes.&lt;/p&gt;

&lt;p&gt;The filesystem has room for more data, but it can't allocate another inode for a new file.&lt;/p&gt;

&lt;p&gt;How do you end up here?&lt;/p&gt;

&lt;p&gt;Usually, something has been creating a ridiculous number of small files.&lt;/p&gt;

&lt;p&gt;A cache directory is a good example.&lt;/p&gt;

&lt;p&gt;Imagine an application writing thousands of tiny cache files and never cleaning them up. Each file barely uses any disk space, but each one consumes an inode.&lt;/p&gt;

&lt;p&gt;Keep doing that long enough, and eventually you hit the limit.&lt;/p&gt;

&lt;p&gt;One command I find useful when investigating this is:&lt;/p&gt;

&lt;p&gt;sudo du --inodes -x -d 2 /var/cache&lt;/p&gt;

&lt;p&gt;Instead of showing disk usage in bytes, it counts inode usage.&lt;/p&gt;

&lt;p&gt;The -x option keeps the scan on the same filesystem, which is helpful when you're trying to find where the inodes went.&lt;/p&gt;

&lt;p&gt;I wouldn't immediately run it across the entire server, though. Scanning millions of files can take a while. It's better to start with directories you already suspect.&lt;/p&gt;

&lt;p&gt;Once you've found the problem, the fix depends on what's creating those files.&lt;/p&gt;

&lt;p&gt;Maybe an application isn't rotating its cache. Maybe a cleanup job stopped working. Maybe somebody left a script running that generates temporary files indefinitely.&lt;/p&gt;

&lt;p&gt;Removing unnecessary files gets the service working again, but unless you fix the underlying cause, you'll probably be dealing with the same incident next week.&lt;/p&gt;

&lt;p&gt;One thing worth mentioning&lt;/p&gt;

&lt;p&gt;This particular problem is common on filesystems such as ext4, where the inode count is generally determined when the filesystem is created.&lt;/p&gt;

&lt;p&gt;Other filesystems handle inode allocation differently, so inode exhaustion isn't always the explanation.&lt;/p&gt;

&lt;p&gt;And No space left on device can have other causes too.&lt;/p&gt;

&lt;p&gt;That's why I wouldn't jump straight to deleting files just because df -i is the next command in this example.&lt;/p&gt;

&lt;p&gt;First, make sure you're checking the filesystem where the failure actually happens. /tmp, for example, might be mounted separately.&lt;/p&gt;

&lt;p&gt;Why I like this scenario&lt;/p&gt;

&lt;p&gt;What makes this interesting to me isn't the solution itself. Once you know about inode exhaustion, it's a fairly straightforward problem to recognize.&lt;/p&gt;

&lt;p&gt;It's the assumption people make at the beginning.&lt;/p&gt;

&lt;p&gt;The error says there's no space, so you check free space. The numbers look fine, and suddenly the obvious explanation doesn't work anymore.&lt;/p&gt;

&lt;p&gt;That's where troubleshooting actually starts.&lt;/p&gt;

&lt;p&gt;I've been trying to build more exercises around situations like this. Not necessarily difficult commands or obscure Linux features, but problems where you need to question your first assumption.&lt;/p&gt;

&lt;p&gt;Because knowing which command to run is useful.&lt;/p&gt;

&lt;p&gt;Knowing why you're running it is much more useful.&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>sre</category>
      <category>education</category>
    </item>
  </channel>
</rss>
