DEV Community

dpm_bush
dpm_bush

Posted on

Deleted a Huge File but Disk Space Didn’t Come Back? Find the Open File Handle

You deleted a large log file, but df -h still shows the filesystem as full. The file is gone from its directory, yet the space has not returned.

A common cause is a process that still has the deleted file open. On Linux, removing a filename unlinks it from the directory; it does not immediately erase the file’s data if a process still holds a reference to it. The blocks become available only after the final reference closes.

This is especially common with logs, temporary files, and files replaced during a deployment. Here’s how to confirm the cause and recover the space safely.

First, distinguish filesystem space from directory usage

Check the filesystem containing the affected path:

df -h /
Enter fullscreen mode Exit fullscreen mode

Use the relevant mount point instead of / if the full filesystem is somewhere else, such as /var or /srv.

Then compare that with visible directory usage:

sudo du -xhd1 /var 2>/dev/null | sort -h
Enter fullscreen mode Exit fullscreen mode

df reports blocks used by the filesystem. du totals space reachable through directory entries. The -x option keeps du on one filesystem, while -d1 limits the output to one directory level. On systems whose du doesn’t support -d, use --max-depth=1.

If df reports much more usage than you can account for with du, an open-but-deleted file is one possibility. It is not the only one: reserved filesystem blocks, snapshots, and mount-point confusion can also explain differences.

For more on interpreting that comparison, see how df and du differ.

Find deleted files that are still open

On many Linux distributions, lsof can list open files whose directory entries have been removed:

sudo lsof +L1
Enter fullscreen mode Exit fullscreen mode

The +L1 selector shows open files with fewer than one link—that is, unlinked files still held open. Look at the COMMAND, PID, USER, FD, and SIZE/OFF columns.

A line might resemble:

COMMAND   PID USER  FD   TYPE DEVICE SIZE/OFF NLINK NODE NAME
worker   2184 app   7w   REG  253,0  8.0G     0  ...  /var/log/worker.log (deleted)
Enter fullscreen mode Exit fullscreen mode

Here, process 2184 still has the deleted log open. The 7w descriptor indicates a writable file descriptor; w means the process opened it for writing. A large SIZE/OFF value suggests it may account for substantial space, but confirm the filesystem and file before taking action.

If lsof is unavailable, /proc can help locate deleted open files. This loop prints process IDs, descriptor numbers, and link targets for matching descriptors:

sudo find /proc/[0-9]*/fd -type l -lname '* (deleted)' -print 2>/dev/null
Enter fullscreen mode Exit fullscreen mode

The output identifies candidates, not their size or whether they are safe to close. lsof is usually easier to interpret.

Reclaim the space without guessing

The safest general fix is to make the owning process close the file. How to do that depends on the service:

  1. Identify the process and the service that owns it.
  2. Check the service’s documentation for a supported log-reopen or reload mechanism.
  3. Use that mechanism, or arrange a controlled restart if appropriate.
  4. Recheck df -h and sudo lsof +L1.

A service may support a reload that closes and reopens log files without stopping the whole application. Don’t assume that every service handles reloads the same way. Check its configuration and documentation first.

For a systemd-managed service, inspect its state and recent messages before acting:

systemctl status example.service
sudo journalctl -u example.service --since '30 minutes ago'
Enter fullscreen mode Exit fullscreen mode

Replace example.service with the actual unit name. If you need help reading unit state, this guide to systemctl status explains the fields that matter. For broader log filtering, see the journalctl guide.

If there is no supported reopen mechanism, a planned restart may release the handle, but consider the service’s role and impact first. A restart can interrupt requests, jobs, or connections. On a critical system, follow the service’s operational procedure rather than improvising.

Avoid truncating /proc file descriptors as a first fix

You may find advice to empty a deleted file by writing to /proc/<pid>/fd/<fd>. That can reclaim blocks in some situations, but it is risky: the descriptor might not belong to the file you think it does, and truncating it can damage data or confuse the application.

Prefer the application’s own log-rotation or reopen procedure. If you do consider manipulating a descriptor directly, first verify the process, descriptor target, and consequences for that specific program. Do not use a broad command that writes to every matching descriptor.

Also, killing the process with kill -9 is rarely the right first step. It prevents cleanup and can interrupt work. If a restart is necessary, use the service manager or the application’s normal shutdown path.

Prevent the problem from recurring

A log rotation rule can rename or remove a log file, but the application must also release the old file and open the new one. That is why rotation configurations commonly signal a service or invoke a service-specific reopen command. Check that the rotation policy and application behavior agree.

When investigating future disk alerts, include open deleted files in the checklist:

df -h
sudo du -xhd1 /var 2>/dev/null | sort -h
sudo lsof +L1
Enter fullscreen mode Exit fullscreen mode

Run du against the affected filesystem’s directories, not just /var, if the full mount is elsewhere. If lsof shows a large deleted file, focus on its owning process and use a supported way to release it. The key distinction is simple: deleting a name removes a directory entry; closing the last open reference is what lets Linux reclaim the file’s storage.

I'm building SSHFlow — an SSH client that organizes terminals, SFTP, code editing, and server tools into workspaces.

Top comments (0)