DEV Community

özkan pakdil
özkan pakdil

Posted on • Originally published at ozkanpakdil.github.io on

Delete files in linux best way

Let say you have bunch(5 millions) of small files in your linux in some folder. when you try to delete them can be very tricky.

Everybody knows rm -rf /tmp/somefolder but if you run this on a server which already has internet load it may freeze it. and make it stop serving pages.

So what are the options. first and easiest way

find /tmp/somefolder/ -type f -mtime +30 | xargs rm -f

Enter fullscreen mode Exit fullscreen mode

this also can freeze or eat a lot of resources to run.

Then ionice comes to scene and with ionice you can make it run smoothly bash ionice -c3 find /tmp2/cache/ -type f -mtime +30 | xargs rm -fthis way for sure your system does not get freeze. but the problem is if you run this in every hour this may overrun and can 2 procceses clash which result as slow server. so what we need to do watch the system and clean whenever server has time.

#!/bin/bash

while true
do
    SYSLOAD=`uptime | awk '{print $10+0}'`
    alreadyrunning=`ps -ef|grep find|grep -v grep`
    if [$SYSLOAD -eq 0] && [-z "$alreadyrunning"]
    then
        #echo run delete. server has almost 0 load.
        date
        ionice -c2 find /tmp/somefolder/ -type f -mtime +90 | head -n 10000 |xargs -rd '\n' rm -f
    fi
    sleep 30
done

Enter fullscreen mode Exit fullscreen mode

This script checks every 30 seconds server load and if it is 0 run 10k delete. BTW server load never hits to 0. it runs when ever its below 1. it took me 3 days to clean 5.5 million files :)

Top comments (0)