Idk how often that happens to you, but for me at work I have some folders with many thousands files in them, and sometimes I repeat my old mistake and try to ls in such a directory, to be met with a terminal freeze, which - depending on the folder size - is either for just 10 seconds, or pretty much forever, until I manage to stop it with ctrl-c.
I decided to benchmark ls against cat with a matching number of files / lines of text.
I understand that such a comparison is not fair - cat doesn't have to deal with the file system all that much.
Now, a few words on possible reasons for ls being slow. One is the need to load some spread-out data into RAM. This can hopefully be mitigated by warm-up runs when benchmarking.
Another two issues I found when googling for "why is ls slow": usually ls tabulates the output, which is a costly process; and also ls colors the output, meaning one has to request info on permissions on all the files one encounters to chose the right colors. To avoid those issues one can benchmark ls <dir> | cat, which just prints files/dirs one per line.
With that in mind, here is some data:
| #files/lines |
cat time[ms] |
ls time[ms] |
|---|---|---|
| 20K | 0.9 ± 0.4 | 38.2 ± 3.9 |
| 40K | 1.4 ± 0.3 | 79.9 ± 7.3 |
| 60K | 1.5 ± 0.4 | 131.5 ± 11.6 |
| 80K | 1.5 ± 0.2 | 172.0 ± 6.6 |
At the very least the growth seems to be linear. Also, here one sees the times about one fifth of a seconds at most, while I was talking about 10 seconds before. There are two reasons for this. First, at work I got some special remotely mounted drive; so operating it takes extra time. Second, for my benchmark I used empty files. Turns out file size actually affects speed of ls. For example if instead of creating empty files I write into each one 10K characters, I get the following table:
| #files |
ls time[ms] |
|---|---|
| 20K | 42.7 ± 3.5 |
| 40K | 90.9 ± 2.6 |
| 60K | 153.3 ± 10.1 |
| 80K | 214.3 ± 37.4 |
The funniest part is that these new results are kinda inconsistent, even though I do warm-up runs. Oh well 🤷
Conclusion - I got no clue how linux filesystem works, however, from a user perspective, this experiment makes me wonder if ls could be faster...
Top comments (0)