Add progress bars to any command

Have you ever tried to compress a huge file, waited a bit in front of your terminal and wondered if you should go get a coffee or if it's just about to finish?

After spending some time dealing with huge files and trying to figure out how much I advanced, I ended up discovering tricks that I decided to industrialize. Enter Spybar.

It's a simple Python script that you can get with a

pip install spybar

If you prefix a command with it, it will start it and display the progress bar. You can also attach to an existing PID. Everything is explained in the readme.

This article is however not about the usage of the tool. It's about how it works.

Let's first state that it's only compatible with Linux. It could be that there is some way to do this in other operating systems but this article is not about that.

In Linux, there is a special directory, /proc which contains a directory for each running process.

Suppose that you are working on process 42, you can list all the files open by a process by doing

ls -lsh /proc/42/fd

When you open a file, in C, you get an integer which is the file handler. All those integers are listed in the fd directory. They all are symbolic links to the actual file that they open. Using ls, once you've located the file you want, you can note down its number. Suppose that you're interested in number 3.

There is a different folder which contains some meta-information about those handlers. Which you can simply access by doing

cat /proc/42/fdinfo/3

You'll get something alike of this:

pos:    569573376
flags:  0104000
mnt_id: 28

Including the pos line which indicates us where exactly the handler is pointing to.

Then you just need to know the size of the file and there you go, you know both the current position and the file size, that's all you need to generate a progress bar.

After doing this for a while I figured that I'd write that little tool so here we are. Thank you for reading!

Comments

[Ben Sinclair]

We also have the pv command to view progress through pipes. It doesn't do the same kind of magic to find file sizes that this tool does, but it works without /proc and is pretty neat.

[Rémy 🤖]

It is also fairly nice, and there is the progress utility that works using the same mechanism also. But I wanted some fun and a different UX :)

[Moises Jafet]

@xowap , by sharing this tool, you made the world a much better place; thank you!

[Rémy 🤖]

Always happy to make other people happy :)