How to spy a process

Have you ever stuck with a program running for a long time or wondering what the process is doing? No worries! You are not alone, same thing happen me a lot.

Most recently I have to spy on a long running php process to figure out why/where its taking so much time.

Here is my approach to spy any process in linux environment.

Find out the process ID by running a ps command and grep for specific program. In my case it was a php program so I type following:

$ ps aux | grep "some_program.php"

Then strace the process. You need root access.

$ strace -p PID

Don't worry if it dumps a lot in the standard output. It trace all the system calls (read,write,open,close,poll,recvfrom etc) that the process is doing. Look for any interesting pattern.

To see which files are opened by the process you can try lsof command. This helps me to figure out which file paths are used by the program.

$ lsof -p PID

To further debug the process try gdb.
First, attach the program with gdb.

$ gdb -p PID

Now you are in gdb shell. type cont and wait for a while. and then hit ctrl + c to stop it.

> cont

It will stop and return back to gdb shell. Now type bt it will dump the backtrace. I can see some php library function calls.

> bt

These few commands helps me a lot to deep dive into a process and figure out whats going on. This is specially helpful when you don't know much about the underlying codebase.

To learn more about strace check out following two articles.

• strace zine by Julia Evans
• What is php doing by Derick Rethans