How file descriptors can bring your service down.

Over the past year I've completed the Linux Foundation's Essentials of Linux Administration course. Today I'm going to focus on file descriptors and how your app creates them, and should close them.

Let's start with a very high level definition of how a linux OS sees its world.

Everything in a running linux system is a file.

A file descriptor is a file which the linux kernel creates and deletes for every file opened and closed by a process.

The linux kernel allows for a finite number of file descriptors!

Let's see this in action. Start your app and get the PID.

java -jar app.jar
ps -ef | grep app.jar

Now cd into processes directory and do an ls.

cd /proc/3535
ls -al

This is all the information about your running app that the kernel needs.

Let's focus on the file descriptors directory

ls -al /proc/3535/fd
ls -al /proc/3535/fd | wc -l

You'll see there are number of open file descriptors.

Use your app for a bit and come back to this directory and execute the above command again.

How many descriptors are there now?

Go back to your app again and use it some more.

How many are there now?

If this number keeps going up you're heading for trouble!

Eventually you're app won't be able to do anything as it can't open anymore file descriptors.

From my experience the main culprits are i/o streams.

So, how do we solve this?

Easy!

Close your streams.

Java

Try-with-resources to the rescue. Open all you're streams in a try-with-resources block and the jvm will handle closing the steam for you

//...

    try (InputStream data = new InputStream()){
        // ...
    }

Python

Python has a similar concept

# ...

    with open('mine.txt', r) as f:
        # ...

I'm sure other languages have the same concept so feel free to comment with it and I'll update the post.

So, always close your streams so that your app doesn't eventually fall over.