loading...

Catching every single exception with Python

joshuaschlichting profile image Joshua Schlichting ・4 min read

When it comes to exceptions, we all know that sticky situations can be handled by simply looking before you leap, and for others, the good ol'try/except does the trick. But, what will come of an exception you've missed? If you're writing in any modern OOP language, then it's likely that your program will print out some traceback data to STDERR immediately before closing. That's it.

In a production environment, this is not acceptable. Failure simply isn't an option.

Now, what if I told you there was another way? What if you could catch an exception you never explicitly tried to catch, and do something graceful before your program's demise?

Allow me to introduce sys's excepthook!

sys.excepthook(type, value, tb)

The interpreter calls this function immediately before a program crashes as the result of an unhandled exception.

So, what we can do with this? We can create a function that accepts the same arguments and assign it to sys.excepthook so that a call to sys.excepthook is actually a call to our own function.

Example:
import sys

def my_exception_hook(type, value, tb):
    """
    Intended to be assigned to sys.exception as a hook.
    Gives programmer opportunity to do something useful with info from uncaught exceptions.

    Parameters
    type: Exception type
    value: Exception's value
    tb: Exception's traceback
    """
    error_msg = "An exception has been raised outside of a try/except!!!\n" \
                f"Type: {type}\n" \
                f"Value: {value}\n" \
                f"Traceback: {tb}"
    # We're just printing the error out for this example, 
    # but you should do something useful here!
    # Log it! Email your boss! Tell your cat!
    print(error_msg)

sys.excepthook = my_exception_hook

# Let's do something silly now
x = 10 + "10"

I saved this in a file called "testfile.py". Let's run that and see what we get.

josh@debian:~/tmp$ python3 testfile.py
An exception has been raised outside of a try/except!!!
Type: <class 'TypeError'>
Value: unsupported operand type(s) for +: 'int' and 'str'
Traceback: <traceback object at 0x7fac25f231c8>

This is great. We have some details about our exception that we can log and/or alert the developers with. We know that not only was an exception raised without a try/except, but we also know it was a TypeError due to a + being used to perform addition/string concatenation (depending on the intentions of the developer). With that said, it would be nice to know more about the error, and we can learn more by using information extracted from the traceback argument, tb.

Extracting details from the traceback

To extract information from our traceback, we'll be importing Python's traceback module. Once we've imported that, we'll be using the extract_tb() function to extract information from our traceback. For the purposes of this article, we'll only be concerning ourselves with passing extract_tb() our traceback, but know that you can limit how much information is returned using the limit parameter.

extract_tb() returns an instance of traceback.StackSummary which carries all of the information about the traceback. Luckily, this class has a format() method that returns to us a list of string, each representing a frame from the stack.

Now, let's rewrite our function above to give us more information about our exception by extracting the details from the traceback object, our tb parameter.

import sys
import traceback

def my_exception_hook(type, value, tb):
    """
    Intended to be assigned to sys.exception as a hook.
    Gives programmer opportunity to do something useful with info from uncaught exceptions.

    Parameters
    type: Exception type
    value: Exception's value
    tb: Exception's traceback
    """

    # NOTE: because format() is returning a list of string,
    # I'm going to join them into a single string, separating each with a new line
    traceback_details = '\n'.join(traceback.extract_tb(tb).format())

    error_msg = "An exception has been raised outside of a try/except!!!\n" \
                f"Type: {type}\n" \
                f"Value: {value}\n" \
                f"Traceback: {traceback_details}"
    print(error_msg)

sys.excepthook = my_exception_hook

# Let's do something silly now
x = 10 + "10"

Let's run it and see what we get back!

josh@debian:~/tmp$ python3 testfile.py
An exception has been raised outside of a try/except!!!
Type: <class 'TypeError'>
Value: unsupported operand type(s) for +: 'int' and 'str'
Traceback:   File "testfile.py", line 19, in <module>
    x = 10 + "10"

Great! We can see the file this happened in, the line it happened on, and a glimpse of the faulty logic that caused the exception!

I'm not doing much with the exception's data here, but it's just an example. I'd like to encourage you to log this exception somewhere. Also, consider settings aside a special exit code for these types of exceptions.

In summation...

Make no mistake about this post - I am NOT suggesting you substitute your proper try/except blocks with this method. This method is what I would use as an emergency back up. If my code ends up here somehow, I still consider it a loss, but at least I may be able to learn about it this way, rather than losing the exception details to whatever (if anything) is watching STDERR.

Discussion

pic
Editor guide
Collapse
rhymes profile image
rhymes

It's more like a catchall error handler, you can still re-raise the exception to make the program behave like it normally does, for example by adding something like:

raise type(value)

to the last line of the hook function.

Another possible usage for this is error logger that send errors to third party services, like for example what sentry-python does here or to intercept Ctrl-C globally, so that you might want to exit a command line script gracefully by releasing whatever resources you might want to release.

Collapse
rafaacioly profile image
Rafael Acioly

Nice!

I'm facing some weird problem with exception handler when it comes to catch the message from CancelledError from asyncio package, even if i do this;

try:
    something()
except asyncio.CancelledError as error:
    logger.info(f"Cancelled error:{error}")
except Exception as error:
    # something

The logged message is aways: "Cancelled error:" (empty error message), have you faced this problem?

Collapse
moshe profile image
Moshe Zada

If your use case is logging the exception use logger.exception("Cancelled error") which logs the exception and the stacktrace

Collapse
nikoheikkila profile image
Niko Heikkilä

This is cool! I think many frameworks use this or similar method to log uncaught exceptions to file, post them to error tracker like Sentry, and so on.

It's often a good choice to let the app crash but capture the information. Service managers like systemd will reboot the app back to life, anyway.

Collapse
pinutz23 profile image
Jannik Wempe

Didn't even know about that...
Learned something new in just a few minutes... I love it, thanks! :-)

Collapse
diegocastrum profile image
Diego Castro

Really nice post, thank you very much!!

Collapse
bl41r profile image
David Smith

Interesting post, but yikes, I would never actually do this!

Collapse
ar10642 profile image
Andrew Richardson

Shouldn't the program stop rather than try to soldier on in an error state? Or am I misunderstanding what this does?

Collapse
rohansawant profile image
Rohan Sawant

I have been Pythoning for a bit now, and I was not even remotely aware something like this existed!

🔥