Context Manager and the "with" Statement in Python

Introduction

According to the Python official document, the context manager is defined as below.

A context manager is an object that defines the runtime context to be established when executing a with statement. The context manager handles the entry into, and the exit from, the desired runtime context for the execution of the block of code. Context managers are normally invoked using the with statement (described in section The with statement), but can also be used by directly invoking their methods.

Typical uses of context managers include saving and restoring various kinds of global state, locking and unlocking resources, closing opened files, etc.

Source: Python Official Documentation

In this blog post, I will confirm how the context manager in Python works with code examples. There are two ways to implement a context manager: class-based and function-based.

How Class-Based Context Manager Works

To define a context manager, we need to implement the .__enter__ and .__exit__ special methods in our classes.

Method	Description
.__enter__(self)	Enter the runtime context and return either this object or another object related to the runtime context. The value returned by this method is bound to the identifier in the as clause of with statements using this context manager.
.__exit__(self, exc_type, exc_val, exc_tb)	Exit the runtime context and return a Boolean flag indicating if any exception that occurred should be suppressed. If an exception occurred while executing the body of the with statement, the arguments contain the exception type, value and traceback information. Otherwise, all three arguments are None.

Source: Python Official Documentation

Code Example: Class-Based Context Manager

Example 1: Normal end

class ContextManager(object):

    def __init__(self):
        print("call __init__")

    def __enter__(self):
        print("call __enter__")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("call __exit__")
        if exc_type is None:
            print('exit normally')
        else:
            print(f'exc_type={exc_type}')
            return True  # Surpress the exception

    def work(self):
        print("start work")
        # raise Exception() # Exception occurred
        print("end work")


if __name__ == '__main__':
    try:
        with ContextManager() as cm:
            cm.work()
    except Exception as e:
        print("Exception is propagated to the caller")

Execution result:

call __init__
call __enter__
start work
end work
call __exit__
exit normally
yosuke@Yosuke-Hanaoka python-sandbox % 

Example 2:
An exception occurs in the "with" block and .__exit__() return True.

    def work(self):
        print("start work")
        raise Exception() # Exception occurred
        print("end work")

Execution result:
The exception is suppressed in .__exit__() and NOT propagated to the caller.

call __init__
call __enter__
start work
call __exit__
exc_type=<class 'Exception'>
yosuke@Yosuke-Hanaoka python-sandbox % 

Example 3:
An exception occurs in the "with" block and .__exit__() return False

    def __exit__(self, exc_type, exc_value, traceback):
        print("call __exit__")
        if exc_type is None:
            print('exit normally')
        else:
            print(f'exc_type={exc_type}')
            return False  # Propagate the exception

Execution result:
The exception is NOT suppressed in .__exit__() and propagated to the caller.

call __init__
call __enter__
start work
call __exit__
exc_type=<class 'Exception'>
Exception is propagated to the caller
yosuke@Yosuke-Hanaoka python-sandbox % 

How Function-Based Context Manager Works

Decorating an appropriately coded generator function with @contextmanager, we can automatically get a function-based context manager that provides .__enter__() and . __exit__().

The processing before and after yield will be .__enter__() and .__exit__(). If there is an object to return by __enter__, we can return with yield. The object returned by yield is bound to the identifier in the as clause of with statements using this context manager.

If any exception occurs in the "with" statement, it can be caught in the except clause of the context manager function, as we can see below.

Code Example: Function-Based Context Manager

Example 1: Normal end

from contextlib import contextmanager

@contextmanager
def context_manager_func():
    print('__enter__')
    try:
        yield "test"
    except Exception as e:
        print("Exception is caught")
    finally:
        print('__exit__')


if __name__ == '__main__':
    try:
        with context_manager_func() as cm:
            print("start work")
            # raise Exception() # Exception occurred
            print("end work")
    except Exception as e:
        print("Exception is propagated to the caller")

Execution result

__enter__
start work
end work
__exit__
yosuke@Yosuke-Hanaoka python-sandbox %

Example 2:
An exception occurs in the "with" block.

if __name__ == '__main__':
    try:
        with context_manager_func() as cm:
            print("start work")
            raise Exception("")
            print("end work")
    except Exception as e:
        print("Exception is propagated to caller")

Execution result:
The exception is caught in the except clause of the context manager function.

__enter__
start work
Exception is caught
__exit__
yosuke@Yosuke-Hanaoka python-sandbox % 

Example 3:
An exception occurs in the "with" block, and the context manager function doesn't catch it.

@contextmanager
def context_manager_func():
    print('__enter__')
    try:
        yield "test"
    # except Exception as e:
    #     print("Exception is caught")
    finally:
        # Always done last, whether an exception occurs or not.
        print('__exit__')

Execution result:
The exception is NOT suppressed in the context manager function and propagated to the caller.

__enter__
start work
__exit__
Exception is propagated to caller
yosuke@Yosuke-Hanaoka python-sandbox % 

Reference:

Python 3.12.0 documentation: Statement Context Managers
Python 3.12.0 documentation: Context Manager Types
PEP 343 – The “with” Statement
Context Managers and Python's with Statement