Timothy rushed into Margaret's office looking distressed. "The library's file system is corrupted. I've been processing catalog updates all week, and now the system reports hundreds of files are locked—I can't access them, and neither can anyone else."
Margaret examined his code and immediately spotted the problem. Throughout his scripts, Timothy had opened files but forgotten to close them. Each unclosed file handle remained locked, consuming system resources until the entire system ground to a halt.
"You need the Automatic Door System," Margaret said, leading him to a section of the library where ornate chambers featured doors that closed and locked themselves the moment visitors stepped out. "These are context managers—Python's solution to resource management."
The Resource Leak Problem
Timothy's file processing code looked reasonable at first glance:
def update_catalog_entry(filename, book_data):
file = open(filename, 'w')
file.write(book_data)
# Forgot to close the file!
return "Updated"
# After hundreds of calls, system runs out of file handles
for book in books:
update_catalog_entry(f"catalog/{book.id}.txt", book.data)
Each call opened a file but never closed it. The operating system kept those files locked, waiting for Python to release them. Eventually, the system exhausted available file handles and refused to open more files.
"Even if you remember to close files," Margaret explained, "exceptions can prevent cleanup code from running."
def process_catalog_file(filename):
file = open(filename, 'r')
data = file.read()
result = risky_processing(data) # Might raise an exception!
file.close() # Never executes if exception occurs
return result
If risky_processing() raised an exception, the function exited immediately—the close() call never executed, and the file remained locked.
The With Statement Solution
Margaret showed Timothy Python's elegant solution:
def process_catalog_file(filename):
with open(filename, 'r') as file:
data = file.read()
result = risky_processing(data)
return result
# File automatically closed here, even if exception occurred!
The with statement created a context—a protected zone where the file was open. When execution left that zone (normally or via exception), Python automatically closed the file. The cleanup was guaranteed.
"The with keyword," Margaret explained, "means 'I'm entering a managed context. When I leave, clean up automatically.'"
The Scope Boundary
Timothy needed to understand an important limitation:
# The resource is only valid inside the with block
with open('catalog.txt', 'r') as file:
data = file.read() # ✓ Works fine
print(f"Read {len(data)} characters")
# Outside the block - file is closed!
print(file.read()) # ✗ ValueError: I/O operation on closed file
The resource became invalid the moment execution left the with block. This was by design—the whole point was guaranteed cleanup at the block's boundary.
The Enter and Exit Protocol
Timothy wanted to understand how the automatic closure worked. Margaret revealed the magic:
# What really happens with 'with'
file = open(filename, 'r')
file.__enter__() # Called when entering the with block
try:
data = file.read()
result = risky_processing(data)
finally:
file.__exit__(None, None, None) # Always called when leaving
The with statement called two special methods:
-
__enter__(): Executed when entering the context, returns the resource -
__exit__(): Executed when leaving the context (for any reason), performs cleanup
The finally block ensured cleanup happened whether the code succeeded or raised an exception.
The Exception Handling Advantage
Margaret demonstrated the critical benefit—exception safety:
# Without with - exception leaves file open
file = open('catalog.txt', 'w')
file.write(dangerous_operation()) # Exception here!
file.close() # Never reached
# With with - file closes regardless
with open('catalog.txt', 'w') as file:
file.write(dangerous_operation()) # Exception here!
# File automatically closed before exception propagates
The context manager's __exit__ method received information about any exception that occurred, allowing it to handle cleanup appropriately before the exception propagated.
Multiple Context Managers
Timothy discovered he could manage multiple resources simultaneously:
# Nested with statements - verbose
with open('input.txt', 'r') as input_file:
with open('output.txt', 'w') as output_file:
data = input_file.read()
output_file.write(process(data))
# Python 3.1+ - cleaner syntax
with open('input.txt', 'r') as input_file, \
open('output.txt', 'w') as output_file:
data = input_file.read()
output_file.write(process(data))
Both files were guaranteed to close properly, even if processing failed midway through.
Common Context Manager Patterns
Margaret showed Timothy where context managers appeared throughout Python:
File operations:
with open('catalog.txt', 'r') as file:
contents = file.read()
# File automatically closed
Database transactions:
import sqlite3
connection = sqlite3.connect('library.db')
try:
with connection:
cursor = connection.cursor()
cursor.execute("INSERT INTO books VALUES (?, ?)", (title, author))
cursor.execute("UPDATE catalog SET count = count + 1")
# Transaction automatically committed (or rolled back on exception)
finally:
connection.close()
"Notice," Margaret pointed out, "the database connection's context manager commits or rolls back the transaction, but doesn't close the connection. We still need the finally block for that."
Thread locks:
from threading import Lock
catalog_lock = Lock()
with catalog_lock:
# Critical section - only one thread at a time
update_shared_catalog()
# Lock automatically released
Temporary files and directories:
from tempfile import TemporaryDirectory
import os
with TemporaryDirectory() as temp_dir:
# temp_dir is the path to a temporary directory
temp_file = os.path.join(temp_dir, 'working_data.txt')
with open(temp_file, 'w') as f:
f.write('Process this data')
process_files_in(temp_dir)
# Directory and all its contents automatically deleted
The Context Manager Protocol in Detail
Timothy examined how __exit__ handled exceptions. While file objects already implemented the context manager protocol, Margaret showed him a custom implementation to understand the mechanics:
class FileManager:
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
self.file = None
def __enter__(self):
self.file = open(self.filename, self.mode)
return self.file # This becomes the 'as' variable
def __exit__(self, exc_type, exc_value, traceback):
if self.file:
self.file.close()
# exc_type: The exception class (or None)
# exc_value: The exception instance (or None)
# traceback: The exception traceback (or None)
# Return True to suppress the exception
# Return False (or None) to let it propagate
return False
with FileManager('catalog.txt', 'w') as f:
f.write('data')
# __exit__ called with exception info if one occurred
"This example is for understanding," Margaret explained. "Files already work as context managers. But knowing how to build your own helps when managing custom resources like database connections, API clients, or hardware devices."
The __exit__ method received three arguments describing any exception that occurred. Returning True suppressed the exception, while returning False (or None) allowed it to propagate normally.
The Suppression Decision
Margaret showed Timothy when to suppress exceptions:
class IgnoringContextManager:
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
if exc_type is ValueError:
print(f"Ignoring ValueError: {exc_value}")
return True # Suppress ValueError only
return False # Let other exceptions propagate
with IgnoringContextManager():
raise ValueError("This error is suppressed")
print("Execution continues") # This runs!
with IgnoringContextManager():
raise TypeError("This error propagates") # Program crashes here
"But understand," Margaret cautioned, "suppressing exceptions is rare in practice and often indicates a design problem. Most context managers return False, allowing exceptions to propagate naturally. Only suppress specific, expected exceptions that you genuinely want to ignore—and document why you're doing it."
The Comparison: Manual vs Automatic
Timothy compared the approaches:
# Manual resource management - error-prone
file = open('catalog.txt', 'r')
try:
data = file.read()
process(data)
finally:
file.close()
# Context manager - clean and safe
with open('catalog.txt', 'r') as file:
data = file.read()
process(data)
The with statement was shorter, clearer, and impossible to get wrong. It eliminated an entire class of bugs—resource leaks.
Timothy's Context Manager Wisdom
Through exploring the Automatic Door System, Timothy learned essential principles:
With statements guarantee cleanup: Resources are released even when exceptions occur.
The protocol is simple: Implement __enter__ (acquire resource) and __exit__ (release resource).
Use for any resource management: Files, locks, database connections, network sockets—anything that needs cleanup.
Resources are invalid outside the block: Once the with block ends, the resource is closed and unusable.
Multiple resources work together: Stack multiple with statements or use comma separation.
Exception information flows through: __exit__ receives details about any exception.
Most managers don't suppress exceptions: Returning False from __exit__ is the standard pattern. Suppression is rare.
The 'as' variable comes from enter: Whatever __enter__ returns becomes available in the with block.
Not all context managers close connections: Some (like database connections) handle transactions but leave the connection open.
Timothy's mastery of context managers transformed his resource management. No more leaked file handles, no more forgotten cleanup, no more corrupted system states. The Automatic Door System ensured that every chamber closed properly behind him, every resource released, every lock freed. The library's file system recovered, and Timothy's code became bulletproof against resource leaks.
Aaron Rose is a software engineer and technology writer at tech-reader.blog and the author of Think Like a Genius.
Top comments (0)