Abhinav Pasham

Posted on Aug 7

ASYNC CONTEXT MANAGERS

#backend #database #python #tutorial

Why Does Python Need Async Context Managers?

Understanding Resource Pooling with a Fake Database Connection

INTRODUCTION

While learning asyncio, I came across something called an Async Context Manager.

Initially, I thought it was just another way of writing try...finally.

Then I started learning about database connection pools.

That's when another question came to my mind.

Every request needs a database connection. Does every request create a new connection?

If that's true, creating and destroying database connections for every request would be expensive.

So how do frameworks like FastAPI and libraries like asyncpg manage thousands of requests efficiently?

This led me to two important concepts:

Resource Pooling
Async Context Managers (async with)

In this article, I'll explain the problem they solve and how they work together.

What You Will Learn

Why resource pooling exists
Why creating database connections repeatedly is expensive
How connection pools work
Why Python introduced async context managers
How async with works internally
Building a fake resource pool

Prerequisites

Before reading this article, you should understand:

Coroutines
Event Loop
await
Classes
asyncio

The Problem

Suppose you're building a backend.

Whenever a request arrives,

it needs to query the database.

Initially, I thought the flow looked like this.

Request

↓

Create Database Connection

↓

Execute Query

↓

Close Connection

Seems perfectly fine.

Now imagine

1000 users send requests simultaneously.

Request 1

Request 2

Request 3

...

Request 1000

If every request creates a brand new database connection,

the server has to

Open a network connection
Authenticate the user
Allocate memory
Establish communication

for every request.

Creating database connections is expensive.

So another question came into my mind.

Instead of creating new connections every time, why can't we reuse existing ones?

That's exactly why Resource Pooling exists.

What is Resource Pooling?

Instead of creating a connection for every request,

the application creates a small number of reusable connections.

Imagine

Pool

Connection 1

Connection 2

Connection 3

Now,

when a request arrives,

it doesn't create a new connection.

It simply borrows one.

Request

↓

Take Connection 2

↓

Execute Query

↓

Return Connection 2

↓

Pool

The next request can reuse the same connection.

The Next Question

After understanding connection pools,

another question came to my mind.

Suppose I borrow a connection.

How do I make sure it's always returned to the pool?

Imagine this code.

conn = await pool.acquire()

await conn.execute(query)

await pool.release(conn)

Looks fine.

But what if

await conn.execute(query)

raise Exception()

The exception occurs before

await pool.release(conn)

The connection is never returned.

After enough requests,

every connection remains occupied.

Eventually,

the pool becomes empty.

New requests have no available connections.

This is called a connection leak.

Python's Solution

Python introduced

async with

Instead of writing

conn = await pool.acquire()

try:
    await conn.execute(query)
finally:
    await pool.release(conn)

we simply write

async with pool as conn:

    await conn.execute(query)

Much cleaner.

More importantly,

the connection is always returned,

even if an exception occurs.

How Python Achieves This

When Python sees

async with pool as conn:

it automatically performs

Acquire Connection

↓

Execute Block

↓

Release Connection

Internally,

Python converts it into

conn = await pool.__aenter__()

try:

    await conn.execute()

finally:

    await pool.__aexit__()

This means

__aenter__() acquires the resource,

while

__aexit__() releases it.

Building a Fake Resource Pool

Let's build a simple version.

class FakeConnection:

    async def execute(self, query):

        print(f"Executing: {query}")

        await asyncio.sleep(2)

        print("Query Completed")

Now we create a resource pool.

class ResourcePool:

    async def __aenter__(self):

        print("Acquiring Connection...")

        await asyncio.sleep(1)

        self.connection = FakeConnection()

        print("Connection Acquired")

        return self.connection

    async def __aexit__(self, exc_type, exc, tb):

        print("Releasing Connection...")

        await asyncio.sleep(1)

        print("Connection Released")

Using it becomes very simple.

async with ResourcePool() as conn:

    await conn.execute(
        "SELECT * FROM users"
    )

    print("Working with database...")

Execution Flow

Program Starts

↓

async with

↓

__aenter__()

↓

Acquire Connection

↓

Return Connection

↓

Execute Query

↓

Exit async with

↓

__aexit__()

↓

Return Connection

↓

Program Ends

Real-world Use Cases

Resource pooling is used almost everywhere.

Database Connections (asyncpg)
HTTP Client Sessions (aiohttp)
Redis Connections
WebSocket Connections
File Handles

Instead of creating expensive resources repeatedly,

applications reuse them.

Advantages

Reuses expensive resources
Improves performance
Prevents connection leaks
Automatic cleanup
Cleaner code
Easier error handling

Conclusion

Initially, I thought every request created and destroyed its own database connection.

Later I learned that this approach doesn't scale.

Instead, applications maintain a pool of reusable connections.

But simply reusing connections isn't enough.

We also need a reliable way to return them back to the pool, even when something goes wrong.

That's exactly why Python provides Async Context Managers.

Once I understood that async with automatically acquires and releases resources, the idea of connection pooling became much easier to understand.

In the next article, we'll explore another interesting concept:

How can a coroutine produce values one at a time while still performing asynchronous operations?

That's where Async Generators come in.

DEV Community