Inside an Async Web Server

Frameworks such as FastAPI and aiohttp make HTTP servers straightforward to build. They also hide most of the machinery that explains why asynchronous servers handle large numbers of concurrent connections efficiently.

Beneath routing, middleware, validation, and dependency injection, a server follows a small cycle for every connection: read a request, dispatch it to application code, write a response, and either wait for another request or close the connection.

Most of that lifecycle is waiting. The server waits for a client to send bytes, for a socket to accept more output, and often for a database or upstream service. An event loop can use those waiting periods to make progress on other connections without assigning an operating-system thread to each one.

This article builds a minimal HTTP/1.1 server directly on asyncio.start_server(). It is intentionally not production-ready. Its purpose is to expose the architecture that frameworks normally keep out of sight: one Task per connection, non-blocking stream I/O, flow control, keep-alive, and cooperative scheduling.

The Server's Core Responsibilities

Once the framework layers are removed, the connection handler has four responsibilities:

1. Read and parse one HTTP request.
2. Select an application handler.
3. Serialize and write an HTTP response.
4. Decide whether to read another request from the same connection.

The application handler may perform CPU work, but network servers commonly spend far more time waiting on I/O. A conventional blocking architecture can overlap that waiting with multiple threads or processes. An asynchronous architecture instead represents each connection as a lightweight Task that suspends whenever its next operation cannot proceed.

This distinction matters at scale. A Task is not an operating-system thread: it is a Python object containing coroutine state. Thousands of Tasks can wait on sockets without requiring thousands of thread stacks or equivalent scheduler activity. The kernel monitors socket readiness, and the event loop resumes only the Tasks that can make progress.

Async is therefore not a shortcut to faster computation. It is a resource-efficient way to coordinate large amounts of concurrent waiting.

Accepting Connections with start_server()

The listening side of the server is compact:

async def main():
    server = await asyncio.start_server(handle_connection, "127.0.0.1", 8899)
    async with server:
        await server.serve_forever()

asyncio.start_server() creates and binds a listening socket, then registers it with the event loop. For each accepted connection, it invokes handle_connection() with a StreamReader and StreamWriter. Because the callback is a coroutine function, asyncio schedules the returned coroutine as a Task.

Each connection consequently has an independent execution path. When one handler waits for a request, a timer, or an asynchronous database call, its Task suspends and the loop can run another connection's Task.

The qualification around database work is important: awaiting helps only when the operation itself integrates with asynchronous I/O. Calling a synchronous database client inside an async def handler still blocks the event-loop thread.

Parsing an HTTP Request from a Byte Stream

TCP provides an ordered stream of bytes, not HTTP messages. A read may return part of a request, exactly one request, or bytes belonging to several requests. StreamReader supplies buffering helpers that let the parser work with protocol boundaries rather than TCP packet boundaries.

An HTTP/1.1 request starts with a request line, followed by headers, a blank line, and an optional body:

async def read_request(reader):
    line = await reader.readline()
    if not line:
        return None  # client closed the connection
    method, path, version = line.decode().strip().split(" ")

    headers = {}
    while True:
        header_line = await reader.readline()
        if header_line in (b"\r\n", b""):
            break
        name, _, value = header_line.decode().partition(":")
        headers[name.strip().lower()] = value.strip()

    body = b""
    if "content-length" in headers:
        body = await reader.readexactly(int(headers["content-length"]))

    return {"method": method, "path": path, "version": version, "headers": headers, "body": body}

readline() may suspend several times before a complete line is available. readexactly() does the same until it has collected the requested number of body bytes. A single low-level recv() is never guaranteed to return an entire request body, regardless of how the client wrote it.

This parser is suitable only as a teaching example. It does not limit line, header, or body sizes; validate the request target or HTTP version; reject malformed headers; support chunked transfer encoding; preserve duplicate headers; or handle conflicting message-length fields. Those omissions are security boundaries in a public server, not optional protocol polish.

Dispatching Requests and Preserving Connections

A minimal router can map request paths to coroutine functions:

ROUTES = {}

def route(path):
    def wrap(fn):
        ROUTES[path] = fn
        return fn
    return wrap

@route("/")
async def index(req):
    return make_response("200 OK", b"hello from the toy server\n")

@route("/slow")
async def slow(req):
    await asyncio.sleep(1)  # simulate an I/O-bound handler: DB call, upstream request, etc.
    return make_response("200 OK", b"done sleeping\n")

async def handle_connection(reader, writer):
    try:
        while True:
            req = await asyncio.wait_for(read_request(reader), timeout=30)
            if req is None:
                break

            handler = ROUTES.get(req["path"])
            resp = await handler(req) if handler else make_response("404 Not Found", b"not found\n")

            writer.write(resp)
            await writer.drain()

            if req["headers"].get("connection", "").lower() == "close":
                break
    except (asyncio.IncompleteReadError, ConnectionResetError, asyncio.TimeoutError):
        pass
    finally:
        writer.close()
        await writer.wait_closed()

The loop implements persistent connections: after sending a response, the handler returns to read_request() instead of closing the socket. That avoids a new TCP connection for every request.

For HTTP/1.1, persistence is the default unless the client requests Connection: close. A complete implementation must also account for HTTP/1.0 semantics, parse comma-separated connection tokens, and ensure every response has an unambiguous message boundary—typically Content-Length for a small fixed body.

A minimal response serializer can make that boundary explicit:

def make_response(status, body):
    headers = (
        f"HTTP/1.1 {status}\r\n"
        f"Content-Length: {len(body)}\r\n"
        "Content-Type: text/plain; charset=utf-8\r\n"
        "\r\n"
    )
    return headers.encode("ascii") + body

The 30-second timeout limits how long an idle or extremely slow client can retain a connection Task while sending no complete request. Real servers apply several independent controls: header timeouts, body timeouts, keep-alive timeouts, maximum request sizes, connection limits, and often per-client limits.

With the loop above, three requests can reuse the same connection:

/ -> HTTP/1.1 200 OK (same connection)
/ -> HTTP/1.1 200 OK (same connection)
/ -> HTTP/1.1 200 OK (same connection)

drain() Applies Backpressure

writer.write(resp) passes bytes to the transport, which attempts to write them and buffers anything the socket cannot currently accept. The method does not wait for the remote client to receive those bytes.

await writer.drain() participates in flow control. It returns quickly while the transport's write buffer remains below its high-water mark. If the buffer grows too large, it suspends the connection Task until enough buffered data has been handed to the operating system for the buffer to fall below the low-water mark.

That distinction prevents a fast producer from allocating memory indefinitely while writing to a slow client. drain() is not a delivery acknowledgement and does not guarantee that the peer has read the response. It bounds producer pressure against the local transport buffer.

For tiny responses, the buffer may never reach its limit, so drain() often appears to do nothing. It becomes essential when responses are large, streamed, or sent to slow consumers.

Measuring Concurrent Waiting

The /slow handler provides a controlled way to observe scheduling. It spends one second waiting on an asynchronous timer, which approximates the scheduling shape of an asynchronous database or upstream request without introducing external variability.

The client opens five connections concurrently:

async def fetch(path, n):
    reader, writer = await asyncio.open_connection("127.0.0.1", 8899)
    writer.write(f"GET {path} HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n".encode())
    await writer.drain()
    data = await reader.read()
    writer.close()
    await writer.wait_closed()
    return n, data.decode().splitlines()[0]

async def main():
    start = time.monotonic()
    results = await asyncio.gather(*[fetch("/slow", i) for i in range(5)])
    print(f"took {time.monotonic() - start:.2f}s total")

A representative run looks like this:

client 0: HTTP/1.1 200 OK
client 1: HTTP/1.1 200 OK
client 2: HTTP/1.1 200 OK
client 3: HTTP/1.1 200 OK
client 4: HTTP/1.1 200 OK
5 concurrent /slow requests (each sleeps 1s) took 1.01s total

All five handlers start, reach await asyncio.sleep(1), and suspend. The event loop registers their timers and remains available to accept connections or run other callbacks. Roughly one second later, the timers become ready and the five Tasks continue.

The correct comparison is not a thread-per-request server: five threads blocked concurrently for one second can also finish in approximately one second. The contrasting implementation is a sequential blocking server, or an async server whose handlers call time.sleep(1) on the single event-loop thread. Either would process these sleeps serially and take roughly five seconds.

The result demonstrates concurrency, not higher execution speed. Async reduces the resources required to keep many waiting operations in flight. Whether it improves throughput depends on the workload, downstream capacity, connection limits, and how consistently handlers avoid blocking the loop.

When Async Stops Helping

The event loop can switch Tasks only when running code suspends. Replace the simulated I/O with a blocking call and every connection stalls:

@route("/blocking")
async def blocking(req):
    time.sleep(1)  # blocks the event-loop thread
    return make_response("200 OK", b"done\n")

The same applies to CPU-heavy Python code. A long computation that contains no await monopolizes the loop just as effectively as time.sleep(). The usual remedies are an asynchronous client for I/O, asyncio.to_thread() for blocking I/O that cannot be replaced, and a process pool or external worker for sustained CPU-bound work.

Async also introduces its own failure modes. Unbounded concurrency can overwhelm databases and upstream APIs even when the event loop remains responsive. Semaphores, bounded queues, connection pools, timeouts, and admission control are part of a production async architecture because cheap Tasks do not make downstream capacity infinite.

In the Real World

The minimal server exposes the scheduling model, but it leaves out the work that makes an HTTP implementation safe and interoperable:

• complete request-line and header validation;
• request-line, header, body, and connection limits;
• chunked transfer encoding and robust message framing;
• HTTP pipelining behavior and ordered responses;
• TLS, shutdown coordination, and signal handling;
• structured access and error logging;
• overload protection and graceful connection draining;
• defenses against slow clients and request smuggling;
• correct cancellation and exception reporting;
• HTTP/2 or HTTP/3 protocol support.

Frameworks and protocol servers such as aiohttp, Uvicorn, Hypercorn, and their underlying HTTP libraries handle these concerns so application code does not have to. Alternative event loops such as uvloop can improve throughput for some workloads, but they are an optimization to benchmark, not a substitute for sound protocol handling or non-blocking application code.

The architecture underneath remains compact. The listener turns accepted connections into Tasks. Stream operations suspend those Tasks when sockets cannot make progress. The kernel reports readiness to the event loop. Completed Futures place Tasks back on the ready queue. Each Task then runs until it reaches another suspension point.

That cycle is what allows one event-loop thread to coordinate many open connections—and what makes a single blocking handler capable of delaying all of them.