Building a Video Metadata Service with Litestar and Async Python

Our video metadata endpoint used to spend most of its time waiting. A single request to fetch a video's details fanned out to a thumbnail CDN, a transcript store, and our own SQLite catalog, and the old synchronous WSGI worker sat blocked on each network call one after another. Under load, the worker pool filled with threads that were doing nothing but sleeping on sockets. On DailyWatch we serve free video discovery to a global audience, and the metadata service is the hot path behind every card, every search result, and every watch page. When it stalls, everything downstream feels it. This is the story of rebuilding that service on Litestar, the async Python framework formerly known as Starlite, and the specific decisions that made it fast, typed, and boring to operate.

Our main site runs on PHP 8.4 with SQLite FTS5 behind LiteSpeed and Cloudflare, and that stack is not going anywhere. But the metadata aggregation layer, the piece that stitches together data from a dozen upstreams before the PHP frontend renders anything, was the wrong job for a blocking runtime. Async I/O is exactly the shape of that problem, and Litestar gave us a way to express it without dragging in a heavyweight framework.

Why the aggregation layer needed to be async

The metadata service does not do heavy CPU work. It does not transcode, it does not run models. What it does is wait: on HTTP calls to upstream providers, on disk reads, on a couple of Redis lookups. That is the textbook case where async wins. A synchronous worker handling a request that makes three 80ms network calls in sequence is busy for 240ms but computing for maybe two of those milliseconds. Multiply that by real traffic and you are paying for idle threads.

The measurements that pushed us over the line were concrete:

• p95 latency on the aggregation endpoint was 310ms, and profiling showed 94% of that was I/O wait.
• We were running 32 sync workers to sustain 400 requests per second, and memory footprint was climbing.
• Adding a new upstream (transcripts) would have added another serial blocking call to every request.

With async, those three upstream calls happen concurrently. The request is bound by the slowest single call, not the sum. That alone took our theoretical floor from 240ms to about 90ms per request before any caching.

I evaluated FastAPI and Litestar. Both are solid. Litestar won for us on three points: its dependency injection is a first-class layered system rather than a function-parameter trick, its data-handling layer (DTOs) is built in rather than bolted on, and its plugin architecture made the SQLAlchemy integration genuinely turnkey. FastAPI would have worked, but Litestar's structure matched how we wanted to organize a growing service.

A minimal but real handler

Here is the shape of the aggregation handler. It fans out to three upstreams concurrently using asyncio.gather, merges the results, and returns a typed response. This is close to what actually runs, trimmed for the article.

import asyncio
from dataclasses import dataclass

import httpx
from litestar import Litestar, get
from litestar.exceptions import NotFoundException


@dataclass
class VideoMetadata:
    video_id: str
    title: str
    duration_seconds: int
    thumbnail_url: str
    has_transcript: bool


async def fetch_core(client: httpx.AsyncClient, video_id: str) -> dict:
    resp = await client.get(f"http://catalog.internal/videos/{video_id}")
    if resp.status_code == 404:
        raise NotFoundException(detail=f"video {video_id} not found")
    resp.raise_for_status()
    return resp.json()


async def fetch_thumbnail(client: httpx.AsyncClient, video_id: str) -> str:
    resp = await client.get(f"http://thumbs.internal/{video_id}/best")
    resp.raise_for_status()
    return resp.json()["url"]


async def fetch_transcript_flag(client: httpx.AsyncClient, video_id: str) -> bool:
    resp = await client.get(f"http://transcripts.internal/{video_id}/exists")
    return resp.status_code == 200


@get("/videos/{video_id:str}")
async def get_video(video_id: str, client: httpx.AsyncClient) -> VideoMetadata:
    core, thumb, has_tx = await asyncio.gather(
        fetch_core(client, video_id),
        fetch_thumbnail(client, video_id),
        fetch_transcript_flag(client, video_id),
    )
    return VideoMetadata(
        video_id=video_id,
        title=core["title"],
        duration_seconds=core["duration"],
        thumbnail_url=thumb,
        has_transcript=has_tx,
    )


app = Litestar(route_handlers=[get_video])

Two things worth noting. First, Litestar serializes the VideoMetadata dataclass to JSON automatically via its msgspec-based encoder, which is dramatically faster than the stdlib json module and validates on the way in. Second, the client parameter is injected. It is not a global. That injection is what makes this testable and lets us share a single connection pool across the whole app instead of opening sockets per request.

Dependency injection that shares the connection pool

The single biggest performance mistake people make with async HTTP is creating a new client per request. Each httpx.AsyncClient owns a connection pool; throwing it away means you never get keep-alive reuse and you pay for TLS and TCP setup constantly. Litestar's dependency injection lets you scope one client to the whole application lifetime and hand it to every handler.

from collections.abc import AsyncGenerator

import httpx
from litestar import Litestar
from litestar.di import Provide


async def provide_http_client() -> AsyncGenerator[httpx.AsyncClient, None]:
    limits = httpx.Limits(max_connections=200, max_keepalive_connections=50)
    timeout = httpx.Timeout(5.0, connect=2.0)
    async with httpx.AsyncClient(limits=limits, timeout=timeout) as client:
        yield client


app = Litestar(
    route_handlers=[get_video],
    dependencies={"client": Provide(provide_http_client, use_cache=True)},
)

The use_cache=True flag is the important part. Without it, the generator runs per request and you are back to the per-request-client anti-pattern. With it, Litestar resolves the dependency once and reuses the yielded client, so the keep-alive pool stays warm. The async with block still runs its cleanup on shutdown, closing the pool cleanly.

The explicit timeouts matter more than they look. A 2-second connect timeout and 5-second total means a single hung upstream cannot pin a request forever. In an async world one slow upstream does not block the event loop, but it can still hold a request open and consume a connection slot, so bounding it is non-negotiable. When a video has no thumbnail service response in time, we would rather serve a placeholder than hang the card.

Handling partial failure without failing the whole request

The naive asyncio.gather above has a flaw: if any single upstream raises, the whole request fails. But a missing thumbnail should not blow up a metadata response. The core catalog call is required; the thumbnail and transcript flags are enhancements. We split the fan-out into a hard dependency and soft dependencies, and we degrade gracefully on the soft ones.

import asyncio
import logging

from litestar import get
from litestar.exceptions import NotFoundException

logger = logging.getLogger("metadata")

DEFAULT_THUMB = "https://cdn.dailywatch.video/placeholder.webp"


async def safe(coro, fallback, label: str):
    try:
        return await coro
    except Exception as exc:  # noqa: BLE001 - deliberate degrade path
        logger.warning("soft dependency %s failed: %s", label, exc)
        return fallback


@get("/videos/{video_id:str}")
async def get_video(video_id: str, client) -> VideoMetadata:
    # Required: if this fails, the request fails.
    core = await fetch_core(client, video_id)

    # Optional: degrade instead of failing.
    thumb, has_tx = await asyncio.gather(
        safe(fetch_thumbnail(client, video_id), DEFAULT_THUMB, "thumbnail"),
        safe(fetch_transcript_flag(client, video_id), False, "transcript"),
    )

    return VideoMetadata(
        video_id=video_id,
        title=core["title"],
        duration_seconds=core["duration"],
        thumbnail_url=thumb,
        has_transcript=has_tx,
    )

This pattern is worth internalizing. The NotFoundException from fetch_core still propagates and Litestar turns it into a clean 404. But a flaky thumbnail service produces a warning log and a placeholder, and the user still gets their video. In production this took our error rate on the endpoint from tracking the union of all upstream error rates down to just the catalog's error rate, which is the only one we truly control.

A subtlety: asyncio.gather with return_exceptions=False cancels sibling tasks when one raises. By wrapping each soft call in safe, no exception escapes to gather, so nothing gets cancelled prematurely. If you instead used return_exceptions=True, you would get exception objects back and have to type-check every result, which is uglier and easy to get wrong.

Where SQLite still fits

Our catalog of record is the same SQLite database that powers the PHP frontend, with FTS5 for search. The metadata service does not own that data; it reads a replica. People are often surprised that we run SQLite behind a service handling hundreds of requests per second, but for a read-mostly workload it is superb. The trick is that async Python and SQLite need care, because the SQLite driver is synchronous C and will block the event loop if you call it directly.

We use aiosqlite, which runs the blocking calls in a thread pool, and we keep the database in WAL mode so readers never block on the single writer. The writer is the PHP cron that ingests new videos; the Python service only reads.

import aiosqlite
from litestar import get

DB_PATH = "/var/data/catalog.db"


async def search_videos(query: str, limit: int = 20) -> list[dict]:
    async with aiosqlite.connect(DB_PATH) as db:
        db.row_factory = aiosqlite.Row
        await db.execute("PRAGMA query_only = ON;")
        cursor = await db.execute(
            """
            SELECT v.video_id, v.title, v.duration
            FROM videos_fts f
            JOIN videos v ON v.rowid = f.rowid
            WHERE videos_fts MATCH ?
            ORDER BY rank
            LIMIT ?
            """,
            (query, limit),
        )
        rows = await cursor.fetchall()
        return [dict(r) for r in rows]


@get("/search")
async def search(q: str) -> list[dict]:
    return await search_videos(q)

The videos_fts MATCH ? clause hits the FTS5 index and ORDER BY rank uses FTS5's built-in BM25 ranking. This is the exact same index the PHP side queries, so search results are consistent across the stack. PRAGMA query_only = ON is a cheap safety belt that guarantees this connection can never write, which matters when your service is supposed to be read-only against a shared file.

One warning from experience: if you have any write path in your async SQLite service, serialize your writes through a single connection or a lock. SQLite allows one writer at a time, and hammering it with concurrent writes from an async app produces database is locked errors that are miserable to debug. For us the answer was simply to keep all writes in PHP and let Python read.

The comparison point that made async concrete

To make the async argument to the rest of the team, I wrote the same fan-out in Go, because everyone understands that goroutines are cheap and concurrent. The point was not to switch languages; it was to show that Litestar's async model gives us the same concurrency shape Go gives you for free, in the language our tooling already speaks.

package main

import (
    "encoding/json"
    "net/http"
    "sync"
)

type Metadata struct {
    VideoID   string `json:"video_id"`
    Thumbnail string `json:"thumbnail_url"`
    HasTx     bool   `json:"has_transcript"`
}

func getVideo(id string) Metadata {
    var wg sync.WaitGroup
    m := Metadata{VideoID: id, Thumbnail: "placeholder.webp"}

    wg.Add(2)
    go func() { defer wg.Done(); m.Thumbnail = fetchThumb(id) }()
    go func() { defer wg.Done(); m.HasTx = fetchTranscript(id) }()
    wg.Wait()
    return m
}

func handler(w http.ResponseWriter, r *http.Request) {
    id := r.URL.Query().Get("id")
    json.NewEncoder(w).Encode(getVideo(id))
}

The Go version launches goroutines and waits on a WaitGroup; the Python version launches coroutines and waits on asyncio.gather. Same idea, same latency win. The difference is that we already had Python tooling, msgspec validation, and Litestar's DI, so staying in Python cost us nothing in concurrency and saved us a rewrite of shared logic. Seeing the two side by side is what convinced people that async Python was not a compromise here.

Operational details that actually mattered

The framework choice was maybe 30% of the win. The rest was operational discipline. A few things paid off out of proportion to the effort:

• Run under an ASGI server with real worker management. We run Litestar under uvicorn with --workers set to the core count, fronted by the same LiteSpeed and Cloudflare edge that serves the PHP site. Cloudflare caches the metadata responses with a short TTL, so the origin only sees cache misses.
• Cache at the edge, not just in the app. Because our responses are keyed cleanly by video ID, Cloudflare's cache does most of the work. The async origin is there to make the misses cheap, not to serve every hit. We set Cache-Control: public, max-age=60, stale-while-revalidate=300 so the edge can serve slightly stale data instantly while it refreshes in the background.
• Bound every upstream. Timeouts, connection limits, and the soft-dependency degrade pattern together mean no single upstream can take the service down. This is the difference between a bad five minutes and a bad night.
• Keep the writer single. SQLite in WAL mode with exactly one writer (the PHP cron) and many async readers has been rock solid. We did not need a bigger database; we needed to respect SQLite's concurrency model.
• Lean on msgspec. Litestar's default encoder validates and serializes far faster than pydantic v1 or stdlib json. For a service whose whole job is shuffling JSON, that shows up in the p50.

The results after the migration: p95 dropped from 310ms to 96ms, we went from 32 sync workers to 8 async workers for the same traffic, and adding the transcript upstream cost us zero additional latency because it joined the existing concurrent fan-out instead of adding a serial hop.

What I would tell someone starting today

Litestar is not magic and async is not free. If your service is CPU-bound, none of this helps and you should reach for processes and a queue instead. But if your service is what most metadata and aggregation services are, a thing that mostly waits on other services, then the async fan-out model is the right tool and Litestar is a clean, well-structured way to express it in Python.

Start with the dependency injection and the shared HTTP client, because getting the connection pool right is the difference between fast and pathological. Add explicit timeouts before you add features. Split hard dependencies from soft ones so partial failures degrade instead of cascade. And do not throw out the boring parts of your stack that already work; our SQLite FTS5 catalog and LiteSpeed edge did not need replacing, they needed a faster, concurrent reader in front of them. The migration paid for itself in a week of reduced latency complaints, and the service has been quiet since, which is the highest compliment I can pay a piece of infrastructure.