DEV Community

Cover image for FastAPI SSE Streaming for Real-Time Network Dashboards: Lessons from netpulse Integration
Leo Martinez
Leo Martinez

Posted on

FastAPI SSE Streaming for Real-Time Network Dashboards: Lessons from netpulse Integration

The Problem: Real-Time Router Fleet Monitoring Without WebSocket Overhead

When integrating gnacho/netpulse into our multi-tenant infrastructure monitoring stack, we needed a lightweight mechanism to stream router health metrics (CPU, memory, WireGuard peer status, AdGuard Home block counts) to browser dashboards without the complexity of WebSocket handshakes or bidirectional chat protocols.

Server-Sent Events (SSE) emerged as the natural fit: unidirectional push over plain HTTP, browser-native EventSource API, automatic reconnection logic. But production deployment revealed three critical failure modes that FastAPI's documentation glosses over.

Architecture Context

The netpulse backend exposes a /events endpoint that aggregates telemetry from multiple OpenWrt routers via their ubus JSON-RPC interface. Each client dashboard maintains a persistent SSE connection, receiving JSON payloads every 5 seconds:

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio
import json

app = FastAPI()

async def event_generator(router_ids: list[str]):
    """Async generator yielding SSE-formatted payloads."""
    while True:
        # Fetch telemetry from SQLite or live ubus poll
        metrics = await fetch_router_metrics(router_ids)
        payload = json.dumps({
            "routers": metrics,
            "timestamp": int(time.time())
        })
        yield f"data: {payload}\n\n"
        await asyncio.sleep(5)

@app.get("/events")
async def stream_events(router_ids: str):
    ids = router_ids.split(",")
    return StreamingResponse(
        event_generator(ids),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "X-Accel-Buffering": "no"  # Disable nginx buffering
        }
    )
Enter fullscreen mode Exit fullscreen mode

Failure Mode 1: Unclosed Database Connections in Infinite Loops

Our first production incident occurred 6 hours after deployment: the connection pool exhausted with 200+ idle SQLite connections. The root cause was subtle:

# BROKEN: Connection opened per iteration
async def event_generator(router_ids: list[str]):
    while True:
        async with get_db_session() as db:
            metrics = await db.execute(select(Router).where(...))
        yield f"data: {json.dumps(metrics)}\n\n"
        await asyncio.sleep(5)
Enter fullscreen mode Exit fullscreen mode

Each async with block acquired a new connection but the generator's infinite loop prevented garbage collection. Fix: hoist the session acquisition outside the loop or use a singleton connection with explicit health checks:

async def event_generator(router_ids: list[str]):
    async with get_db_session() as db:
        while True:
            metrics = await db.execute(select(Router).where(...))
            yield f"data: {json.dumps(metrics)}\n\n"
            await asyncio.sleep(5)
Enter fullscreen mode Exit fullscreen mode

Failure Mode 2: Client Disconnections Leave Zombie Generators

When a browser tab closes or the client network drops, FastAPI does not automatically stop the async generator. Our monitoring showed 50+ orphaned generators still polling the database after clients disconnected.

The solution requires explicit cancellation checks using asyncio.CancelledError:

async def event_generator(router_ids: list[str]):
    try:
        async with get_db_session() as db:
            while True:
                metrics = await db.execute(select(Router).where(...))
                yield f"data: {json.dumps(metrics)}\n\n"
                await asyncio.sleep(5)
    except asyncio.CancelledError:
        # Client disconnected - cleanup resources
        await db.close()
        raise
Enter fullscreen mode Exit fullscreen mode

Crucially, FastAPI's StreamingResponse does raise CancelledError when the client disconnects, but only if you're awaiting something. A synchronous time.sleep(5) would block indefinitely.

Failure Mode 3: Synchronous Blocking Calls in Async Generators

During load testing with 100 concurrent SSE clients, we observed 2-second stalls where all connections froze. Profiling revealed a synchronous requests.get() call inside the generator:

# BROKEN: Blocks the event loop
async def event_generator(router_ids: list[str]):
    while True:
        response = requests.get(f"http://{router_ip}/ubus")  # BLOCKING
        yield f"data: {response.text}\n\n"
        await asyncio.sleep(5)
Enter fullscreen mode Exit fullscreen mode

The fix requires using httpx.AsyncClient or wrapping the blocking call in asyncio.to_thread():

import httpx

async def event_generator(router_ids: list[str]):
    async with httpx.AsyncClient() as client:
        while True:
            response = await client.get(f"http://{router_ip}/ubus")
            yield f"data: {response.text}\n\n"
            await asyncio.sleep(5)
Enter fullscreen mode Exit fullscreen mode

Production Hardening: Pydantic Validators for SSE Payloads

To prevent malformed JSON from breaking the event stream, we wrapped all outbound data in Pydantic v2 models:

from pydantic import BaseModel, Field, field_validator
from datetime import datetime

class RouterMetrics(BaseModel):
    router_id: str
    cpu_percent: float = Field(ge=0.0, le=100.0)
    memory_mb: int = Field(ge=0)
    wireguard_peers: int = Field(ge=0)
    timestamp: datetime

    @field_validator('cpu_percent')
    @classmethod
    def clamp_cpu(cls, v: float) -> float:
        return min(max(v, 0.0), 100.0)

    model_config = {
        "json_encoders": {
            datetime: lambda dt: int(dt.timestamp())
        }
    }
Enter fullscreen mode Exit fullscreen mode

This caught a bug where negative CPU percentages from buggy router firmware would cause JSON serialization to fail mid-stream.

Benchmark: SSE vs. Polling in Multi-Tenant Scenarios

When comparing netpulse's SSE implementation against a naive 5-second polling REST API, we measured a 70% reduction in database query load (since the generator maintains a single long-lived connection per client rather than repeated short-lived requests). Network bandwidth dropped by 40% due to eliminating HTTP request headers on every poll.

For latency-sensitive telemetry where sub-second updates matter, SSE's persistent connection eliminated the cold-start overhead of TLS handshakes and TCP slow-start. In our tests routing through a multi-provider relay gateway (similar to production topologies using baseURL: "https://b-lost.com/v1" for model inference proxying), time-to-first-byte remained under 50ms even with 200 concurrent SSE streams.

Open Questions for High-Scale Deployments

One unresolved trade-off: FastAPI runs SSE generators in the same event loop as regular HTTP handlers. Under extreme load (500+ concurrent SSE clients), does the constant asyncio.sleep() wakeup churn degrade latency for non-streaming endpoints? Our profiling showed a 15ms p99 increase in regular REST API response times, suggesting a need for dedicated worker processes or load balancer affinity rules.

Another edge case: how do you handle schema evolution when clients maintain 8-hour SSE connections? Netpulse currently forces a client reload on backend restart, but a production system might need versioned event types or graceful degradation when new fields appear mid-stream.

What strategies have you used to manage long-lived async generators in production? Have you hit connection pool limits or event loop contention with high SSE concurrency?

Top comments (0)