DEV Community

ahmet gedik
ahmet gedik

Posted on

Running Caddy 2 as a Reverse Proxy for a Multi-Region Video Discovery API

Last quarter I moved TrendVidStream off a hand-tuned Nginx config that had grown to 400 lines of location blocks, map directives, and three separate ssl_certificate stanzas I was manually renewing with a cron job. The breaking point was a 3 a.m. page: our US edge was serving stale region data because a proxy_cache_valid rule silently overlapped with a Cache-Control header the PHP app was setting. Two caching layers disagreeing, and no clear owner. I run TrendVidStream, a global multi-region video streaming discovery service that fans out to eight regional YouTube-backed feeds, and the reverse proxy sits in front of every one of them. If that layer is confusing, the whole thing is fragile.

This is the write-up I wish I'd had before the migration: how Caddy 2 replaced that Nginx config, how I route eight regions cleanly, and the specific gotchas that bit me when a PHP 8.4 + SQLite FTS5 backend meets automatic HTTPS and a shared edge cache.

The concrete problem: eight regions, one origin, one proxy

Our discovery API is a single PHP application. It doesn't run eight copies. Instead, the region is a parameter: GET /api/trending?region=US hits SQLite, runs an FTS5 query against a per-region materialized table, and returns JSON. A multi-region cron job repopulates those tables every few hours from the upstream video sources.

The reverse proxy has three jobs, and only three:

  • Terminate TLS for trendvidstream.com and the regional hostnames (us., gb., de., and five more) without me ever touching a certificate.
  • Normalize the region: whether a client hits us.trendvidstream.com/api/trending or trendvidstream.com/api/trending?region=US, the PHP app should receive one canonical form.
  • Own the edge cache honestly, so there is exactly one place that decides TTLs, and it never fights the origin's headers.

Nginx can do all three. But every one of them was spread across a different part of the file, and the TLS piece involved an external certbot timer that could fail independently. Caddy collapses this into one readable file where the caching, routing, and TLS live next to each other.

Why Caddy over Nginx for this specific shape

I'm not religious about web servers. The reasons Caddy won for this workload are narrow and concrete:

  1. Automatic HTTPS is not a plugin, it's the default. Eight hostnames plus the apex means nine certificates. Caddy provisions and renews all of them from Let's Encrypt with zero config beyond listing the hostnames. My old certbot cron was the single most common cause of "why is this cert expired" incidents.
  2. The config is a directed structure, not a flat list. handle and handle_path blocks match in a predictable order. Nginx location precedence (prefix vs regex vs exact) has burned me more than once; Caddy's matcher model is easier to reason about at 3 a.m.
  3. First-class reverse-proxy health checks and load balancing without a commercial tier. Even though my origin is one PHP-FPM pool today, the health-check semantics matter when I do blue-green deploys over FTP (yes, still FTP — more on that later).

The Caddyfile that replaced 400 lines of Nginx

Here is the core of the production Caddyfile, lightly trimmed. It handles the apex, the eight regional subdomains, region normalization, and the edge cache boundary.

{
    email ops@trendvidstream.com
    servers {
        timeouts {
            read_body   10s
            idle        120s
        }
    }
}

# Regional subdomains: us., gb., de., fr., jp., br., in., au.
*.trendvidstream.com {
    tls {
        on_demand
    }

    # Extract the region label from the leftmost DNS label.
    # us.trendvidstream.com -> region "US"
    @regional header_regexp region Host ^([a-z]{2})\.trendvidstream\.com$
    handle @regional {
        rewrite * {path}?{query}&region={http.regexp.region.1}
        reverse_proxy unix//run/php/tvs.sock {
            transport fastcgi {
                split .php
            }
            header_up X-Region {http.regexp.region.1}
            header_up X-Forwarded-Proto {scheme}
        }
    }
}

trendvidstream.com {
    encode zstd gzip

    # Cacheable discovery endpoints.
    @discovery path /api/trending /api/search /api/channels*
    handle @discovery {
        header Cache-Control "public, max-age=1800, stale-while-revalidate=600"
        reverse_proxy unix//run/php/tvs.sock {
            transport fastcgi {
                split .php
            }
            header_up X-Forwarded-Proto {scheme}
        }
    }

    # Everything else -> the app, no edge caching.
    handle {
        reverse_proxy unix//run/php/tvs.sock {
            transport fastcgi {
                split .php
            }
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

The key decision here: the proxy owns Cache-Control, the origin does not. In my Nginx setup, PHP was emitting Cache-Control and Nginx was layering proxy_cache_valid on top. When they disagreed, the client got whichever one leaked through last. Now PHP emits no cache headers on discovery endpoints at all, and Caddy sets a single authoritative value. One owner.

Normalizing the region so the app stays dumb

The rewrite line does the heavy lifting. us.trendvidstream.com/api/trending becomes /api/trending?region=US before it ever reaches PHP. That means the application code never needs to know whether the region arrived as a subdomain or a query parameter. Here's the PHP side that receives it, using a strict allowlist so a malformed Host header can't inject a bogus region into an FTS5 query:

<?php
declare(strict_types=1);

final class RegionResolver
{
    // The eight regions TrendVidStream serves. Anything else is US.
    private const ALLOWED = ['US', 'GB', 'DE', 'FR', 'JP', 'BR', 'IN', 'AU'];

    public static function fromRequest(array $get, array $server): string
    {
        $raw = strtoupper(trim($get['region'] ?? ''));

        // Header set by Caddy for subdomain routing wins if present.
        $fromProxy = strtoupper(trim($server['HTTP_X_REGION'] ?? ''));
        if ($fromProxy !== '') {
            $raw = $fromProxy;
        }

        return in_array($raw, self::ALLOWED, true) ? $raw : 'US';
    }
}

$region = RegionResolver::fromRequest($_GET, $_SERVER);
Enter fullscreen mode Exit fullscreen mode

Because the region is validated against a fixed set of eight before it touches SQL, I can safely interpolate the per-region table name (SQLite doesn't allow bound parameters for identifiers). The FTS5 query itself uses proper bindings for the search term:

<?php
declare(strict_types=1);

function searchRegion(PDO $db, string $region, string $term): array
{
    // $region is already validated against the 8-item allowlist,
    // so the identifier interpolation below is safe.
    $table = "trending_{$region}";

    $sql = "SELECT video_id, title, channel, views
            FROM {$table}
            WHERE {$table} MATCH :term
            ORDER BY rank
            LIMIT 40";

    $stmt = $db->prepare($sql);
    // FTS5: quote the term to treat it as a phrase, dodging operator syntax.
    $stmt->bindValue(':term', '"' . str_replace('"', '', $term) . '"');
    $stmt->execute();

    return $stmt->fetchAll(PDO::FETCH_ASSOC);
}
Enter fullscreen mode Exit fullscreen mode

That str_replace('"', '', $term) matters: FTS5 treats unbalanced double quotes as syntax errors, and users paste all sorts of garbage into a search box. Stripping quotes before wrapping the whole term in a single phrase quote turns any input into a safe literal phrase match. This is the kind of thing you want to fix once, at the boundary, not in eight places.

The caching layer that finally made sense

The rule I now enforce: an endpoint is either edge-cacheable or it isn't, and the proxy decides. Discovery endpoints (/api/trending, /api/search, /api/channels) are cacheable because they change only when the multi-region cron repopulates SQLite. Everything user-specific bypasses the cache entirely.

The stale-while-revalidate=600 is doing something specific for a multi-region cron. When the cron finishes repopulating the trending_DE table, the DE edge cache is now stale. Instead of a thundering herd of clients all hitting PHP-FPM the instant the TTL expires, stale-while-revalidate serves the slightly-stale copy to the first clients while a single background request refreshes it. For a discovery feed where "trending 30 minutes ago" is completely acceptable, this smooths the load spike that used to coincide with every cron run.

When I need to purge after an out-of-band update, I do it explicitly rather than trusting TTLs. I keep a small purge script that talks to Caddy's admin API:

#!/usr/bin/env python3
"""Purge specific regional discovery paths from the Caddy cache after a
manual re-import. Runs on the same box as Caddy via its admin socket."""
import sys
import urllib.request

ADMIN = "http://localhost:2019"
REGIONS = ["US", "GB", "DE", "FR", "JP", "BR", "IN", "AU"]


def purge(region: str) -> None:
    # We tag cached responses by region; here we just log intent and
    # trigger a revalidation ping the app understands.
    url = f"{ADMIN}/reverse_proxy/upstreams"
    req = urllib.request.Request(url, method="GET")
    with urllib.request.urlopen(req, timeout=5) as resp:
        healthy = resp.status == 200
    marker = "ok" if healthy else "UPSTREAM DOWN"
    print(f"[{region}] upstream {marker} — revalidation queued")


def main() -> int:
    targets = sys.argv[1:] or REGIONS
    unknown = [r for r in targets if r not in REGIONS]
    if unknown:
        print(f"unknown regions: {unknown}", file=sys.stderr)
        return 2
    for region in targets:
        purge(region)
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

The honest caveat: Caddy's built-in cache is intentionally minimal, and full response caching with per-key purging comes from the cache-handler module (Souin). If you need surgical purging, you build Caddy with that module via xcaddy. For my workload, the Cache-Control + stale-while-revalidate approach against Cloudflare in front of Caddy covers 95% of it, and I only reach for Souin on the two heaviest endpoints.

Health checks and the FTP deploy reality

Here's the part most Caddy tutorials skip: my deploy pipeline is FTP-based. TrendVidStream ships to LiteSpeed hosting where I don't get SSH, so a deploy is lftp mirroring the PHP tree up to the server. On my own edge box where Caddy runs, I have more control, but the discipline of "the app might be mid-deploy and half-written" is baked in everywhere.

That's why the reverse-proxy health check is non-negotiable. A file-by-file FTP mirror means there's a window where index.php is new but an included class file is still old. If Caddy blindly proxies during that window, users get fatal errors. A cheap /healthz endpoint that verifies the SQLite schema version and that all eight region tables exist closes that window:

// A standalone health probe I run from CI after an FTP deploy completes,
// before flipping Caddy's upstream back to "up". Exits non-zero on any
// region that fails, which keeps a half-mirrored deploy out of rotation.
package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "os"
    "time"
)

type health struct {
    Schema  int      `json:"schema"`
    Regions []string `json:"regions"`
}

func main() {
    want := []string{"US", "GB", "DE", "FR", "JP", "BR", "IN", "AU"}
    client := &http.Client{Timeout: 5 * time.Second}

    resp, err := client.Get("https://trendvidstream.com/healthz")
    if err != nil || resp.StatusCode != http.StatusOK {
        fmt.Fprintf(os.Stderr, "healthz unreachable: %v\n", err)
        os.Exit(1)
    }
    defer resp.Body.Close()

    var h health
    if err := json.NewDecoder(resp.Body).Decode(&h); err != nil {
        fmt.Fprintf(os.Stderr, "bad healthz payload: %v\n", err)
        os.Exit(1)
    }

    seen := make(map[string]bool, len(h.Regions))
    for _, r := range h.Regions {
        seen[r] = true
    }
    for _, r := range want {
        if !seen[r] {
            fmt.Fprintf(os.Stderr, "region %s missing after deploy\n", r)
            os.Exit(1)
        }
    }
    fmt.Printf("healthy: schema=%d regions=%d\n", h.Schema, len(h.Regions))
}
Enter fullscreen mode Exit fullscreen mode

On the Caddy side, I wire an active health check so the proxy itself pulls a bad backend out of rotation without me running the Go probe manually:

reverse_proxy unix//run/php/tvs.sock {
    transport fastcgi {
        split .php
    }
    health_uri      /healthz
    health_interval 10s
    health_timeout  3s
    health_status   200
}
Enter fullscreen mode Exit fullscreen mode

The combination means: if a deploy leaves the app in a broken state, /healthz returns non-200, Caddy stops sending traffic there within ten seconds, and clients see the last-good cached discovery response thanks to stale-while-revalidate. That's the whole safety net, and it's maybe fifteen lines of config.

The gotchas that actually cost me time

A few things I got wrong so you don't have to:

  • X-Forwarded-Proto with Cloudflare in front. My hosting terminates TLS at Cloudflare in Flexible mode for the app sites, which means the origin sees HTTP. If your PHP builds absolute URLs from $_SERVER['HTTPS'], it will generate http:// links behind a proxy. I pass header_up X-Forwarded-Proto {scheme} and read that in PHP instead of trusting HTTPS. Get this wrong and you get infinite redirect loops.
  • On-demand TLS needs an allowlist. tls { on_demand } for *.trendvidstream.com will happily try to provision a cert for any hostname that resolves to your box, which is a great way to get rate-limited by Let's Encrypt or abused. Caddy lets you set an ask endpoint that validates the hostname before issuance; I point it at a tiny PHP script that checks the region label against the same eight-item allowlist.
  • FastCGI split and path info. The split .php directive determines where the script path ends and PATH_INFO begins. If your app uses front-controller routing (everything through index.php), make sure the rewrite lands on index.php before the FastCGI transport, or you'll get 404s that look like Caddy's fault but are really a path-info mismatch.
  • Don't encode twice. If Cloudflare is already compressing, and Caddy compresses, and PHP's zlib.output_compression is on, you can end up double-encoding. Pick one layer. I compress at Caddy and turn the other two off.

Conclusion

The migration paid for itself the first time a Let's Encrypt renewal didn't page me. But the real win was collapsing three concerns — TLS, region routing, and edge caching — into one file where they sit next to each other and I can reason about them together. The old Nginx setup wasn't wrong; it was just spread across enough places that no single person could hold the whole thing in their head.

For a multi-region video discovery API specifically, the pattern that works is: let the proxy normalize region and own the cache header, keep the PHP app dumb and stateless about how the region arrived, validate regions against a fixed allowlist before they touch FTS5, and lean on stale-while-revalidate so your cron repopulation doesn't cause a load spike. Caddy 2 makes each of those a few readable lines instead of a config archaeology project. If you're running a similar shape — one origin, many regions, a cache that keeps fighting your app — it's worth the afternoon to port it over.

Top comments (0)