Building a Video CDN Edge Cache Layer with Varnish and ESI for Trending Pages

#varnish #caching #php #performance

On TopVideoHub our trending pages are 95% identical for every visitor in a region and 5% deadly personal. The video grid for "Japan trending music this week" is the same JSON blob for a million people. But the top-right corner shows the viewer's saved-videos count, their language toggle state, and a CJK search box that remembers the last query. For a long time we cached the whole page for 60 seconds and accepted that logged-in users saw a stale saved-count, or we skipped caching entirely and hammered PHP 8.4 + SQLite FTS5 on every request. Both were wrong. The fix was Edge Side Includes (ESI) in front of TopVideoHub: cache the expensive 95% aggressively at the edge, and let a tiny uncacheable fragment fill in the personal 5%.

This article is the actual Varnish + ESI setup we run, the mistakes that cost us a week, and the code that stitches a mostly-static trending page together from fragments with different TTLs. It assumes you have a backend that can render both a full page and standalone fragments — ours is PHP behind LiteSpeed, but the pattern is stack-agnostic.

Why a full-page cache breaks on aggregation sites

A video aggregator's cost is not bandwidth — Cloudflare handles the thumbnails and HLS segments. The cost is assembly. Rendering /trending/jp/music means:

Running an FTS5 query with our CJK tokenizer to rank ~40 candidate videos
Hydrating each with view counts, channel metadata, and localized titles
Building the language switcher for 7 CJK + SEA locales
Rendering the personalized header

The first three are identical for everyone hitting that URL in that region. The fourth is per-user. A naive Cache-Control: max-age=60 cache is forced to pick the TTL of its most volatile part. So the whole page inherits the personalized header's "never cache" requirement, and you throw away the 40ms FTS5 query result on every hit.

ESI inverts this. The cached object stored at the edge is not HTML — it's a template with holes. Varnish caches the template with a long TTL, and each hole (<esi:include>) is a separate cacheable object with its own TTL. The personalized header becomes a fragment with TTL 0; the video grid becomes a fragment with TTL 60s; the language switcher becomes a fragment with TTL 24h. One request, three cache policies.

The page as a composition of fragments

Here's the skeleton our PHP emits for a trending page when Varnish asks for the outer template. Note it contains almost no data — just <esi:include> tags pointing at fragment URLs.

<?php
// GET /trending/jp/music  (the ESI skeleton)
// Varnish caches THIS for 300s; the holes fill independently.

header('Content-Type: text/html; charset=utf-8');
header('Surrogate-Control: content="ESI/1.0"'); // tell Varnish to parse ESI
header('Cache-Control: s-maxage=300, max-age=0');

$region = 'jp';
$cat    = 'music';
?>
<!doctype html>
<html lang="ja">
<head><meta charset="utf-8"><title>Trending Music — Japan</title></head>
<body>
  <esi:include src="/_frag/header?region=<?= $region ?>" />
  <esi:include src="/_frag/langswitch?region=<?= $region ?>" />
  <main>
    <esi:include src="/_frag/grid?region=<?= $region ?>&cat=<?= $cat ?>" />
  </main>
  <esi:include src="/_frag/footer" />
</body>
</html>

The important header is Surrogate-Control: content="ESI/1.0". Varnish only parses ESI tags when explicitly told to, per-response. You do not want to blindly parse ESI on every response — that's an injection risk if user-generated content can contain the literal string <esi:include>. Emit the surrogate header only on templates you control.

The VCL that ties it together

This is the core of our default.vcl (Varnish 7.x). It enables ESI processing only when the backend opts in, and it strips cookies from fragment requests that must be cacheable while preserving them for the personalized header.

vcl 4.1;

backend php {
    .host = "127.0.0.1";
    .port = "8080";        # LiteSpeed / php-fpm listener
    .connect_timeout = 2s;
    .first_byte_timeout = 5s;
}

sub vcl_recv {
    # The personalized header fragment must never be cached from a shared object.
    if (req.url ~ "^/_frag/header") {
        return (pass);     # always go to backend, respects the session cookie
    }

    # All other fragments and skeletons are anonymous — drop cookies so they
    # hash into one shared cache object per URL+region.
    unset req.http.Cookie;

    # Normalize the region so ?region=jp and ?region=JP share one object.
    if (req.url ~ "[?&]region=") {
        set req.url = std.tolower(req.url);
    }
    return (hash);
}

sub vcl_backend_response {
    # Only parse ESI when the backend asked us to via Surrogate-Control.
    if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
        set beresp.do_esi = true;
        unset beresp.http.Surrogate-Control;
    }

    # Grid fragments: short TTL, allow stale serving while we refetch.
    if (bereq.url ~ "^/_frag/grid") {
        set beresp.ttl = 60s;
        set beresp.grace = 30s;
    }
    # Langswitch / footer: rarely change.
    else if (bereq.url ~ "^/_frag/(langswitch|footer)") {
        set beresp.ttl = 24h;
    }
    # The skeleton itself.
    else if (beresp.http.Cache-Control ~ "s-maxage") {
        set beresp.ttl = 300s;
    }
}

sub vcl_deliver {
    set resp.http.X-Cache = obj.hits > 0 ? "HIT" : "MISS";
}

Two things here caused most of our early pain.

First, cookie stripping must be per-URL. If you unset req.http.Cookie globally, the personalized header can never see the session and everyone looks logged out. If you never strip it, every fragment hashes per-user and your hit rate collapses to zero. The return (pass) on /_frag/header before the global unset is what makes both work.

Second, grace. When a grid fragment's 60s TTL expires, Varnish would normally block the request while it refetches from PHP. With set beresp.grace = 30s, Varnish serves the slightly-stale fragment instantly and refreshes it in the background. On a trending page that updates every few minutes anyway, 30s of staleness is invisible, and it means an FTS5 hiccup never becomes a user-facing latency spike.

Rendering the fragments

Each fragment is a normal route that returns a partial. The trick is that a fragment must be renderable standalone — it can't assume outer-page state. Here's the grid fragment, the expensive one, in PHP:

<?php
// GET /_frag/grid?region=jp&cat=music

$region = preg_replace('/[^a-z]/', '', $_GET['region'] ?? 'us');
$cat    = preg_replace('/[^a-z0-9]/', '', $_GET['cat'] ?? 'all');

// This is the whole point: the costly query lives behind a cacheable URL.
$db = new PDO('sqlite:' . __DIR__ . '/../data/videos.db');
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

// FTS5 with our CJK trigram tokenizer, ranked by a trending score.
$sql = <<<SQL
    SELECT v.id, v.title, v.channel, v.views, v.thumb
    FROM videos_fts f
    JOIN videos v ON v.id = f.rowid
    WHERE f.region = :region
      AND (:cat = 'all' OR f.category = :cat)
    ORDER BY v.trend_score DESC
    LIMIT 40
SQL;

$stmt = $db->prepare($sql);
$stmt->execute([':region' => $region, ':cat' => $cat]);
$rows = $stmt->fetchAll(PDO::FETCH_ASSOC);

// Fragment TTL is set by Varnish VCL, but we also send a matching header
// so the fragment is correct even if served without Varnish (dev, origin pull).
header('Cache-Control: s-maxage=60, stale-while-revalidate=30');
header('Content-Type: text/html; charset=utf-8');

echo '<div class="grid">';
foreach ($rows as $r) {
    printf(
        '<a class="card" href="/v/%d"><img src="%s" loading="lazy" alt="%s"><h3>%s</h3><span>%s views</span></a>',
        $r['id'],
        htmlspecialchars($r['thumb'], ENT_QUOTES),
        htmlspecialchars($r['title'], ENT_QUOTES),
        htmlspecialchars($r['title'], ENT_QUOTES),
        number_format((int)$r['views'])
    );
}
echo '</div>';

Because this URL is stable and anonymous, Varnish stores exactly one copy per region+cat and serves it to everyone. Our FTS5 query runs once per minute per region-category, not once per request. On the JP music page that took us from ~4,000 query executions/minute at peak to under 50.

The personalized header, by contrast, is deliberately tiny and always hits the backend:

<?php
// GET /_frag/header?region=jp   (return (pass) in VCL — never shared-cached)

session_start();
$uid   = $_SESSION['uid'] ?? null;
$saved = 0;

if ($uid !== null) {
    $db = new PDO('sqlite:' . __DIR__ . '/../data/videos.db');
    $saved = (int)$db->query(
        'SELECT COUNT(*) FROM saved WHERE uid = ' . (int)$uid
    )->fetchColumn();
}

header('Cache-Control: private, no-store');
header('Content-Type: text/html; charset=utf-8');

if ($uid !== null) {
    printf('<header class="me">Saved: <b>%d</b> · <a href="/logout">Sign out</a></header>', $saved);
} else {
    echo '<header class="me"><a href="/login">Sign in</a></header>';
}

This fragment does a single indexed COUNT, no template rendering of the grid, no FTS5. It's cheap precisely because everything expensive has been lifted out into the cacheable fragments around it.

Cache invalidation without a thundering herd

TTL-based expiry is fine for the grid because trending data is naturally time-bounded. But sometimes we need targeted purge — a video gets pulled for a copyright claim and must vanish from every regional grid immediately. Varnish bans handle this. We tag every fragment response and ban by tag when our cron fetcher detects a takedown.

First, tag the response in the fragment (add to the grid fragment's headers):

// A space-delimited surrogate key set: one token per video plus region+cat.
$keys = 'grid ' . $region . ' ' . implode(' ', array_map(
    fn($r) => 'v' . $r['id'], $rows
));
header('xkey: ' . $keys);   // requires the xkey (surrogate keys) vmod

Then a purge is a single HTTP call from our takedown handler, in Go (our cron tooling is a mix of PHP and small Go binaries):

package main

import (
    "fmt"
    "net/http"
)

// purgeVideo evicts every cached fragment that referenced a given video id
// across all regions in one shot, using the xkey surrogate-key ban.
func purgeVideo(varnishAddr string, videoID int) error {
    url := fmt.Sprintf("http://%s/", varnishAddr)
    req, err := http.NewRequest("PURGE", url, nil)
    if err != nil {
        return err
    }
    // The VCL maps PURGE + xkey-purge header to xkey.purge(key).
    req.Header.Set("xkey-purge", fmt.Sprintf("v%d", videoID))

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return err
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        return fmt.Errorf("purge failed: status %d", resp.StatusCode)
    }
    return nil
}

func main() {
    if err := purgeVideo("127.0.0.1:6081", 918273); err != nil {
        fmt.Println("purge error:", err)
    }
}

The matching VCL that turns that request into a surrogate-key ban:

sub vcl_recv {
    if (req.method == "PURGE") {
        if (!client.ip ~ purge_acl) {     # ACL defined elsewhere
            return (synth(403, "Forbidden"));
        }
        if (req.http.xkey-purge) {
            set req.http.n-gone = xkey.purge(req.http.xkey-purge);
            return (synth(200, "Purged " + req.http.n-gone + " objects"));
        }
    }
}

This is dramatically better than a URL-based ban, which forces Varnish to walk its entire object list and test a regex against each — O(n) and a real problem at a few hundred thousand objects. Surrogate keys purge in near-constant time and only touch the objects that actually referenced that video, so a takedown doesn't cold-start every regional grid at once.

Where this sits relative to Cloudflare

We run Cloudflare in front of Varnish, and it's worth being precise about who does what, because doubling up caches naively just doubles your staleness bugs.

Cloudflare caches the final assembled HTML per region using cache rules keyed on the region path, plus all static assets and HLS segments. It has no idea ESI exists — Varnish assembles the page before it leaves the origin, so Cloudflare sees ordinary HTML.
Varnish does the ESI assembly and holds the fragment objects. It's the only layer that understands the composition.
LiteSpeed + PHP 8.4 only ever renders a fragment or a skeleton, never a whole personalized page from scratch.

The subtle rule: the skeleton and grid can be cached by Cloudflare, but any page that includes the live personalized header must not be — otherwise Cloudflare would serve one user's saved-count to everyone. We solve this by keeping personalized pages on a path prefix (/me/...) that a Cloudflare cache rule marks bypass, while anonymous trending pages (the overwhelming majority of traffic, and the ones that matter for SEO) go through the full Cloudflare cache. Anonymous visitors get an edge HIT at Cloudflare and never touch Varnish; logged-in visitors get a Cloudflare bypass, a Varnish HIT on the grid, and a Varnish pass on their header.

Numbers and gotchas

After rolling this out region by region, the origin PHP request rate for trending pages dropped roughly 20x at peak, and p95 latency on a cold regional page went from ~380ms to ~40ms because the FTS5 query result is shared. A few hard-won notes:

ESI includes are sequential by default. Varnish fetches fragments one after another unless you're careful. Keep the number of includes small (we use four) and make sure no fragment itself does something slow like an external HTTP call — that latency is on the critical path.
Never emit the ESI surrogate header on user-generated HTML. If a comment body can contain <esi:include src="file:///etc/passwd"> and you have ESI parsing on, that's an SSRF/LFI. Parse ESI only on templates you generate.
Match fragment Cache-Control to Varnish TTL. In dev we run without Varnish, and if the fragment's own headers lie about freshness you get bugs that only reproduce in prod. Keep them consistent.
Grace is not optional at scale. Without it, every TTL expiry is a synchronous origin fetch, and a synchronized expiry across regions is a thundering herd. Grace + background refresh turns expiry into a non-event.
CJK URL normalization matters. We lowercase region codes in VCL so region=JP and region=jp don't fork the cache. Watch for percent-encoded query params doing the same thing.

Conclusion

ESI isn't glamorous and Varnish has a reputation for being fiddly, but the mental model is simple and it's the right tool for exactly this shape of problem: a page that is mostly the same for everyone with a small personal part bolted on. Instead of letting the volatile 5% dictate the cache policy of the whole page, you give each piece the TTL it deserves. The expensive FTS5 aggregation gets computed once a minute and shared by everyone in a region; the saved-count stays live and per-user; and a copyright takedown purges in constant time via surrogate keys. If your aggregation site is currently choosing between "cache everything and show stale user state" or "cache nothing and melt the database," fragment-level caching is the third option you actually want.