Building a Video CDN Edge Cache Layer With Varnish and ESI Fragments

#varnish #caching #php #performance

Our trending page for the Japanese region was doing 40,000 requests a minute during prime time, and every single one of them was rebuilding the same HTML. The video grid changes maybe once every fifteen minutes when the crawler finishes a pass. The header, the language switcher, the category rail — those change almost never. But because one small strip on the page shows your recently watched items, we were treating the entire document as private and uncacheable. We were burning PHP-FPM workers to re-render 95% static markup so we could stamp in 5% of personalized content.

That is the exact problem Edge Side Includes solve, and after moving TopVideoHub to a Varnish layer with ESI fragments, our origin request rate dropped by roughly 88% while the personalized strip stayed fresh. This is a walkthrough of how we structured it — the VCL, the fragment boundaries, the cache-key discipline, and the CJK-specific gotchas that almost nobody writes about.

Why a full-page cache was never going to work

We run an Asia-Pacific video aggregator. A single trending page mixes content with wildly different lifetimes:

The trending grid — recomputed when our regional crawler finishes, every 10–15 minutes. Identical for every anonymous visitor in a region.
The category navigation and language switcher — changes when we add a category, so effectively daily.
The "continue watching" strip — unique per visitor, driven by a cookie.
The trending-in-your-language rail — shared across everyone who picked ja, ko, or zh-Hant, but different between those groups.

A classic full-page cache (LiteSpeed page cache, or Cloudflare's edge cache) forces an all-or-nothing decision. The moment one strip is personalized, the whole page becomes Cache-Control: private and you are back to rendering everything on every hit. The insight behind ESI is that a page is not one cache object — it is a composition of fragments, each with its own TTL and its own cache key. Varnish assembles them at the edge, so the client sees one seamless HTML document but the origin only ever renders the pieces that actually expired.

The fragment boundary map

Before touching VCL, draw the boundaries. Getting this wrong is the single biggest mistake teams make — they either fragment too finely (dozens of tiny ESI includes, each a subrequest) or they park a personalized element inside a shared fragment and poison the cache for everyone.

Our page became four fragments:

Fragment	Vary on	TTL	Cacheable
`/_frag/shell`	nothing	24h	shared
`/_frag/trending?region=JP`	region	10m	shared
`/_frag/lang-rail?lang=ja`	lang	10m	shared
`/_frag/continue`	session cookie	0 (pass)	private

The shell is the outer skeleton and contains the <esi:include> tags. Three of the four fragments are shared and cacheable; only the continue strip is personalized, and it is the only thing that ever needs the visitor's cookie. That is the whole trick: isolate the private surface area down to the smallest possible fragment.

Rendering fragments from PHP

On the origin, each fragment is just a lightweight route. We are on PHP 8.4, and the important detail is that a fragment endpoint must emit the correct Cache-Control and Surrogate-Control headers so Varnish knows how to treat it. Here is the shell route emitting the ESI markup, plus a shared trending fragment.

<?php
declare(strict_types=1);

// GET /_frag/shell
function renderShell(string $region, string $lang): void {
    // Tell Varnish this response contains ESI to process.
    header('Surrogate-Control: content="ESI/1.0"');
    header('Cache-Control: public, max-age=86400');
    header('Content-Type: text/html; charset=utf-8');

    $region = htmlspecialchars($region, ENT_QUOTES);
    $lang   = htmlspecialchars($lang, ENT_QUOTES);

    echo <<<HTML
    <!doctype html>
    <html lang="{$lang}">
    <head><meta charset="utf-8"><title>TopVideoHub — Trending</title></head>
    <body>
      <esi:include src="/_frag/nav?lang={$lang}" />
      <main>
        <section class="continue">
          <esi:include src="/_frag/continue" />
        </section>
        <section class="grid">
          <esi:include src="/_frag/trending?region={$region}" />
        </section>
        <aside class="rail">
          <esi:include src="/_frag/lang-rail?lang={$lang}" />
        </aside>
      </main>
    </body>
    </html>
    HTML;
}

// GET /_frag/trending?region=JP
function renderTrending(PDO $db, string $region): void {
    // Shared across all anonymous visitors in this region.
    header('Cache-Control: public, max-age=600');
    header('Content-Type: text/html; charset=utf-8');

    $stmt = $db->prepare(
        'SELECT video_id, title, thumb_url
           FROM trending
          WHERE region = :region
          ORDER BY rank ASC
          LIMIT 24'
    );
    $stmt->execute([':region' => $region]);

    foreach ($stmt as $row) {
        $title = htmlspecialchars($row['title'], ENT_QUOTES, 'UTF-8');
        printf(
            '<a class="card" href="/w/%s"><img src="%s" alt="%s" loading="lazy"><span>%s</span></a>',
            rawurlencode($row['video_id']),
            htmlspecialchars($row['thumb_url'], ENT_QUOTES),
            $title,
            $title
        );
    }
}

Notice the shell sends Surrogate-Control: content="ESI/1.0". That header is how the origin explicitly opts a response into ESI processing — Varnish will only parse <esi:include> tags on responses that carry it. The trending fragment does not send it, because it contains no includes; it is a leaf. Leaf fragments that never emit ESI should not pay the parsing cost.

The VCL that ties it together

This is Varnish 7.x VCL. The goals: enable ESI only on responses that ask for it, strip cookies from shared fragments so they cache, pass the continue fragment straight through, and build a cache key that separates regions and languages without exploding the object count.

vcl 4.1;

backend origin {
    .host = "127.0.0.1";
    .port = "8080";
}

sub vcl_recv {
    # The personalized fragment must never be cached.
    if (req.url ~ "^/_frag/continue") {
        return (pass);
    }

    # Shared fragments and the shell: drop cookies so they cache.
    # We only keep the language preference, which is part of the key.
    if (req.url ~ "^/_frag/(shell|nav|trending|lang-rail)"
        || req.url == "/") {
        unset req.http.Cookie;
    }

    return (hash);
}

sub vcl_hash {
    hash_data(req.url);
    # Region and lang already live in the query string of shared
    # fragments, so req.url covers them. For the root document we
    # fold in the resolved region header set by our edge geo lookup.
    if (req.url == "/") {
        hash_data(req.http.X-Geo-Region);
        hash_data(req.http.X-Lang);
    }
}

sub vcl_backend_response {
    # Turn on ESI parsing only when the origin opted in.
    if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
        set beresp.do_esi = true;
        unset beresp.http.Surrogate-Control;
    }

    # Shared fragments: give them a grace window so we can serve
    # slightly stale content while a single request refreshes.
    if (bereq.url ~ "^/_frag/(shell|nav|trending|lang-rail)") {
        set beresp.grace = 30s;
    }

    # Never store the private fragment.
    if (bereq.url ~ "^/_frag/continue") {
        set beresp.uncacheable = true;
        set beresp.ttl = 0s;
    }
}

sub vcl_deliver {
    set resp.http.X-Cache = obj.hits > 0 ? "HIT" : "MISS";
}

A few decisions worth calling out:

Cookie stripping is scoped by URL, not global. If you unset req.http.Cookie for everything, the continue fragment loses the session it needs. We strip only on shared fragments, and the private fragment reaches the origin with its cookie intact via return (pass).
beresp.grace = 30s lets Varnish serve a stale-but-recent fragment while exactly one background request revalidates. During a crawler-driven cache miss storm, this is what stops all 40k requests from stampeding the origin at once.
The hash stays coarse. Region and language are the only two dimensions in the key for shared fragments. We deliberately do not vary on User-Agent or on the full cookie — every extra dimension multiplies the object count and shreds your hit ratio.

The CJK trap: byte length vs. character length

Here is the part that bit us and that almost no ESI tutorial mentions. When Varnish assembles fragments, it is splicing byte streams. If the origin declares a Content-Length that does not match the actual UTF-8 byte count, the fragment gets truncated mid-character and you get a broken tofu box in the middle of a Japanese title.

The usual culprit is code that measures string length with a multibyte-aware function and then uses that number as a byte count. mb_strlen('トレンド') is 4, but the byte length is 12. If any middleware sets Content-Length from the character count, Varnish reads 4 bytes and throws away the rest of the fragment. The fix is to never hand-compute Content-Length for fragments — let PHP and Varnish use chunked transfer — and if you must compute it, use strlen() on the raw UTF-8 bytes, not mb_strlen().

<?php
declare(strict_types=1);

function emitFragment(string $html): void {
    // WRONG for CJK: mb_strlen counts characters, not bytes.
    // header('Content-Length: ' . mb_strlen($html));

    // Correct: strlen() returns the raw byte count of the UTF-8 string,
    // which is what Varnish and the HTTP layer actually need.
    header('Content-Length: ' . strlen($html));
    header('Content-Type: text/html; charset=utf-8');
    echo $html;
}

The same class of bug shows up when you truncate a title for a card. Slicing with substr() at a byte offset can cut a three-byte CJK character in half, and now your cached fragment contains an invalid UTF-8 sequence that some browsers render and others reject. Always truncate with mb_substr() for display, but measure Content-Length with strlen(). They are answering two different questions.

There is a search-layer echo of this too. Our search runs on SQLite FTS5 with a custom CJK tokenizer, because the default tokenizer treats a run of Han characters as a single token and destroys recall. The fragment that renders "related searches" pulls from FTS5, and we cache it per normalized query. The normalization has to happen before the cache key is built, or 東京 and a full-width variant produce two cache entries for the same intent.

Invalidation: the hard half

Caching is easy; invalidation is the job. When our crawler finishes a regional pass, the trending fragment for that region is stale and must be purged — but only that region's fragment, not the shell and not the other 40 regions.

We use Varnish bans keyed by URL pattern. The crawler, after committing new rankings for a region, fires an HTTP BAN request. Here is the ban handler in VCL plus a Go snippet from the crawler that triggers it, because our crawler is a Go service and the origin renderer is PHP.

acl purgers {
    "127.0.0.1";
    "10.0.0.0"/24;   # crawler subnet
}

sub vcl_recv {
    if (req.method == "BAN") {
        if (!client.ip ~ purgers) {
            return (synth(403, "Forbidden"));
        }
        # Ban every cached object whose URL matches the pattern
        # sent in the X-Ban-Url header, e.g. "^/_frag/trending\?region=JP"
        ban("obj.http.X-Url ~ " + req.http.X-Ban-Url);
        return (synth(200, "Banned"));
    }
}

sub vcl_backend_response {
    # Stash the URL on the object so bans can match against it.
    set beresp.http.X-Url = bereq.url;
}

sub vcl_deliver {
    # Don't leak the internal header to clients.
    unset resp.http.X-Url;
}

package cache

import (
    "fmt"
    "net/http"
    "time"
)

var client = &http.Client{Timeout: 3 * time.Second}

// PurgeRegion bans the trending fragment for one region after a crawl.
func PurgeRegion(varnishAddr, region string) error {
    // Anchor the pattern so "JP" cannot also match "JPX" etc.
    pattern := fmt.Sprintf(`^/_frag/trending\?region=%s$`, region)

    req, err := http.NewRequest("BAN", "http://"+varnishAddr+"/", nil)
    if err != nil {
        return err
    }
    req.Header.Set("X-Ban-Url", pattern)

    resp, err := client.Do(req)
    if err != nil {
        return fmt.Errorf("ban request failed: %w", err)
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        return fmt.Errorf("ban rejected: status %d", resp.StatusCode)
    }
    return nil
}

Two lessons from running this in production:

Anchor your ban patterns. An unanchored region=JP pattern also bans region=JPX if such a region exists, and worse, a sloppy pattern can accidentally ban the shell. Bans are a regex against every object; a broad one is a self-inflicted cache flush.
Bans are lazy, not eager. Varnish does not walk the cache and delete matching objects immediately — it checks the ban list when an object is next requested. If you fire thousands of overlapping bans, the ban list grows and every lookup gets slower. Keep bans specific and let the ban lurker thread clean them up.

Where this sits relative to Cloudflare and LiteSpeed

We still run Cloudflare in front and LiteSpeed as the PHP SAPI behind Varnish. People ask why we bother with Varnish when Cloudflare already caches at the edge. The answer is that Cloudflare gives us global POPs and TLS, but it does not do ESI assembly with per-fragment TTLs and instant regional bans the way Varnish does. So the layering is: Cloudflare caches the fully-assembled anonymous page with a short TTL and respects our Cache-Control; Varnish, sitting at our regional origin, does the ESI composition and holds the fragments with independent lifetimes; LiteSpeed only ever renders a fragment on a genuine miss. Cloudflare sees a mostly-static page, Varnish sees mostly-cached fragments, and the PHP layer sees a fraction of the original load.

The personalized continue fragment is the one thing that flows all the way to origin on every request, and because it is a tiny isolated fragment — a single query against a per-user table — it renders in under a millisecond. We deliberately keep Cloudflare from caching the assembled page for logged-in sessions by having the shell response carry Cache-Control: private only when a session cookie is present, while the shared fragments underneath it stay public in Varnish.

Conclusion

The mental shift that made all of this work was giving up on the idea of "the page" as a cache unit. A page is a composition, and each part of that composition has its own truth about how long it stays valid. Once you fragment along those lines — shell, shared regional content, shared language content, and the one genuinely private strip — the caching strategy writes itself, and the origin stops rendering the same markup forty thousand times a minute.

If you take three things from this: isolate personalization into the smallest possible fragment so everything else can be shared; measure Content-Length in bytes with strlen() and never in characters when you serve CJK; and anchor every ban pattern so an invalidation stays surgical. Those three details are the difference between an ESI layer that quietly saves your origin and one that serves broken, truncated, or over-flushed pages during your busiest hour.