DEV Community

Daniel Pertu
Daniel Pertu

Posted on

Our about page named the wrong company for weeks, and the age header was the only thing that admitted it

On 3 October I changed the company named as the publisher of all my apps, in
every place the repository says it. On 8 October I went to quote that page in
something I was writing, loaded it, and the deployed page still named the
previous company.

Nothing was broken. The page was valid HTML, prerendered, fast, cached at the
edge, and wrong. There was no error, no warning and no difference in the markup
between a correct page and a stale one, because a stale prerendered page is not
a degraded version of the current one. It is a perfect copy of an older one.

The only part of the response that knew anything was a header I had never
thought of as a correctness signal.

What a prerendered page is actually promising

$ curl -sI https://pub-trivia.app/about
age: 256676
cache-control: public, max-age=0, must-revalidate
etag: "85fd9d84b78aa8f7a392abe6ec81faf8"
x-nextjs-prerender: 1
x-vercel-cache: HIT
Enter fullscreen mode Exit fullscreen mode

x-nextjs-prerender: 1 means this HTML was produced by a build, not by this
request. That is the entire explanation of the bug, and it is a feature. 70 of
the 72 URLs in our sitemap answer with that header, which is why the marketing
site is quick. The content of all 70 is a function of the last build, not of the
last commit.

So the question "is the deployed about page correct" has nothing to do with
main. A commit is not a deploy, and I had made the change and not shipped it.
When I measured the cached object on 8 October, its age was about 24 days: the
pages had been sitting at the edge since a build in the middle of September, and
every request since had been served from that copy.

That is what makes this class of staleness so quiet. It is not a cache holding a
page past its freshness window. It is the freshness window never having applied
in the first place, because nothing had produced a newer page to replace it with.

The contradiction in those two headers

Look at the pair again:

age: 256676
cache-control: public, max-age=0, must-revalidate
Enter fullscreen mode Exit fullscreen mode

The cache-control is the instruction to your browser, and it says: do not
reuse this without checking. Meanwhile the edge has been holding the object for
256,676 seconds, which is 2.97 days, and will keep holding it. Two different
caches, two different policies, one response, and the only one the headers
describe is the one that is not doing the caching.

You can watch the edge's copy age in real time. Two requests, five seconds apart:

age1=256671 age2=256676 delta=5
Enter fullscreen mode Exit fullscreen mode

It tracks the wall clock exactly, because nothing is refreshing it. And the ETag
works as advertised, which is the part of the stack that behaved perfectly all
along:

$ curl -s -o /dev/null -w "%{http_code} %{size_download}\n" \
    -H 'If-None-Match: "85fd9d84b78aa8f7a392abe6ec81faf8"' \
    https://pub-trivia.app/about
304 0
Enter fullscreen mode Exit fullscreen mode

A conditional request gets a 304 and zero bytes. The caching was never the
problem. The content was three weeks old and the caching was doing precisely
what it was told.

Age is not the age of the content

This is the trap I walked into while trying to audit the thing. Compare two
pages from the same build:

/about                            age: 256676
/guides/how-to-host-a-pub-quiz    age: 229324
Enter fullscreen mode Exit fullscreen mode

Same build, same deployment, 27,000 seconds apart. age is how long this edge
node
has held this object, which starts when someone first asked that node for
that URL. So it is a measurement of traffic, not of content. A popular page
pulled into a node early carries a bigger number than an identical-vintage page
nobody requested until later, and both are the same age in the only sense that
matters.

Which means age is a lower bound on staleness and nothing more. If you want to
know whether the deployed HTML matches your repository, there is exactly one
reliable method, and it is to read the deployed HTML:

$ curl -s https://pub-trivia.app/about | grep -o "CogniPrep" | wc -l
Enter fullscreen mode Exit fullscreen mode

That now returns a non-zero count, because the build finally happened: the
cached copy dates from 9 October, and it has the right company name in it. Six
days between the commit and the page.

The two pages that cannot go stale

The same curl on two other URLs comes back completely different:

$ curl -sI https://pub-trivia.app/
age: 0
cache-control: private, no-cache, no-store, max-age=0, must-revalidate
x-vercel-cache: MISS
Enter fullscreen mode Exit fullscreen mode

No x-nextjs-prerender header, age: 0 on every request, and a cache-control
that forbids storing it anywhere. The homepage and
the pricing page are the only two URLs in the
sitemap rendered per request, because the price shown has to be resolved from the
request itself. They are more expensive and they are structurally incapable of
having this bug.

So the freshness model across the site is: two pages that are always current and
70 that are exactly as current as the last build. And the content most likely to
be wrong is concentrated in the 70, because the per-request pages carry prices
and the prerendered ones carry everything that changes rarely: company details,
legal copy, the guides,
the about page itself. Rarely changing copy is
copy nobody reloads, which is copy nobody notices is stale.

What I actually changed about how I work

Not the caching. The caching is right and I would not weaken it to make a
company name arrive four days sooner.

What changed is that I no longer quote my own repository when I write about what
the site does. If a sentence in a post says "the page says X", it gets checked
with curl against the live URL first, because for 70 of 72 URLs the repository
is a statement of intent and the deployed HTML is the fact. That rule has already
caught two things this month, this being the second.

The other change is cheap and I recommend it to anyone running a prerendered
marketing site: pick five strings that must be true of the deployed HTML, your
company name among them, and have something fetch the live pages and check them
on a schedule. Not a test of your code, a test of your deployment. The HTML gave
me no signal at all, the headers only told me when an edge node last went
shopping, and the one thing that would have told me plainly in under a day is a
grep nobody had written.

Run the commands above against our pages if you like. The ages will be larger
than the ones printed here, which is itself the point.

Top comments (0)