DEV Community

Cover image for The Audit's Blind Spot: I Weighed the Build, Not the Page
Gabriel Abreu
Gabriel Abreu

Posted on Originally published at codewithgabo.com

The Audit's Blind Spot: I Weighed the Build, Not the Page

I published a post called "I Audited My Own Portfolio and Found 20 Problems". It
was an inventory: I went through my own site — a React 19 + Vite SPA with Sanity
as the CMS — wrote down everything that was wrong with it, fixed what mattered, and put the before and after numbers next to each item. If you haven't read it, the
only part that matters here is the methodology, and one line of it in particular:

I went through the build output chunk by chunk in build/assets/.

I called that the step that hurts and the one most people skip. I still think
that is true. It is also the step that guaranteed I would miss the largest thing
wrong with the site.

The step that worked

Weighing the build output worked exactly as advertised. Finding 1 of that audit
was an unoptimized PNG of a developer illustration on /gabriel-abreu, my contact page, 993 KB, sent
to every visitor who landed there. It went to 23 KB. A second image, the cutout
of me that sits in three different greetings, went from 358 KB to 45 KB.

Those two are bundled assets. A component imports one:

import p from "../assets/developer-illustration.webp";
Enter fullscreen mode Exit fullscreen mode

Vite follows that import, hashes the file, and emits it into build/assets/.
After the build it is a file on disk with a size. Listing the directory finds it.
Sorting the listing by size finds it first. There is no way to ship it and not
have it show up in that step.

So the method was sound within its domain: both of those images are bundled assets, and the step found both.

On August 23 I opened the blog index in a browser and watched what it actually
requested. Sixteen post covers, 9.88 MB.

None of that could have appeared in the audit. Not because I was sloppy that day
— because of where those bytes come from.

Two lifecycles

A bundled asset exists at build time. An import makes it a build input, the
bundler makes it a build output, and anything that reads the build output sees
it.

A CMS image is never a build input. Nothing imports it. It arrives as a string in
a query result, after the page has already loaded. This is the query that feeds
the blog index:

*[_type == "post"] | order(publishedAt desc){
  title,
  slug,
  mainImage{ asset->{ _id, url } },
  publishedAt
}
Enter fullscreen mode Exit fullscreen mode

That runs in a useEffect after mount. The url that comes back points at
cdn.sanity.io, React puts it in an img src, and the browser fetches it from a
CDN I do not build. At no point in the lifecycle of that image does a file land
in build/assets/.

Two different lifecycles, and my audit's most rigorous step could only observe
one of them. Not "did not happen to observe." Could not, by construction. The
scale was accurate. It was not weighing the whole load.

What was actually on the page

Once I measured with a browser's real Accept header instead of a directory
listing:

/allpost, 16 post covers ........ 9.88 MB
one post page, 4 images ......... 4.54 MB
worst single image .............. 3.13 MB  (2160x2700 PNG, rendered ~250px wide)
another cover ................... 2.27 MB  (rendered at 433x227)
Enter fullscreen mode Exit fullscreen mode

The two images the audit caught came to 1,351 KB between them. One route was
serving 9.88 MB.

Every one of my dev.to cross-posts closes with a link to /allpost. The heaviest route I measured was the page I send people to.

urlFor was called once

Sanity ships an image-URL builder, urlFor, that applies CDN transformations —
width, fit, format. Before the fix, urlFor was called exactly once in the application. Not once per component. Once. It was in OnePost.tsx, inside the
PortableText serializer for images embedded in the body of an article:

const src = value?.asset
  ? urlFor(value).width(1600).fit("max").auto("format").url()
  : value?.url;
Enter fullscreen mode Exit fullscreen mode

Width capped, fit: max so nothing upscales, auto: format so modern browsers
get WebP. That is the correct call. It applies to images an author dropped into
the middle of a paragraph — the images a reader is least likely to notice.

AllPosts.tsx contained zero references to it. So did the post header. Those
cards rendered mainImage.asset.url — the original upload, straight from the
CDN — and let CSS scale it into a 250px box. The browser downloads all 3.13 MB first
and then paints it small.

It made the site look like a site that sized its images.

The fix

The queries feeding those cards project asset->{url}, not the reference object
urlFor wants, so going through the builder meant rewriting the queries. Instead
the helper takes the URL the query already returns:

export function sizedImage(url: string | undefined | null, width: number): string {
  if (!url) return "";
  // Non-Sanity URLs (the BlockNote editor stores upload-endpoint URLs directly)
  // do not understand these parameters.
  if (!url.includes("cdn.sanity.io")) return url;
  const sep = url.includes("?") ? "&" : "?";
  return `${url}${sep}w=${width}&fit=max&auto=format&q=75`;
}
Enter fullscreen mode Exit fullscreen mode

auto=format negotiates on the request's Accept header, so a modern browser gets
WebP and an older one gets the original format. fit=max only ever scales down,
so a small source is never blown up. The cdn.sanity.io check is there because
my BlockNote editor stores upload-endpoint URLs directly and those do not
understand the parameters — appending them would have broken images that were
working.

Measured the same way afterward, those sixteen covers came to 182 KB, 56 times
less than before. The 2160x2700 PNG is 24 KB.

That is the number for those images on those routes, measured with one browser on one day. I have not measured load time on a real connection, and I am not
going to tell you the site is fast now.

The scope is the claim I did not audit

An audit answers the question you point it at. Mine asked: what is heavy in what
I ship? The answer was correct.

But "what I ship" is a boundary drawn by the bundler, and the browser does not
know that boundary exists. It requests what the page tells it to request, from
wherever. My method treated a build artifact as a stand-in for a page load, and
those two things overlap on a site like this without being the same set.

What made that step feel rigorous is the same thing that made it narrow: it was
mechanical. A directory, a list of files, sizes, sorted. A measurement that can
be exhaustive is exhaustive over its own domain and silent about everything else,
and it does not feel silent — it feels finished. Twenty findings, all real, and 9.88 MB still going out on the route I advertise.

The list of findings is the part of an audit that gets checked. The scope is a
claim too — a claim about where problems are allowed to be — and it is the one
that ships unexamined. Next time I will write that claim down next to the
method, in the same document: here is what I measured, and here is what this
measurement cannot see.

I write up the things I break and fix at codewithgabo.com.

Top comments (5)

Collapse
 
heinrichneb profile image
Heinrich Neb

"The scope is a claim too - and it ships unexamined" is the sentence this post exists for, and the class is bigger than frontends. My twin from the ops side: a disk audit with du -x that was exhaustive, sorted, and finished-feeling - and structurally silent about the mounted volume sitting right next to it, because -x means "my own filesystem" and the flag's domain boundary had quietly become my world boundary. Same disease: the instrument's edge mistaken for the system's edge, and the mechanical thoroughness is exactly what makes it feel safe. Your fix generalizes too: writing the scope claim next to the method is preregistration - we've made it a standing rule that every measurement writes down what it cannot see, in the same document, before results exist, because afterwards the blind spot always looks obvious and never gets written. One question in the spirit of your last post: does a check now exist that fails when the next lifecycle appears - say, a route-level byte budget measured in a real browser - or does the browser measurement live in your habits, where the build listing used to live?

Collapse
 
gabbs279 profile image
Gabriel Abreu

It lived in my habits. It doesn't any more, and building it answered your
question in a way I didn't expect.

npm run budget: Chrome over CDP, navigate each route, sum
encodedDataLength. No domain boundary to be silent about — it counts what
the page fetched, from anywhere, by any mechanism.

It found the third lifecycle on the first run.

Home page: 3533 KB. Of that, 2.6 MB is nine PNGs in public/images/,
referenced by string path from a data file. Not bundled, so build/assets/
never saw them. Not from the CMS, so the image assertion I wrote last week
never saw them either. Static files copied verbatim, sitting in the exact
gap between the two checks I already had. Your du -x, one level up: two
instruments, each exhaustive over its own domain, and the thing lives
between them.

Your preregistration framing is what I'd have needed to catch it earlier.
Writing "this measures the bundler's output and cannot see anything
fetched at runtime" next to the method would have made the gap visible
while the method still felt rigorous — which is the only moment it's ever
invisible.

One note on the instrument, since it nearly repeated the disease. Resource
Timing was the obvious way to measure this from inside the page. It would
have reported zero: a cross-origin response without Timing-Allow-Origin
reports encodedBodySize 0, and every image here is cross-origin. The check
would have passed by measuring nothing. Green and blind, in the tool built
to catch green and blind.

The budget for the home page stays at 900 KB, so the check is red right
now. Setting the ratchet above 3533 would make the check agree with the
bug.

Collapse
 
heinrichneb profile image
Heinrich Neb

The Resource Timing detail is the part I will be repeating, and I do not think you are giving it enough weight by calling it a note.

You reached for the obvious instrument. It reports zero for cross-origin responses without Timing-Allow-Origin. Every image was cross-origin. The check would have passed by measuring nothing - green and blind, in the tool built to catch green and blind. That is not an anecdote about one API, that is a law with a testable form: an instrument that returns a number for the thing it cannot see is worse than one that errors, and most measurement APIs are the first kind.

Which makes your preregistration framing load-bearing rather than tidy. "This measures the bundler's output and cannot see anything fetched at runtime" is a sentence that costs nothing to write and would have caught both the gap and the instrument that nearly hid it. I would go one step further and preregister the failure mode of the instrument, not just its scope: not "CDP sees everything the page fetched" but "CDP reports encodedDataLength; if a response never reaches the network layer it is invisible here." The second sentence is checkable. The first is a promise.

The budget staying red at 3533 against a 900 KB target is the right call and the harder one. Setting the ratchet above the bug is the single most common way a check dies, and it never looks like a decision - it looks like housekeeping. The tell is that nobody writes "we accepted 3533" in the commit; they write "adjusted budget to reflect current state," which is the same sentence a broken check would produce.

Thread Thread
 
gabbs279 profile image
Gabriel Abreu

You are right that it is a law, and I have the second specimen: the budget itself, three days later.

Your sentence — an instrument that returns a number for the thing it cannot see is worse than one that errors — is what the byte budget did to me. Not Resource Timing this time. The tool I built to avoid Resource Timing.

One Chrome, one tab per route. The HTTP cache is per profile. So the shared JS bundle was counted once, on whichever route happened to be measured first, and every route after it reported 0 KB of its own. Four of five routes, green, reporting zero. The check was measuring the order I ran it in.

It did not error. It returned a number. The number was 0, which reads as "nothing to see" and means "I cannot see."

Fix is a whole browser per route with its own --user-data-dir. Target.createBrowserContext would be lighter; a second Chrome is impossible to get subtly wrong, which is the entire job of that file.

The second one is closer to your ratchet point. AdSense measured between 247 KB and 644 KB across runs of the same route on the same afternoon. A budget over the total goes red or green depending on what Google decided to load that second — and a check that fails at random is one you learn to ignore, which is its own way of dying. So budgets now enforce first-party bytes only. Third-party is still measured and printed, because not enforcing it is not a reason to stop looking at it.

That narrowing is a scope claim, so by your own rule it goes next to the method: this enforces bytes from codewithgabo.com and cannot fail on a third-party regression, however large. Which is a real blind spot I am choosing, and now it is written where the next person will read it.

On the ratchet: I did raise it, and your tell is exact. The home page went from 3533 KB to 469 KB of first-party, and the commit that raised the number says ratchet the home page to what it now weighs — legitimate, the bytes actually left. But the commit for the cache bug I deliberately titled the check was measuring measurement order, not "adjusted budget calculation," because you are right that the second sentence is what a broken check produces and nobody would ever look at it again.

Thread Thread
 
heinrichneb profile image
Heinrich Neb

The specimen is accepted, and it's a better one than mine because the instrument that lied was the one built to stop the last instrument from lying. "The check was measuring the order I ran it in" belongs next to the law itself - every piece of state that survives between two measurements is an input you never declared, and a per-profile HTTP cache is exactly such an input. Your fix (one browser per route) is the blunt version, and blunt is right here; your commit-message discipline is the part most people skip, and it's what keeps the ratchet honest.

Two thoughts to push it one step further. First: your instrument now deserves its own admission gate - a self-test that measures two routes in one order, then the reversed order, and demands the same numbers. A commutativity probe. It would have caught the cache bug on day one, and it will catch the next order-shaped input you haven't met yet. Second, on AdSense variance: enforcing first-party only while still printing third-party is exactly the right split - a check may refuse to gate on what it cannot measure stably, but it must not stop looking. You wrote the blind spot down where the next person will read it; that sentence is the difference between a scope decision and a silent hole. That's the whole discipline, applied twice in one incident.