The flicker problem nobody warns you about when you build an A/B testing tool

fabienc974 — Tue, 30 Jun 2026 06:00:08 +0000

I spent 14 months building Jarstak, a visual A/B testing editor for e-commerce, alone. This is part of a series about that. There's already a general post about the whole journey. This one is about a single problem that ate more of those 14 months than anything else: flicker.

If you've ever used an A/B testing tool, you've seen it. The page loads, you see the original headline for a split second, then it snaps to the variant. That snap is flicker. It's also called FOOC — Flash Of Original Content. And it's the most underestimated problem in this entire category of tools.

Underestimated because it looks cosmetic. It isn't. If the visitor sees the original before the variant, two things break:

Your experiment is contaminated. The visitor was exposed to both versions. Your conversion data now measures "original then variant," which is neither cell of your test.
It looks broken. On a checkout page, a flash of the wrong price erodes trust faster than any layout you could test.

So flicker isn't a polish item you get to later. It's the thing that decides whether your tool produces valid data at all. Here's what's actually going on, and the real code I ended up with.

What the browser is actually doing

The browser doesn't wait for your JavaScript. It parses HTML top to bottom and paints as soon as it has something to paint. Your variant code — the thing that rewrites the headline — runs after that first paint. So the sequence is:

Browser paints the original DOM.
Your script runs and mutates the DOM.
Browser repaints with the variant.

Step 1 is the flicker. The only way to kill it is to make step 1 paint nothing visible until your script is done. You hide the page, you change it, you reveal it. Simple in one sentence, and that sentence hides every hard problem in this post.

The naive version is: hide <html>, run your code, show <html> on load. It works in a demo. In production it fails in three ways — the script gets blocked by an ad blocker and the page stays invisible forever; the script loads late and you hide a page that already painted; the framework hydrates after you reveal and flashes anyway. Everything below is the cost of handling those three.

The pre-hide, line by line

This is the core. It runs as an IIFE injected at the very top of the bundle, before anything else. I trimmed the commented-out debug logs for readability — the structure is verbatim from the runtime.

(function __jarstakRobustAntiFlicker() {
    var t0 = performance.now();
    // Tell the inline sentinel the script actually loaded (more on that later)
    window.__jarstakLoad = 1;

    try {
        // Skip entirely inside the editor/preview iframe — hiding there is pointless
        // and would just make the editing surface blink
        var urlParams = new URLSearchParams(window.location.search);
        if (urlParams.has('jarstak_preview') || urlParams.has('jarstak_simulate') ||
            urlParams.has('jarstak_editor_iframe') || urlParams.has('jarstak_picker') ||
            window.self !== window.top) {
            window.__jarstakBodyHidden = false;
            window.__jarstakHideMethod = 'editor-skip';
            return;
        }

First decision: don't hide if there's nothing to hide for. The editor renders the customer's page inside an iframe so you can click elements to edit them. Anti-flicker there is not just useless, it's actively wrong — it would hide the surface you're trying to edit.

        var isLateLoad = document.readyState !== 'loading';

Second decision, and this one took me embarrassingly long to learn: if the document already finished parsing, do not hide. If readyState isn't 'loading', the browser has already painted. Hiding now means flashing the original and then a blank page on top of it. You've made it worse. So on late load, the runtime bails out with hideMethod = 'late-load' and reveals nothing.

When it does decide to hide, the global case is this:

        var style = document.createElement('style');
        style.id = 'jarstak-body-hide';
        style.textContent =
            'html.jarstak-loading{opacity:0!important;pointer-events:none!important}' +
            'html.jarstak-loading *{animation-play-state:paused!important}' +
            'html.jarstak-ready{opacity:1!important;pointer-events:auto!important;transition:opacity 0.15s ease-out!important}';

        // Insert at the very start of <head> so nothing can override it before we reveal
        var target = document.head || document.documentElement;
        target.insertBefore(style, target.firstChild);

        document.documentElement.classList.add('jarstak-loading');
        window.__jarstakBodyHidden = true;
        window.__jarstakHideMethod = 'global';

Two things worth pausing on.

opacity:0, not display:none and not visibility:hidden on the root. display:none would collapse layout and trigger a full reflow on reveal — you'd trade a content flash for a layout jump. opacity:0 keeps the page laid out and composited; revealing it is a cheap GPU operation. pointer-events:none is there so a visitor can't click a button they can't see during that window. And animation-play-state:paused freezes any CSS animations so they don't burn their first frames while hidden and then jump.

The interesting case isn't global hide, though. It's the one that avoids hiding the whole page at all.

Hide only what changes

Hiding the entire page is a sledgehammer. If your test only swaps one headline, you've blanked the whole viewport to protect one <h1>. The second time a visitor comes back, we know exactly which selectors the test touches — we cached them from the last visit:

        cache = localStorage.getItem('jarstak_variant_cache_v1');
        if (cache) {
            var data = JSON.parse(cache);
            if (data && Array.isArray(data.targetSelectors)) {
                cachedSelectors = data.targetSelectors.filter(Boolean);
            }
        }

        if (cachedSelectors.length > 0) {
            var earlyStyle = document.createElement('style');
            earlyStyle.id = 'jarstak-early-hide';
            var earlyTarget = document.head || document.documentElement;
            earlyTarget.insertBefore(earlyStyle, earlyTarget.firstChild);
            // visibility:hidden, NOT opacity — these elements keep their box,
            // so the rest of the page renders normally around the hidden slots.
            // No reflow when we reveal, and nothing else on the page is touched.
            earlyStyle.textContent = cachedSelectors.join(',') + '{visibility:hidden!important}';
            window.__jarstakHideMethod = 'targeted';
            return;
        }

Here visibility:hidden is the right choice, for the opposite reason it was wrong above. These are specific elements, not the root. visibility:hidden keeps their layout box, so the page renders fully — header, footer, images, everything — with just the headline slot reserved and blank. The visitor barely perceives it. And because the box is preserved, revealing causes no reflow.

There's even a jarstak_no_tests_v1 flag with a 5-minute TTL: if the last visit found zero active tests, we skip masking entirely. The whole design is a ladder from "hide nothing" up to "hide everything," and most visits land near the bottom.

While we're in there, since the cache already knows the variant ID, we fire a speculative preload so the variant script downloads in parallel with the bundle instead of after it:

        if (data && data.variantId && !document.getElementById('lj-preload-variant')) {
            var __ljVarUrl = serverOrigin + '/c/' + siteId + '/r/' + encodeURIComponent(data.variantId) + '.js';
            var __ljVarLink = document.createElement('link');
            __ljVarLink.rel = 'preload';
            __ljVarLink.as = 'script';
            __ljVarLink.href = __ljVarUrl;
            __ljVarLink.setAttribute('fetchpriority', 'high');
            // Best-effort: if the server reassigned the variant since last visit,
            // the browser just ignores the preload. Not harmful.
            document.head.insertBefore(__ljVarLink, document.head.firstChild);
        }

The comment is the point: a preload that turns out wrong is free. So you preload aggressively and don't worry about being right.

The three timeouts, and why they have to disagree

Hiding the page is a bet: my script will finish and reveal in time. Every bet needs a fallback, because the alternative to a fallback here is a permanently blank page on someone's storefront. So every hide path arms a timeout that force-reveals.

But there isn't one timeout. There are three, in three different places, and the non-obvious part is that they have different durations on purpose.

Timeout 1 — the inline pre-mask snippet (1000ms). When a customer installs through Google Tag Manager, the external script can't run early enough — GTM injects it too late to beat first paint. So the install snippet itself hides the page inline, before the external script even exists:

// Inline snippet, pasted into <head> above everything else
(function(){
  document.documentElement.classList.add('__jsk_hide');
  window.__jarstakPreMask = Date.now();
  window.__jarstakLoad = 0;
  setTimeout(function(){
    document.documentElement.classList.remove('__jsk_hide');
    window.__jarstakPreMask = 0;
    if(!window.__jarstakLoad){
      window.__jarstakBlocked = true;
      navigator.sendBeacon('//<origin>/api/v1/blocked', '{"siteId":"<id>"}');
    }
  }, 1000);
})();

This snippet hides immediately and reveals itself after 1000ms no matter what. It's the outermost safety net: even if the external bundle never arrives, the page un-hides in one second.

Timeout 2 — the inline sentinel (1500ms). Separate inline script, also above everything. Its only job is to detect that the external jarstakup.js got blocked — by an ad blocker, by CSP, by a network failure:

(function(){
  window.__jarstakLoad = 0;
  setTimeout(function(){
    if(!window.__jarstakLoad){
      window.__jarstakBlocked = true;
      // Un-hide whatever might be hiding the page...
      document.documentElement.classList.remove('jarstak-loading', '__jsk_hide');
      // ...and exclude this visitor from the A/B sample entirely.
      navigator.sendBeacon('//<origin>/api/v1/blocked', '{"siteId":"<id>"}');
    }
  }, 1500);
})();

It's inline because an inline script can't be blocked by the same ad blocker that blocks the external one. If the bundle never sets __jarstakLoad = 1, the sentinel knows the visitor never received the test — so it doesn't just reveal the page, it pulls the visitor out of the sample. A blocked visitor counted in your results is a poisoned cell.

1500ms, longer than the pre-mask's 1000ms, because the sentinel is the last line of defense — it should only fire after the snippet's own reveal has already happened.

Timeout 3 — the runtime, once the bundle is alive (800ms). When the external script does load and takes over, it arms its own, shorter timeout:

// Safety timeout: reveal after 800ms max (fail-open if the variant scripts stall)
window.__jarstakRevealTimeout = setTimeout(function() {
    window.__jarstakRevealBody && window.__jarstakRevealBody('timeout');
}, 800);

800ms is shorter than 1000 and 1500 on purpose. Once the bundle is running, it's in control and should resolve faster than the dumb inline nets. The inline timeouts exist for the case where the bundle isn't there. The runtime timeout exists for the case where it is, but a variant script is slow. They're a cascade: 800 (smart) inside 1000 (snippet net) inside 1500 (blocked detector). Each one assumes the inner one failed.

That layering is the whole architecture. And it's exactly what one bug quietly broke.

The bug: the timeout became the happy path

The runtime's 800ms is a fail-safe. It's supposed to almost never fire. The normal flow is: variant applies → call reveal → done, in well under 800ms.

Except for a while it didn't. Pages were revealing at exactly 800ms, every time. Not crashing — the timeout was catching it, so nothing errored. It just meant a visible blank page for 800ms on every single load. The fail-safe had silently become the primary path, and because it worked, nothing told me.

The reason was a missing handoff. The bundle has two layers: the anti-flicker IIFE that does the hiding, and the core module that applies the actual variant. The core module had its own reveal logic for one mode and called the IIFE's reveal for others — but in the pre-mask and global cases, it finished applying the variant and just... didn't tell the IIFE. So __jsk_hide and jarstak-loading stayed on until the 800ms timeout swept them away.

The fix is one conditional, and I left the comment in because it's the kind of thing I'd want the next person to understand instantly:

// Relay to the IIFE reveal handler for pre-mask and global modes.
// Without this call, __jsk_hide (pre-mask) or jarstak-loading (global) are
// only lifted by the 800ms timeout, causing a needless blank page.
if (window.__jarstakRevealBody && (window.__jarstakBodyHidden || window.__jarstakHideMethod === 'pre-mask')) {
    window.__jarstakRevealBody('natural');
}

The reveal handler itself routes by mode, because pre-mask and global hid the page in different ways and have to be undone differently:

if (window.__jarstakHideMethod === 'pre-mask') {
    // GTM mode: drop the snippet's class
    document.documentElement.classList.remove('__jsk_hide');
    window.__jarstakPreMask = 0;
} else {
    // global/targeted: transition jarstak-loading → jarstak-ready
    document.documentElement.classList.remove('jarstak-loading');
    document.documentElement.classList.add('jarstak-ready');
}

What made this bug nasty is the thing that makes all fail-open systems hard to debug: the fallback hid the failure. I built three timeouts so the page would never stay blank, and they did their job so well they masked the fact that the real reveal was never happening. The lesson I took from it: if your safety net is load-bearing, measure how often it fires. A fail-safe that triggers on the happy path isn't safety, it's a bug wearing a costume.

That's why every reveal now reports its reason:

performance.measure('jarstak:flicker', 'jarstak:hide', 'jarstak:reveal');

The reveal is tagged natural, timeout, css-only, dom-complete, and so on, sampled at 10% to a metrics endpoint. The number I watch most isn't the median flicker duration. It's the share of reveals tagged timeout. If that climbs, a relay is broken again somewhere — and this time it'll tell me instead of hiding.

A couple of edge cases that earned their code

Applying changes without causing your own flicker. Once the page is hidden, you still have to mutate it, and mutations cause reflows. So changes are split: pure CSS applied immediately, DOM changes batched into a single requestAnimationFrame, and only then reveal — with the reason naming which path ran:

// CSS changes apply instantly (no RAF wait)
applyCssChanges();
// DOM changes batched into one frame to minimize reflow
requestAnimationFrame(function() {
    applyDomChanges();
    window.__jarstakRevealBody('all-complete');
});

SSR hydration. I originally detected Next/Nuxt via __NEXT_DATA__ and friends and skipped masking for them. That was wrong — those frameworks render on the server and then hydrate, and the hydration pass is itself a flash of un-varied content. The detection got deleted. The masking stays through hydration, watched by a MutationObserver on a throwaway sentinel node with a 2s ceiling.

What I'd ask the room

The thing I never fully resolved: the entire approach is a bet that hiding-then-revealing is faster than the flash it prevents. On a fast connection with a cached bundle, you sometimes hide a page that would've painted the variant in time anyway — so you added a few milliseconds of blank to prevent a flash that wasn't going to happen. The honest fix would be to predict, per visit, whether masking will be net-positive before you commit to it.

If you've shipped anti-flicker in production — for A/B testing, feature flags, theming, anything that mutates the DOM before paint — how do you decide when not to hide? Is anyone modeling that decision instead of always-hide-with-a-timeout? I'd genuinely like to know if there's a better answer than the cascade of fallbacks I ended up with.

DEV Community: fabienc974