DEV Community: Ivan Yatsenko

Crystal in 2026: a 7 MB binary, zero dependencies, and five traps

Ivan Yatsenko — Wed, 12 Aug 2026 09:30:22 +0000

I spent a few days writing a satellite ground station daemon in Crystal, with an empty dependency list and a hard rule against third-party code. It works, it ships as one file, and it sits at 1.9 MB of memory at rest.

This is what the language was like to use, and what it cost.

The project is kozai: it reads orbital elements, propagates them with SGP4/SDP4, predicts passes over a ground station, serves a JSON API and an offline web interface, and drives a rotator and a radio through hamlib. About 9,000 lines of source and 6,400 lines of specs, on Crystal 1.21.0. None of that matters here except as the load under which the language was tested — this is a report on the tool, not on the satellites.

What the language actually delivers

The headline claim of a compiled language with a garbage collector is that you get Ruby's ergonomics and a binary at the end. In 2026 that claim holds, and the numbers are the part worth quoting:


Docker image, `FROM scratch`	7.41 MB
Static binary, musl, arm64	6.9 MB
Dynamic binary, release	1.9 MB
Memory at rest, 2 satellites	1.9 MB
Memory at rest, 97 satellites	4.3 MB
Memory after a day of serving, 97 satellites	19.3 MB, flat
Build steps before `crystal build`	none
Runtime files outside the binary	none

The last two rows are the ones that changed how the project was built. There is no Node in this repository, no bundler, no asset pipeline, and no postinstall. The web interface — HTML, CSS, JavaScript, and a 66 KB SVG of the world's coastlines — is read at compile time by {{ read_file(...) }} and lives inside the executable (src/assets.cr). Deploying is scp.

The standard library covered the whole surface of a network daemon with six imports: http/server, http/client, json, log, socket, option_parser. That list is not an aspiration; CI fails if a seventh appears.

The type system earned its keep in the numerical core. Predicting a week of passes for a hundred satellites is on the order of ten million propagator calls, and the hot loop allocates nothing: positions and satellite state are structs, and propagation failures are reported through an enum instead of an exception, because raise allocates. A spec propagates 200,000 steps on the near-earth branch and 50,000 through the deep-space integrator, and asserts that heap growth is zero (spec/allocation_spec.cr). It passes. Getting that from a GC'd language, without writing anything that looks like C, is the reason to be here.

So much for the brochure. Here are the five things that cost me time.

Trap 1: the inline `rescue` does not filter by type

This one is specific to Crystal, and it is the one I would warn a newcomer about first.

Crystal has a suffix rescue, inherited in spirit from Ruby:

value = risky_call rescue fallback

In a block form, rescue IO::Error means "catch this class of error". In the suffix form it does not. The suffix form has no type filter at all: it catches everything, and the thing on the right is the value returned on failure.

So this line, which closed a socket in a mock server without caring whether it was already closed:

socket.close rescue IO::Error

does not mean "catch IO errors". It means "catch every exception, including the ones that indicate a bug, and on failure evaluate to the class object IO::Error". The code reads as if it were correct. It compiles, it type-checks, and it will happily swallow the failure you needed to see. The fix is the block form, which does filter:

private def close_quietly(socket : TCPSocket) : Nil
  socket.close
rescue IO::Error
end

I did not find this by reasoning about it. Ameba, the linter, found it. That is the useful lesson: the trap is invisible during review precisely because it looks like the block form, so run the linter and believe it.

Two smaller sharp edges live next door.

A macro cannot be expanded inside a rescue clause. The parser rejects it. I needed the rescue list to depend on a compile-time flag, because a build without OpenSSL has no OpenSSL::Error type, and naming a type that does not exist will not compile. The way through is an alias, declared once (src/catalog.cr):

{% if flag?(:without_openssl) %}
  alias TransportError = IO::Error | Socket::Error
{% else %}
  alias TransportError = IO::Error | Socket::Error | OpenSSL::Error
{% end %}

and then rescue ex : Error | TransportError at the call site.

Exceptions from the standard library are easy to under-catch. The same loader missed OpenSSL::SSL::Error, so a TLS failure killed the daemon instead of falling back to its cache — the exact opposite of the offline behaviour the project exists to guarantee. It surfaced only when the binary ran inside a FROM scratch image, where OpenSSL could not find a CA bundle. A dependency this project deliberately has none of would not have helped; reading the error hierarchy would have.

Trap 2: the fiber stack pool looks exactly like a memory leak

This is the one that nearly went into a release note as a defect in Crystal's standard library. It would have been wrong.

The daemon is meant to run for weeks unattended, so I put it under continuous request load and sampled memory. The live heap, measured after a forced GC.collect, grew linearly: 0.21 MiB per minute, about 300 MB per day. That is a leak by any reasonable reading.

I isolated it. Thirty lines, a bare HTTP::Server with one ErrorHandler and not a single line of my project, and the shape reproduced: roughly 75 KiB retained per request when each request arrived on a new TCP connection. At that point I had a clean reproduction against the standard library and a draft sentence about a leak in HTTP::Server.

The sentence was wrong, and one more measurement showed why. Instead of extrapolating the line, I asked whether it saturates:

first 250 requests:  +13.4 MiB
250 → 500:            +2.4 MiB
500 → 750:            −6.8 MiB     ← memory comes back
beyond:               11–20 MiB, no trend

2000 requests on a single connection:  −0.07 MiB

It is not a leak. Crystal pools the stacks of finished fibers, and this server runs one fiber per connection. A server that has handled a burst of concurrent connections holds more live data than one that just started, up to the high-water mark of concurrency it has ever seen — and then it stops. Thirteen minutes of a perfectly straight line in a container was the pool filling up slowly, because I was sampling once a minute.

Two things follow, and both generalise beyond Crystal.

RSS tells you nothing here. Boehm does not return pages to the operating system unless it is built with USE_MUNMAP, so resident memory cannot fall and its flatness is not evidence of anything. Measure the live heap after a forced collection.

"Zero growth" is the wrong acceptance criterion; "reaches a plateau" is the right one. Restated that way, the soak is a clean pass: over 13.8 hours the live heap climbed from 4.9 MB to 19.2 MB during the first four hours, then held between 19.19 and 19.37 MB for the remaining 9.8 hours and 576 samples. The residual trend is 14 KB/hour — 250 times below the fill rate, and the same size as the scatter between consecutive samples. RSS over the same period sat at 12.3–14.2 MB.

If you are writing a long-running Crystal service, budget an afternoon for this and do not report the first curve you see.

Trap 3: the standard library links C you did not ask for

"Zero dependencies" means an empty dependencies: block in shard.yml. It does not mean the binary contains no C. The runtime stands on Boehm, libc and libm — that is the language, not your supply chain. What surprised me is how much C arrives through ordinary require lines.

require "yaml" links libyaml. For a config file of a few dozen keys that is a poor trade, so configuration is parsed by hand.
Regular expressions link PCRE2. TLE parsing is by fixed columns anyway — the format demands it — but the point is that one =~ in a cold path pulls a C library into a binary meant to be static.
HTTP::Server links OpenSSL for its TLS support whether or not you use TLS. This is the one you cannot deduce from the source you wrote. A -Dno_network build removes the HTTPS client and still links OpenSSL; you need -Dwithout_openssl as well, and the only way to discover that the first time is to build the thing and run ldd. The project now prints a compile-time notice if you pass one flag without the other.

The same pressure shows up in small places. Static assets are served with an ETag derived from their bytes, and the obvious way to compute one is a digest from the standard library — which links a C library, for a checksum whose collisions do not matter. The next obvious thing is String#hash, and that is a trap of its own: Crystal seeds it randomly per process, so every restart would invalidate every browser cache. The ETag is therefore a hand-rolled 64-bit FNV-1a, eight lines in src/assets.cr. Twice now, "use the standard library" has been the wrong answer for reasons that have nothing to do with quality.

Because these are properties of the product rather than preferences, CI enforces them (the purity job in .github/workflows/ci.yml): shard.yml must declare no runtime dependencies, src/ must contain no lib blocks, no require "yaml", no regular expressions, and no import outside the allowed six.

That job also taught me something about enforcement. Its first version grepped for .scan( and .match(, which flagged the project's own Passes.scan — a false positive that would have had someone rename working code to satisfy a grep. It now matches on the constructs (Regex, =~, a slash immediately after the parenthesis) rather than on method names. A purity check that produces false positives does not get tightened; it gets ignored.

And the honest footnote: libpcre2 is in the binary regardless, because OptionParser uses regular expressions internally. The codebase contains none. The dependency is the standard library's, not mine, and I cannot remove it without giving up argument parsing.

Trap 4: HTTP/2 is not in the standard library, and for me that was the same as absent

Issue #2125, "HTTP/2 support", was opened on 8 February 2016 and is still open. Ten years is long enough that most people read it as "Crystal has no HTTP/2", and that reading is now wrong — which is worth knowing before you rule the language out.

The gap is filled outside the standard library, by ysbaddaden/http2 from Julien Portalier, a Crystal core contributor. Its status list has HPACK, frames and streams, flow control per stream and per connection, HTTP/1-to-HTTP/2 upgrades, server connections, integration into HTTP::Server, and a green run against h2spec 2.6.0. Adding it to an existing server is one require:

require "http2/server"

The remaining unchecked box is HTTP::Client; the author described in the issue what that would take. gRPC is in the same position — there is a pure-Crystal implementation, and it is somebody's shard rather than a stdlib module.

So the honest form of this trap is narrower than "no HTTP/2", and it is the form that actually bit me: a shard is not the standard library. This project's whole premise is an empty dependency list, so a solution distributed as a shard is a solution it cannot take, however good it is. For anyone without that rule the cost is one dependency. For anyone with it, protocol support that lives outside stdlib is support that does not exist.

Worth knowing either way: browsers require TLS for HTTP/2 even on localhost, so the shard also means certificates and bind_tls, not just a require.

For this project it cost nothing: a ground station serves a handful of clients on a LAN, and HTTP/1.1 with a keep-alive is more than enough. Settle it up front anyway. Whether the protocols you need live in the standard library or in somebody's shard is a question for the week you pick the language, not for the month you discover the answer.

Trap 5: the ecosystem lags the compiler

Crystal releases move faster than the tools around them, and you will feel it at the edges rather than in the language.

Ameba 1.6 does not build against Crystal 1.21 — the compiler's lexer API changed underneath it. The fix is to pin the development version by tag, which is what shard.yml does:

development_dependencies:
  ameba:
    github: crystal-ameba/ameba
    version: 1.7.0-dev

Pinned to a tag rather than a branch, so a checkout stays reproducible. Note the shape of the problem: the one linter everybody uses needed a pre-release to work with the current compiler.

Cross-building on CI is where the days go. The arm64 + musl jobs failed before they started: actions/checkout is a JavaScript action, GitHub builds Node for Alpine only on x64, and the job died on an arm64 Alpine runner before reaching a single build step. The toolchain had to move inside docker run. Four defects in the release pipeline in total, none of which could appear locally, and all of which appeared on the first push.

libm differs between platforms, and your tests must know it. The full SGP4 verification set passes on glibc and on musl, but not bit-for-bit: the worst position disagreement is 8.26 × 10⁻⁸ km on one and 8.29 × 10⁻⁸ km on the other, both 0.083 mm, because the two libms differ in the last place of their trigonometric functions. For scale, the two published reference implementations disagree with each other by 7 × 10⁻⁸ km, so this is the noise floor and nothing else. State numerical tolerances physically; a bitwise comparison would fail a correct implementation built against the other libc.

Would I use it again

For this shape of project, without hesitating. A daemon that has to be one file, start instantly, hold single-digit megabytes, run on a single-board computer with no network and no sysadmin, and still be readable a year later — Crystal is close to ideal, and I do not know a language that would have been meaningfully better. The zero-dependency rule was sustainable only because the standard library is good enough to make it sustainable.

Against that: a standard library without HTTP/2 or gRPC, so anything modern on the wire means taking a dependency; a linter that needs a pre-release to match the compiler; cross-compilation that has to be learned the hard way; and a hiring pool of approximately nobody. If any of those are load-bearing for you, the decision is made.

The project: github.com/VanyaNeytrino/kozai. Every number above is measured and reproducible from that repository — the memory figures in the README's "No dependencies" section, the allocation guarantee in spec/allocation_spec.cr, the purity rules in .github/workflows/ci.yml.

I blocked accessibility overlays on 56 Shopify stores and measured what changed

Ivan Yatsenko — Sun, 09 Aug 2026 07:19:05 +0000

Median change to the underlying markup: zero — in every one of six runs, across three samples drawn at different times.

That sentence needs about two thousand words of context before it means anything, including the three occasions where I nearly published a number that was wrong. Here they are.

Start with what it does not measure: whether a widget helps anyone. It measures whether a page's markup changes when the widget runs. A tool with a genuinely useful toolbar — bigger text, a reading mode, a contrast switch — scores zero here and may still be worth having. The full limitations are below, and they are load-bearing.

The comparison that cannot work

Accessibility overlay widgets attach a script to your site and are sold as a way to improve its accessibility. The obvious way to test that is to measure sites that have one against sites that don't.

I did that first. It produced a clean, statistically significant result pointing in an interesting direction, and it is worthless.

In a pool of 233 German Shopify storefronts, the 17 stores with an overlay installed had a median of 25 violation nodes; the 216 without had 14. The overlay group carried 78.6% more, and the difference clears the usual bar (Mann–Whitney, z = 2.48, p = 0.013). Read that as a fact about which stores buy a widget, not about what a widget does. Group A is 17 stores and a median over 17 is fragile; the international contour, with 65 stores against 71, shows no statistically detectable difference (5.7%, z = −0.90, p = 0.369). Nor can I say why the German groups differ. Overlay stores do carry heavier pages, a median of 3,068 DOM nodes against 2,714, but that is 13% more markup against 78.6% more violations, so page weight accounts for a small part of the gap at best. The conclusion of this article rests on the paired test in the next section, not on either of these numbers.

The problem is structural. Nobody installs an accessibility widget at random. You go looking for one because you have a problem, or because a lawyer's letter arrived, or because your category is one where this gets noticed. The stores that bought a widget are not a control group for the stores that didn't — they are the stores that had a reason. That comparison can never separate the effect of the tool from the reason someone reached for it, no matter how many sites you throw at it or how small the p-value gets.

I could have published the 78.6%. It is a real number, it is significant, and it flatters a thesis people enjoy. It is also uninterpretable, and dressing it up with a p-value would only have made it more convincing than it deserves.

The comparison that does work

Measure the same page twice, a minute apart, and change exactly one thing: whether the browser is allowed to load the overlay's script.

ON   → load the page as served, wait, run axe-core
OFF  → load the page again, abort every request to the overlay's CDN, wait, run axe-core

Same URL. Same viewport, 1440×900. Same locale and Accept-Language. Same settle delay. Same axe-core version and the same rule set. Same network position — a German exit node throughout — seconds apart, so the same CDN edge and the same storefront state. Fresh browser context each time, so nothing carries over.

The difference between those two numbers is what the widget does to the page. There is no selection left to worry about, because the store is its own control.

I ran this on 56 distinct stores — a German sample of 16, an international sample of 20, and a fresh sample of 20 that had never been measured, drawn deliberately after the first results existed so that the finding had a chance to fail against data it wasn't derived from. Two of the three samples were measured twice, and one vendor had to be measured again after a bug described below, which is how 56 stores produce 92 store-runs and 179 page pairs.

What came out

Nodes removed by the widget, per run. Read "removed" as OFF minus ON: positive means the widget took violations away.

run	pairs	stores	nodes removed	of	share	median per store
German	14	7	12	468	2.6%	0.0%
English, run 1	35	18	106	1,920	5.5%	0.0%
English, run 2	35	18	99	1,921	5.2%	0.0%
Fresh, run 1	37	19	41	1,343	3.1%	0.0%
Fresh, run 2	35	18	17	1,215	1.4%	0.0%
Accessibly, re-measured	23	12	−3	905	−0.3%	0.0%
all six	179	56	272	7,772	3.5%	0.0%

The widgets in the sample were UserWay (23 stores), accessiBe (15), Accessibly (12), EqualWeb (4) and AudioEye (2). Results are not broken down by product: the split is uneven and no per-vendor number here would survive its own sample size. Nothing in these figures should be attached to any one named product. The Accessibly row exists only because that vendor had to be re-measured, not to single it out.

Two summary figures, and they disagree on purpose.

3.5% is node-weighted: every violation node counts equally, so one large storefront with hundreds of findings can carry a whole run. 0.0% is the median across stores: every store counts once, regardless of size. The median came out at zero in all six runs.

The rest of the shape:

56 of 92 store-run observations showed no change whatsoever — 61%.
Per-run share ranged from −0.3% to 5.5%. That spread, on samples this size, is the honest measure of how much a single run can wander.
In every run some stores measured worse with the widget enabled than with it blocked — 1, 3, 2, 2, 4 and 2 stores respectively. That is mostly run-to-run noise rather than a widget effect. Across the two English runs, 3 different stores came out worse and only 2 did so both times; across the two fresh runs, 4 different stores and again only 2 both times.

In one sentence: on most of these storefronts, blocking the widget's script changed nothing an automated checker could see, and where something did change it was small and did not reproduce reliably between runs on the same stores.

The three times I nearly published something false

All three were caught by re-running measurements I had already written down as results. All three push the number in the flattering direction. If you repeat this work, you will hit them.

Lazy loading

An early run showed one German storefront dropping from 46 violation nodes to 21 — a 54% improvement, exactly the kind of number that ends up on a slide.

It was not the widget. The two measurements used different settle delays, 5 seconds and 20 seconds. Almost the entire difference is a single rule: image-alt fell from 31 to 7, while the page kept growing between the two measurements. The storefront's own lazy-loading was still working at 5 seconds, and the placeholder images it had not yet swapped out were being counted as images with no alt text. What I had measured was how long I was willing to wait.

Re-run as an actual pair — widget allowed and widget blocked, both at the same delay: 46 and 46.

This is why the settle delay is not a detail you tune for convenience. Any asymmetry in how long you wait becomes a measurement of page weight wearing an accessibility result's clothes.

Pages that never rendered

The fresh sample nearly ended the project. Run 1 gave −17 nodes; run 2, on the identical sample under identical settings, gave +116. A result that flips sign between consecutive runs is not a result.

Per store, though — before any exclusion — 16 of 20 were stable within three nodes, and three stores produced 92% of the variance. One of them returned a single violation node on the widget-enabled side against 56 on the blocked side: the page had not rendered at all, and it had no <title>. That one blank page contributed +98 of that run's +116.

So I wrote down a validity rule — before checking which stores it would remove:

A pair is discarded if one side has ≤ 3 violation nodes while the other has ≥ 15, or if the two sides disagree about whether the page had a <title>.

It removes 4 pairs across the whole study. With it applied, the fresh sample gave 3.1% and 1.4% — inside the range the original samples had already produced. The finding survived contact with data it had not been built on.

I mention the order — rule first, then look — because doing it the other way round is the whole game. A threshold chosen after you have seen which points it excludes is not a threshold, it is an opinion with arithmetic attached.

A vendor that was never actually switched off

This is the one that should have killed the study, and the only reason it did not is that I checked a field I had been collecting and never using.

The OFF side works by aborting every request whose URL contains one of a list of overlay hosts. My list contained accessiblyapp.com. Accessibly serves its script from cdn.accessibly.app. Those two strings do not match, so for every pair of every run, that vendor's widget loaded normally on both sides. Thirty pairs that I was counting as ablations were the same page measured twice.

Nothing looked wrong. Those pairs produced small differences clustered around zero — which is exactly what the study concluded — so the bug was holding the headline up rather than knocking it down. A result that agrees with you is the hardest kind to audit.

What exposed it: ablation.mjs records widgetInDom on the ON side, whether the widget's own button or panel is present after the delay. Comparing that field between ON and OFF, Accessibly never disappeared — 30 pairs out of 30. Then a second surprise: after fixing the host list, it still never disappeared, because accessibly-trigger and accessibly-config are rendered into the page by the Shopify app itself and sit there whether the script loads or not. widgetInDom is a vendor-dependent proxy and cannot be used as a validity filter — a conclusion I reached only after briefly believing the opposite. What settles it is the request log: with the corrected host list the script request is aborted, and that is the thing being manipulated.

The 30 bad pairs are excluded and those 12 stores were measured again with the list fixed. Properly switched off, Accessibly removes −3 nodes across 23 pairs — three nodes worse with the widget running. The median per store is 0.0%, like everything else.

The full accounting, since a denominator you cannot reconstruct is not worth much:

	pairs
attempted	192
lost — one side hit the 40 s navigation timeout	3
discarded by the validity rule	3
excluded — the block never engaged for this vendor	30
re-measured with the host list corrected (24 attempted, 1 discarded)	+23
analysed	179

The lesson is not "check your regexes". It is that a manipulation has to be verified, not assumed. I had written in my own method notes that a pair where the widget never ran is not testing anything, and then never enforced it.

Half the problem is one rule

Across the wider survey — 729 pages, 367 distinct stores, 18,383 violation nodes — the findings sort like this. (233 of those stores sit in the German contour and 136 in the international one; that sums to 369 rather than 367 because two stores appear in both.) All of these counts are lower bounds: axe-core decides only the part of WCAG a machine can decide, and one node is one element failing one rule, not a measure of how much it hurts.

bucket	nodes	share
deterministic fix, no meaning to invent	3,037	16.5%
detection deterministic, fix is generated text (alt, link name)	5,824	31.7%
fixable but the visual design changes (contrast, target size)	9,219	50.1%
needs a human	303	1.6%

One rule, color-contrast, is 50.1% of everything found. Every confident claim of the form "N% of accessibility problems can be fixed automatically" is, underneath, a claim about how you treat contrast — and contrast cannot be fixed without changing what the site looks like, which is a decision belonging to whoever owns the brand.

I ran a separate test on that: if a handful of colour pairs generate most of a store's contrast findings, contrast is fixable centrally in a theme rather than node by node. Across 990 contrast nodes in 30 stores, the median store needs 2 colour pairs to cover 80% of its contrast findings — promising. But the share of stores clearing the coverage bar I had set in advance came out at 39.6% against a 40% threshold.

That is four nodes out of 990. I left the threshold where it was and recorded the result as undecided. A test that lands this close hasn't told you the answer is no; it has told you your sample is too small to ask.

One more thing about those contrast numbers: 132 of the 990 nodes (13.3%), across five stores, have a computed contrast ratio below 1.1 — text almost exactly the colour of its background. That is the signature of an element that is invisible for some unrelated reason, not of unreadable text. I did not inspect them visually, and they are included in the totals above.

For completeness: how common are these widgets

Not very. In a straight scan of German Shopify storefronts, 16 of 886 had one — 1.8%. In a global sample, 370 of 5,326 — 6.9%.

Which raises a fair objection to the numbers further up: 17 of the 233 measured German stores had an overlay, and that is 7.3%, not 1.8%. Both are correct, and the gap is deliberate. At 1.8% a random draw of 233 stores would contain about four overlay stores, which is not enough to compare anything. So overlay stores were deliberately over-sampled into the measured set — roughly four times their natural rate.

That is fine for the ablation, where each store is its own control and the sample's composition is irrelevant. It is one more reason the store-against-store comparison in the first section should not be read as representative of anything.

Limitations

Read this section before quoting any number above. Several of these are large enough to change what you should conclude.

This measures DOM repair and nothing else. The ablation answers exactly one question: does the source markup contain fewer machine-detectable violations when the widget runs? It does not answer whether a widget helps anyone. An overlay can provide a genuinely useful toolbar — bigger text, a reading mode, a contrast switch — and leave the underlying markup untouched. Such a product scores 0.0% here and may still be valuable to a real person. Nothing in this article should be read as a claim that any of these tools does not work.

Automated testing catches a minority of WCAG. axe-core checks the success criteria a machine can decide. Whether a label actually describes its field, whether reading order makes sense, whether a custom widget is operable by keyboard — none of that is in these numbers. A page with zero violations can be unusable; a page with many can be fine.

A node is not a severity. A missing alt on a decorative icon and a missing name on the checkout button both count as one.

Nobody tried to buy anything with a screen reader. These are scanner counts, not user research. That is a different study and a more important one.

Checkout was never measured. Two pages per store: homepage and one product page. Shopify hosts checkout, and testing someone's checkout without permission is not something I was willing to do — so the single most important page in a store is absent from every number here.

The German contour is now very small. Most of the German ablation sample ran Accessibly, so removing the unblocked pairs cut it from 16 stores to 7. Treat that row as an indication, not a country-level result.

One network position, one moment. All measurements ran from a German exit node between 5 and 8 August 2026, in headless Chromium with no profile or prior consent state. Geo-routing, CDN edges and consent banners all change what a scanner sees. This matters for the survey figures. It matters much less for the ablation, because both sides of every pair were measured from the same position seconds apart — that is the point of pairing.

Blocking a CDN is not uninstalling. Aborting requests to the overlay's host is the closest thing to a controlled off-switch available from outside a site, but a page could in principle behave differently when a script fails to load than when it was never referenced. And, as the third episode above shows, the block only works if your host list is right — which is now something the code checks rather than assumes.

Small samples. 92 store-runs, 179 pairs, three samples. The spread between runs is the honest error bar, and it is why the headline is a median across stores rather than any single run's total.

None of this is legal advice or a compliance assessment. Nothing here establishes whether any site meets any legal requirement in any jurisdiction. Conformance is a judgement about a whole product, and a scanner is not competent to make it.

What is not published

No store domains, no raw HTML, no screenshots. These stores did not agree to be measured, and a list of named sites with their violation counts is a list of targets, not a methods note.

The cost is real and worth stating plainly: you cannot check my arithmetic against my inputs. You can only re-run the method on a sample you draw yourself. The code, the exact versions, the delays, the sampling procedure and the validity rule are all published so that this is possible:

https://github.com/VanyaNeytrino/overlay-ablation

Run it twice on the same sample before you believe any number it gives you — and check that the widget was actually switched off before you believe it did nothing.

Measured with axe-core 4.12.1 via @axe-core/playwright, Playwright 1.62.1, Chromium 151.0.7922.34, tags wcag2a wcag2aa wcag21a wcag21aa, viewport 1440×900, 12,000 ms settle for ablation and 5,000 ms for the survey.