How we test 102 browser-only tools: unit, property, oracle, and real-browser tests

xiuhai shi — Mon, 26 Oct 2026 13:44:20 +0000

A static tool site has no server to catch mistakes. If the Base64 decoder is wrong,
there is no API error, no 500, no server log — just a user who quietly leaves. On
yizhiweb.top the correctness of the engine is the product,
so the test suite has to play the role a backend usually plays.

The site is 102 tools and 162 pages, built with Astro as static files. Every engine
is pure TypeScript with no runtime dependencies. This is the four-layer test setup
we use, the bugs each layer actually caught, and what it still does not cover.

Layer 1: unit tests

Each engine is a pure module: input in, output out, no DOM. The test runner bundles
the engine barrel with esbuild and runs it in Node. That keeps the fast loop fast —
no browser, no build of the site.

Typical cases are boring on purpose: empty strings, one character, the boundary just
below and just above a limit, and the exact error message for invalid input. Boring
tests are cheap and they catch the majority of regressions when an engine is edited.

Layer 2: property and fuzz tests

Examples prove the cases you thought of. Property tests look for the cases you did
not.

For every codec we assert a round-trip: decode(encode(x)) === x for generated
inputs, including empty input, maximum-length input, non-BMP Unicode, and malformed
byte sequences. For hashing and formatting we assert invariants: output length,
character set, stable formatting, and no NaN/undefined leaking into user-facing
strings.

The fuzz layer runs tens of thousands of generated cases. It is not a proof, but it
is very good at finding crashes and silent corruption — the two failure modes that
matter most for a local tool.

Layer 3: oracle tests

This is the layer that gives the most confidence per line of test code: compare the
implementation against an independent reference.

Hashing / HMAC — compare against Node's crypto for the same input.
AES-256-GCM encryption — encrypt in the browser engine, decrypt with Node's crypto; then the reverse. The envelope is self-describing, so the test also checks tamper rejection and wrong-passphrase rejection.
Punycode — compare against Node's built-in punycode implementation, including non-BMP characters.
TOTP — the six RFC 6238 test vectors.
Subnet maths — a second, deliberately naive implementation that manipulates the address as a 32-character bit string, compared with the production version that uses integer arithmetic.
Financial and date maths — closed-form values and hand-computed boundary cases (leap years, ISO week 53, month-end rollovers).

The point is not that the reference is perfect. The point is that the two
implementations fail differently. When they disagree, one of them is probably wrong,
and the test tells you exactly where.

Layer 4: real-browser E2E, plus a coverage gate

The last layer runs Chrome against the built site. It clicks every control on every
tool page, types into inputs, toggles options, checks empty and invalid states, and
verifies that Copy buttons copy what they claim to copy.

That suite is currently more than 1,500 assertions. It also enforces a simple rule:
every control and every Copy button must be covered by a test spec. If a new
button appears without a test, the coverage gate fails. This is the only way we have
found to stop the test suite from drifting behind the UI as tools are added.

Four bugs the layers caught

1. A diff engine that could exhaust memory. The text-diff page had a comment
claiming the algorithm was capped. It was not. A 20,000-line comparison took about
7 seconds and 1.5 GB; 50,000 lines was roughly 10 GB and would crash the tab. The fix
was to trim the common prefix and suffix first, and fall back to a linear comparison
above a hard limit. The same case dropped from 7,023 ms to about 10 ms. The page now
reports when the result was linearised, and the FAQ says so instead of pretending the
result is always a minimal diff.

2. A Copy button that lied. The button showed "Copied!" before checking whether
the Clipboard API actually succeeded. On browsers where navigator.clipboard is
missing or permission is denied, it reported success and copied nothing. The fix
wraps writeText with a feature check, adds a hidden-textarea fallback, and only
shows success after the write resolves. The E2E suite now tests the normal path, the
missing-API path, and the rejected-permission path.

3. Emoji counted as seven characters. The text counter used String.length, so a
family emoji built from a ZWJ sequence was counted as seven. For a text tool, that is
a real correctness bug. The counter now reports both code units and user-perceived
characters using Intl.Segmenter, and the page explains the difference.

4. Small lies in formatting and file detection. formatNumber(-0) returned
"-0"; the EAN-13 renderer ignored the required quiet zone; CSV export did not
escape headers containing commas; an image format fallback reported the requested
format instead of the actual output MIME type. None of these would crash anything.
All of them would make a user distrust the tool.

What this does not cover

Tests cannot tell you whether a tool is useful, whether the copy is clear, or whether
the page answers the query someone typed into Google. They also cannot prove the
absence of a bug — they only make the common failure modes expensive to reintroduce.

Two practical limits are worth stating:

Browser APIs vary. Clipboard, canvas, file readers and Intl differ across browsers and versions. The E2E suite runs one Chrome build; it is not a browser matrix.
Heavy dependencies are lazy-loaded. Image processing and other large modules sit outside the base page bundle. That keeps the first load small, but it means the E2E suite must wait for the lazy module before asserting — a real source of flaky tests if not handled explicitly.

The trade-off

A static, browser-only tool site gives up server-side validation, per-user logs and
easy A/B testing. In exchange it gets a strong privacy story: input never leaves the
device, and there is no account or backend to breach. For a certain class of tools —
encoding, hashing, formatting, image conversion, calculators — that trade is worth
it. The test suite is what makes the trade defensible.

The full method, including the limits above, is published at
yizhiweb.top/methodology. The tools are at
yizhiweb.top.

Disclosure: I build and maintain yizhiweb.top. The article describes its test
setup; it is not a paid placement.

DEV Community: xiuhai shi