Benchmarking regex patterns properly: P95/P99, equivalence checks, real engines

Every developer has been here: two regex patterns that both work. One is a dense one-liner, the other reads like prose. Which do you ship?

"Use the simpler one" is a fine heuristic until the pattern runs over a 50 MB log file or every request in a hot path. Then you want data. The problem is that regex benchmarks done casually lie to you three ways:

1. The engine problem — your JS benchmark measures JavaScript's RegExp, but the pattern ships to a PCRE2-based backend. Different engine, different behavior, different speed.
2. The clock problem — a fast regex finishes in microseconds, far below performance.now() resolution. You're measuring timer noise.
3. The equivalence problem — the faster pattern is faster because it matches less. You optimized a bug.

RegexTester.dev's built-in Benchmark Suite was designed against those three traps. Here's how it works.

Real engines, identical conditions

Up to 5 alternative patterns run sequentially under identical input, flags and engine — the same engines the tester itself uses: PCRE2 and RE2 as WebAssembly in a dedicated worker, .NET's System.Text.RegularExpressions as a real interpreter, and native JS RegExp (including the normalize-to-JS path used for Python, Java and other flavors).

Sub-clock-precision timing

Each sample times K consecutive executions and records time ÷ K, with K auto-calibrated so every sample spans between 0.35 ms and 12 ms. This batch amplification resolves sub-microsecond patterns with sub-nanosecond effective resolution — far below browser clock precision.

You get median, mean, P95, P99, ops/sec and the relative speedup of the fastest pattern, with a ±RSE column that tells you when a number is too noisy to trust.

Equivalence checking

Speed means nothing if the output changed. The suite records each pattern's match count on the benchmark input and raises an "Output differs" warning when counts diverge — a pattern that's faster because it matches less gets flagged before you ship it.

Reading the charts

Latency histograms and timelines make the distribution shape visible: a tight spike means stable performance; a long tail means some inputs hit pathological backtracking even when the median looks fine.

Export

Results export as Markdown (for PRs and docs) or JSON (for your own tooling), with the environment (UA, cores, timestamp) baked in so numbers stay comparable.

---

Try it: open regextester.dev/bench, paste two alternatives, press Run. Sixty seconds later you'll know which one to ship — and whether they really do the same thing.

Full write-up: regextester.dev/blog/regex-benchmark-suite. Free, no signup, everything runs locally.