DEV Community: Daniel

Fifty Green Tests, Two Real Bugs, and the Oracle One `npm install` Away

Daniel — Mon, 31 Aug 2026 16:27:34 +0000

A QR code cannot hide a bug from you, because checking it costs nothing. You point a phone at it. A URL pops up or it doesn't.

That's why I picked a QR encoder for a hackathon whose entire rule is standard library only: no packages, an empty dependency manifest, verified at submission. I rebuilt qrcode, the library nearly every JavaScript project reaches for, in 844 lines of Node and an empty dependency manifest. By the end I had fifty tests, all green, and two bugs that not one of them could see. A phone found the first in four seconds, on day three, the first time I bothered to try. A tool one command away would have found both on the first afternoon. I just wouldn't run it, because running it felt like cheating.

The encoder was the easy part. This is a write-up about the hard part, which was verification, which I also got wrong.

What `npm install qrcode` actually installs

qrcode gets 24 million downloads a week. Before rebuilding it I looked at what it was.

npm install qrcode writes 29 packages to disk, 2.4 MB. Three of them do the actual work: qrcode itself, pngjs for image output, and dijkstrajs, a shortest-path graph algorithm it uses to choose the most compact way to split text across QR's encoding modes. The other 26 are yargs and its dependency tree, there for a command-line interface: cliui, string-width, wrap-ansi, emoji-regex, y18n, camelcase, and down. They install on your server whether or not you ever run qrcode from a terminal. Call require('qrcode') and you have shipped an internationalized argument parser and everything under it.

This isn't a complaint about qrcode. It's that "one dependency" was a fiction. The number was 29, and I hadn't read a line of any of them, which is the normal condition of every Node service I've worked on: a few thousand packages of strangers' code, unread, running as me. The polite word is "dependency." The accurate word is borrowed trust. I trust these people because auditing them is more work than the feature is worth.

The hackathon's premise is that we borrow far more than we need. So how much of the 29 is load-bearing?

The parts you build, and the one you don't

Most of it. The specification (ISO/IEC 18004) is a stack of small closed problems: Reed-Solomon error correction over GF(2⁸), which is two lookup tables and polynomial division; the module matrix with its finder squares and timing lines; eight mask patterns and a penalty function to choose one; fifteen and eighteen bits of metadata wrapped in BCH codes. None of it needs anything outside std.

The one place I stopped was DEFLATE. PNG compresses its pixels with zlib, and a DEFLATE encoder is a hackathon project by itself. node:zlib is standard library; I called deflateSync and wrote the rest of the PNG by hand: signature, chunks, CRCs. That's the single asterisk on "zero dependencies." It's the opening line of my README and the last time I'll mention it, because it isn't where this goes.

Fifty green tests and a phone that said no

By the end of day two I had a test suite I trusted. It wasn't lazy.

Galois-field arithmetic, checked against the values printed in the standard. Reed-Solomon output, checked byte for byte against the standard's worked example: encode "01234567" at version 1, level M, and Annex I gives you the exact codewords. The matrix, checked by an invariant: total modules minus fixed-pattern modules must equal the data-bit count, for every version. Format and version bits, checked against the standard's BCH tables. And a round-trip: a small decoder in the test file that walked the data path and pulled the bytes back, asserting decode(encode(text)) === text for every text, every error level, every mask.

Fifty tests. Green. I rendered "01234567", held up my phone, and got nothing. Bigger, more light, a printout, a different scanner app. Nothing scanned.

The bug a scanner reads first

QR format information is fifteen bits (mask, error level, ten of BCH check) written twice for redundancy, one copy in an L around the top-left finder square. A scanner reads it before it reads anything else.

This placed the first six:

for (let i = 0; i <= 5; i++) modules[8][i] = bit(i);

modules[row][col]. That runs along row 8. It should run down column 8:

for (let i = 0; i <= 5; i++) modules[i][8] = bit(i);

Row and column, swapped, in both copies. Every bit a scanner needs to begin was on the wrong axis.

The round-trip test didn't catch it because it never read the format bits. They're fixed-function modules, and the data-path walk skips them by definition. The payload round-tripped perfectly, and the payload was never where the bug was.

The BCH test didn't catch it because it checked the value, not the address. My fifteen bits were the correct fifteen bits. I had verified what they were and assumed that told me where they went.

Fifty tests, one blind spot

The tests were not weak. They were the wrong shape.

Every one was rigorous about computation: fields, polynomials, BCH. Computation is the part of a QR code that looks hard, and it's the part I'd worked to understand. None checked placement, because by the time I wrote tests, placement felt beneath checking. Put the bits where the figure shows them. I wasn't going to get that wrong.

A test suite is built from its author's model of where the danger is. Mine faithfully encoded mine. The bug lived in the exact gap between "I checked this closely" and "this didn't need checking," and fifty green marks correctly reported that everything I'd thought to test was fine.

That generalizes past QR codes. A passing suite is evidence about the parts you thought to check. Its silence is not evidence.

The oracle I refused to install

qrcode was one npm install away the whole time. I could have generated the same code with both, in a scratch directory that never touched my submission, and diffed the matrices. The transposed bits would have shown immediately.

I didn't, because on a zero-dependency project, installing the thing I was replacing felt like cheating. Not against a rule (there is no such rule), but against the point. I was doing this properly. From the spec. Myself.

That reflex is the same one the hackathon criticizes, pointed inward. "Don't blindly trust a stranger's package" and "don't blindly trust your own code" are one statement. Both kinds of trust need an outside check, and I had argued myself out of the best one on the table.

Writing this, I finally ran the diff. The broken build differs from qrcode on "01234567" by twelve modules, all in the format region, all visible in the first comparison anyone would run. A phone in four seconds, or one diff on day one. I took neither.

The diff also found a second bug.

The second bug, the one that scanned fine

With the format bits fixed, my codes scanned, and I thought I was done. Then I diffed qrkit against qrcode properly: every byte-mode input I could think of, versions 1 through 10, all four error levels. 47 of 48 came back byte-identical. The one that didn't was the string "zero dependency": same version, same mask, same mode, 124 modules different.

It was the padding. When your data doesn't fill the symbol, the spec tops it up with two alternating codewords, 0xEC and 0x11, and it says the first is always 0xEC. Mine:

while (bytes.length < capacity) bytes.push(pad[bytes.length % 2 === 0 ? 0 : 1]);

I picked the pad byte from its absolute position in the stream. When the real data ended on an even byte count the first pad came out 0xEC, correct by accident. When it ended odd, like "zero dependency" does, the first pad came out 0x11 and the entire tail inverted.

This bug cannot break a scan. Pad codewords are discarded before the decoder reads your data. The round-trip test walked over these exact bytes on every run and could not see it, because "correct" meant "the text comes back," and the text always came back. Wrong, and fully functional. Only a byte comparison against an implementation I didn't write could surface it, and that was the comparison I'd spent three days avoiding.

The fix counts the pad bytes instead of the stream bytes, plus a fifty-first test so it can't come back:

for (let p = 0; bytes.length < capacity; p++) bytes.push(pad[p % 2]);

The spec that disagrees with itself

There's a third source of truth here, and it's also unreliable.

Mask selection is meant to be deterministic: apply eight masks, score each, keep the lowest. But the rule that penalizes finder-lookalike patterns is loose enough that implementations disagree on what counts. qrkit, whose scoring I ported from Nayuki's reference implementation, and qrcode, which rolls its own, choose different masks on three of every eight inputs I tested. Both scan.

And a 2008 note by Johan Persson works through the standard's own example, the "01234567" I'd been testing against, and shows the mask it picks is not the lowest-scoring one by the standard's own rules. The reference fails the reference.

So the spec is an oracle you cross-check too. There's no document that hands you the answer. There's a quorum: the spec, its examples, other implementations, a physical scanner. Where they disagree, you decide. I lost an evening treating my mask disagreement with qrcode as a defect before I understood that.

The scorecard, told honestly

Measure	Result
`npm install qrcode`	29 packages, 2.4 MB
`yargs` and its subtree	26 of the 29
`qrkit`	0 dependencies, one 844-line file
Format matrix vs `qrcode`, all 4 EC levels by 8 masks	32 / 32 byte-identical
Byte-mode output vs `qrcode`, versions 1 to 10 by 4 EC levels, after the fix	48 / 48 byte-identical
Mask choice matching `qrcode` when both choose freely	5 / 8
Encode speed, `qrkit` vs `qrcode`	677 µs vs 271 µs
Bugs shipped	2
found by 50 unit tests	0
found by a phone	1
found by a diff available on day one	2

The speed number held at 2.5x across four runs. That's not measurement noise, qrkit is genuinely slower. It re-scores all eight masks with the full penalty function on every build; qrcode has had years to not do that. The value here is the third row of the table, not the seventh.

What I didn't do

I wrote this list before I wrote the README, because the failure mode of a solo project is finding out your limits were load-bearing.

Byte and numeric encoding modes only, no alphanumeric, no kanji, so an all-caps string that qrcode packs tighter comes out larger from qrkit. Versions 1 through 10, not 40. No ECI, no structured append. No decoder: reading a code back out of a photo is thresholding, perspective correction, and error location, and it's the same large, fragile surface that would have sunk the project if I'd tried it in a weekend.

What I'd tell someone building one of these

Reach for the oracle you didn't write first, before the first unit test. A phone, a reference implementation, a printed example. If you're rebuilding X, npm install X in a scratch directory and diff. That's the control group.
Treat a green suite as a map of the ground you thought to cover. Ask what class of bug it structurally cannot see.
"Different function" is not "independent implementation." An encoder and a decoder written by one person from one reading of one spec agree with each other, not with the world.
Some bugs never fail a functional test, because the function still works. Conformance is a separate question, and only a foreign implementation answers it.
Write down what you didn't do.

Where trust actually comes from

I expected the lesson to be about dependencies, that I'd finish having shown you don't need the tree.

That isn't it. Deleting the dependencies bought me nothing; an empty manifest is just a different set of things to be wrong about. Writing it myself bought me less than I thought, because every check I built came from the same head as the bug. My test file is 769 lines against 844 of source. I spent nearly as long proving the thing as building it, and I still shipped two defects, because the proof and the code had the same author and the same blind spot.

The bugs died when the code met something with no stake in whether I was right: a phone camera, a committee's standard, a stranger's hand arithmetic from 2008, and the library I was too proud to install.

References

ISO/IEC 18004, QR Code bar code symbology specification. Current revision 2024; the 01234567 worked example and codeword tables cited here are in Annex I of the 2015 revision. Paid standard.
Johan Persson, "A note on minor errors in the International QR Barcode standard" (2008). Source for the claim that the standard's own worked example does not select the lowest-penalty mask under its own rules, and for the hand-computed matrix qrkit is tested against. https://www.coastalmonitoring.org/resources/jpclass/QR/qr-comment.pdf
Project Nayuki, "QR Code generator library." The reference implementation qrkit's mask-penalty scoring follows. https://www.nayuki.io/page/qr-code-generator-library
qrcode (npm), MIT. The package qrkit replaces and is diffed against. https://github.com/soldair/node-qrcode. Download figure for the week of 23 to 29 August 2026, from https://www.npmjs.com/package/qrcode.
qrkit, MIT. https://github.com/Dannyamah/qrkit

Built for the Zero Dependency Hackathon by Hackathon Raptors. #hackathonraptors

140 Bugs Were Hiding in One Function, and My Tests Couldn't See Any of Them

Daniel — Mon, 10 Aug 2026 09:57:25 +0000

Anyone can port a library. Point a translator at the source, clean up the output, get it to compile, and you have something that looks like a port. The actual engineering problem is different and much harder: proving that the new code means the same thing as the old code, across thirty algorithms, hundreds of edge cases, and a test suite written by people who were not thinking about you.

This is the story of porting textdistance, a Python library for measuring string similarity, to Rust. The result is textdistance-rs. The porting took a fraction of the time. Everything else: the differential fuzzing, the 140 divergences, the 35 year old threshold I violated, the floating-point drift at the 15th decimal place, is what this writeup is actually about.

Thirty algorithms and one architectural bet

The original textdistance covers a lot of ground: edit-based distances (Levenshtein, Damerau-Levenshtein, Hamming, Jaro-Winkler), token-based measures (Jaccard, Sørensen-Dice, cosine, Tversky), sequence-based methods (LCS, Ratcliff-Obershelp), phonetic algorithms (MRA, Editex), and compression-based distances built on normalized compression distance. Over thirty algorithms in total, all reimplemented in Rust.

But before writing a single algorithm, I had to make the decision that shaped everything downstream: how does the existing Python test suite (397 tests I did not write) talk to the Rust code?

The obvious answer is PyO3: wrap every algorithm in a #[pyclass], build a native extension, and the Python tests import Rust directly. The answer I chose instead was a subprocess CLI. The Rust core is a standalone binary that speaks JSON over stdin/stdout, and a thin Python adapter shells out to it:

// The entire cross-language surface is one struct.
#[derive(Deserialize)]
struct Request {
    algorithm: String,
    s1: String,
    s2: String,
    qval: Option<usize>,
    external: Option<bool>,
}

fn dispatch(req: &Request) -> Response {
    match req.algorithm.as_str() {
        "hamming" => hamming::distance(&req.s1, &req.s2),
        "levenshtein" => levenshtein::distance(&req.s1, &req.s2),
        "jaro_winkler" => jaro::winkler_similarity(&req.s1, &req.s2),
        // …thirty more arms
        _ => Response::error("unknown algorithm"),
    }
}

The reasoning: PyO3 binds you to Python's ABI, complicates the build for anyone who just wants the Rust crate, and turns every signature mismatch into a compile-and-link problem. A JSON pipe is crude, but it is debuggable with your eyes. You can replay any failing case by piping a JSON string into the binary from your shell. When the goal is verifying behavioral equivalence against a foreign test suite, that inspectability is worth a lot.

It was a speed decision, and I knew it. What I did not know was that this bet had quietly closed a door I would walk straight into later on.

The bug that was older than the port

The single worst debugging session of the project traced back to Jaro-Winkler, and specifically to a paper published in 1990.

Jaro similarity comes from Matthew Jaro's 1989 work on record linkage for census data. William Winkler's 1990 refinement adds a bonus for strings that share a common prefix, on the empirical observation that clerical typos rarely occur in the first few characters of a name. My first implementation applied that prefix boost the way the formula usually gets quoted:

// What I wrote first: boost applied unconditionally.
let jaro = jaro_similarity(s1, s2);
jaro + (prefix_len as f64) * P * (1.0 - jaro)

It looks right. It passes the examples people paste into blog posts. And it is wrong, because Winkler's actual method only applies the boost when the base Jaro score exceeds a threshold of 0.7 in his published work. Below that, the strings are too dissimilar for a shared prefix to mean anything, and boosting them inflates scores for garbage matches:

// What Winkler actually specified in 1990.
let jaro = jaro_similarity(s1, s2);
if jaro > BOOST_THRESHOLD { // 0.7
    jaro + (prefix_len as f64) * P * (1.0 - jaro)
} else {
    jaro
}

Combined with a second, smaller mistake: the original library treats two empty strings as perfectly similar (1.0), and my port returned 0.0. This one function accounted for 140 divergences in fuzz testing. One commit fixed all of them.

The lesson that stuck with me: I was not diverging from the Python library. I was diverging from a 35-year-old paper that the Python library had implemented faithfully and I had implemented from memory. When you port a library, you inherit its citations, not just its code.

Proof by ten thousand disagreements

That raises the obvious question: how did I even know there were 140 divergences? The unit tests for Jaro-Winkler passed. Example-based tests check the inputs somebody thought of, and nobody thinks of "ab" vs "aaab" with a coincidental prefix and a sub-threshold Jaro score.

The answer was differential fuzzing — running both implementations on the same randomized inputs and diffing the outputs. The technique goes back to McKeeman's work on differential testing for compilers: when you have two implementations that claim to compute the same function, disagreement between them is a test oracle you get for free. You don't need to know the correct answer; you only need to know they should match.

The harness is embarrassingly simple, which is the point:

from hypothesis import given, strategies as st
import textdistance          # the Python original
import textdistance_rs       # the port, via the JSON pipe

@given(st.text(), st.text())
def test_jaro_winkler_parity(s1, s2):
    expected = textdistance.jaro_winkler(s1, s2)
    actual = textdistance_rs.jaro_winkler(s1, s2)
    assert abs(expected - actual) < 1e-9, (s1, s2, expected, actual)

Hypothesis, Python's property-based testing library, descended from Claessen and Hughes' QuickCheck, generates the adversarial inputs: empty strings, single characters, Unicode combining marks, strings that are prefixes of each other. Every property-based failure gets shrunk to a minimal reproducing case, which is how a six-hour bug hunt eventually collapsed into a two-character counterexample I could reason about on paper.

After the Jaro-Winkler fix and a handful of smaller ones, the fuzzer reached zero divergences across every algorithm and input class it could generate. That number, not the passing unit tests, is the claim I actually stand behind. But notice the caveat hiding in that sentence: zero divergences on everything that could cross the pipe. Two things could not.

The door I closed on myself

Several of textdistance's alignment algorithms — Needleman-Wunsch, Smith-Waterman, Gotoh — accept a sim_func parameter: an arbitrary Python callable that scores the similarity of two characters. The test suite exercises it:

# From the original test suite. This cannot be serialized.
alg = NeedlemanWunsch(sim_func=lambda a, b: 1 if a == b else -1)

A lambda cannot be serialized into JSON and sent down a pipe to a Rust subprocess. It is a closure over a live Python interpreter. There is no workaround that isn't a horror, no eval-strings-in-Rust, no RPC-callback-per-character scheme that would be slower than just running the Python.

So those tests fail. 22 of them, permanently, by architecture rather than by bug. This is the direct, foreseeable cost of the subprocess decision from the beginning of this article. PyO3 would have handled it natively: a #[pyfunction] can hold a reference to a Python callable and invoke it from Rust mid-algorithm. I traded that capability for build simplicity and debuggability, and for 28 algorithms the trade was clearly right. For these 4, I built a wall and then walked into it.

If I ran this project again, I would keep the subprocess architecture for development and verification, it earned it's keep during fuzzing, and add a PyO3 layer at the end specifically for callable-taking algorithms. That hybrid was always possible. I just didn't see the need until the tests told me.

The 15th decimal place

The other place the port compromised was subtler, and it's the result I'm least happy reporting: the "pure Rust" story does not fully hold for the compression-based algorithms.

Normalized compression distance, from Cilibrasi and Vitányi's work on clustering by compression, approximates the information distance between strings using a real compressor:

NCD(x, y) = (C(xy) − min(C(x), C(y))) / max(C(x), C(y))

By construction, the output is exquisitely sensitive to the compressor's exact byte counts and the arithmetic done on them.

The original library computes arithmetic-coding probabilities using Python's Fraction, exact rational arithmetic, no rounding. My Rust implementation used f64. The fuzzer caught divergences on the order of 1e-15: not wrong in any way a human would notice, but not equal, and for NCD "close" is doing real work in the formula, because those values get subtracted and divided in ways that can amplify the drift.

I tried rational arithmetic in Rust. I tried reordering operations to match Python's evaluation order. What shipped is the honest defeat: for the arithmetic-coding NCD path, the final distance computation moved back into the Python adapter, where Fraction guarantees exact parity. Rust still does the heavy lifting; Python does the last arithmetic mile. Eight algorithms carry that asterisk, and I'd rather document the asterisk than pretend it isn't there.

What the port actually proved

None of these four things — the JSON pipe, the Winkler threshold, the fuzzing harness, the sim_func wall were about writing Rust. They were about the much slower work of finding every place a fast, plausible looking reimplementation quietly disagrees with the thing it's replacing, and then deciding, for each one, whether to fix it, work around it, or write it down and ship anyway.

That's the actual deliverable here, more than the crate itself: not thirty algorithms in Rust, but a documented account of where equivalence held, where it took a 35 year old paper to restore, and where it simply didn't. A port you can't interrogate isn't really verified, it's just translated. This one, at zero fuzz divergences and two known, named exceptions, is the former.

References

life4/textdistance — the original Python library.
Len3hq/textdistance-rs — the Rust port discussed throughout.
Jaro, M. A. (1989). Advances in record-linking methodology as applied to matching the 1985 census of Tampa, Florida. Journal of the American Statistical Association, 84(406), 414–420.
Winkler, W. E. (1990). String Comparator Metrics and Enhanced Decision Rules in the Fellegi-Sunter Model of Record Linkage. Proceedings of the Section on Survey Research Methods, American Statistical Association.
McKeeman, W. M. (1998). Differential Testing for Software. Digital Technical Journal, 10(1), 100–107.
Claessen, K., & Hughes, J. (2000). QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming (ICFP '00).
Cilibrasi, R., & Vitányi, P. M. B. (2005). Clustering by Compression. IEEE Transactions on Information Theory, 51(4), 1523–1545.

The full port and the DECISIONS log this writeup is pulled from is up at Len3hq/textdistance-rs if you want to dig in yourself.
Written for Hackathon Raptors, worth a follow if you're into this kind of writeup.

DEV Community: Daniel

Fifty Green Tests, Two Real Bugs, and the Oracle One `npm install` Away

What npm install qrcode actually installs

The parts you build, and the one you don't

Fifty green tests and a phone that said no

The bug a scanner reads first

Fifty tests, one blind spot

The oracle I refused to install

The second bug, the one that scanned fine

The spec that disagrees with itself

The scorecard, told honestly

What I didn't do

What I'd tell someone building one of these

Where trust actually comes from

References

140 Bugs Were Hiding in One Function, and My Tests Couldn't See Any of Them

Thirty algorithms and one architectural bet

The bug that was older than the port

Proof by ten thousand disagreements

The door I closed on myself

The 15th decimal place

What the port actually proved

References

What `npm install qrcode` actually installs