Anish Prakash

Posted on Aug 30

Everything Was Working Exactly As Documented. The Answer Was Still Wrong.

#rust #security #showdev #opensource

I spent a weekend building a supply-chain auditor for dependency lockfiles. It has no dependencies. That part is a gimmick — it was the rule of the Zero Dependency Hackathon, where the whole event is one constraint: your dependency manifest ships empty, standard library only.

The gimmick is not what I want to write about.

What I want to write about is that almost every bug I hit that weekend had the same shape. A mechanism was working exactly as its documentation said it would, and the answer coming out the other end was wrong anyway. Not broken — working, and wrong. By Sunday I'd seen it four times, and by the fourth I realised it was also a decent description of how supply-chain attacks get past everybody.

Here's the tour.

The rule that cried wolf

The tool is called stranger. You point it at a package-lock.json, Cargo.lock, or requirements.txt and it reconstructs the transitive dependency graph and tells you what's in there: how much of the tree nobody chose, what runs code during install, what arrived without an integrity hash, and which names sit one keystroke away from something popular.

That last one is the reason the project exists. Roughly 19.7% of packages suggested by AI coding models don't exist, the invented names repeat, and attackers pre-register them. It's a supply-chain attack where the victim never even makes a typo — the model does it for them.

So: bounded Damerau-Levenshtein against a list of widely-installed package names, flag anything within two edits. Damerau rather than plain Levenshtein because the most common typo is a transposition — axois, lodahs, python-dotnev — and Levenshtein charges 2 for a swap, the same as two unrelated edits, which puts real squats outside a distance-1 ball.

I ran it against a real 665-package tree. It reported:

etag is 2 edits from tar
depd is 2 edits from del
exit is 2 edits from next
bser is 2 edits from bson Every one of those is true. Every one is worthless. Those are Express's own dependencies.

The edit-distance implementation was correct. The corpus was fine. The rule was doing precisely what I wrote it to do, and the output was garbage — because "within two edits" is a meaningless threshold when the name is four characters long. Two edits on etag is half the string.

Three gates fixed it, and only one of them is interesting:

Distance relative to length. d=1 needs ≥5 characters, d=2 needs ≥8. Two edits on python-dotenv is a typo; two edits on etag is a different word.
Compare like with like. @jest/core is not a near-miss for @types/node just because core and node are two apart.
The third one is the next section. After the fix, that same 665-package tree reports zero high-severity findings, which is the correct answer for an ordinary project. I later ran it across every real lockfile on my machine — about 1,030 packages across two Next.js projects it had never seen — and got zero false positives.

The lesson isn't "tune your thresholds." It's that for a security tool, precision beats recall, badly. A rule that fires on Express's own dependencies is a rule people turn off, and a disabled rule catches nothing. A scanner at 100% recall and 5% precision protects no one, because after the third false alarm nobody reads the output. I knew that in the abstract. I did not act like I knew it until I saw etag accused of impersonating tar.

What you reach for when you can't reach the network

Gate three is the one I'd keep if I could only keep one, and it exists purely because the hackathon rules wouldn't let me have a network.

The obvious way to know whether safer-buffer is a real package or a squat is to ask the registry for its download count. I couldn't. The rules put "projects that need a running third-party service" out of scope — and honestly, an auditor that phones a registry to tell you your registry is dangerous has a credibility problem anyway.

So what popularity signal exists inside a lockfile?

In-degree. How many packages in this tree independently depend on that one.

A typosquat gets pulled in by one mistaken import — the developer, or the model, that reached for the wrong name. safer-buffer gets pulled in by half the registry. So: if a suspicious-looking name has three or more independent dependents, demote the finding to informational. Not delete it — the evidence is weaker, not absent — but stop shouting.

Here's what I didn't expect. In-degree is a better signal than download counts would have been. A typosquat has a download count too; that's the entire point of registering it. What a squat cannot manufacture is thirty unrelated packages, maintained by different people, choosing to depend on it. Download counts measure how many people got fooled. In-degree measures how many maintainers made an independent judgement.

I would never have written that rule with a network available. I'd have called the API, gotten a number, shipped it, and had a worse tool. The constraint didn't tax the design; it produced the design.

That's the honest version of "constraints breed creativity," which I've always found a slightly smug thing to say. The mechanism isn't mystical. Removing the obvious option forces you to actually look at your problem, and sometimes the second-best-looking option turns out to be first.

Three layers of a deterministic build that wasn't

The event offered a +5 bonus for a reproducible build: compile twice, produce byte-identical output, publish both hashes.

On Linux it worked first try. ELF carries no link timestamp; with codegen-units=1 and stripped symbols, two builds matched. I wrote "verified byte-identical" in the README and moved on.

Then I ran the same check on Windows.

Layer one. DIFFER at byte 249 — the COFF TimeDateStamp. PE executables embed a link timestamp. Obvious, documented, and there's a documented fix: MSVC's /Brepro, which replaces that field with a hash of the file contents instead of the clock.

Layer two. With /Brepro, the builds still differed. 69 bytes, across three sites, and the differing values weren't clock-shaped — 0x5E131510 versus 0x87A635AA. Those are hashes. /Brepro was working.

It took me longer than I'd like to admit to see it. MSVC also emits a CodeView debug directory containing a PDB GUID, and that GUID is regenerated on every link. /Brepro hashes the whole file. Including the random GUID.

The deterministic timestamp was faithfully, correctly reflecting a nondeterministic input.

That's the sentence I keep coming back to. /Brepro did not malfunction. It computed exactly the hash it promised to compute, of exactly the bytes it was given, and the result was garbage because one of those bytes was noise. /DEBUG:NONE drops the debug directory, and both builds settle byte-identical.

Layer three. Fixed, committed, and I sent myself the verification command. It failed:

LINK : fatal error LNK1181: cannot open input file 'C:\Program Files\Git\Brepro.obj'

Git Bash. MSYS rewrites any argument that looks like a Unix path into a Windows path before the program sees it, so -C link-arg=/Brepro reached link.exe as C:/Program Files/Git/Brepro. The flag was correct. The shell edited it in transit. MSYS_NO_PATHCONV=1 turns that off, and the Makefile now exports it so nobody else has to find out.

Three layers. Three mechanisms — a linker flag, a hash function, a shell — each doing exactly what its documentation says, composing into a wrong answer.

The pattern, and why it's the same problem the tool is about

Somewhere in layer two it occurred to me that I was debugging the exact failure mode my program is supposed to detect.

An integrity hash in a lockfile is a /Brepro. It is a correct hash, honestly computed, over content that nobody pinned. When chalk and debug were compromised in September 2025 — utilities with 2.6 billion combined weekly downloads — the attacker phished a maintainer and published through the front door. Every hash matched. Every signature verified. The mechanism worked perfectly and told you nothing, because it was measuring the wrong thing: not "is this the code you reviewed" but "is this the code the registry served."

Same shape as the GUID. Same shape as etag being two edits from tar — a true statement, correctly computed, that answers a question nobody asked.

This is, I think, the actually useful thing I took from the weekend, and it's why I stopped trying to make stranger a scanner that tells you a package is bad. It can't. It has no CVE feed and no advisory database, because both need a network. What it can tell you is the shape of your exposure:

        665  packages in the tree
         22  you chose
        643  arrived with them

  #######################################. 97% of your tree is code nobody chose

  max depth 9  ·  1 run install scripts  ·  32 duplicated  ·  8 cycles

Nothing there is a vulnerability. All of it is true whether or not anything has gone wrong yet, and none of it depends on a mechanism that might be faithfully reporting a value someone else controls.

My favourite output from the whole build came from running it on my own Next.js project — nine hops from a single dev dependency:

eslint-config-next → eslint-plugin-import → array-includes → es-abstract
  → typed-array-byte-offset → reflect.getprototypeof → which-builtin-type
  → is-async-function → async-function@1.0.0

Nobody chose async-function. Nobody reviewed it. Nobody is watching its releases. It ships anyway.

What the standard library made painful

The honest accounting, because a post that says the constraint was free is lying.

Writing a JSON parser is easy. Writing a correct one is not. 569 lines to replace serde_json — and serde_json doesn't arrive alone, it brings serde, serde_derive, syn, quote, proc-macro2: five crates and two procedural macros that execute arbitrary code in your build, to read a text file.

The parser was maybe two hours. The strictness was the rest of the day, and it's where hand-rolled parsers quietly go wrong:

Surrogate pairs. "\ud83d\ude00" is one emoji. A lone \ud800 must be an error, not U+FFFD.
Raw control characters inside strings are invalid JSON. Accepting them means accepting input other parsers reject — which is a fun way to disagree with your own CI.
The number grammar. 01 is not a number. Neither is .5, 1., or +1. Leading zeros in particular are how an octal reading gets smuggled past a lenient reader.
A depth cap, because a recursive-descent parser meeting 100,000 nested [ is a stack overflow, and this program reads files an attacker may have written. I tested it against a 30-case must-reject corpus modelled on JSONTestSuite. Getting the n_ cases right is the entire difference between "reads my file" and "is a JSON parser."

serde_json is also faster than mine, and has a Deserialize derive so every field access isn't an explicit get("version").and_then(as_str). I'd still use it in production. That isn't the same as needing it here.

Where I stopped, and said so. The Cargo.lock reader is not a TOML implementation — no inline tables, no dotted keys, no datetimes. It's the subset Cargo emits, and where the subset ends it returns "inline tables are outside the Cargo.lock subset this reader implements" rather than guessing. A parser that silently mis-reads input it doesn't understand is worse than one that refuses it.

Where I just lost. The report is a fixed 78 columns. Terminal width detection needs either an unsafe ioctl — the file is #![forbid(unsafe_code)] — or shelling out to stty, which is a hidden dependency on an external binary. So it doesn't adapt. That's a real limitation, not a solved problem, and it's in the README as one.

The package I made look unnecessary

serde_json, for the reasons above. But the one that actually surprised me was chalk — 319 million weekly downloads.

ANSI colour is \x1b[31;1m. Those escape codes have been stable since 1979. They are four characters.

What chalk actually sells is not the codes. It's the answer to when to emit them, because writing colour into a pipe corrupts every downstream grep. That's a real problem, and it's worth something. It is worth about three lines:

match args.value("color") {
    Some("always") => return true,
    Some("never") => return false,
    _ => {}
}
if std::env::var_os("NO_COLOR").is_some() { return false; }
std::io::stdout().is_terminal()

Explicit flag wins, then NO_COLOR, then TTY detection. IsTerminal has been in std since Rust 1.70. Three hundred and nineteen million weekly downloads for a decision tree with three branches.

I'm not being smug about it — I've installed chalk without thinking, too. That's the point. The registry is full of packages that exist because at some moment it was easier to add a line to a manifest than to think for ninety seconds, and each one is a maintainer account, a publish token and a release pipeline you've quietly agreed to trust.

The bug my own evidence caught

One more, because it's the most embarrassing and therefore probably the most useful.

Before submitting, I ran the tool across every real lockfile on my machine and committed the output as a field test — the idea being that rules tuned on two fixtures are a weak basis for claiming precision.

Reading my own evidence file, I found stranger reporting:

core-js is a one-liner carried as a dependency

core-js is a polyfill suite of roughly 150,000 lines.

The advice was defensible — modern runtimes make it unnecessary — but that's a different argument from the one the rule was making, and the rule was making its argument in a way I couldn't defend. core-js and es6-promise came off the list, the finding got retitled to "is a standard-library call carried as a dependency," and there's now a test that fails if any large library gets added back.

A rule is worth as much as its least defensible entry. core-js was going to be the line a reviewer quoted.

The field test wasn't confirmation. It was the thing that caught the defect — which is the only reason to run one.

What's actually in the box

4,549 lines of Rust in a single file, 64 tests on the standard-library harness, no test crate. Ten sections: an RFC 8259 JSON parser, a Cargo.lock reader, a requirements.txt reader, bounded Damerau-Levenshtein with confusable-glyph folding (cha1k and chalk reduce to the same skeleton), an embedded corpus of 454 npm names and 50 genuinely-trivial packages, a graph engine that reproduces Node's own resolution algorithm, nine offline rules, an ANSI reporter, an argument parser, and the tests.

$ cargo tree
stranger v1.0.0

That's the whole tree. Cargo.lock has one [[package]] block and it's this program. The tool audits itself as part of the dependency proof and reports zero packages, which is a slightly smug thing to put in a CI job and I did it anyway.

Repo: github.com/AnishPrakash/stranger

The thing I'd tell past-me on Friday

Not "avoid dependencies." I'm not against libraries, and neither is the event — the rules say as much. serde_json is better than what I wrote. chalk solves a real problem.

What I'd say is: the mechanisms you're trusting are almost never lying to you, and that's exactly why they're dangerous. /Brepro computed an honest hash of a random number. npm audit returns clean on a package that was compromised this morning through the front door. An integrity hash matches a tarball that was republished under the same version. Edit distance correctly reports that etag is two characters from tar.

Every one of those is a correct answer. Whether it's a useful one depends entirely on whether the question it answers is the question you were asking — and that's not something the mechanism can tell you. You have to go and look.

Which, incidentally, is the whole argument for occasionally building the thing yourself. Not because your version will be better. Because you can't inspect a question you never had to ask.

Built for the Zero Dependency Hackathon run by Hackathon Raptors, August 2026. Track A, Rust, standard library only.
Tag: @raptorsdev

DEV Community