Repo: https://github.com/Yash-vs9/bindery
Demo video: https://youtu.be/9wFCPcWEZzM
@partnerships_raptors
I built bindery for the Zero Dependency Hackathon: a tool that turns a folder
of Markdown into a full documentation site, complete with live reload, full
text search, PDF export, and hand rendered diagrams. One binary. Go standard
library only. Empty manifest, no require block, nowhere for a dependency to
even hide. The whole premise of this event is that half of today's code gets
written by an AI that confidently invents package names that don't exist, and
that the registry those hallucinated packages point at added something like
four hundred and fifty thousand malicious entries last year alone. My plan
was to prove the opposite was possible. Build something genuinely useful using
only what the language already gives you, no imports, no excuses, and see how
far that actually gets you.
What I didn't expect was that the AI helping me build it would hallucinate
something far more interesting than a fake package name. Over one long
weekend it hallucinated a protocol constant, misread a standard library
upgrade's silent behavior change, drew a graph with an arrow pointing at
nothing, and then, near the very end, wrote a test that flagged its own
correct code as a security hole. Every single one of those looked completely
fine on the surface. Every single one compiled. Every single one passed the
tests that existed at the time. Catching them, one at a time, ended up being
the actual story of this project, more than the feature list ever was.
The bug that would have failed silently forever
Bindery's live reload works over a WebSocket I wrote by hand, because Go's
standard library has no WebSocket implementation at all, none, not even an
experimental one. The opening handshake needs one very specific ingredient: a
GUID defined in RFC 6455, concatenated with the client's Sec WebSocket Key,
hashed with SHA1, and base64 encoded into the response. Get one character of
that GUID wrong and nothing crashes. No error. No stack trace. No panic to
grep for in a log file. The socket just quietly, politely, never opens, and
you sit there refreshing a browser that never reloads, with absolutely no clue
why, because every layer above the handshake is working exactly as designed.
That's exactly what happened. The AI wrote the GUID from memory and it was
wrong. Not obviously wrong, not gibberish, not a string that looks like a
placeholder. It had the right shape, the right length, the right hyphen
pattern of a UUID, and it sat right next to a code comment claiming it was
"transcribed directly from the RFC." It wasn't. One character had drifted
from the front of the final group to the back of it, the kind of transposition
a human proofreader skims straight past because the overall silhouette of the
string looks correct.
The only reason this got caught at all is that I refused to accept "it
compiles" as evidence and insisted on a test that pinned the actual output
against the worked example the RFC itself publishes, including the
intermediate SHA1 digest, not just the final base64 string. That test failed
on its very first run. And here's the part that actually worried me: when it
failed, I went back to check the "correct" value from memory too, and my own
recollection didn't match either. Neither the code nor my own head could be
trusted at that moment, which meant there was exactly one thing left to do,
which was go fetch the actual RFC text and check both of us against it like a
referee. The code was wrong. My memory of what should replace it was also
wrong in a different way. Only the primary source settled it.
That felt like the entire hackathon compressed into one bug. An AI can write
code fast and with total, unwavering confidence. It genuinely cannot tell you,
from the inside, when that confidence is misplaced, because a hallucinated
constant reads exactly the same as a correct one. The only real defense is a
test that checks against ground truth published somewhere outside the model's
own head, not a test that checks whether the output merely looks plausible.
When the standard library itself quietly broke me
Go 1.27 shipped a new package, encoding/json/v2, and I reached for it to
serialize bindery's search index because it's newer and faster and the
hackathon's own capability notes flagged it as a fresh, real standard library
answer worth using. What none of us caught until much later is that the old
encoding/json sorted map keys before writing them out as JSON, quietly, as an
implementation detail nobody ever had to think about, and the new package
does not carry that behavior forward. It writes map keys in whatever order
Go's runtime happens to iterate them internally, and Go deliberately
randomizes that order between runs, on purpose, specifically so nobody
accidentally depends on it.
Bindery's search index is, structurally, a map of words to the list of pages
containing them. So every single time I rebuilt the site, that JSON came out
with its keys in a different byte order. Same words, same postings, same
content in every meaningful sense, completely different bytes on disk. That
kind of bug is invisible right up until the exact moment you try to prove
something stronger than "it looks right," which is precisely what one of this
event's bonus challenges asks for: build the same source twice, hash both
outputs, and they had better match exactly.
I wrote a test that built the identical search index nine times in a row and
compared every byte of the result. It failed on the very first run. The
compiled binary itself was perfectly reproducible, byte for byte, hash for
hash. The data that binary produced when it actually ran was not. A check that
only hashes the executable would have printed a clean green REPRODUCIBLE and
sailed straight past a real, silent hole in the actual claim being made. The
fix turned out to be a single line, a Deterministic option the new package
happens to expose if you know to look for it, but finding the gap in the
first place meant refusing to treat "reproducible" as a property of the build
system alone, when it's really a property of every single piece of output
that build system ever touches.
Looking at the actual picture instead of trusting the code
Bindery renders diagrams straight out of Markdown fences, written in a small
subset of Mermaid's syntax, but with zero JavaScript shipped to the browser
and zero external layout library involved. It's a hand written graph layout
algorithm underneath: assign every node a layer based on the longest path to
it, order the nodes within each layer to reduce how many lines cross each
other, then draw boxes and arrows as raw SVG text, no image library required
anywhere in the chain.
The first version compiled cleanly, passed every unit test I had written for
it, and quietly produced a diagram where an arrow pointed directly at nothing,
floating in empty space. A loop in the graph, something as mundane and common
as "the file watcher notices a change, does its work, and goes back to
waiting for the next one," completely broke the layering, because my layer
assignment logic had quietly assumed the graph could never contain a cycle,
and just kept pushing the same node deeper and deeper on every single pass
whenever one actually existed.
None of my tests caught this, for a very specific and slightly embarrassing
reason: not one of them actually rendered the SVG and looked at it. They
checked node counts. They checked edge counts. They checked that specific
strings like "marker-end" showed up somewhere in the output. All of that
passed while the picture itself was visibly, obviously broken to anyone who
bothered to open it. I only found the bug because I exported that SVG to a
PNG and actually looked at it, the exact same way any real human being would
look at it before ever putting it in a demo video for other people to judge.
And once I started actually looking instead of just asserting, I found a
second problem sitting right next to the first one: an edge that spanned more
than one layer was being drawn as a perfectly straight line directly through
the boxes sitting between its two endpoints, which is completely correct
coordinate math and completely wrong to look at.
This is the pattern I kept running into over and over across this whole
project. Correctness according to the tests I had already thought to write is
not the same thing as correctness. Sometimes the only honest verification
step is opening the actual output the way a real person eventually will.
Three Windows bugs, and none of them were logic bugs
Once the core parser was solid, I set up continuous integration across Linux,
macOS, and Windows, because a judge running your submission on the wrong
operating system and hitting a wall is entirely on you, not on them. Windows
failed. Not once, but three separate times, on three genuinely different,
completely unrelated causes, and looking back at all three together taught me
something I didn't expect going in: every single one of them was a hidden
assumption baked into a test, never a real defect in what bindery actually
does.
The first failure took down an entire suite of end to end tests all at once,
every single one with the identical error, "executable file not found."
Windows requires a literal dot exe extension to execute a binary at all, full
stop, no exceptions, and my test harness built the shared test binary with a
bare name and no extension whatsoever, because that's simply what works
without a second thought on Linux and macOS. The fix was two lines checking
the current operating system. The lesson underneath it was bigger: an entire
category of failures, every single test in that file, traced back to one
completely wrong assumption about what "runnable" even means on a different
platform.
The second failure was subtler and, honestly, more interesting to me. A test
asserted that a missing directory produced the literal Unix error string "no
such file or directory" on the program's stderr. Windows reports the
identical underlying failure with entirely different wording, something like
"the system cannot find the file specified," because that's simply how its
own error reporting is worded at the operating system level. Bindery's actual
behavior was completely correct on both platforms the entire time: right exit
code, empty stdout, a real and useful error message on stderr. My test was
quietly checking English phrasing rather than the actual behavior it claimed
to be verifying.
The third was the strangest of the three. Windows checks out Git repositories
with a setting called autocrlf turned on by default, which silently rewrites
every LF line ending to a CRLF pair inside anything Git considers a text
file, including Go source files, including the exact multi line raw string
literals my code embeds for the theme's CSS and JavaScript. Without an
explicit gitattributes file forcing LF everywhere regardless of platform,
those embedded byte for byte strings could have silently gained an invisible
carriage return before every newline purely because of which operating system
happened to check the repository out, an entirely different compiled binary
built from an identical git commit depending solely on which machine cloned
it.
Three genuinely different root causes, three real fixes, and not one line of
bindery's actual logic changed for any of them. What changed, every single
time, was an assumption quietly baked into a test or a build script that
nobody had ever bothered to question because it had simply never been wrong
before that specific afternoon.
Sourcing real data instead of letting anything guess
The PDF export needed accurate character widths so that lines of text
actually wrap correctly instead of running off the edge of the page or
leaving awkward gaps. The tempting shortcut here, the one I almost took, was
to just ask for typical widths for a handful of common fonts and trust
whatever number came back with total confidence. Given everything above, I
didn't trust that shortcut for one second longer than it took to think of it.
Instead I went and fetched the actual published Adobe Font Metrics files for
Helvetica, Helvetica Bold, Helvetica Oblique, and Courier, the same core
fourteen fonts every single PDF reader on the planet is contractually
guaranteed to already have installed, parsed the real character width tables
directly out of those files, and cross checked a handful of well known values
by hand against numbers I could actually verify independently, like the fact
that Helvetica's capital M is exactly eight hundred and thirty three
thousandths of an em wide, and Courier is uniformly six hundred thousandths
across every character because it's a monospace font by definition. A test
now asserts both of those specific facts on every single run, which means a
corrupted metrics table fails a build loudly instead of silently producing a
subtly misaligned document that only a human proofreading a PDF would ever
notice, months later.
The exact same discipline showed up again chasing the very last percentage
point of CommonMark conformance. Bindery sat at six hundred and fifty one out
of six hundred and fifty two official spec examples for a while, one single
stubborn failure away from a perfect score, and the one remaining case needed
full Unicode case folding, specifically the capital sharp S character folding
down to two lowercase letters rather than one. Go's standard library only
implements simple case folding, one character to one character, by design,
so there was no shortcut sitting there waiting to be called. I generated the
actual exception table directly from the Unicode Consortium's own published
CaseFolding.txt file, rather than trusting anyone's memory, mine or the
model's, of which few dozen code points actually need special handling. That
last one percent turned into a genuine, documented standard library gap
instead of a fudged number, and bindery now passes all six hundred and fifty
two.
Catching my own test lying to me
Near the very end of the weekend, continuous integration's fuzz testing found
an input that supposedly broke bindery's HTML escaping entirely: a raw HTML
tag containing a literal angle bracket sitting inside one of its own
attribute values. My own property based test flagged it immediately as a
serious injection vulnerability, exactly the kind of finding that makes your
stomach drop a little right before a submission deadline.
Except it genuinely wasn't a vulnerability at all. CommonMark explicitly
requires that raw HTML written directly by a document's own author pass
straight through completely untouched, no escaping applied whatsoever,
because that is precisely what "raw HTML support" means as a documented
Markdown feature, and it's the identical behavior every other fully
conformant Markdown renderer on earth exhibits, the same behavior already
certified by that six fifty two out of six fifty two conformance number. My
test was the thing that was wrong, not bindery's renderer. It had been
scanning the entire rendered page for anything shaped even remotely like an
HTML attribute, instead of checking only the small, specific handful of
attributes bindery itself actually constructs from untrusted user input, like
an href or a src.
I did not simply loosen the check and quietly move on with my day, because a
test you can no longer fully trust is meaningfully worse than having no test
at all sitting there giving you false confidence. I deliberately, temporarily
disabled the real escaping function on purpose, confirmed the newly narrowed
test still correctly caught a genuinely leaked angle bracket when escaping
was actually broken on purpose, and only then put the original code back and
reran absolutely everything from scratch. Trusting the fix meant proving it
both ways, not just one.
What all of this actually proves
None of the four bugs above were "the AI is simply bad at writing code." The
code compiled cleanly every single time. It passed whatever tests already
existed at that moment every single time. That is precisely what makes this
particular failure mode so genuinely dangerous: confident, fluent,
well formatted, plausible looking code that is subtly wrong in a way that
looks, on the surface, exactly like code that is subtly right. A hallucinated
constant does not announce itself. A silently changed default in a standard
library upgrade does not announce itself. A layout algorithm that quietly
assumes away cycles does not announce itself either, right up until you
actually look.
The fix was never simply "trust the AI less" as some vague, generalized
posture. It was building layers of verification that check against something
genuinely outside the code itself: an RFC's own published worked example, the
same search index built nine separate times and compared byte for byte, an
actual rendered picture opened and looked at with human eyes, a real escaping
function deliberately broken on purpose and used as a canary, official
published Unicode data pulled directly from its actual source rather than
recalled from memory by anyone, human or otherwise. Every single one of those
catches happened because I refused to accept "it compiles and the existing
tests pass" as the actual finish line, and kept pushing one layer further
each time.
Bindery ends up at six hundred and fifty two out of six hundred and fifty two
on the official CommonMark conformance suite, with a completely empty
dependency manifest, and a reproducible build proven two entirely separate
ways, the compiled binary and its actual data output both. But if I'm honest,
the number I'm most genuinely proud of out of this whole weekend isn't six
fifty two. It's four. Four real, distinct bugs, caught not because I somehow
wrote flawless code on the first attempt, but because I built a project whose
entire founding premise forced me to stop trusting anything I hadn't gone and
independently verified myself, including the AI helping me, including my own
memory, including the very tests I had written to protect myself from exactly
this.
That's the actual lesson sitting underneath a zero dependency hackathon, once
you've lived inside one for a full weekend. It was never really about the
packages. It was about learning, the hard way, four separate times in one
weekend, what it actually costs you to trust something you never bothered to
check.
If you want to see any of this for yourself rather than just take my word for
it, the full source is public at https://github.com/Yash-vs9/bindery, and
there's a five minute walkthrough of it actually running, live reload, search,
the diagrams, the PDF export, and the reproducible build check all included,
at https://youtu.be/9wFCPcWEZzM.
Top comments (0)