This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.
The bug wasn't in my code. It was in the thing I trusted to tell me my code was fine.
I build and sell ebooks. Not the kind of software most people picture when they hear "bug" — no server, no users, no stack trace at 3am. Just a document that has to render correctly on a stranger's Kindle.
But I lost two full days to two different bugs this year, and when I finally sat down and looked at both of them side by side, they were the same bug wearing two costumes. Both times, the code was fine. Both times, the thing I trusted to check the code was quietly lying to me. And the gap between "what my tools told me" and "what was actually true" is exactly where both bugs were hiding.
If you've ever shipped something with a green test suite and watched it break in production anyway, this is your bug too. Mine just happened to live in an ebook.
Bug one: I rebuilt a document by eye, and my eye is an idiot
Here's the setup. I had a finished 18-page PDF — a product, already built, already looked good. What I did not have anymore was the code that generated it. The source was gone. Lost to a container reset. All that survived was the output.
I needed to make edits. So I did the obvious thing: I opened the PDF, looked at it, and started rebuilding the source to match what I saw.
This went badly in a way I didn't notice for hours.
I looked at the body text and figured it was about 9.6pt. It was actually 11px. I looked at the chapter titles and called them 27pt. They were 30px. I eyeballed the rail widths, the margins, the colors — all slightly off, all in ways that looked close enough that nothing screamed at me. And three diagrams I couldn't quite make out? I quietly replaced them with my own versions and told myself they were "unreconstructable."
They were not unreconstructable. That's the part that stings.
Because here's what I forgot: a rendered PDF is not a picture of the source. It contains the source. Every font size is sitting right there in the file as a Tf operator. Every color is an rg operator. Every position is a text matrix. The exact values I was squinting at and guessing — 11px, 30px, the real colors, the real diagram coordinates — were all in the file, as numbers, the entire time I was eyeballing them.
I wasn't missing information. I was ignoring it in favor of a guess.
The fix was to stop looking and start reading:
import zlib, re
d = open('lost_source.pdf', 'rb').read()
# who made this file? (tells you which engine to reproduce with)
for m in re.finditer(rb'/(Producer|Creator)\s*\(([^)]*)\)', d):
print(m.group(1).decode(), '->', m.group(2).decode())
# pull the real values out of the page streams
for s in re.findall(rb'stream\r?\n(.*?)endstream', d, re.S):
try:
c = zlib.decompress(s.strip(b'\r\n')).decode('latin-1')
except Exception:
continue
for m in re.finditer(r'/F\d+ ([\d.]+) Tf', c): # font sizes
print('font size:', m.group(1))
for m in re.finditer(r'([\d.]+) ([\d.]+) ([\d.]+) rg', c): # colors
print('color:', m.groups())
The moment I ran that, the guessing stopped. The fonts were exact because I was no longer estimating them — I was copying them out of the file. The rebuild became a transcription instead of an impression. And the "unreconstructable" diagrams turned out to be perfectly reconstructable the second I read their actual coordinates instead of squinting at their shapes.
Lesson, stated plainly: if you're reconstructing something from its output, the output usually still carries the real values. Measure them. Do not trust your eye when the file will just tell you the number.
Bug two: every test passed, and the thing was still broken
Different day, different project — an EPUB this time. There's a small design element between sections of the book: a kicker line of text, and a thin decorative rule centered under it. On my machine it looked perfect. In my test renders it looked perfect. On an actual Kindle, the rule jammed itself against the left margin like it was scared of the middle of the page.
So I fixed it. Then I fixed it again. Then again. Five times.
- Fix 1 & 2: I used flexbox to center it, plus a tall min-height so it'd sit in the middle. The Kindle ignored both. Collapsed straight to the top.
-
Fix 3: Classic
margin: 0 autoon the element. On older Kindle software,margin: autois silently treated asmargin: 0. It didn't center. It did nothing. - Fix 4: I made the rule a 1px-tall element with a background color. The Kindle didn't paint the background. The rule didn't shift — it vanished.
-
Fix 5, the one that worked: three em-dash characters —
———— in a paragraph withtext-align: center.
That last one is almost embarrassingly simple, and why it worked is the entire point of this post. It worked because it's the exact same mechanism the book's chapter titles already used, on the same device, successfully, the whole time. I didn't need to invent a centering technique. I needed to reuse the one that was already proven inside this exact file, on this exact reader.
But here's the real bug, the one underneath all five failed fixes:
Every single test render passed. My local renderer was more capable than a Kindle. It honored flexbox. It honored margin: auto. It painted hairline backgrounds. So every broken fix got a green light from my tools and a red light from the actual device. I wasn't debugging the bug. I was debugging against a renderer that didn't share the Kindle's limitations — which meant my tests were confidently telling me "fixed" four times in a row while real readers would've seen it broken.
I only made progress when I stopped trusting the lenient renderer and forced my test to reproduce the actual failure — I disabled margin: auto in my check on purpose, to make my tool as dumb as the target device. The instant my test could fail the way the Kindle failed, it caught the bug immediately.
Lesson, stated plainly: a passing test is only worth as much as the conditions it runs under. If your test environment is more forgiving than reality, a green check isn't proof — it's a nicer-looking version of the bug.
The two costumes, one bug
Line them up:
| What I trusted | What was actually true | Where the truth was hiding | |
|---|---|---|---|
| Rebuild by eye | My visual estimate of the PDF | Exact font/color/position values | Inside the PDF's own operators |
| The centering rule | A passing test render | The Kindle's real behavior | In the actual failure condition |
Same shape both times. I trusted a convenient signal — my eye, my green test — instead of measuring the real thing. And both bugs lived precisely in the gap between the two.
This is not an ebook lesson. It's the thing under most bugs that survive more than one fix:
- The log that looks clean because it isn't logging the thing that's actually failing.
- The test that passes because it mocks away the exact condition that breaks in prod.
- The metric that's green because it's measuring something adjacent to what you care about.
- The "it works on my machine" that is true, and useless, because your machine is more forgiving than the one that matters.
When a bug survives your fix, the first thing to distrust isn't your code. It's the signal that's telling you your code is fine. Go make your check as harsh as reality — measure the real value, reproduce the real failure — and the bug usually stops hiding.
What I'm proud of
Not the fixes. The fixes were small — a regex and three em-dashes. What I'm proud of is that I stopped treating each bug as a code problem and started treating it as a trust problem, and that reframe turned two separate two-day slogs into one lesson I now reach for automatically. The next time something survives a fix, I don't touch the code first. I go interrogate whatever told me the code was fine.
That's the resilience win. Not harder code — harder checks.
Check out my website: www.tagzauthor.com
Support TagzAuthor: ko-fi.com/tagzauthor
My author page: Amazon Bookstore
Top comments (0)