Five Times Pixbu's Tests Measured the Wrong Thing

enes sevinc — Tue, 22 Sep 2026 11:33:51 +0000

Originally published on HackerNoon. Two product details in that version have since gone out of date; this one is corrected and re-measured.

I'm a solo dev shipping Pixbu — a self-care app where a pixel creature grows as you look after yourself. It's live on both stores in six languages, from one Flutter codebase, with 2,699 automated tests.

This post isn't about the features. It's about a habit that saved the launch: whenever I caught myself knowing something about my own app, I measured it instead. I was wrong often enough that it stopped being embarrassing and started being a process.

The bug that taught me the pattern

Pixbu's creature evolves through six stages. Each stage should be visibly bigger than the last — that's the entire emotional payoff of the app.

I replaced the whole art set. Tests stayed green. Ship it, right?

Then I actually counted the drawn pixels in each stage:

baby 6572 → teen 5627 → adult 5840 → elder 6244 → mythic 6338

The creature shrank 14% at the first evolution — the transition users see most. The single moment the app exists to deliver was running backwards.

Why were the tests green? Because the guard read the stage sizes from a metadata file, and the metadata hadn't been re-measured from the new art yet. The test wasn't lying. It was answering a different question than the one I thought I'd asked.

That's the pattern. Not "I made a mistake." The pattern is: the check I trusted was measuring something adjacent to the thing I cared about.

Once I started looking for that shape, it was everywhere.

Four more, all real

1. I measured the wrong dimension and found nothing

Hats have to sit on the creature's head across five body shapes. They were floating. I measured the head's bounding box across every frame — and found almost no variation, ±1 px. Dead end.

The bounding box includes the ears, and the ears are the tallest thing on the creature. I was measuring ear tips and calling them skulls.

The measurement that worked was counting filled pixels per row: at ear height the count is 8; at the skull it jumps to 73. Completely unambiguous once you count the right thing. That fixed 514 frames.

A null result is not evidence of no problem. It can just as easily be evidence that your instrument points somewhere else.

2. A guard test that asserted nothing

I had a test checking that a critical function was actually called. Green for weeks. Then I mutated the source — wrapped the call in if (false) — and the test stayed green. It was checking that the call text existed in the file, not that it ran.

Now I mutate every guard I write. If a test can't fail, it isn't a test; it's a comment that costs CI time. Three of mine died in a single afternoon this way.

There's a second floor to this basement. Two of my mutations never reached the file — a line-ending mismatch meant the patch silently matched nothing, the test ran against unmodified source, and passed. A green mutation test that proved nothing about a guard that proved nothing. So now the mutation script also asserts that the mutation was applied. You have to verify your verifier's verifier, at least once.

3. "There's no way to check that" is a guess wearing a lab coat

I wrote in my own notes: "there is no API way to confirm a store update was approved." I had reasons. They were even individually true.

An hour later I found it — I'd been reading the store page's status and never its content. The live version string, the update date, and the release notes all change at approval. Three signals, sitting in the HTML the whole time.

Same class, more expensive: one network timeout made me conclude I had no network access to my git remote. I stopped pushing. Eight days and 95 commits later — including the entire source of a version already live in 177 countries — I tested it properly: 10 out of 10 connections fine. One failed reading is not a capability assessment.

Now, if I'm about to write "impossible," "unsupported," or "can't be tested," I have to attach the date, the exact command, and its output. Undated impossibility claims are treated as stale and can't block anything.

4. The dashboard was showing me fake money

Filling in a hackathon submission, I opened my revenue dashboard: $1,573 in revenue, 136 customers. Encouraging!

There was a toggle at the top of the page reading "Sandbox data." It was on. Every figure on that screen came from test purchases I had made myself, on my own phone.

I hadn't misread a number. I'd misread which world the number was from — and I'd have cheerfully quoted it in a public submission. The habit that saved me was boring: before using a figure anywhere, read what the screen says about the figure.

What this actually costs

Almost nothing, and it isn't discipline. It's three questions:

Am I measuring the thing, or something next to it? (bounding box vs skull)
Can this check fail? (mutate it and see)
Is this a measurement or a memory? (if there's no date and no command output, it's a memory)

The third one is the cheapest and catches the most. Most of my false certainties weren't wrong when written — they were right once, and then the world moved.

Where Pixbu stands

Live on the App Store and Google Play from one Flutter codebase, in English, Turkish, Spanish, Portuguese, German and French. Local-first, no account, no backend — server cost $0. No advertising SDK at all; not "ads turned off," the library simply isn't in the app. The paid tier unlocks species, scenes, widget themes and a signature animation for each creature.

There's no pet death, no decay, no punishment for missing a day, and — the one I'll defend hardest — the recovery is never the thing you pay for. Break a streak and you get a week to repair it by hugging your creature and doing one small task. That path is free, and it is the only path; there is no button that sells it to you.

The pet sleeps at night (23:00–06:00). Tap it and it mumbles "shh… I think I was dreaming about you." That line shipped the same night I wrote it. I checked it on my own phone at 00:13 — which, now that I think about it, is the same habit as everything above.

Pixbu is free on Google Play and the App Store. Site · 29-second demo.

DEV Community: enes sevinc