A few months into building StareBrain — a confirm-before-execute layer for Android, show the user exactly what's about to happen, get a confirm, then execute — I thought I'd solved the hard part. Plan integrity: does execution match exactly what was confirmed, byte-for-byte, via a parameter-bound hash. Execution-time validity: are the underlying facts still true right before dispatch. Post-dispatch evidence: what can I actually prove happened once the action left my boundary.
All real problems. All still partially unsolved. But this week I found a much dumber one hiding underneath all of them: I've been testing the confirmation screen itself for the wrong thing.
The test I was running
For two weeks, my bar for a confirmation screen was legibility. Can someone read "Send text to Sarah: running late" and understand it in about two seconds. Clean copy, clear hierarchy, no jargon. I treated every screen that passed a two-second read as done.
Why that's the wrong test
The reframe came from an unrelated thread about AI course-approval gates — someone testing whether students could tell when an AI-generated lesson plan had a problem baked into it. The insight ports over cleanly: legibility only proves someone can read the sentence. It proves nothing about whether they'd notice if the sentence were describing the wrong thing.
A confirmation screen's actual job isn't to be readable. It's to be the last place a wrong plan gets caught before it becomes a real action. Those are different properties, and I'd been optimizing for the one that's easy to test.
The test that actually matters
Seed a deliberately wrong state on purpose: a calendar slot that's already filled, a contact that's been swapped, a time that's off by an hour, a message going to the wrong "Sarah." Put a real person in front of that screen and see whether they catch it before they hit confirm — not whether they can parse the sentence, but whether they'd stop it.
This is a meaningfully harder test to build than a readability check. It needs a harness that can inject a known-wrong value into an otherwise-normal-looking plan, a set of testers who don't know which run is corrupted, and an honest count of what got caught versus what got waved through.
Where this connects to the bigger unsolved piece
This actually sharpens a gap I've been circling for a while: confirming a specific action ("send this text") is a different sentence than disclosing what the app can do overall. A user might correctly read and approve a slightly-wrong action without ever getting a chance to notice it conflicts with their actual intent, because the interface never had to show them enough context to catch it. Error-detection testing is one of the few ways I know of to find out whether that gap is theoretical or actually costing people something.
Where it's still unsolved
I don't have this built yet. Every confirmation screen shipped so far has been validated against states I already knew were correct — which tells me the screen is readable, not that it's protective. I don't yet have a harness for seeding corrupted states, I don't have a test-user pool, and I don't have a baseline for what "acceptable catch rate" even means for a consumer app doing SMS and calendar actions.
Concrete next step
Before shipping another confirmation screen, build the smallest possible version of this: one action type (calendar booking), one deliberately corrupted field (wrong time), a handful of real testers who don't know it's corrupted, and an honest count of who caught it. That's the whole experiment. No new architecture needed — just a different question asked of the same screen.
If anyone's run adversarial tests like this on a critical-action UI — wrong amount, wrong recipient, wrong time — I'd genuinely like to know what your catch rate looked like, and what surprised you about what people missed.
Building StareBrain in public, one corrupted test case at a time.
Top comments (0)