TL;DR — My last post was about catching drift: the moment my config tool's idea of "valid" silently diverges from the real Pyxel framework. This post is about the other half of the same system — what happens the instant a human hits one of those checks. A safety net has two jobs: it has to catch you, and it has to talk to you when it does. Perfect code chases the first and forgets the second. I dug into ~60 years of error-message research to work out what a net that talks back should actually say, and mapped each finding onto a concrete rule I now hold my own diagnostics to.
The myth I had to give up first
For a long time I treated "write better code" as the goal, as if enough care up front would let me arrive at software that simply didn't go wrong. That belief survives right up until you maintain a tool whose correctness depends on something you don't own. My browser-based configuration tool exists to help scientists write YAML that ESA's Pyxel detector-simulation framework will accept. Pyxel is the source of truth. My copy of that truth can drift, users will make mistakes I can't anticipate, and no amount of cleverness on my side changes either fact.
So I stopped aiming for code that never fails and started building a net that assumes it will. In the drift post I wrote about the catching layers — freshness checks, bounds tests, API introspection, per-file coverage gates. But a net that catches you silently and then hands you a stack trace is only half a net. The half nobody budgets for is the part that speaks.
Here's the thing I didn't expect: the "speaking" half turns out to be the more researched of the two, and the research is unambiguous.
What the research actually says
I went looking for evidence rather than opinion, because "be nicer to your users" is easy to say and easy to skip. The literature is older and firmer than I expected.
People read error messages — and it's expensive. The comfortable myth is that developers skip the message and jump straight to the code. Barik et al. put 56 developers in front of an eye-tracker and found the opposite: people genuinely read error messages, they allocate around a quarter of their visual attention to them, and the difficulty of reading a message is comparable to the difficulty of reading source code itself. Your error text isn't a footnote to the real work — it is real work you're imposing on someone.
Better messages measurably help. When Becker and colleagues rewrote cryptic compiler errors into enhanced ones, the group receiving them made fewer errors overall, fewer errors per person, and — the metric I care about most — fewer repeated errors, the signature of someone stuck in a loop. The effect has been debated in the literature, which is itself a useful lesson: rewording alone isn't magic. What helps is a message that names the likely cause and the next action, not just prettier phrasing.
It's fundamentally an interface problem. Traver's HCI analysis reframes the compiler as an interface between machine and human, and the error message as the primary channel of that interface. The message has to bridge the gap between the system's internal state and the user's mental model of what they were trying to do. When it fails, it usually fails because it describes the machine's world ("expected identifier") instead of the user's world ("this needs a variable name").
Readability is relative. Later work found that whether a message reads as "clear" depends heavily on the reader's experience and language familiarity — experts and novices don't even agree on what counts as unreadable. There is no universally clear message, only a message clear to a specific audience. Mine is scientists, not compiler engineers, and that changes every wording decision.
And there's a 40-year-old checklist waiting for us. Nielsen's ninth usability heuristic — help users recognize, diagnose, and recover from errors — has been the same since the 1990s: plain language, precise identification of the problem, and a constructive, actionable path out, without blaming the person. His fifth heuristic pairs with it: the best error is the one you prevented from being possible at all.
Mapping the studies to rules I actually hold myself to
Findings are only worth the searches if they change what I do on Monday. Here's the translation, each rule tied to the study behind it and to what it looks like inside my tool.
Rule 1 — Treat the message as a first-class output, budgeted like code
From: Barik (reading a message costs as much as reading code).
If the message is as expensive to read as the code, it deserves as much care as the code. In practice that means the module that turns raw validator noise into human diagnostics is not a "nice to have" I bolt on at the end — it sits behind the same per-file coverage gate as my parsing and bounds logic. The safety net's voice is part of the net, so it gets tested like the rest of it.
Rule 2 — Frontload the fix, because you're spending someone's attention
From: Barik (reading is expensive) plus Nielsen's guidance to frontload the important content so users can scan.
Every message is a withdrawal from someone's attention budget. So the fix goes first, not buried under a paragraph of context. Compare:
Error: instance.detector.optics[2] failed schema validation:
data.cutoff_wavelength must be <= 15
against:
cutoff_wavelength is 18.0 but dark_current_rule07 only accepts 1.7–15.0 µm.
Lower it to within that range, or check you meant a different model.
Same underlying failure. The second one hands back most of those minutes.
Rule 3 — Enhance with cause and action, not just nicer words
From: Becker (enhanced messages reduce repeated errors) plus the debate around it (rewording alone is weak).
A repeated error means someone is stuck in a loop the message failed to break. So a good diagnostic in my tool answers three questions in order: what's wrong, why it's wrong here, and what to do next. The raw AJV output answers only the first, in the machine's dialect. My job is to add the other two — and to resist the temptation to think prettier phrasing is the whole job.
Rule 4 — Speak the user's domain, not the parser's
From: Traver (the message must bridge system state and the user's mental model).
My users think in detectors, wavelengths, and pipeline stages, not in "tokens," "instances," or "discriminated unions." So when a model is placed under the wrong pipeline stage, the message says exactly that in Pyxel's own vocabulary rather than surfacing the JSON-schema machinery underneath. The parser's world leaks out constantly; part of the tool's job is to translate it back into the world the scientist actually inhabits.
Rule 5 — Write for your reader's level, and prove it
From: the readability research (clarity is relative to experience and familiarity).
There's no such thing as an objectively clear message, so "clear to a scientist who is not a career programmer" becomes the target. Practically, that means plain sentences, a low reading grade, and jargon only where it's their jargon (cutoff wavelength, yes; discriminator, no). This is checkable — you can run message text through a readability tool the same way you'd lint code — which turns "be clearer" from a vibe into a measurement.
Rule 6 — Recognize, diagnose, recover — and prevent
From: Nielsen's heuristics 9 and 5.
This is the spine that holds the other rules together, and it maps cleanly onto the layers I already have:
- Recognize — surface the problem at the seam where it lives, visibly, so it can't be missed.
- Diagnose — say precisely what and where, in the user's terms (Rules 3–5).
- Recover — give an actionable next step, ideally with the valid range or the nearest correct option (Rule 2).
- Prevent — the best error message is the one that never has to appear. My bounds warnings and schema freshness checks are heuristic 5 in disguise: they stop whole classes of invalid config before they're ever submitted.
Rule 7 — Never blame, and never claim certainty you don't have
From: Nielsen (constructive, non-accusatory) plus my own drift work (a check that can silently discover nothing is worse than no check).
Tone is a design decision. A message that reads as "you did this wrong" produces the person who stops trusting the tool; a message that reads as "here's what happened and how to move" produces the person who fixes it and moves on. And there's a deeper honesty rule that connects straight back to the drift post: when my tool can't verify freshness because Pyxel is unreachable, it says so plainly rather than pretending everything is current. The invariant isn't "the tool is always right." It's "the tool never lies about what it knows." An error message that overstates its own certainty is just a politely-worded version of the silent check that discovers nothing.
The synthesis: a net catches and speaks
Put the two posts side by side and the shape is clear. Perfect code is a single strand — get everything right and you never fall. It's also a fantasy, because in any system whose truth lives somewhere else, you don't control whether you fall. A safety net is the honest alternative, and it's made of two things:
- The catching layer — the drift tests, the validation, the coverage gates. This decides whether you fall.
- The speaking layer — the error messages. This decides whether the fall teaches you something or just hurts.
I used to pour all my effort into the first strand and call it engineering. The research quietly makes the case that the second strand is where a huge share of your users' time, frustration, and trust actually lives — and that it's been well-understood for decades while most of us kept treating it as an afterthought.
You don't need code that never fails. You need a net that catches you, and then has the decency to tell you — in your own language, without blame, with the fix in hand — exactly what just happened.
Takeaways
- Perfect code is the wrong target; a safety net is the right one. When you don't own the source of truth, you can't prevent every fall — so design for the fall.
- The message is not a footnote. Reading it costs as much as reading code, so budget and test it like code.
- Frontload the fix. You're spending someone's attention; give the answer before the context.
- Enhance with cause and next step, not just softer wording — a repeated error is a message that failed to break the loop.
- Speak the user's domain and the user's reading level, and measure both instead of guessing.
- Recognize, diagnose, recover, prevent — Nielsen's 40-year-old checklist still maps onto every layer of a modern config tool.
- Never blame, and never claim certainty you can't back up. An over-confident message is the friendly face of a check that silently discovers nothing.
References
- Barik, T. et al. Do Developers Read Compiler Error Messages? ICSE 2017.
- Becker, B. A. An Effective Approach to Enhancing Compiler Error Messages. SIGCSE 2016; and An Exploration of the Effects of Enhanced Compiler Error Messages for Computer Programming Novices (2015).
- Becker, B. A. et al. Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research. ITiCSE-WGR 2019.
- Denny, P., Becker, B. A. et al. On Designing Programming Error Messages for Novices: Readability and its Constituent Factors. CHI 2021.
- Traver, V. J. On Compiler Error Messages: What They Say and What They Mean. Advances in Human-Computer Interaction, 2010.
- Nielsen, J. 10 Usability Heuristics for User Interface Design (Heuristics 5 and 9); Nielsen Norman Group, Error-Message Guidelines.
Top comments (0)