Most tools that analyse text for reliability show you a score. A number, a colour, sometimes a badge. The interface answers the question "should I trust this?" in the most compressed form available.
I want to describe what happens to an interface when you decide not to answer that question, and why several of the resulting design problems turn out to be harder than the analysis itself.
The problem with the number
A score is an excellent piece of interface design in every respect except one: it is the wrong output.
It scans instantly. It sorts. It fits in a badge. Users understand it without instruction. Everything about it is good product design, which is why nearly every tool in this space converges on it.
The problem is that it collapses a document into a scalar, and the collapse discards exactly the information the reader needs to disagree with you. A 3.5 tells you the verdict. It does not tell you which sentences produced it, which means you cannot check it, which means accepting it is a matter of trust.
And as soon as it is a matter of trust, you are back to the position where the tool only works on people who already agreed.
What the alternative has to display
If the output is structure rather than a verdict, the interface has to show, for a document of arbitrary length:
- which sentences are verifiable factual claims- which are opinions phrased in factual form- which claims carry attribution and which do not- which comparisons lack a baseline- what a complete treatment would include that this one omits
That is not a badge. That is an annotated document, and annotated documents have well-known usability problems: they are slow to read, visually noisy, and they compete with the text they annotate.
Three design problems and what I have concluded about them
Density. Annotate every sentence and the page becomes unreadable. Annotate selectively and you have made an editorial judgement about what matters — which is a verdict wearing a different hat.
The resolution I have settled on is layering: the default view marks only categories, and detail appears on demand per sentence. The reader controls the density. This is slower for someone who wants a summary and better for someone who wants to check, which is the correct trade for this tool and the wrong one for most tools.
Colour. The obvious encoding is traffic lights: green for verified, red for unsupported. It is also the encoding that reintroduces the verdict, because red reads as "bad" regardless of what the legend says. An unattributed claim is not false; it is unattributed. The colour says otherwise.
I have not solved this cleanly. Using non-evaluative hues helps — categories that do not map onto good and bad — but readers arrive with the traffic-light convention already installed, and the interface cannot fully override it.
The absence problem. The most important output is often what is missing: the source that was not cited, the counterexample that was not mentioned, the baseline that was not given. Absence has no position on the page. You cannot highlight a sentence that does not exist.
Everything I have tried here is unsatisfying. Summary panels listing omissions read as a separate document. Inline markers at the point where something should have been are guesses about where that is. This remains open.
What the interface gains
Three things, which I think justify the costs.
Errors become local. When a classification is wrong, it is wrong next to the sentence it describes. The reader sees the sentence and the label together and can judge the mismatch immediately. Compare this to a wrong score, which is wrong globally and invisibly.
Disagreement stops being binary. A reader who rejects one classification out of forty has rejected one classification. In a score interface, the same disagreement means rejecting the score, and the score is the whole product.
The reader keeps the judgement. This is the design goal that everything else serves. The output is input to a decision the reader makes, not a decision presented for acceptance.
The honest cost
Users ask for the number. Repeatedly, and reasonably — they have a question and they want it answered.
Declining to answer is a real product cost, and dressing it up as principle does not make it smaller. Some proportion of people will use a tool that gives them a verdict instead, and that proportion is not small.
There is also a sensitivity that has to be surfaced in the interface itself: analysis depends on how much text is submitted. Three paragraphs produce an assessment of three paragraphs. The same article in full can read quite differently, because structure is a property of the whole and a fragment is not the whole. An interface that hides this produces confident readings of things that were not there.
We show that limitation in the product rather than in a footnote, and publish the analyses — including unflattering ones — at faktoskop.pl. Stating failure modes is not a disclaimer; for this category of tool it is part of the output.
Question for the community
For those who have designed interfaces that deliberately withhold a summary the user is asking for: how did you handle the pressure to add it? Every user research session I have run includes at least one person asking for the number, and I have never found a framing that makes the absence feel like a feature rather than a gap.
Top comments (0)