Grading a wrong answer by how wrong it is: ICD-10 proximity scoring in Postgres

Thiago Longo Moraes — Mon, 21 Sep 2026 22:01:43 +0000

Most quiz apps have one bit of output: right or wrong. For a medical diagnosis game, one bit throws away almost everything interesting.

A physician who answers unstable angina when the case is a myocardial infarction has done something very different from one who answers bacterial meningitis. The first is a neighbour on the differential. The second is a different organ system entirely. Collapse both into "wrong" and the learner gets no signal about which mistake they made.

We build NEXO, a daily clinical reasoning game. Players answer with an ICD-10 diagnosis, and we score the answer by how far it sits from the correct code on the ICD-10 tree. This post is about how that is implemented, why it lives in the database, and a tautology that quietly killed one of the five levels for a while.

The ICD-10 code is already a tree

An ICD-10 code carries its own hierarchy in its characters:

G40.1
│││ └── subdivision
││└──── category    (G40, epilepsy)
│└───── block       (G4x, episodic and paroxysmal disorders)
└────── chapter     (G, diseases of the nervous system)

So you do not need a graph traversal or a distance matrix to know that G40 and G43 are closer than G40 and J18. You need a prefix comparison. Walk up from the full code, one character at a time, and the first level where the two codes agree is the proximity.

That gives five outcomes:

Prefix match	Feedback	Meaning
full code	exact	correct answer
3 characters	category	right category, wrong subdivision
2 characters	block	right block of related conditions
1 character	chapter	right chapter, wrong disease
none	distant	wrong branch entirely

Why this runs on the server

The scoring is a Postgres function, not client code. That is deliberate.

The client never holds the correct answer and never decides whether a guess was right. It sends a string; it gets back a verdict. Putting the comparison in the app would mean shipping the answer to the device, which for a daily game with a global leaderboard means shipping tomorrow's answers to anyone willing to read a memory dump.

The function is SECURITY DEFINER with a pinned search_path, and it takes a row lock on the game session before it touches anything:

SELECT * INTO v_sessao
FROM sessoes_jogo
WHERE id = p_sessao_id
  AND usuario_id = v_uid
FOR UPDATE;

The FOR UPDATE matters more than it looks. Without it, two submissions racing from a flaky connection can both read the same attempt count, both pass the six-attempt cap, and both append to the attempts array.

Normalising before comparing

ICD-10 codes reach us in several shapes. G40.1, G401, g40.1. Before any comparison, both sides are stripped to alphanumerics and upper-cased:

v_guess_norm   := upper(regexp_replace(p_tentativa, '[^A-Za-z0-9]', '', 'g'));
v_correto_norm := upper(regexp_replace(v_caso.cid10,  '[^A-Za-z0-9]', '', 'g'));

Normalising here and not at the edge means the rule holds no matter which client is calling. It also means the prefix comparison below can be plain left() on a clean string.

The comparison

IF v_guess_norm = v_correto_norm THEN
  v_feedback := 'exato';
ELSIF left(v_guess_norm, 3) = left(v_correto_norm, 3) THEN
  v_feedback := 'categoria';
ELSIF left(v_guess_norm, 2) = left(v_correto_norm, 2) THEN
  v_feedback := 'grupo';
ELSIF left(v_guess_norm, 1) = left(v_correto_norm, 1) THEN
  v_feedback := 'capitulo';
ELSE
  v_feedback := 'distante';
END IF;

Four comparisons, no lookup table, no ICD-10 reference data at query time. The hierarchy is in the string.

The bug: a condition that was always true

The version before this one tried to express "same numeric block" as a range check instead of a prefix check:

ELSIF left(v_guess_norm, 1) = left(v_correto_norm, 1)
  AND (left(v_guess_norm, 2) >= left(v_correto_norm, 1) || '0'
   AND left(v_guess_norm, 2) <= left(v_correto_norm, 1) || '9')
THEN v_feedback := 'grupo';
ELSIF left(v_guess_norm, 1) = left(v_correto_norm, 1)
THEN v_feedback := 'capitulo';

Read the range condition carefully. If the first characters already match, then left(guess, 2) is that letter followed by one more character. The bounds are that same letter followed by 0 and by 9. Every digit falls inside. So for any two codes in the same chapter, the grupo branch always fired.

Which made the capitulo branch dead code. The fifth level existed in the schema, in the client, in the copy on the website, and never once reached a player.

Nothing crashed. No error was logged. Players got a slightly-too-generous verdict and there was no way to notice from the outside, because "same block" and "same chapter" both render as an amber near miss. It only surfaced when someone went looking for how often each level fires and found one of them at exactly zero.

The fix was to delete the cleverness and compare prefixes.

What I would take from this

A tautology is the quietest kind of bug. It does not throw. It does not return null. It returns a plausible answer, forever. The only signal was a distribution with a zero in it, and you only see that if you go looking.

Put the hierarchy in the identifier when you can. ICD-10, and a lot of other clinical coding, is designed so that the code is the path. That turns "how related are these two diagnoses" from a graph problem into string slicing, which is fast enough to run inside the transaction that records the attempt.

Boolean ranges deserve a truth table. Both branches of that ELSIF chain read fine in review. The failure only shows up if you write down what the range actually evaluates to when the first character matches.

NEXO is a daily clinical reasoning game for physicians and medical students. One fictional case a day, six clues in order, and an answer given as an ICD-10 code. The daily case is free.

nexo.wiki.br · App Store

Every case is fictional and written for teaching. NEXO is not medical advice and is not for use with real patients.

I used the ICD-10 tree as a scoring function

Thiago Longo Moraes — Mon, 21 Sep 2026 18:41:33 +0000

Most medical quiz apps score you as right or wrong. That throws away the part that matters. In clinical reasoning, mistaking cholecystitis for appendicitis and mistaking it for a myocardial infarction are not the same mistake. One is a near miss inside the acute abdomen. The other is a different organ system.

So when we built NEXO, a daily diagnosis game for physicians and medical students, we made the scoring reflect that.

The idea

ICD-10 is a tree. A code like I21.0 sits inside category I21, inside group I20-I25, inside the chapter for diseases of the circulatory system. That hierarchy is already a distance metric. We just had to use it.

When a player submits a guess, we compare the guessed code against the answer and return the deepest node they share:

exact code         I21.0  vs  I21.0
same category      I21.9  vs  I21.0
same group         I20    vs  I21.0
same chapter       I50    vs  I21.0
different chapter  J18    vs  I21.0

Five levels instead of two. A player who lands on "same chapter" learns they were reasoning in the right system and missed the specific entity. A player who lands on "different chapter" learns something else entirely. Both are wrong answers, and they are not worth the same.

Keeping the answer secret

The daily case is the same for every player worldwide, and the answer has to stay hidden until the day rolls over. The comparison runs server side against the stored code. The client never receives the answer, only the level. That kept the API small: send a code, receive an enum.

The harder problem: cases that hold up

Generating a plausible clinical case with an LLM takes seconds. Generating one a physician will not tear apart is a different problem. A clue that sounds right but does not belong to the diagnosis is worse than no clue at all.

We built NEXO Core for that. Every disease, syndrome, exam, clinical finding and ICD-10 code is a node. Edges connect what clinical practice treats as neighbors: pneumonia connects to its etiological agent, its radiological pattern, its differentials. The graph currently holds around 11,500 concepts and 26,000 edges, drawn from 27 reference textbooks.

Every generated case crosses that graph before approval. Each clue has to appear among the neighboring concepts of the proposed diagnosis. A clue that does not, fails. It is a structural check, not a human reading it over and nodding.

Stack

React Native with Expo for the app
Supabase for Postgres, auth, storage and edge functions
Next.js on Vercel for the marketing site and the case management backoffice
RevenueCat and StoreKit 2 for subscriptions
Sentry for crash reporting

649 cases across 25 specialties, in six languages.

What I would do differently

Start with the graph, not the cases. We wrote a few hundred cases before the graph existed and had to re-audit all of them against it afterwards. That audit found problems a read-through had missed.

NEXO is live on iOS and the daily case is free forever, with no ads. The site is at nexo.wiki.br and the app is on the App Store.

Happy to answer questions about the graph or the scoring.

DEV Community: Thiago Longo Moraes