DEV Community

Thiago Longo Moraes
Thiago Longo Moraes

Posted on

Grading a wrong answer by how wrong it is: ICD-10 proximity scoring in Postgres

Most quiz apps have one bit of output: right or wrong. For a medical diagnosis game, one bit throws away almost everything interesting.

A physician who answers unstable angina when the case is a myocardial infarction has done something very different from one who answers bacterial meningitis. The first is a neighbour on the differential. The second is a different organ system entirely. Collapse both into "wrong" and the learner gets no signal about which mistake they made.

We build NEXO, a daily clinical reasoning game. Players answer with an ICD-10 diagnosis, and we score the answer by how far it sits from the correct code on the ICD-10 tree. This post is about how that is implemented, why it lives in the database, and a tautology that quietly killed one of the five levels for a while.

The ICD-10 code is already a tree

An ICD-10 code carries its own hierarchy in its characters:

G40.1
│││ └── subdivision
││└──── category    (G40, epilepsy)
│└───── block       (G4x, episodic and paroxysmal disorders)
└────── chapter     (G, diseases of the nervous system)
Enter fullscreen mode Exit fullscreen mode

So you do not need a graph traversal or a distance matrix to know that G40 and G43 are closer than G40 and J18. You need a prefix comparison. Walk up from the full code, one character at a time, and the first level where the two codes agree is the proximity.

That gives five outcomes:

Prefix match Feedback Meaning
full code exact correct answer
3 characters category right category, wrong subdivision
2 characters block right block of related conditions
1 character chapter right chapter, wrong disease
none distant wrong branch entirely

Why this runs on the server

The scoring is a Postgres function, not client code. That is deliberate.

The client never holds the correct answer and never decides whether a guess was right. It sends a string; it gets back a verdict. Putting the comparison in the app would mean shipping the answer to the device, which for a daily game with a global leaderboard means shipping tomorrow's answers to anyone willing to read a memory dump.

The function is SECURITY DEFINER with a pinned search_path, and it takes a row lock on the game session before it touches anything:

SELECT * INTO v_sessao
FROM sessoes_jogo
WHERE id = p_sessao_id
  AND usuario_id = v_uid
FOR UPDATE;
Enter fullscreen mode Exit fullscreen mode

The FOR UPDATE matters more than it looks. Without it, two submissions racing from a flaky connection can both read the same attempt count, both pass the six-attempt cap, and both append to the attempts array.

Normalising before comparing

ICD-10 codes reach us in several shapes. G40.1, G401, g40.1. Before any comparison, both sides are stripped to alphanumerics and upper-cased:

v_guess_norm   := upper(regexp_replace(p_tentativa, '[^A-Za-z0-9]', '', 'g'));
v_correto_norm := upper(regexp_replace(v_caso.cid10,  '[^A-Za-z0-9]', '', 'g'));
Enter fullscreen mode Exit fullscreen mode

Normalising here and not at the edge means the rule holds no matter which client is calling. It also means the prefix comparison below can be plain left() on a clean string.

The comparison

IF v_guess_norm = v_correto_norm THEN
  v_feedback := 'exato';
ELSIF left(v_guess_norm, 3) = left(v_correto_norm, 3) THEN
  v_feedback := 'categoria';
ELSIF left(v_guess_norm, 2) = left(v_correto_norm, 2) THEN
  v_feedback := 'grupo';
ELSIF left(v_guess_norm, 1) = left(v_correto_norm, 1) THEN
  v_feedback := 'capitulo';
ELSE
  v_feedback := 'distante';
END IF;
Enter fullscreen mode Exit fullscreen mode

Four comparisons, no lookup table, no ICD-10 reference data at query time. The hierarchy is in the string.

The bug: a condition that was always true

The version before this one tried to express "same numeric block" as a range check instead of a prefix check:

ELSIF left(v_guess_norm, 1) = left(v_correto_norm, 1)
  AND (left(v_guess_norm, 2) >= left(v_correto_norm, 1) || '0'
   AND left(v_guess_norm, 2) <= left(v_correto_norm, 1) || '9')
THEN v_feedback := 'grupo';
ELSIF left(v_guess_norm, 1) = left(v_correto_norm, 1)
THEN v_feedback := 'capitulo';
Enter fullscreen mode Exit fullscreen mode

Read the range condition carefully. If the first characters already match, then left(guess, 2) is that letter followed by one more character. The bounds are that same letter followed by 0 and by 9. Every digit falls inside. So for any two codes in the same chapter, the grupo branch always fired.

Which made the capitulo branch dead code. The fifth level existed in the schema, in the client, in the copy on the website, and never once reached a player.

Nothing crashed. No error was logged. Players got a slightly-too-generous verdict and there was no way to notice from the outside, because "same block" and "same chapter" both render as an amber near miss. It only surfaced when someone went looking for how often each level fires and found one of them at exactly zero.

The fix was to delete the cleverness and compare prefixes.

What I would take from this

A tautology is the quietest kind of bug. It does not throw. It does not return null. It returns a plausible answer, forever. The only signal was a distribution with a zero in it, and you only see that if you go looking.

Put the hierarchy in the identifier when you can. ICD-10, and a lot of other clinical coding, is designed so that the code is the path. That turns "how related are these two diagnoses" from a graph problem into string slicing, which is fast enough to run inside the transaction that records the attempt.

Boolean ranges deserve a truth table. Both branches of that ELSIF chain read fine in review. The failure only shows up if you write down what the range actually evaluates to when the first character matches.


NEXO is a daily clinical reasoning game for physicians and medical students. One fictional case a day, six clues in order, and an answer given as an ICD-10 code. The daily case is free.

nexo.wiki.br · App Store

Every case is fictional and written for teaching. NEXO is not medical advice and is not for use with real patients.

Top comments (0)