A symptom checker that only reads one message and immediately names a diagnosis is not doing anything a doctor would recognize as diagnosis. A real intake asks a follow-up question, one that actually narrows things down, not a random one. Building the Neo4j knowledge graph project, that follow-up question turned out to be the single most interesting piece of logic in the whole app, more interesting than the graph schema itself, because it is a small, greedy algorithm making a real decision under uncertainty, three times in a row, before giving up and committing to an answer.
Why the naive versions of this don't work
The obvious bad version is a fixed checklist, ask about fever, then cough, then fatigue, in that order, every time, regardless of what the patient already said. That wastes questions on symptoms that do not actually distinguish between the diseases still under consideration. A slightly less bad version asks about a random unconfirmed symptom, which at least avoids a fixed order but still has no reason to expect the answer will be useful.
What actually needs to happen is closer to how a doctor thinks: given everything the patient has said so far, what one additional piece of information would eliminate the most remaining possibilities. That is a real algorithmic question, and answering it requires two things: a ranked list of current candidate diseases, and a way to pick the most informative next question from that ranked list.
The actual two-step algorithm
Step one, rank candidate diseases by how well they match confirmed symptoms so far.
MATCH (s:Symptom)-[:INDICATES]->(d:Disease)
WHERE s.name IN $confirmed
WITH d, count(s) as match_score, collect(s.name) as matched
ORDER BY match_score DESC
RETURN d.name as disease, match_score, matched
LIMIT 5
This finds every disease connected to at least one confirmed symptom, counts how many confirmed symptoms each one matches, and returns the top five, ranked by match count. Early in a conversation, with only one or two symptoms confirmed, this list is going to be broad, plenty of diseases share a single common symptom like fatigue or headache. That breadth is expected and fine, it is exactly what the next step is meant to narrow down.
Step two, and this is the actual clever part, find the symptom that appears most often across the current candidates, excluding anything already asked about.
MATCH (s:Symptom)-[:INDICATES]->(d:Disease)
WHERE d.name IN $diseases AND NOT s.name IN $exclude
RETURN s.name as symptom, count(d) as freq
ORDER BY freq DESC, symptom ASC
LIMIT 3
$diseases is the current candidate list from step one. $exclude is every symptom already confirmed or already ruled out, so the same question never gets asked twice. The query counts, across only the current candidates, how many of them share each remaining symptom, and returns the most common one first. This is a real, if simple, information-theoretic idea: the symptom shared by the most current candidates is the one whose answer, yes or no, is most likely to split the candidate list meaningfully, confirming it strengthens several candidates at once, ruling it out eliminates several at once. A symptom relevant to only one candidate barely moves the needle either way.
The loop asks, evaluates, and re-ranks, up to three times, and this is enforced explicitly, not left implicit. The actual stopping condition in the application code is direct:
if st.session_state.question_count >= 3 or not next_symptom or not candidates:
st.session_state.stage = "diagnosed"
Three clarifying questions, or an earlier exit if the algorithm runs out of useful questions to ask, whichever comes first. This cap is a genuine product decision, not a technical limitation, more questions might narrow the candidate list further, but real users lose patience with an interrogation, and three rounds was the balance chosen between accuracy and not being annoying.
Answers are not restricted to a fixed yes or no. A patient can respond with plain free text instead of tapping a button, and that response gets analyzed separately, first checked for whether it confirms or denies the specific symptom asked about, and second, scanned for any other symptom the patient volunteered that was not explicitly asked for:
prompt = f"""
The doctor asked the patient if they have the symptom: "{current_symptom}".
The patient responded: "{user_text}".
Analyze the patient's response and extract:
1. Did they confirm having "{current_symptom}"? (true, false, or null if unclear).
2. Did they mention any other symptoms they have, from this allowed list:
{json.dumps(all_symptoms)}
"""
That second part matters more than it might look like. A patient answering "no, but I do have chest pain" should not have that volunteered detail thrown away just because it was not the specific symptom asked about, so any additional confirmed symptoms found in the response get added straight into the confirmed set, feeding directly back into the next round's candidate ranking. There is also a plain keyword-matching fallback for when no LLM API key is configured, checking for words like "yes" or "no" directly, which keeps the app functional in a degraded but still usable way without an API dependency.
Where this algorithm is honestly limited
This is a greedy heuristic, not a real diagnostic reasoning system, and it is worth being direct about the gap. It optimizes one step at a time, picking whichever single question looks most useful right now, with no lookahead into how the answer might interact with a question two steps later. It also treats every disease-symptom connection as equally weighted, a genuinely rare, highly specific symptom and a common, low-specificity one count identically in the match score, when a real diagnostic process would weight a rare, specific symptom far more heavily, since it carries much more information. None of this makes the algorithm wrong for what it is being asked to do, narrow a candidate list reasonably well in three questions or fewer, it just means calling it "diagnostic reasoning" would be overselling a fairly simple, honest heuristic.
What this means practically
A hard question limit is a legitimate product decision, not a compromise to apologize for. Optimizing for "most useful next question" only matters if the user actually sticks around to answer it.
Ranking a next question by how many current candidates share it is a simple, effective approximation of information gain, without needing the full statistical machinery of a real Bayesian update.
Free-text answers should be scanned for more than the literal yes or no you asked for. Discarding volunteered information because it did not fit the expected answer shape wastes a genuinely useful signal the user already gave you.
Be honest in the actual product about what kind of reasoning this is. A greedy, unweighted heuristic can produce a genuinely useful conversation without needing to be dressed up as more sophisticated than it actually is.
Conclusion
"Are you also experiencing X" is not a random or scripted question in this project, it is the output of a two-step query: rank the current candidate diseases by symptom overlap, then find the single remaining symptom shared by the most of them, repeated up to three times, with every answer, confirmed, denied, or volunteered, feeding back into the next round. It is a small algorithm, honestly closer to a smart checklist than real medical reasoning, but it is the specific piece of this project that actually behaves like it is thinking, rather than just retrieving.
Top comments (0)