I gave an AI tutor a map of 8,117 ways students get maths wrong

Parth Mishra — Sun, 18 Oct 2026 13:05:10 +0000

Two students both score 40% on a fractions quiz.

Asha gets them wrong because she can't find a lowest common multiple — a skill from a different topic, two steps below the question she missed. Ben gets them wrong because when he makes an equivalent fraction, he changes only the denominator.

Same score. Opposite problems. A chat tutor sees two 40%s, explains the question again, and forgets both of them by tomorrow.

That's the problem I picked for Graph Hacks (WeMakeDevs × FalkorDB). The result is Unlearn: a tutor that finds the prerequisite that actually broke, remembers it next session, and shows the database query behind every claim.

👉 Live demo: https://unlearn-ejdh.onrender.com
👉 Code: https://github.com/pm2m1/unlearn

Why a graph?

The interesting question isn't "which answer was wrong", it's "why". And "why" is a chain of connections:

wrong option → misconception → the skill it shows up in → the skill that skill depends on → the student → their class.

That's a graph. I used the Eedi Misconceptions Graph (CC BY 4.0): 8,117 real misconceptions and 2,258 maths skills, collected from real teaching. I took a fractions slice of 40 skills, added 57 hand-checked "this skill requires that skill" edges, 120 multiple-choice questions where every wrong option is tagged with the misconception it reveals, and 8 teaching strategies.

Everything lives in FalkorDB. Each class gets its own graph, copied from a template with GRAPH.COPY, so one class's memory never leaks into another's.

How it finds the root cause

A Diagnoser agent asks a few adaptive questions. A correct answer rules out that skill and everything below it; a wrong one adds evidence. Then one query walks up to four steps down the prerequisite chain and ranks the unmastered skills by weighted misconception evidence — deeper wins ties, because the deepest broken skill is the real cause.

Simplified (the real one is in api/unlearn/queries.py):

MATCH (s:Student {id:$sid})
MATCH path = (t:Construct {id:$target})-[:REQUIRES*1..4]->(p:Construct)
OPTIONAL MATCH (s)-[e:SHOWED]->(:Misconception)-[w:SEEN_IN]->(p)
WITH p, length(path) AS depth, sum(e.count * e.confidence * w.w) AS evidence
RETURN p.name, depth, evidence
ORDER BY evidence DESC, depth DESC LIMIT 1

For Asha, it lands on "lowest common multiple", two levels below the fraction question. For Ben, it lands on "equivalent fractions". The graph panel lights the path up in amber.

Memory that lives in the graph, not the prompt

Every answer, diagnosis and tutoring attempt is written back as nodes and edges. When Asha comes back, the Tutor agent opens with "Last time you…" — straight from a recall query. Agents hand work to each other through task nodes in the same graph, and the Tutor learns which teaching strategy works by updating a success rate on it.

The teacher stays in charge

The Teacher view runs graph algorithms as tools: algo.WCC groups students who share the same confident misconceptions, PageRank finds the keystone skills everything depends on, and betweenness finds misconceptions that leak across topics. The Teacher agent can only propose actions, regroup these students, assign this practice. Nothing changes until a human clicks Approve.

There's also a chat. The LLM can only call fixed graph tools (no raw Cypher), and every name it says is checked against the rows those tools returned. If it invents something, the answer is thrown away and a plain template answers instead. The same tools are exposed over MCP.

Does it actually work?

I compared four setups on 90 simulated students (3 seeds), same questions, same local model (qwen 27B via Ollama) for the baselines:

	Root cause found	Top-1 misconception	Remembered next session	Time
A. LLM, no memory	32% [22–42]	7%	0%	~15 s
B. LLM + vector RAG	37% [27–47]	7%	9%	~7.5 s
C. Unlearn (graph)	72% [63–81]	48%	57%	~50 ms

Brackets are 95% bootstrap intervals; the root-cause gap doesn't overlap.

The honest part: the students are simulated, and the simulator uses the same option-to-misconception map Unlearn uses — that favours Unlearn. So I stress-tested it: with 20% of answers replaced by random noise, Unlearn still beats every baseline's clean score. At 40% noise it stops winning everywhere. Treat these numbers as an upper bound, not a classroom result.

I also tried pasting the whole graph into the LLM's prompt (30 students): it reached 40% root cause against Unlearn's 63% on the same students but with only 30 students the intervals overlap, so I won't claim that one.

What broke

My own data. I reviewed every hand-written prerequisite edge and distractor one by one. I found 11 problems (2 bad edges, 9 misleading options), fixed them, and re-ran the whole eval.
I scored a baseline unfairly. The "graph in the prompt" baseline was only credited for answers from a short list. Fixing that raised its score. Embarrassing, but better found by me than by a judge.
FalkorDB races. Copying a class graph while another process recreated it threw "key is already set"; on the free tier, GRAPH.COPY once failed with "could not fork". Both are now retried and documented.
A UI regression made the Teacher tab blank whenever one slow query timed out, all four cards loaded together. Now each loads on its own.
The chat couldn't find "Rosa" because tools want student ids, not names. It confidently said it had no records. Fixed with a name lookup.
Free-tier reality: Gemini's free quota runs out fast, so the live site falls back to recorded and template answers.

What's next

The misconception graph is real; the students aren't. The next step is validating on real student responses, like Eedi's NeurIPS 2020 Education Challenge data, and growing beyond fractions.

Stack

FalkorDB 4.20.7 · Python/FastAPI · Cytoscape.js · fastembed · Ollama / Gemini · MCP · Render.

Built for Graph Hacks (WeMakeDevs × FalkorDB) AI helped me with Documentation, reviewed myself.

DEV Community: Parth Mishra