There's a good conversation on DEV right now about not being a proxy between people and your AI. I want to look at the other end of that problem: what happens when the AI is the product, and the person reading its answer is a 15-year-old revising for a national exam.
I'm building Ezoato, a web and mobile app that collects real Togolese exam papers (CEPD, BEPC, BAC, university exams, competitive entrance exams), works offline, has every paper checked by a human, and adds an AI tutor on top. It's a prototype. There are no users yet, so this isn't a "here are our results" post. It's the set of rules I'm designing around before the tutor talks to a single student, because they're much harder to bolt on later.
Why "mostly right" isn't good enough here
In most chat products, a hallucination costs the user a few minutes. In exam prep the cost is different:
- The student can't tell. They're asking because they don't know the answer. A confident wrong explanation looks exactly like a right one.
- Wrong method sticks. A student who learns a wrong method for a math problem will repeat it on exam day.
- Trust doesn't come back. If students, parents, or teachers catch the tutor inventing a "correction" once, the whole archive looks suspect, including the papers that were carefully verified.
So the tutor's job isn't "always answer." It's "answer only from what we can show, and say clearly when we can't."
Rule 1: The verified paper is the source, not the model
The archive is the asset. Every paper goes through human verification: the right year, the right series, the scan is legible, and the official correction is attached when one exists.
The tutor works on top of that material. It doesn't replace it:
- A question to the tutor is always tied to a specific paper and exercise (
BAC 2 · série D · année · exercice 3), not a free-floating prompt. - The context sent to the model is the text of that exercise and, when it exists, its verified correction.
- The answer shows where it came from: "Based on the verified correction for exercise 3" or "No verified correction for this exercise, this is an explanation of the method, not the official answer."
This is basic retrieval-augmented generation. The important part isn't the retrieval step. It's that the source label is part of the UI, not hidden in logs.
Rule 2: Three answer modes, chosen by the data, not by the model
Instead of letting the model decide how sure it sounds, the app decides the mode before calling it, based on what's actually available for that exercise:
| What we have | Mode | What the student sees |
|---|---|---|
| Verified official correction | Explain | Step-by-step explanation that follows the correction, with a "Verified correction" label |
| Exercise text only, no correction | Guide | Method, hints, and questions back to the student; no final answer shown as "the" answer |
| Exercise missing, illegible, or not verified yet | Refuse | "This paper isn't verified yet" plus a link to the scan; no generated content |
The mode is a plain value in code, set by the backend from the paper's verification status. The prompt changes with the mode, but the model never gets to upgrade itself from Guide to Explain.
type TutorMode = "explain" | "guide" | "refuse";
function pickMode(ex: Exercise): TutorMode {
if (ex.status !== "verified") return "refuse";
return ex.verifiedCorrection ? "explain" : "guide";
}
It's boring code on purpose. Boring code is easy to audit.
Rule 3: Check the output against the correction, not just the input
Grounding the input isn't enough. Models can still drift: a different final number, a skipped step, a unit that changes halfway through.
For exercises with a verified correction, the plan is a cheap check after generation:
- Extract the final results from the tutor's answer (numbers, chosen option, key expressions).
- Compare them with the final results in the verified correction.
- If they don't match, don't show the generated explanation. Fall back to the verified correction itself and log the mismatch for a human to review.
This won't catch every subtle error in the reasoning. It does catch the worst failure, a confident explanation that ends on the wrong answer, and it costs almost nothing compared with the generation.
Rule 4: Offline means the tutor degrades, the archive doesn't
Ezoato is offline-first because data is expensive and connections drop. The AI tutor needs a network (at least for now), so the design has to be honest about that:
- Papers and verified corrections are available offline. That's the core promise and it must never depend on the AI.
- When there's no connection, the tutor button says so plainly ("The tutor needs internet. The verified correction is below.") instead of spinning.
- Questions asked offline can be queued and answered when the phone reconnects, with the same mode rules applied at that moment, not when the question was asked.
The worst outcome would be a student who can't open the correction because the AI part is loading.
Rule 5: Humans stay in the loop where it matters
Human verification is the slow, expensive part, and it's also the whole point. The tutor should feed it, not route around it:
- Every Refuse and every output mismatch becomes a task in the review queue: "this paper needs a correction" or "this explanation drifted."
- Students can flag an explanation as confusing or wrong with one tap. A flag doesn't remove anything automatically. It sends the item to a human reviewer.
- The papers that get the most tutor questions are the first ones to get a verified correction. Usage tells reviewers where to spend their time.
What I'm deliberately not doing (yet)
- No "solve any photo" feature. It's an impressive demo, but it removes the link to a verified source, which is exactly what Rule 1 protects.
- No invented exam predictions ("this will come up this year"). There's no honest data behind that.
- No shipping the tutor before the archive is solid. If I had to cut one, I'd ship the verified offline archive without AI, not the other way around.
The takeaway
If your AI feature answers people who can't check its work, such as students, patients, or first-time users, the most important design choices happen outside the model:
- Tie every question to a verified source.
- Let your data pick the answer mode, not the model's confidence.
- Check the output against the source when you can.
- Make the non-AI path work without the AI.
- Turn every refusal into work for a human, not a dead end.
I'll write a follow-up once the prototype has real students using it, with what actually broke. If you've built something similar, especially for low-connectivity users, I'd like to hear how you handled the "no verified answer" case.
Ezoato: « Archive. Révise. Excelle. »
Top comments (1)
Grounding helps most when the tutor can show the exact source behind a claim and abstain when that source does not support the answer. I’d separate retrieval confidence from answer confidence too: a fluent explanation should never conceal that the evidence set is thin or conflicting.