Ask the model for something your code can check

Michi Yamamoto — Wed, 16 Sep 2026 13:01:48 +0000

Every AI feature eventually has to deal with the model being wrong. Four projects with public code handled it the same way, and none of them relied on the prompt to do it. Each one changed what the model hands back, so plain code could check it before anything happened.

They're hackathon winners, but you don't need to know the events. Every detail below comes from their code.

1. Ask for coordinates, not a direction

Gilbeot is an on-device walking assistant for older adults in Korea. You photograph a confusing corner, and a small vision model (Gemma 4 E2B) tells you which way to go. A comment in the code (in Korean) notes that small vision-language models often reverse left and right. Here, a flipped word means walking the wrong way.

So the model also returns the x-coordinates of the arrow's tip and tail, and code settles the direction. Roughly:

if (tipX == null || tailX == null) return sentence; // nothing to check
if (Math.abs(tipX - tailX) < 0.05) return sentence;  // too close to call
const dir = tipX < tailX ? "left" : "right";
return replaceWord(sentence, opposite(dir), dir);    // coordinates win

This only makes the sentence agree with the coordinates. It can't prove the model found the right arrow. But "which number is smaller?" is a question code always answers correctly, and "left or right?" wasn't.

2. Let the model choose where, not what

Sentinel scans MCP servers for security issues and asks GPT to review candidate findings in structured JSON. The host throws out any review that cites lines it wasn't shown, returns finding IDs that don't match the batch, or proposes a probe plan that doesn't fit the tool's input schema.

The probe plan is the clever part. The model decides the order of four predefined probes and which input fields they target. The values are placeholder tokens from a fixed set, so the model never writes the payload. A rejected review is retried. If nothing passes, those findings stay "needs review," or the scan fails, depending on a setting.

3. Check the call, not the intent

AirBridge streams Windows audio to AirPlay speakers and includes an assistant that can call tools. A local catalog labels every tool read-only, reversible, confirmation-required, or forbidden (arbitrary_shell is on the list, forbidden). Tools missing from the catalog are refused, arguments are range-checked (volume must be 0–100), and a pending confirmation is bound to the exact tool name and arguments.

A refusal isn't an exception. It goes back to the model as the tool's result, so the assistant can explain or ask the user.

4. If the right answer is already known, don't ask

Project Rosie is a prototype that helps veterinary oncologists design personalized mRNA cancer vaccines for dogs. One commit had Gemma write the synthesis specification sent to a contract manufacturer. About five hours later, another commit replaced it with a template.

The commit message explains why. The document goes out verbatim. An invented catalog number or a drifted QC threshold is exactly what makes a formulation scientist dismiss the tool. And the only things that change between patients are case data.

Any check on that document would have compared the model's output with values the pipeline already knew. So the template just writes those values. Gemma kept the work where prose is the point: the clinical report and questions about the case.

The question to ask first

Side by side, the four projects answer one question you can ask before writing any prompt:

What will code check before this output affects anything, and what happens when the check fails?

If the answer is a comparison, an ID, a line the model was shown, or a name on a list, ask the model for exactly that. If the answer is "nothing," the model probably shouldn't produce that output.

None of these checks make the model right. They limit what a wrong answer can do.