DEV Community

Wessam Ibrahim
Wessam Ibrahim

Posted on

Your Prompt Templates Are Tool Calls: How AskUserQuestion's 4-Option Cap Bit Me Three Times

#ai #claude #llm #agents

The same bug hit me in three separate sessions before I fixed it properly. Each time, my orchestrator reached a decision point, tried to present its menu, and burned a turn on a validation error instead of a question:

InputValidationError: {
  "code": "too_big",
  "maximum": 4,
  "path": ["questions", 0, "options"]
}
// abridged; the full payload includes the Zod message
Enter fullscreen mode Exit fullscreen mode

Claude Code's AskUserQuestion tool caps every question at 4 options. My menu had 5.

First strike: the end-of-run menu. Second strike: a blocker-recovery menu. Third strike: the same recovery menu two weeks later, after I thought I'd fixed it. That repetition is the story.

Why this one keeps coming back

A one-off validation error is not worth a blog post. What makes this one worth writing up is why it recurred: the cause wasn't a typo. It was a template.

Suhail, my Claude Code orchestrator, is a set of markdown prompt files, and its menus live in those files as literal option lists: the template says exactly what to present, and the model presents it verbatim. Five options go into one AskUserQuestion call, the schema rejects it, and the round-trip to the model is wasted. In my runs the model then retried with four options and continued, which is why, the first two times, I let the retry count as the fix.

The recovery is so cheap that the bug reads as a hiccup, not a defect.

But decision menus are the natural accumulation point of any orchestrator. Every new capability wants a slot: continue, commit, skip, retry, abort, show status. The menu only grows. A 5-option template doesn't fail once; it fails on every run that reaches it, one wasted turn each time, until you fix the template.

The failure mode worse than the error

The wasted turn is the benign version. Suhail's public changelog records the malignant one: the interactive complete-handler menu grew past the cap, and instead of erroring, the presented menu simply lost its last option. The option that got pushed out of reach was Abort.

Abort option in the interactive complete-handler made reachable. The menu previously exceeded the 4-option cap, pushing Abort out of reach; the menu is now split so Abort is always selectable.

That's from Suhail's CHANGELOG, v0.13.0. When the model squeezes an oversized menu down to fit the schema, it decides what to drop, and it dropped the escape hatch. A user staring at that menu had no way to abort the run. No error anywhere.

What the schema actually allows

Verified against the live tool schema, July 2026:

  • 2 to 4 options per question. Five or more fails with the too_big error above; a single option is also invalid. This is schema validation, not model behavior you can prompt around.
  • Up to 4 questions per call. Clustering a big menu into two questions in one call is legal, and it's how Suhail's complete-handler menu got fixed.
  • "Other" is free. Claude Code appends an "Other" free-text option to every question automatically. The schema itself says so: "There should be no 'Other' option, that will be provided automatically." A slot spent on "something else" is a slot wasted.
  • multiSelect exists. If your options aren't mutually exclusive, one multi-select question can replace several yes/no ones.

The fix

The code-level fix is trivial: trim to 4, or split into two questions. The durable fix started with noticing where the bug lived. It was in the template file, not in any tool call. So the rules I now apply to every prompt file that scripts a menu:

  1. Bake the cap into the template, next to every menu. Suhail's complete-handler now reads "AskUserQuestion clustering two questions in one call (4-option cap per question)" right where the menu is defined. The annotation travels with the menu, so the next capability I bolt on hits the cap at edit time, not at runtime.
  2. Never spend a slot on an escape hatch you get for free. "Other" is auto-provided; catch-all options come out of the template entirely.
  3. Decide the drop order in the template, not at runtime. When a menu wants a fifth option, the author cuts or splits. Left to the model, the cut can land on Abort.

And because menus accumulate, the annotation has to be on every menu. While fact-checking this post I grepped Suhail's main branch and found two more menus already at five options, plus an instruction telling the model to add an "Other" option the tool already provides. That cleanup shipped as v1.1.1 while this post was in draft. The accumulation never stops; the cap has to be part of how menus get written.

The takeaway

When a model builds tool calls from your prompt files, every canonical list in those files is a tool call waiting to be validated, and the failure is either a wasted turn or, worse, a silently trimmed menu with the important option gone. Lint your templates against the limits of the tools they feed, because the model will follow your doc verbatim, straight into the validator.

The schema will win.

The recurring menu here is from Suhail, the orchestrator I use daily against production Expo/Supabase repos. Tool limits verified against Claude Code's live AskUserQuestion schema (v2.1.216) and docs as of July 2026.

Top comments (0)