Asking your LLM classifier for a one-sentence reason doubles the bill

#ai #llm #api #beginners

You build a small classifier: an LLM reads each support ticket and puts it in one of five queues. While testing, you want to know why it picked a queue, so you change the prompt to return JSON with a reason field next to the category. It's helpful, nobody removes it, and it ships. From then on every ticket costs about twice as much, and the queue it lands in is exactly the same.

Why one sentence weighs as much as the whole prompt

Model APIs bill per token, a chunk of text about three-quarters of an English word. Input tokens are what you send: instructions plus the ticket. Output tokens are what the model writes back. Output is priced higher, usually five or six times the input price, because the model generates it one token at a time.

A classifier is the extreme case of that split. The input is a few hundred tokens and the useful output is one word. The category billing is a single token. A sentence explaining it is about 35. At a 6x price ratio, 36 output tokens cost as much as 216 input tokens, and my whole prompt was 219.

I wrote a realistic prompt: 119 tokens of routing rules for five categories, and an 83-token ticket about a double card charge and a late parcel. Then I measured three output styles with tiktoken, OpenAI's open-source library for counting tokens (other vendors' tokenizers count slightly differently). The format instruction adds a few input tokens; the rest is output.

PRICE_IN, PRICE_OUT = 2.00, 12.00   # GPT-5.6 list price, $ per 1M tokens
MONTH = 300_000                     # 10,000 tickets a day for 30 days

styles = {                          # (input tokens, output tokens), measured
    "label only":                 (209, 1),
    "label + one-sentence reason": (219, 36),
    "step by step, then label":   (220, 84),
}
base = None
for name, (tin, tout) in styles.items():
    cost = (tin * PRICE_IN + tout * PRICE_OUT) / 1e6
    base = base or cost
    print(f"{name:28} ${cost:.6f}/call  x{cost / base:.2f}  "
          f"output {tout * PRICE_OUT / 1e6 / cost:.0%}  ${cost * MONTH:,.0f}/month")

GPT-5.6 at its list price, the one the vendor publishes, is $2 per million input tokens and $12 per million output. The prices come from the catalog of the LLM gateway I work on, read on 7 November 2026.

Output style	Per call	vs label	Output's share	Per month
Label only	$0.00043	1.00x	3%	$129
Label + one-sentence reason	$0.00087	2.02x	50%	$261
Step by step, then label	$0.00145	3.37x	70%	$434

With a bare label, output is 3% of the bill. One sentence of reason makes it half. "Think step by step" (asking the model to write its reasoning before the answer) makes it 70%, and the router still reads one word at the end.

It's the price ratio, not the vendor

I ran the same token counts on four more models. GPT-5.6-mini has the same 6x ratio, so it shows the same 2.02x and 3.37x, just with smaller numbers: $12.90 a month becomes $26.10. Claude Haiku 5.5, Claude Sonnet 5.5 and Gemini 3.8 Flash price output at 5x input, so the reason field costs 1.86x and step by step costs 2.99x. A cheaper model shrinks the dollars, but not the multiplier. The multiplier comes from two numbers you control: how many tokens the model writes, and how many it reads.

That also tells you when this matters less. If your prompt is 5,000 tokens of context and the answer is a label, a 36-token reason adds a few percent. The shorter the input, the more an explanation dominates, and classifiers usually have short inputs.

Ask for reasons on a sample

You don't have to choose between a cheap classifier and one you can debug. Ask for the reason on a small random share of calls, log those, and read them when you want to know how the classifier thinks.

import random

LABEL = "Answer with the category name only."
REASON = 'Answer as JSON: {"category": "...", "reason": "<one sentence>"}'

def output_format(audit_share=0.02):
    """Pick the format instruction for this call: reasons only on a sample."""
    return REASON if random.random() < audit_share else LABEL

Your code then parses either a bare word or JSON, depending on which format it asked for. At a 2% sample, GPT-5.6 costs $131.64 a month instead of $129 for labels only, versus $261 for reasons on every ticket. At 5% it's $135.60. That still gives you 200 to 500 explained decisions a day, more than anyone will read. Also set max_tokens, the API's cap on how many output tokens one call may produce, to something small like 5 for label-only calls, so a model that starts chatting can't run up the bill.

What this doesn't solve

Writing reasoning first can make the answer more accurate. On hard, ambiguous tickets, step by step may pick the right queue more often, and I didn't measure accuracy here, only cost. Before you drop it, label a hundred of your own tickets by hand and compare both prompts. If step by step fixes enough mistakes, 3x may be a fair price.

Reasoning models think in hidden tokens before answering, and those are billed as output even when you ask for one word. Cutting the visible reason doesn't remove that; picking a non-reasoning model or a low reasoning setting does. The sample also only shows you how the model explains itself on 2% of tickets, not why it got a particular ticket wrong. And no gateway can make this decision for you, including the one I work on: it sees the prompt you send, not whether you needed the explanation.

The takeaway

In a classifier, the answer is one token and everything else the model writes is extra. Ask for labels by default, cap the output, and collect explanations from a sample when you actually want to read them.

DEV Community