DEV Community

Cover image for Telling an AI to "be concise" can make it generate MORE tokens. I benchmarked it across 5 models.
Pedro Santos
Pedro Santos

Posted on

Telling an AI to "be concise" can make it generate MORE tokens. I benchmarked it across 5 models.

For a while I'd been sticking "be concise" on prompts and assuming it saved me money. Then I actually measured the output instead of eyeballing the answer.

On two of the five models I tested, "be concise" made the model generate more tokens than saying nothing at all.

The experiment

Two versions of the same five tasks:

  • One with no system prompt.
  • One with a system prompt that literally said Be concise. Keep your answer short.

I counted provider output tokens (what the model actually generates, and the part you pay 5x for), not how short the answer looked. Five current Claude models, run through the CLI.

Model No prompt "Be concise." Change
Haiku 4.5 3288 2666 −19% (fewer)
Sonnet 5 353 453 +28% (MORE)
Opus 4.8 278 394 +42% (MORE)
Fable 5 1340 1019 −24% (fewer)
Opus 5 591 555 −6% (fewer)

So it's not that "be concise" always backfires. It's that it's unreliable. Sometimes it trims, sometimes it pads, and which one you get depends on the model.

My guess at why: on some models a "keep it short" instruction triggers a little meta-preamble ("Here's a concise summary:") or a restatement of the question, and on tasks that were already short, that overhead costs more than the instruction saves.

Why it matters: output tokens are the expensive slice. On every current Claude model, output costs 5x the input token. If your "make it cheaper" prompt is quietly adding output on the exact model you use, you're paying more to feel efficient.

What actually reduced the output

If you want fewer output tokens reliably, restyling the prose isn't enough. You have to change what goes into the answer at all.

That's the idea behind a small skill I ended up building, called NoYap. Instead of "say it shorter", it's a set of rules for what belongs in a response:

  • Answer first. Result on line one.
  • Delete whole sections that don't earn their place (recaps, narration, generic "there may be edge cases" caveats) before tightening wording.
  • Only claim something was verified if it actually ran.
  • Only state a risk if it's specific and real.
  • Ask one question if blocked, then stop.

Same five tasks, same measurement. I ran it against a plain baseline plus two existing terseness tools as reference points: Caveman (compresses the prose into a terse persona) and Ponytail (makes the model write less code).

Skill Haiku 4.5 Sonnet 5 Opus 4.8 Fable 5 Opus 5
Baseline (no prompt) 4732 1134 1334 2933 1078
Caveman 5219 951 1879 2622 951
Ponytail 5554 886 978 2277 992
NoYap 4152 592 823 1229 745
NoYap vs baseline −12% −48% −38% −58% −31%

Benchmark chart Caveman vs Ponytail vs NoYap vs Baseline

NoYap was below baseline on all five models. (Note: these baselines come from a separate, longer run than the "be concise" table above, so don't line the two tables up column by column. They're different experiments.)

What it looks like

A migration task: the migration drops a legacy column, the code was changed, tests were not run, and old v1 API clients still read that column.

Plain agent, 158 tokens:

The migration is incomplete and will break v1 API clients.

Blockers:

  1. Tests haven't run — we haven't verified the code changes work
  2. v1 API compatibility — clients still read the status column you removed; they'll get null/error responses

Next steps:

  • If this is a controlled deprecation: add a deprecation window, keep the column populated for one release, communicate to clients
  • If v1 must stay compatible: restore the column or add a compatibility shim
  • Either way: run tests first

Which approach fits your release strategy?

NoYap, 35 tokens:

Migration drops the legacy status column. Code updated, not run.

Risk: v1 API clients still read status, they break until upgraded.

Same facts. It states what happened, flags the one real risk with the mechanism (who breaks and why), and admits nothing was verified. It doesn't invent two remediation plans I didn't ask for.

Try it

The runner, the raw results, and the skill are here: https://github.com/pedrosantospt/noyap

In Claude Code it's two lines:

/plugin marketplace add pedrosantospt/noyap
/plugin install noyap@noyap
Enter fullscreen mode Exit fullscreen mode

NoYap Logo

Top comments (1)

Collapse
 
alexshev profile image
Alex Shev

This is a great reminder that instruction wording is part of the system, not decoration. "Be concise" can change planning behavior, not just output length. I would love to see this paired with task type, because summarization, coding, and reasoning probably react very differently.