DEV Community

Cover image for Gemini Accepted "thinkingBudget: 0". So Why Was It Still Thinking?
Prikshit
Prikshit

Posted on Edited on Originally published at Medium

Gemini Accepted "thinkingBudget: 0". So Why Was It Still Thinking?

An unexpected Gemini API result, a small experiment with reasoning tokens, and a lesson about verifying LLM behavior before production.

1. The Unexpected Result

Imagine configuring an AI model to minimize its reasoning effort. You set:

{
  "thinkingConfig": { "thinkingBudget": 0 }
}
Enter fullscreen mode Exit fullscreen mode

Your expectation is simple: either the model accepts the setting and behaves accordingly, or the API rejects it.

But what happens when the request succeeds and the model still spends more than a thousand thinking tokens?

That is what a small Gemini API experiment showed.

The Firebase AI Logic thinking docs say that Gemini 4.x models do not support thinking budget and "will return a 400 error if it's set". The Firebase release note for gemini-4-hydrogen says the same: "thinkingBudget is not supported (there's no backwards-compatibility; the request will fail with a 400 error)."

Yet a REST test against the Gemini Developer API, using gemini-4-hydrogen, returned HTTP 200 for budgets of 0, 1024 and 8192.

Even more interesting: thinkingBudget: 0 still produced 1,290 to 1,466 thinking tokens.

Before calling this a bug, one distinction matters. Firebase AI Logic and the raw Developer API REST interface are different integration paths, and a statement written for one SDK does not automatically describe every endpoint.

Still, the result raises a useful engineering question:

Does a successful API request mean our model configuration is actually doing what we intended?

Not necessarily.

2. Understanding Thinking Tokens

Diagram: prompt, hidden thinking tokens, answer; thoughtsTokenCount plus candidatesTokenCount drive cost and latency

Reasoning-capable language models can spend tokens on internal computation before producing the visible answer. It helps to separate three categories:

  • Input tokens: the instructions, conversation and context sent to the model.
  • Thinking tokens: tokens counted for the model's internal reasoning.
  • Output tokens: the response the user sees.

All three can affect usage, latency and billing, depending on the provider and model.

A setting like thinkingBudget tries to influence how much reasoning the model does. But its exact meaning is model specific. Some models allow thinking to be switched off. Others require a minimum effort, treat a budget as a target rather than a hard limit, or use a different control entirely, such as thinkingLevel.

So developers should not assume that thinkingBudget: 0 means zero thinking tokens, or that any budget is a guaranteed ceiling unless the model's docs explicitly promise it.

3. The Experiment

  • Model: gemini-4-hydrogen
  • Interface: Gemini Developer API, REST (v1beta, generateContent)
  • Environment: a fresh free tier project, November 17, 2026
  • Conditions: no thinking config, and thinkingBudget of 0, 1024 and 8192
  • Runs: planned 3 per condition. The free tier's limit of 20 requests per day cut some conditions short, so they have 1 or 2 runs.

Two questions:

  1. Would the API accept requests containing thinkingBudget?
  2. How many thinking tokens would each response report?

The key measurement was the usage metadata field:

{
  "usageMetadata": {
    "thoughtsTokenCount": 1466
  }
}
Enter fullscreen mode Exit fullscreen mode

(This shows the field to inspect, not a complete response.) Every run also recorded the HTTP status, finish reason, prompt, output and total token counts, and latency.

4. What the Numbers Showed

Infographic titled Gemini thinkingBudget: Docs vs. Real Behaviour. Panels show the documentation gap, the test setup, a bar chart of thinking tokens by configuration (about 2,300 with no config, 1,320 at budget 0, 1,860 at 1024 and 2,200 at 8192), and three validation layers: request, behavior and operational.

Configuration Successful runs Thinking tokens Median latency
No thinking config 3 2,253 to 3,139 13.2 s
thinkingBudget: 0 3 1,290 to 1,466 9.9 s
thinkingBudget: 1024 2 1,792 and 1,924 11.0 s
thinkingBudget: 8192 1 2,204 13.7 s

Every request that got past the free tier quota returned HTTP 200. None returned a 400.

Three things stand out.

First, budget zero did not switch thinking off. Every run with thinkingBudget: 0 reported more than 1,200 thinking tokens.

Second, 1024 was not a hard ceiling. Both successful runs went well past 1,024.

Third, the parameter clearly did something. Budget 0 used roughly half the thinking tokens of no config and had the lowest median latency. So the field was not simply ignored. It was interpreted as a nudge, not a literal limit.

The sample is far too small for statistical claims, and token use depends on the prompt and run conditions. This is an interesting observation, not a universal statement about how Gemini works inside.

5. Why HTTP 200 Isn't Enough

An HTTP 200 tells us the request was processed at the API level. It does not prove that every optional parameter produced the behavior the developer expected.

Picture a team that changes a model setting to cut latency. The deploy succeeds, the API returns 200s, functional tests pass. But nobody measures reasoning tokens or response time. The team believes it shipped an optimization without knowing whether it did.

A stronger approach checks three layers:

Infographic: Docs say 400. REST says 200. Gemini thinkingBudget key takeaway

  • Request validation: did the API accept the configuration?
  • Behavior validation: did the model behave the way the configuration is documented to behave?
  • Operational validation: did the metrics that matter actually improve, such as latency, tokens, quality and cost?

All three matter.

6. Why Model Migration Makes This Urgent

Timeline: gemini-2.5 models migrating to 3.x around December 1; thinkingBudget discouraged in favour of thinkingLevel

Model migrations add another layer. A setting supported by one model may be deprecated, ignored, interpreted differently or rejected by its replacement.

This is not hypothetical right now. Firebase's release notes say that for the Gemini Developer API, starting December 1, 2026, requests to gemini-2.5-pro and gemini-2.5-flash will automatically route to gemini-3.8-flash and gemini-3.5-flash-lite, "which do not support disabling thinking via thinkingBudget: 0". The same notes say all general-use Gemini 3.x models ignore temperature, topK and topP, and that thinking budgets are "strongly discouraged" in favor of thinkingLevel.

So an app that sends thinkingBudget: 0 today to save latency or cost may quietly behave differently after December 1, with no error to warn anyone.

Treat a model migration as a behavioral compatibility change, not an endpoint swap. Before switching, record a baseline for:

  • Thinking and output token usage
  • End-to-end latency
  • Answer quality and task completion
  • API errors and unsupported parameters
  • Cost per successful request

Then run the same evaluation against the replacement model. A migration succeeds only when both functionality and operational behavior still meet your requirements.

7. How I Would Make This Reproducible

Checklist: fixed prompt, several runs per setting, log thoughtsTokenCount, compare against a no-config baseline

The natural next step is a small open-source test utility. Given a model, a prompt set and a list of thinking configurations, it would:

  1. Send repeated requests through the chosen API.
  2. Record the HTTP status and response metadata.
  3. Extract thinking token counts.
  4. Measure end-to-end latency.
  5. Save results as CSV or JSON.
  6. Generate comparison charts.

A simple result schema:

model, api_version, configuration, run_number, http_status,
thought_tokens, output_tokens, latency_ms, timestamp
Enter fullscreen mode Exit fullscreen mode

For a reliable comparison I would run 20 to 30 requests per condition on a paid tier, use the same prompt set, and report distributions rather than single numbers. That would answer what this small test cannot:

  • Is the behavior consistent across runs?
  • Does it depend on prompt complexity?
  • Is the budget ignored, or just interpreted differently?
  • Does fewer thinking tokens reliably mean lower latency?
  • Does answer quality drop?

8. What Android Developers Should Check

Android checklist before you migrate: API path, log thought tokens, switch to thinkingLevel, re-check latency and cost

This experiment did not test the Firebase Android SDK, and that matters. The SDK may validate settings differently, expose different options, or reject parameters before a request ever reaches the model.

A practical Android check should cover:

  • SDK compatibility: does your installed SDK version accept the thinking configuration you set?
  • Request behavior: is the parameter accepted, rejected or transformed on the way out?
  • Usage telemetry: can your app or backend capture thinking token counts?
  • Latency: does changing the thinking setting change response time in a way users notice?
  • Quality: do answers get less reliable with less reasoning?

For production apps, avoid logging raw user prompts or responses. Prefer aggregated metrics, sanitized diagnostics and controlled experiments.

And if the target model recommends thinkingLevel instead of thinkingBudget, test the new setting explicitly. Don't assume it is a one to one replacement.

9. The Bigger Lesson

The takeaway is not that Gemini is broken. The evidence is too thin for that.

The takeaway is that configuration correctness and behavioral correctness are different things. A model can return a perfectly successful response while behaving differently from what you expected.

That is why production AI systems need more than successful API calls and functional tests. They need observability, repeatable evaluations, and regression checks whenever models, SDKs or inference settings change.

Next time you configure a model to use fewer reasoning tokens, don't stop at checking that the request succeeded. Measure what actually happens.

In production AI engineering, the gap between what we configure and what we observe is the gap between an assumed optimization and a verified one.


Experiment note: The numbers come from a small REST test on the Gemini Developer API run on November 17, 2026, on a free tier project, with 1 to 3 successful runs per condition. Some planned runs hit the free tier daily quota. The Firebase Android SDK path was not tested.

Have you seen gaps between documented thinking settings and actual token usage in Gemini, Claude or OpenAI models? I'd like to compare results.

Top comments (0)