DEV Community

Cover image for Gemini 4 Argon thinking_level and Long Decode: A Developer's Guide
AIHubMix
AIHubMix

Posted on

Gemini 4 Argon thinking_level and Long Decode: A Developer's Guide

#ai #api #gemini #llm

Gemini 4 Argon went GA on October 13, 2026, and two settings now decide what each request costs and how long it runs: thinking_level (low, medium or high; default medium) and max_output_tokens (65,536 by default, up to 1,048,576 with long decode). Both are documented. Both have traps.

The engineering problems: every published benchmark ran at high, but a request without a level runs at medium. A limit that's too low returns an empty answer that's still billed. And anything past 64K tokens needs long decode, which runs as background execution and bills every hop. This post covers the settings, a client for AIHubMix, the long decode pattern, and the failure modes with their fixes.

TL;DR

  • Levels: low, medium (default), high. No off switch. Through the OpenAI SDK, reasoning_effort maps minimal/low to low, medium to medium, high to high.
  • Every published score is Argon at high. Nothing has been published at medium or low.
  • max_output_tokens caps thinking plus answer. Use at least 32,768; lower the level, not the limit, to save tokens.
  • The synchronous cap is 65,536. Past that you need long decode: background=True with max_output_tokens above 65,536 on the Interactions API.
  • temperature, top_p and top_k are ignored. Set client timeouts to at least 15 minutes.
  • Long decode bills every token of every 64K hop, and each hop re-reads prior output as input at a 95% cache discount.

The settings at a glance

Setting Gemini 4 Argon
Thinking levels low, medium, high
Default level medium
Turn thinking off Not possible
Default and max output, synchronous 65,536 tokens
Max output with long decode 1,048,576 minus input
Context window 1,048,576, shared by input, thinking, output
temperature, top_p, top_k Ignored
Recommended client timeout At least 15 minutes

Sources: Google's What's new in Gemini 4 Argon and the Gemini thinking guide. The reasoning_effort mapping is from Google's OpenAI compatibility guide.

Choosing a level

Level Token use Google recommends it for
low Fewest tokens, lowest latency Simpler and cost-sensitive work
medium Balanced Most tasks, incl. complex code and agents
high Most tokens, slowest first token The hardest reasoning, math and code

Google's rule: "Run your evaluations at all three levels, then pick the lowest level that meets your quality bar." You'll have to run that evaluation yourself, because the public numbers are all at high: Google evaluated Argon "with the highest thinking settings", and Artificial Analysis and Vals AI both tested at high. [TBD: accuracy and tokens by thinking level]

Comparison caveat: GPT-6 Astra and GPT-6.1 Sol were tested at "max", a level above high. A tie between "Argon (high)" and "Astra (max)" is each model at its ceiling.

What high costs

From Artificial Analysis, its provider tests and Vals AI, all at high:

Measure Gemini 4 Argon (high) For comparison
Output tokens per AA task ~62k GPT-6 Astra (max): ~27k
Output tokens for the AA index 110M Median model: 85M
Output speed 99.5–116.5 tokens/s AI Studio / Vertex AI
Time to first answer token ~45 s Includes thinking
Vals Index time per test 46 min 33 s —

Output pricing includes thinking tokens, which appear in usage as total_thought_tokens. So ~62k output tokens per task is about $0.62 at Google's introductory $10 per million and about $1.24 at $20, the price from December 1. At 99.5–116.5 tokens/s, those 62k tokens take roughly 9 to 10 minutes to generate.

Prerequisites

  • An AIHubMix key in AIHUBMIX_API_KEY. Argon is on AIHubMix as gemini-4-argon.
  • The OpenAI SDK for normal requests, on https://aihubmix.com/v1.
  • The Google GenAI SDK for long decode, on https://aihubmix.com/gemini. The AIHubMix Gemini guide covers the native route and the Interactions API, including background mode, from google-genai 2.0.0. 
pip install -U openai google-genai
export AIHUBMIX_API_KEY="your-aihubmix-key"
Enter fullscreen mode Exit fullscreen mode

The request 

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://aihubmix.com/v1",
    api_key=os.environ["AIHUBMIX_API_KEY"],
    timeout=900,
)

resp = client.chat.completions.create(
    model="gemini-4-argon",
    reasoning_effort="medium",
    max_tokens=32768,
    messages=[
        {"role": "user", "content": "Review this payment service for race conditions in its retry logic.\n\n<source code>"},
    ],
)
print(resp.choices[0].message.content)
print(resp.usage)
Enter fullscreen mode Exit fullscreen mode

What each choice is for:

  • reasoning_effort="medium" names the default explicitly, so a later change of mind is a one-word diff. Use "high" where you want benchmark-level behaviour.
  • max_tokens=32768 is Google's minimum recommendation for apps that set a limit. It caps thinking and answer together.
  • timeout=900 follows Google's advice of at least 15 minutes for hard prompts.
  • No temperature or top_p: Argon ignores them.

Long decode

The long decode guide is the reference; long decode is what Google first announced as Long Decode Continuation. On the Interactions API, the server runs the continuation loop for you: set background=True (which needs store=true) and max_output_tokens above 65,536. It writes in hops of up to 64K tokens, for up to 24 hours, and each hop adds a model_output step.

The pattern below is Google's documented one, pointed at AIHubMix's Gemini endpoint. It hasn't been tested through AIHubMix yet. [TBD: long decode through AIHubMix]

import os
import time
from google import genai

client = genai.Client(
    api_key=os.environ["AIHUBMIX_API_KEY"],
    http_options={"base_url": "https://aihubmix.com/gemini"},
)

interaction = client.interactions.create(
    model="gemini-4-argon",
    input="Write the full test suite for this payment module, with a test for every edge case.\n\n<source code>",
    background=True,
    generation_config={"max_output_tokens": 262144, "thinking_level": "high"},
)
while interaction.status in ("in_progress", "queued"):
    time.sleep(5)
    interaction = client.interactions.get(id=interaction.id)

text = "".join(
    part.text
    for step in interaction.steps
    if step.type == "model_output"
    for part in step.content
    if part.type == "text"
)
print(interaction.status, interaction.usage)
Enter fullscreen mode Exit fullscreen mode

The alternative is client-side continuation on generateContent. Each hop returns up to 64K tokens (32K for non-streaming REST) and ends with finish_reason: CONTINUATION plus a continuation_token; you resend the same request with the token. google-genai 2.30.1 and @google/genai 2.29.1 or later do this automatically. Each request has a 20-minute limit.

If you stream: save the interaction ID and every event_id, reconnect with last_event_id, read to the end before checking status, and set read timeouts of 15 to 20 minutes. After the first 64K tokens, output arrives in chunks of about 5,000 tokens.

When to use it, per Google:

Workload Long decode?
Hard math, science and proofs Yes
The hardest coding and agent tasks Yes
Eval harnesses that stop at the limit Yes
Chat, Q&A, summaries, extraction No, under 32K
Requests for a fixed length No, targets are ignored
Agents with many short tool turns No
Prompts near the 1M input limit No
Latency-sensitive calls No

Failure modes and fixes

Empty or cut-off answers that are still billed

The response hit max_output_tokens while thinking. Status is incomplete, the text is truncated or empty, and the thinking tokens are billed. Fix: raise the limit to at least 32,768, or lower the thinking level. Don't shrink the limit to save money.

400: max_output_tokens greater than 65536 is not yet supported with background=false

You asked for more than 65,536 tokens on a synchronous call. Fix: use background=True, or keep the limit at 65,536.

400: store=true is required for background interactions

Background execution stores the interaction. Fix: don't set store=False on background calls.

Output stops at exactly 64K

background is false, max_output_tokens isn't set above 65,536, or an older SDK isn't continuing on generateContent. Fix: set background=True with a limit above 65,536, or upgrade to google-genai 2.30.1 or later.

Results don't match the published scores

The benchmarks ran at high; your request runs at medium unless you say otherwise. Fix: set reasoning_effort="high" (or thinking_level="high") when you want benchmark behaviour, and measure whether medium is enough.

temperature changes nothing

Argon ignores temperature, top_p and top_k. Fix: remove them; control behaviour with the thinking level and the prompt.

Context compaction starts too early

During long decode, total_input_tokens sums every hop's input, so it overstates the context. Google's example: a 1,000-token prompt plus 200,000 generated tokens runs four hops and reports about 397,000 input tokens for a context of about 201,000. Fix: estimate the context as prompt size plus total_output_tokens plus total_thought_tokens.

The long decode bill

You pay for every token in every hop, including thinking-only hops, and the repeated input in each hop is discounted 95% by implicit caching. In Google's example, at introductory prices, the 200,000 output tokens cost $2.00 and the ~396,000 repeated input tokens at $0.10 per million add about $0.04. Cancelled and failed interactions bill the work already done. Fix: use long decode only for the workloads in the table above, and cap max_output_tokens at what the job needs.

Cache hits disappear mid-session

Argon caches implicitly for prompts of at least 4,096 tokens, at 95% off. The AIHubMix guide notes a cache hit needs the same content, model and parameters, so switching levels between turns may cost you hits. Fix: one level per workflow.

What you can't verify yet

  • How accuracy and token use change at medium and low. All public results are at high. [TBD: accuracy and tokens by thinking level]
  • Whether long decode works end to end through AIHubMix's Gemini endpoint. AIHubMix documents background interactions; Argon's long decode on that route is untested. [TBD: long decode through AIHubMix]

Argon's AIHubMix model page lists its inputs, capabilities and current price.

Keep reading: the Gemini 4 Argon series

Sources

Top comments (0)