Groq Returned Empty Content. The Bug Was Hiding in Reasoning Tokens.

#ai #java #debugging #webdev

This article was originally published on Jo4 Blog.

We use Groq's gpt-oss-safeguard model to classify pages behind freshly created short links. Most pages take a few hundred tokens to score. Some don't. And the ones that don't were silently failing — for weeks — until we noticed the symptom: a small but consistent stream of links stuck in "preview pending" forever.

Here's what we found.

The Problem

The classifier wraps a single Groq chat completion. Send page text, get back a JSON verdict (safe, unsafe, with category codes). For 95% of links, this works in well under a second.

For the other 5%, we'd see this in logs:

WARN  Empty content in Groq response
WARN  Classification failed for shortUrl=xyz123 — preview stays enabled

Empty content. Not a network error, not a rate limit, not malformed JSON. The API returned 200, the choices array had one entry, and choices[0].message.content was "".

What did those pages have in common? They weren't obvious spam. They weren't obvious safe. They were ambiguous — a wellness blog that mentioned medication dosages, a forum thread about firearms law, a satire site quoting violent rhetoric. The kind of content where a human reviewer would also pause.

The Wrong First Guess

Our first instinct: the model is rate-limited or degraded for hard inputs. We added retries. The empty-content rate didn't budge.

Second guess: we're hitting max_tokens. We had set it to 200. Maybe ambiguous pages produce longer verdicts. We bumped it to 400. Empty content rate didn't budge.

The clue we kept missing was sitting in the response body itself, in a field we weren't parsing.

The Root Cause

Groq's response includes a usage block, and usage.completion_tokens_details.reasoning_tokens was the smoking gun:

{
  "choices": [{
    "message": { "content": "" },
    "finish_reason": "length"
  }],
  "usage": {
    "completion_tokens": 200,
    "completion_tokens_details": {
      "reasoning_tokens": 200
    }
  }
}

gpt-oss-safeguard is a reasoning model. Before emitting a single character of content, it spends completion tokens on internal chain-of-thought. Easy pages spend a few dozen reasoning tokens, then emit a 50-token verdict. Ambiguous pages spend several hundred reasoning tokens — and on those, our 200-token budget was being exhausted inside the reasoning phase, leaving zero tokens for content.

The API obediently returned the response. choices[0].message.content was "" because there was nothing left in the budget to write into it. finish_reason was length, not stop — the model didn't decide it was done, the token budget cut it off mid-thought.

We were paying for full inference and getting empty strings.

The Fix

Three changes:

1. Switch from max_tokens to max_completion_tokens — max_tokens is deprecated for reasoning models. Use the correct parameter name so the API enforces the limit you mean.

2. Raise the budget with headroom. We profiled real ambiguous pages: worst case was ~550 reasoning tokens. We set the budget to 1024 — covers worst case plus content with margin to spare.

static final int MAX_COMPLETION_TOKENS = 1024;

String requestBody = objectMapper.writeValueAsString(Map.of(
    "model", modelName,
    "messages", List.of(
        Map.of("role", "system", "content", SAFETY_POLICY),
        Map.of("role", "user", "content", text)
    ),
    "max_completion_tokens", MAX_COMPLETION_TOKENS,
    "temperature", 0.0
));

3. Parse usage and alert when reasoning tokens approach the budget. This is the part that actually prevents the next regression:

if (reasoningTokens != null
    && reasoningTokens > MAX_COMPLETION_TOKENS * 0.8) {
    log.warn("Classifier reasoning tokens near budget: {}/{} — "
        + "consider raising max_completion_tokens",
        reasoningTokens, MAX_COMPLETION_TOKENS);
}

When reasoning crosses 80% of the budget, we log a warning. The next ambiguous page in that distribution is the one that will trip finish_reason=length and return empty content. We'd rather raise the budget before users see stuck previews, not after.

We also added the diagnostic to the empty-content branch:

if (rawContent == null || rawContent.isBlank()) {
    String finishReason = firstChoice.getFinishReason();
    log.warn("Empty content (finish_reason={} completion_tokens={} "
        + "reasoning_tokens={})",
        finishReason, completionTokens, reasoningTokens);
    return ClassificationResult.error(
        "Empty classifier output (finish_reason=" + finishReason + ")");
}

So if it ever happens again, the next person debugging it has the answer in the first log line, not after a week of squinting.

Lessons Learned

For reasoning models, max_tokens is a budget the model spends thinking and speaking. If the budget runs out mid-thought, you get a 200 response with empty content. There is no exception, no error code in the body — just a "" and a finish_reason: length that you have to parse to see.
Profile against the hardest inputs, not the average. Our 200-token budget worked fine on test fixtures because our fixtures were obvious. Ambiguity is what blows the budget.
finish_reason is the field that tells you the truth. stop = model is done. length = the model wanted to keep going and you didn't let it. Treat them as completely different outcomes.
completion_tokens_details.reasoning_tokens is the leading indicator. Don't wait for empty content to alert. Watch reasoning-token usage as a percentage of the budget, and alert before you cross the cliff.
Use the deprecated-API warnings. max_tokens was the wrong field name for reasoning models. The API silently honored it anyway, which made the bug subtler. The right field name is max_completion_tokens.