Ollama v1 OpenAI Compat Drops Think Toggle

This week's headlines buzzed with the promise of running large language models at unprecedented speeds, but the technical details beneath the surface often get lost in the hype. As engineers, we care more about the nuts and bolts - like when a seemingly minor API change can break a pipeline in unexpected ways. That’s exactly what happened to us when working with Ollama’s OpenAI-compatible endpoint and the Qwen3-family models.

We were running a service that relied on Ollama’s /v1 endpoint to interface with Qwen3 models, expecting the think: false toggle to short-circuit the model’s internal reasoning and return a prompt-based response. Instead, we noticed that the models were consistently exhausting their num_predict budget and returning empty content. This was a silent failure - no errors, no logs, just empty strings where we expected output.

After some digging, we realized the issue was not with the model itself, but with the endpoint. The /v1 OpenAI-compatible endpoint silently ignored the think: false parameter for Qwen3-family models. This meant the model would process the entire prompt, perform internal reasoning, and then - with no tokens left to predict - return nothing. It was a classic case of a misaligned expectation between the API and the model's behavior.

To fix this, we pivoted to Ollama’s native /api/chat endpoint, which respects the think: false toggle as intended. Here’s a comparison of the two payloads:

// OpenAI-compat endpoint (fails for Qwen3)
{
  "model": "qwen3",
  "prompt": "What is the capital of France?",
  "think": false,
  "num_predict": 10
}

// Native /api/chat endpoint (works as expected)
{
  "model": "qwen3",
  "messages": [
    {"role": "user", "content": "What is the capital of France?"}
  ],
  "options": {
    "think": false,
    "num_predict": 10
  }
}

The key difference lies in how the think parameter is interpreted and where it's placed in the payload. The /v1 endpoint appears to ignore think: false for certain models, while the native endpoint correctly honors it. We noticed the issue by monitoring the num_predict usage and correlating it with the output length. When the num_predict budget was consumed but no output was returned, it became clear that something was wrong in the reasoning phase.

This experience highlights the importance of testing across endpoints and understanding the nuances of model-specific behavior. While the /v1 endpoint is convenient for compatibility with OpenAI tools, it’s not a one-size-fits-all solution - especially for models like Qwen3 that have unique interaction patterns.

Looking ahead, we’re exploring ways to unify our internal tooling to handle these endpoint differences more gracefully. We’re also considering contributing to the Ollama project to clarify the behavior of the /v1 endpoint for specific models. What would you do if you encountered a similar silent failure in your AI pipeline?