Ollama v1 OpenAI Compat Drops Think Toggle

This week’s discussions around open-source and commercial LLMs brought up a familiar theme: the importance of compatibility and control when deploying models in production. But beneath the surface, there’s a subtle but critical issue that can derail even the most well-intentioned deployments. We recently ran into one such issue while working with Ollama’s OpenAI-compatible endpoint and the Qwen3-family of models.

We’re building a system that relies on fine-grained control over model behavior, including the ability to disable thinking phases for specific use cases. This is particularly important for models like Qwen3, where the think: false toggle is meant to bypass internal reasoning and return results faster. However, we noticed that when using Ollama’s /v1 endpoint, this toggle was being silently ignored for Qwen3-family models. The result? The model would exhaust its num_predict budget on internal reasoning, only to return empty content. It was a silent failure that took us days to trace back.

To understand what was going on, we compared the payloads sent to Ollama’s /v1 endpoint versus its native /api/chat endpoint. Here’s a simplified version of the payloads we used:

// /v1 endpoint (OpenAI-compat)
{
  "model": "qwen3",
  "prompt": "What is the capital of France?",
  "think": false,
  "num_predict": 10
}

// /api/chat endpoint (native)
{
  "model": "qwen3",
  "messages": [
    {"role": "user", "content": "What is the capital of France?"}
  ],
  "options": {
    "num_predict": 10
  }
}

The key difference here is the absence of the think toggle in the native endpoint. While the /v1 endpoint is designed to be OpenAI-compatible, it seems that for Qwen3-family models, the think parameter is either not supported or silently ignored. This led to unexpected behavior where the model would spend all its prediction budget on internal reasoning and return nothing useful.

This highlights a common pitfall when using compatibility layers: they often abstract away important details, which can lead to subtle misconfigurations. In our case, the lack of a clear error message made it difficult to diagnose the issue quickly. We had to dive into the model’s internal behavior and compare responses across endpoints to isolate the problem.

The fix was straightforward: switch from the /v1 endpoint to the native /api/chat endpoint. While this meant giving up some level of OpenAI compatibility, it gave us full control over the model’s behavior and ensured that our system could handle Qwen3-family models reliably.

We’re now working on a wrapper that abstracts away these differences, allowing us to use both endpoints seamlessly while maintaining the same interface for our application logic. We’re also exploring ways to contribute back to Ollama’s ecosystem to improve compatibility with models like Qwen3. What would you do in this situation - switch endpoints, or try to patch the compatibility layer?