Building a Deep Reasoning System Tutorial

Deep reasoning systems do not arrive at answers in a single forward pass. They plan, decompose, reflect, and often invoke external tools before returning a final result. If you are building an agent that writes code, performs multi-step analysis, or solves open-ended research tasks, you need an architecture that treats inference as a stateful loop rather than a stateless completion. This tutorial walks through the core components of a deep reasoning system, shows how to implement a minimal but extensible reasoning loop in Python, and explains why the economics of long-context inference should shape your infrastructure choices.

Architecture Overview

A production-grade system usually separates concerns into five areas:

1. Reasoning engine. The model that produces chain-of-thought, decides when to act, and synthesizes conclusions.
2. Working memory. The message history, tool outputs, and intermediate drafts that fit inside the context window.
3. Tool registry. Executable functions the model can call, such as calculators, search APIs, or code interpreters.
4. Controller. The orchestration loop that repeatedly calls the reasoning engine, dispatches tools, and appends observations back into working memory.
5. Verifier. An optional critique layer that checks consistency, correctness, or completeness before the final answer is returned.

The controller is the piece most developers build themselves. The other four are largely determined by your inference provider and model choice.

Selecting the Reasoning Engine

Not every model is suited for deep reasoning. You want a foundation model that either exposes explicit chain-of-thought or is fine-tuned for agentic execution. Oxlo.ai offers several relevant options across different trade-offs:

• DeepSeek R1 671B MoE for deep reasoning and complex coding tasks.
• DeepSeek V4 Flash, an efficient MoE with a 1M token context window and near state-of-the-art open-source reasoning.
• Kimi K2.6, which supports advanced reasoning, agentic coding, vision, and 131K context.
• Kimi K2 Thinking and Kimi K2.5 for advanced chain-of-thought reasoning.
• GLM 5, a 744B MoE designed for long-horizon agentic tasks.
• Qwen 3 32B for multilingual reasoning and agent workflows.

Because Oxlo.ai is fully OpenAI SDK compatible, switching between these models is a single string change in your client configuration. There are no cold starts on popular models, so the first request in a session behaves like any other.

Implementing the Reasoning Loop

The classic pattern for deep reasoning is a ReAct-style loop: the model reasons, chooses an action, the controller executes it, and the observation is fed back. Below is a minimal but complete example using the Oxlo.ai API endpoint.

import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ["OXLO_API_KEY"]
)

MODEL = "deepseek-r1-671b"  # or kimi-k2.6, deepseek-v4-flash, glm-5, etc.

tools = [
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Evaluate a mathematical expression safely.",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {"type": "string"}
                },
                "required": ["expression"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "search",
            "description": "Run a web search and return top snippets.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string"}
                },
                "required": ["query"]
            }
        }
    }
]

def dispatch_tool(name: str, args: dict) -> str:
    if name == "calculate":
        # In production, run inside a sandbox, not eval.
        try:
            return str(eval(args["expression"]))
        except Exception as e:
            return f"Error: {e}"
    if name == "search":
        # Replace with real search integration.
        return "No results."
    return "Unknown tool."

def deep_reason(user_query: str, max_iterations: int = 5) -> str:
    messages = [
        {"role": "system", "content": "You are a careful reasoning agent. Think step by step, and use tools when facts or calculations are required."},
        {"role": "user", "content": user_query}
    ]

    for _ in range(max_iterations):
        response = client.chat.completions.create(
            model=MODEL,
            messages=messages,
            tools=tools,
            tool_choice="auto"
        )

        message = response.choices[0].message
        messages.append(message)

        # If the model did not request tools, we have a final answer.
        if not message.tool_calls:
            return message.content

        # Otherwise, execute each tool and append observations.
        for tool_call in message.tool_calls:
            name = tool_call.function.name
            arguments = json.loads(tool_call.function.arguments)
            observation = dispatch_tool(name, arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "name": name,
                "content": observation
            })

    return "Reached maximum reasoning depth without a final answer."

if __name__ == "__main__":
    answer = deep_reason("What is the square root of 1764 multiplied by the population of Iceland?")
    print(answer)

This loop is the skeleton of most agentic systems. The model decides when it knows enough to answer, and the controller remains agnostic about domain logic.

Structuring Reasoning with JSON Mode

Unstructured text is hard to parse if you want to extract the model's internal plan or route actions programmatically. Oxlo.ai supports JSON mode, which lets you constrain the output to a schema. You can ask the model to emit an object with fields such as thought, action, and action_input, then parse it reliably before dispatching tools.

To use JSON mode, set response_format={"type": "json_object"} in your request and include a schema description in the system prompt. This pairs well with the reasoning loop when you want the controller, not the model, to decide how tool arguments are validated.

Managing Context and Memory

Deep reasoning traces grow quickly. Every thought, tool call, and observation consumes tokens, and multi-turn agentic workloads can push context windows into the hundreds of thousands of tokens. With token-based providers, cost scales linearly with input length, which means long reasoning sessions become prohibitively expensive.

Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. Unlike token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, Oxlo.ai does not penalize you for carrying full history, large retrieved documents, or verbose chain-of-thought into every turn. For agentic and long-context workloads, this can make deep reasoning far more economical. You can review the details on the Oxlo.ai pricing page.

Because cost is decoupled from context length, you can keep more turns in working memory before compressing or summarizing. When you do need to compress, a simple strategy is to summarize turns older than a threshold into a single system message, while preserving the most recent raw exchanges for accuracy.

Adding Reflection and Verification

A single reasoning pass can hallucinate facts or make arithmetic errors. A robust system adds a verification step. After the loop produces a draft answer, send a second request asking the model to critique its own work.

def verify(question: str, draft: str) -> str:
    prompt = (
        f"Question: {question}\n"
        f"Proposed Answer: {draft}\n\n"
        "Critique this answer. Identify any factual errors, logical gaps, or unstated assumptions. "
        "Respond with CORRECT if it is fully accurate, or explain the flaw."
    )
    response = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

If the verifier finds an issue, you can append the critique to working memory and restart the reasoning loop. This self-correction pattern is especially effective on Oxlo.ai models such as DeepSeek R1 671B MoE and Kimi K2 Thinking, which are explicitly tuned for extended reasoning chains.

Streaming and Production Deployment

In production, users expect to see progress. Oxlo.ai supports streaming responses, so you can emit reasoning tokens as they are generated rather than blocking until the full response is ready.

stream = client.chat.completions.create(
    model=MODEL,
    messages=messages,
    tools=tools,
    tool_choice="auto",
    stream=True
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="")

Because there are no cold starts on popular models, the first chunk arrives immediately even after idle periods. If you need guaranteed throughput or dedicated GPUs, Oxlo.ai offers Enterprise plans with custom sizing and dedicated infrastructure.

Conclusion

Building a deep reasoning system is fundamentally about looping inference, managing long context, and integrating tools under a controller you own. The infrastructure you choose should reward complexity rather than tax it. Oxlo.ai provides the model depth, from DeepSeek R1 and V4 Flash to Kimi K2.6 and GLM 5, alongside a flat per-request pricing model that keeps long reasoning traces affordable. With full OpenAI SDK compatibility, you can point your existing client to https://api.oxlo.ai/v1 and focus on the architecture, not the billing.