The Role of Deep Reasoning in LLM Models: A Deep Dive

We are building a Deep Reasoning Research Agent that tackles ambiguous technical questions by decomposing them into verifiable sub-problems, reasoning through each step, and synthesizing a cited answer. It runs on Oxlo.ai's deep reasoning models and is designed for engineers who need reliable analysis without token-cost surprises.

What you'll need

• Python 3.10 or newer
• The OpenAI SDK: pip install openai
• An Oxlo.ai API key from https://portal.oxlo.ai

Oxlo.ai carries DeepSeek R1 671B, Kimi K2.6, and Qwen 3 32B on a flat per-request pricing model, which makes iterative agent loops predictable even when the reasoning traces grow long.

Step 1: Configure the Oxlo.ai client

Import the OpenAI SDK and point it at Oxlo.ai. This is a drop-in replacement, so the base URL and key are the only changes needed.

from openai import OpenAI
import json
from typing import List, Dict

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY",
)

Step 2: Define the system prompt

The system prompt forces the reasoning model to expose its chain of thought in a parseable format. We will feed this to DeepSeek R1 671B in the next step.

SYSTEM_PROMPT = """You are a deep reasoning engine. Solve the problem through explicit step-by-step analysis.

Rules:
1. Write your raw reasoning inside <thinking>...</thinking> tags.
2. State assumptions explicitly.
3. After reasoning, provide a concise partial answer inside <answer>...</answer> tags.
4. If uncertain, say so. Do not hallucinate facts."""

Step 3: Decompose the problem

We use Qwen 3 32B on Oxlo.ai to break the user question into sub-questions. JSON mode keeps the output structured and easy to iterate over.

def decompose_question(question: str) -> List[str]:
    response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {"role": "system", "content": "You are a query planner. Return a JSON object with a key 'sub_questions' containing a list of strings."},
            {"role": "user", "content": question},
        ],
        response_format={"type": "json_object"},
    )
    result = json.loads(response.choices[0].message.content)
    return result["sub_questions"]

Step 4: Reason through each sub-problem

Now we route each sub-question to DeepSeek R1 671B, a model built for deep reasoning. We parse the thinking and answer blocks from its response so the final synthesis has full visibility into the logic.

def reason_sub_problem(sub_question: str) -> Dict[str, str]:
    response = client.chat.completions.create(
        model="deepseek-r1-671b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": sub_question},
        ],
    )
    raw = response.choices[0].message.content

    thinking = ""
    if "<thinking>" in raw and "</thinking>" in raw:
        thinking = raw.split("<thinking>")[1].split("</thinking>")[0].strip()

    answer = raw
    if "<answer>" in raw and "</answer>" in raw:
        answer = raw.split("<answer>")[1].split("</answer>")[0].strip()

    return {
        "sub_question": sub_question,
        "thinking": thinking,
        "partial_answer": answer,
    }

Step 5: Synthesize the final report

Partial answers need to be merged into a coherent report. Kimi K2.6 on Oxlo.ai handles long context and advanced reasoning well, so we feed it the full set of reasoning traces.

def synthesize_report(question: str, partials: List[Dict[str, str]]) -> str:
    context_blocks = []
    for p in partials:
        block = (
            f"Sub-question: {p['sub_question']}\n"
            f"Reasoning: {p['thinking']}\n"
            f"Partial answer: {p['partial_answer']}"
        )
        context_blocks.append(block)

    context = "\n\n".join(context_blocks)

    response = client.chat.completions.create(
        model="kimi-k2.6",
        messages=[
            {"role": "system", "content": "You are a technical editor. Synthesize the research below into a clear final report with bullet points. Do not introduce facts that are not present in the reasoning traces."},
            {"role": "user", "content": f"Original question: {question}\n\nResearch context:\n{context}"},
        ],
    )
    return response.choices[0].message.content

Step 6: Wire the agent together

The agent class orchestrates the pipeline. Because Oxlo.ai charges per request rather than per token, running three separate model calls in a loop stays predictable even when the reasoning traces grow long.

class DeepReasoningAgent:
    def __init__(self):
        self.client = client

    def research(self, question: str) -> str:
        print(f"Decomposing: {question}")
        sub_questions = decompose_question(question)

        partials = []
        for sq in sub_questions:
            print(f"  Reasoning: {sq}")
            partial = reason_sub_problem(sq)
            partials.append(partial)

        print("Synthesizing final report...")
        return synthesize_report(question, partials)

Run it

Here is a realistic query about model architecture decisions. We instantiate the agent and print the result.

agent = DeepReasoningAgent()

question = (
    "I need to build a real-time coding assistant that handles 100k token contexts. "
    "Should I use a dense transformer or a mixture-of-experts model, and which open weights "
    "options are easiest to self-host on a single A100?"
)

report = agent.research(question)
print("\n=== FINAL REPORT ===\n")
print(report)

Example output:

Decomposing: I need to build a real-time coding assistant that handles 100k token contexts. Should I use a dense transformer or a mixture-of-experts model, and which open weights options are easiest to self-host on a single A100?
  Reasoning: What are the memory and latency trade-offs between dense transformers and MoE architectures for 100k context windows?
  Reasoning: Which open-weight MoE and dense models support 100k+ context and fit on a single A100 80GB?
  Reasoning: What quantization and serving strategies are required to maintain real-time throughput under these constraints?
Synthesizing final report...

=== FINAL REPORT ===

- Memory Trade-offs: Dense models store every parameter in active memory during inference, which scales linearly with sequence length for KV cache. MoE architectures activate only a subset of experts per token, reducing active parameter count and memory pressure at the cost of routing overhead.
- Suitable Open Models: DeepSeek V4 Flash offers a 1M context window and efficient MoE inference, making it a strong candidate for long-context coding. Llama 3.3 70B is a dense alternative with robust general performance but requires aggressive quantization or context pruning to fit on one A100 at 100k tokens.
- Serving Strategy: Use vLLM with FP8 or AWQ quantization for dense models, or a specialized MoE serving framework that caches expert weights. For real-time latency, aim for tensor parallelism across multiple GPUs if possible, or use a smaller active parameter MoE variant.
- Recommendation: Start with an MoE model such as DeepSeek V4 Flash if your primary constraint is fitting 100k tokens on a single A100 80GB, because the sparse activation keeps memory usage lower than an equivalent dense model. Validate end-to-end latency with your expected token generation rate before committing.

Wrap-up

You now have a working deep reasoning agent that decomposes questions, audits its own logic, and synthesizes a final answer. A concrete next step is to wire in a web search tool so the reasoning steps can cite live documentation rather than relying solely on parametric knowledge. Another is to expose the agent through a FastAPI endpoint and stream the partial reasoning traces to the client using Oxlo.ai's streaming response support.