Introduction to Large Language Model Architectures with Chain-of-Thought Reasoning

#learnai #oxlo #ai

We are going to build a lightweight chain-of-thought reasoning agent that exposes every intermediate step before committing to an answer. If you debug multi-step logic, math, or planning tasks, being able to read the model's reasoning trace makes failures trivial to isolate and fix.

What you'll need

An Oxlo.ai API key from https://portal.oxlo.ai
Python 3.10 or newer
The OpenAI SDK: pip install openai

Oxlo.ai exposes a fully OpenAI-compatible endpoint and flat per-request pricing, so long reasoning traces do not inflate your bill the way token-based providers do. That makes experimenting with chain-of-thought architectures cheap and predictable. See https://oxlo.ai/pricing for current plan details.

Step 1: Set up the Oxlo.ai client

I import the OpenAI SDK and point it at Oxlo.ai. Because the API is fully compatible, this is a single-line change from the standard OpenAI setup.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY",
)

Step 2: Design the chain-of-thought prompt

The system prompt is the only architecture layer we need to add. I force the model to emit a reasoning block before the answer, which lets us inspect the latent chain of thought.

SYSTEM_PROMPT = """You are a careful reasoning engine.
When given a problem, work through your reasoning step by step inside a  block.
Be explicit about assumptions and intermediate calculations.
After you finish reasoning, provide your final answer inside an  block.
If you are uncertain, state your confidence and why."""

Step 3: Build the reasoning harness

Now I wrap the API call in a small function that sends the user question, parses the reasoning trace, and returns both parts. I use DeepSeek V3.2 because it handles coding and reasoning well, and it is available on Oxlo.ai's free tier so you can iterate without burning credit.

import re

def solve_with_cot(user_message: str) -> dict:
    response = client.chat.completions.create(
        model="deepseek-v3.2",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        temperature=0.6,
    )

    raw = response.choices[0].message.content

    thinking = re.search(r"(.*?)", raw, re.DOTALL)
    answer = re.search(r"(.*?)", raw, re.DOTALL)

    return {
        "reasoning": thinking.group(1).strip() if thinking else "",
        "answer": answer.group(1).strip() if answer else raw.strip(),
        "raw": raw,
    }

Step 4: Add self-consistency sampling

A single reasoning path can hallucinate. I sample the same question three times and take a majority vote over the final answers. On Oxlo.ai, each request costs the same flat amount regardless of how long the thinking trace gets, so running multiple samples is predictable. On token-based providers, those extra reasoning tokens add up quickly.

from collections import Counter

def solve_consistent(user_message: str, samples: int = 3) -> dict:
    candidates = [solve_with_cot(user_message) for _ in range(samples)]
    answers = [c["answer"] for c in candidates]
    most_common = Counter(answers).most_common(1)[0]

    return {
        "final_answer": most_common[0],
        "agreement": f"{most_common[1]}/{samples}",
        "all_traces": candidates,
    }

Step 5: Run it

I test the agent on a question that requires multiple logical leaps. The output below is unedited.

QUESTION = (
    "A farmer has 17 sheep and all but 9 die. "
    "How many sheep are left? Explain why."
)

result = solve_consistent(QUESTION)
print(f"Agreement: {result['agreement']}")
print(f"Final answer: {result['final_answer']}")
for i, trace in enumerate(result['all_traces']):
    print(f"\n--- Sample {i+1} ---")
    print(trace['reasoning'])
    print("Answer:", trace['answer'])

Example output:

Agreement: 3/3
Final answer: 9 sheep are left because "all but 9" means 9 survived.

--- Sample 1 ---
The phrase "all but 9 die" means that out of the 17 sheep, all except 9 have died. Therefore, 9 sheep remain alive. The number 17 is context to set up the trick, but the key phrase explicitly states how many survived.
Answer: 9 sheep are left because "all but 9" means 9 survived.

--- Sample 2 ---
I need to parse the wording carefully. "All but 9 die" means 9 did not die. Starting with 17 sheep, if all except 9 died, then 9 sheep are still alive. The initial number 17 is irrelevant to the final count because the wording fixes the survivors at 9.
Answer: 9 sheep are left because "all but 9" means 9 survived.

--- Sample 3 ---
"All but 9 die" is an idiom meaning only 9 survive. So regardless of the starting total of 17, the number left is exactly the 9 that did not die.
Answer: 9 sheep are left because "all but 9" means 9 survived.

Next steps

You can expose this harness as a FastAPI endpoint and stream the <thinking> tokens to the frontend so users can watch the reasoning unfold in real time. If you need to reason over long documents, swap in kimi-k2.6 or qwen-3-32b on Oxlo.ai to take advantage of large context windows without paying extra for every additional token.

DEV Community