DEV Community: soohan abbasi

Chain-of-Thought and Beyond: How LLMs Actually Learn to Reason

soohan abbasi — Sat, 16 May 2026 12:30:00 +0000

"The ability to reason step-by-step is not just a feature. It might be the difference between a language model that sounds intelligent and one that actually is."

Introduction: When AI Started Thinking

In 2022, researchers at Google Brain published a paper titled "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models". At the time, nobody quite anticipated it would mark the beginning of a shift that would reshape the entire AI field.

The idea was simple: instead of asking a model to answer directly, give it time to think. Ask it to write out intermediate steps. Accuracy improves dramatically.

That paper now sits at over 10,000 citations. But the question it raised has never been fully answered:

Do LLMs actually think? Or do they create a very convincing illusion of thinking?

That is what this blog is about. And as someone preparing for a PhD in AI, it is a question I keep coming back to.

Part 1: What Is Chain-of-Thought?

Standard Prompting vs. CoT Prompting

Imagine asking a model this:

"Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many does he have now?"

With standard prompting, the model jumps straight to: "11"

With chain-of-thought prompting, the model works through it first:

Roger starts with 5 balls.
2 cans × 3 balls = 6 balls.
5 + 6 = 11 balls.
Answer: 11

Both get the same answer. So what is the point?

The gap shows up on harder problems. Models that reason through steps outperform those that answer directly on multi-step math, symbolic reasoning, and commonsense problems. The more complex the task, the bigger the difference.

Zero-Shot CoT: One Phrase Changes Everything

In the same year, researchers discovered something even more surprising. Simply adding the phrase "Let's think step by step" to a question, without any examples, significantly improved reasoning accuracy.

No demonstrations. No fine-tuning. Just those five words.

This became known as zero-shot CoT. And the obvious follow-up question is: why does this even work?

My Own Experiment: Testing CoT on GSM8K

Before going deeper into the theory, I wanted to test this myself. So I ran a small experiment using an open-source model on a standard benchmark.

Setup:

Model: Qwen 2.5 1.5B Instruct (free, runs on Kaggle GPU)
Dataset: GSM8K (grade school math problems)
Test: Standard prompting vs. "Let's think step by step"
Sample: 10 problems

Results:

Approach	Correct	Accuracy
Without CoT	2/10	20%
With CoT	3/10	30%

[CoT vs No-CoT Results on GSM8K]

Even on a model roughly 360 times smaller than the one used in the original paper, the improvement showed up. A single phrase shifted accuracy by 10%.

A few things stood out from the per-problem breakdown:

Problem 1 was solved correctly with CoT, but not without it. Problem 7 showed the same pattern. Problem 4 was solved correctly either way. But Problem 6 was actually solved correctly without CoT and incorrectly with it. The model overthought a straightforward calculation and got it wrong.

That last observation matters and connects to something I discuss in Part 4.

Quick note: the overall accuracy numbers look low because this model is tiny compared to what the original paper used. The point here is the relative difference, not the absolute numbers.

Part 2: What Is Actually Happening Inside the Model?

More Than Pattern Matching

The common criticism of LLMs is that they are sophisticated autocomplete. They match patterns from training data rather than genuinely reasoning. This criticism is not entirely wrong, but it is incomplete.

Between 2023 and 2024, researchers doing mechanistic interpretability work found some interesting things inside these models.

LLMs contain specific reasoning circuits: groups of neurons and attention heads that work together to perform logical operations. They use something called induction heads, which are attention patterns that identify sequences in context and predict what follows. Some models have developed implicit world models, meaning they internally represent concepts like spatial relationships, time, and causality.

None of this was explicitly programmed. It emerged from training on text.

The picture that comes out of this research is more interesting than "just pattern matching." These models have developed internal structures that support reasoning-like behavior. Whether that constitutes real reasoning is a separate philosophical question, but it is clearly more than autocomplete.

Process Reward Models: Grading the Work, Not Just the Answer

Here is an idea that changed how reasoning models are trained. Instead of grading only the final answer, what if you graded every individual reasoning step?

That is the core of a Process Reward Model (PRM).

In standard training, the model produces an answer and gets told whether it was right or wrong. In PRM-based training, each step in the reasoning chain gets its own score. A wrong step gets flagged early, before it derails the rest of the solution.

OpenAI's 2023 paper "Let's Verify Step by Step" showed that PRMs significantly outperform outcome-based reward models on mathematical reasoning tasks.

This idea became the foundation for something much bigger, which I will cover in Week 12 when we get to test-time compute scaling.

Part 3: OpenAI o1 and DeepSeek-R1

OpenAI o1: Giving Models Time to Think

In September 2024, OpenAI released o1, and the response from the research community was immediate.

The idea behind o1 is straightforward. Give the model more time to think about the inference. Before producing an answer, o1 generates a hidden chain of thought that the user never sees, but the model uses internally. This chain is trained with reinforcement learning: the model gets rewarded for reaching correct answers, which teaches it to develop better internal reasoning strategies.

The results on AIME 2024, a notoriously difficult high school math competition, were striking. GPT-4o scored 12%. o1 scored 74%.

That is not a small improvement. That is a different class of performance, driven almost entirely by letting the model think longer.

DeepSeek-R1: The Open Source Answer

In January 2025, a Chinese startup called DeepSeek released R1, and it caused genuine disruption in the Western AI community.

DeepSeek-R1 matched o1-level performance at a fraction of the training cost. And it was fully open source.

Three technical contributions made this possible.

Group Relative Policy Optimization (GRPO): Standard RLHF needs a separate critic model to score responses, which adds significant overhead. GRPO removes that requirement. Instead, the model generates multiple responses to the same question, compares them against each other, and rewards the best one. No criticism needed.

Warm Start Before RL: Training a model from scratch with pure reinforcement learning is unstable because the model starts random. DeepSeek's approach was to first run supervised fine-tuning to give the model a reasonable starting point, then apply RL on top of that. A sensible idea that turned out to matter a lot.

Emergent Reasoning Behaviors: During training, R1 developed behaviors that were never explicitly programmed. The model began catching its own mistakes mid-reasoning and reconsidering. It started verifying its own answers before finalizing them. It explored alternative solution paths. These behaviors just appeared from the training process. For researchers trying to understand what is happening inside these models, this is genuinely interesting territory.

Part 4: Where CoT Fails

Unfaithful Reasoning

One of the more unsettling findings in recent research is that CoT explanations do not always reflect what the model actually computed.

Anthropic's 2023 research showed that models sometimes produce post-hoc rationalizations. They settle on an answer through some internal process, then construct a reasoning chain that appears to justify it. The explanation and the computation are decoupled.

What the model writes as its reasoning may not be what actually happened.

Reasoning or Memorization?

There is a deeper question underneath CoT performance: is the model actually reasoning, or is it recalling reasoning-shaped patterns from its training data?

Researchers created a symbolic variant of GSM8K where the logic of each problem stayed the same, but surface features like numbers and names were changed. Performance dropped significantly. If the model were truly reasoning about the structure of the problem, this change should not matter. The fact that it does suggests some of the apparent reasoning is memorization in disguise.

The Overthinking Problem

My experiment showed a small version of this. On Problem 6, the model solved it correctly without CoT. With CoT, it added extra steps, got confused, and got it wrong.

Researchers have documented this pattern at scale. Longer reasoning chains are not always better. Past a certain point, additional steps introduce errors rather than correct them. This has been called "overthinking" or the "lost in the middle" problem.

Compositional Generalization

LLMs also struggle when they need to combine reasoning skills in novel ways. They can handle familiar patterns well. But put two familiar patterns together in a configuration the model has not seen, and performance degrades. This suggests the reasoning ability is less flexible and generalizable than it might appear from benchmark numbers.

Part 5: What We Still Do Not Know

CoT has genuinely advanced what language models can do. But there are open questions that the field has not resolved.

Are the Explanations Honest?

When a model shows its reasoning, is that actually what happened computationally? The unfaithful reasoning research says it often is not. We do not have reliable tools to check whether a model's stated reasoning matches its internal computation. This matters a lot if you want to trust the reasoning, not just the answer.

Where Does Reasoning End and Memorization Begin?

The symbolic variant experiments raise a question that nobody has cleanly answered yet. For any given correct reasoning chain, how much of it reflects genuine logical inference versus pattern recall? The boundary is not well defined.

Why Does CoT Work in English and Struggle Elsewhere?

Almost all CoT research was conducted in English. When you apply the same techniques to Arabic, Urdu, or other lower-resource languages, performance drops noticeably. Whether this is primarily a data coverage problem or something more structural about how reasoning transfers across language families is still an open question.

Can We Formally Verify a Reasoning Step?

A calculator gives you a provably correct answer. An LLM gives you a confident one. There is currently no reliable way to formally verify whether an individual step in an LLM's reasoning chain is logically valid. Researchers are exploring integrations with formal theorem provers such as Lean4, but this remains largely unsolved.

Does Interpretability Scale?

Mechanistic interpretability research has produced real insights at small model scales: specific circuits identified, specific behaviors localized. But as models grow to hundreds of billions of parameters, these techniques become computationally impractical. How interpretability research keeps pace with model scale is an open problem.

Papers Worth Reading

Paper	What It Contributes	Venue
Wei et al. (2022)	Original CoT paper	NeurIPS 2022
Kojima et al. (2022)	Zero-shot CoT discovery	NeurIPS 2022
Lightman et al. (2023)	Process Reward Models	OpenAI Tech Report
DeepSeek-AI (2025)	GRPO and DeepSeek-R1	arXiv 2501
Turpin et al. (2023)	Unfaithful reasoning	NeurIPS 2023
Wang et al. (2022)	Self-consistency via majority voting	ICLR 2023

Research Groups Doing Interesting Work Here

Anthropic's interpretability team is doing some of the most rigorous work on understanding what is happening inside these models. DeepMind's Gemini team is pushing multimodal reasoning. MIT's BCS and CSAIL groups are connecting cognitive science with language model research. Peking University's NLP group has produced strong work on multilingual reasoning.

Benchmarks You Should Know

GSM8K covers grade school math with 8,500 problems. MATH is competition-level with 12,500 problems. MMLU covers broad knowledge across many domains. ARC-Challenge focuses on scientific reasoning. BIG-Bench Hard collects 23 tasks specifically designed to be difficult for current models.

Conclusion

Chain-of-thought prompting is one of the more surprising ideas in recent AI research. A single phrase, added to a prompt, unlocks reasoning capabilities that were already there but not being used.

And yet the central question it raised remains unanswered. Do these models actually reason, or do they produce sophisticated simulations of reasoning? The honest answer is that we do not fully know.

The gap between sounding intelligent and being intelligent is where the most interesting work in this field is happening right now.

Next week: Small Language Models. How models like Phi-3 and Gemma became serious competitors to GPT-4, and what the research landscape looks like when you do not need a data center to run your model.

References

1.Wei, J., et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022.

Kojima, T., et al. (2022). Large Language Models are Zero-Shot Reasoners. NeurIPS 2022.
Lightman, H., et al. (2023). Let's Verify Step by Step. arXiv:2305.20050.
DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948.
Turpin, M., et al. (2023). Language Models Don't Always Say What They Think. NeurIPS 2023.
Wang, X., et al. (2022). Self-Consistency Improves Chain-of-Thought Reasoning in Language Models. ICLR 2023.
Elhage, N., et al. (2021). A Mathematical Framework for Transformer Circuits. Anthropic.

Code for this experiment is available on GitHub: Week 01 Code

This is part of a weekly series on AI/ML research. Each post covers theory, recent work, and experiments I run myself.

Connect on LinkedIn Soohan Abbasi

I Built an Offline AI Career Advisor Using Gemma 4 — Here's Exactly How It Works

soohan abbasi — Wed, 13 May 2026 06:04:46 +0000

I Built an Offline AI Career Advisor Using Gemma 4 — Here's Exactly How It Works

A technical walkthrough of GuidanceOS: from model loading to multi-agent orchestration, running entirely on a Kaggle T4 GPU with no internet at inference time.

I teach Computer Science. Over the years, one thing I kept seeing was students who had decent skills but no idea what to do with them. They didn't know what jobs matched their profile, what courses to take next, or how to position themselves for a career. Career guidance platforms exist, sure — but they're mostly behind paywalls, require accounts, and need a stable internet connection.

So I built GuidanceOS for the Gemma 4 Good Hackathon. The goal was simple: a fully offline AI system that takes your resume, figures out your skills, and gives you a complete career analysis — job matches, course recommendations, a 3-month learning plan, and an ATS score — all running locally on a GPU, no API calls at inference time.

Here's exactly how I built it.

The Model Choice: Why Gemma 4 e4b-it

The hackathon required using Gemma 4. Google released four variants: 2B, 4B (edge), 26B MoE, and 31B Dense. I went with gemma-4-e4b-it for a specific reason.

The "e" stands for edge-optimized. The "it" stands for instruction-tuned. On Kaggle's free T4 GPU (15GB VRAM), a naive load of even a 4B model can fail if quantization isn't handled right. With 4-bit NF4 quantization via BitsAndBytes, gemma-4-e4b-it loads in about 8.7GB — leaving headroom for inference.

One problem I ran into immediately: the stable release of Hugging Face Transformers (5.0.0 at the time) didn't recognize the gemma4 architecture. Loading the model threw:

ValueError: The checkpoint you are trying to load has model type `gemma4`
but Transformers does not recognize this architecture.

The fix was straightforward — install Transformers from the GitHub dev branch:

`pip install git+https://github.com/huggingface/transformers.git`

This bumped the version to 5.8.0.dev0, which includes the Gemma 4 model class.

The second issue was GPU memory management. Using device_map="auto" caused BitsAndBytes to split the model across CPU and GPU, which it doesn't allow in 4-bit mode:

ValueError: Some modules are dispatched on the CPU or the disk.
Make sure you have enough GPU RAM to fit the quantized model.

Solution: pin everything to a single GPU.

model = AutoModelForImageTextToText.from_pretrained(
    MODEL_PATH,
    quantization_config=bnb_config,
    device_map="cuda:0",
    dtype=torch.bfloat16,
)

After that, the model loaded cleanly in about 3 minutes and sat at 8.7GB on GPU 0.

The Knowledge Base: TF-IDF Over 130K Records

I used two datasets:

LinkedIn Job Postings — 123,849 jobs with title, description, skills, location, experience level, and salary
Coursera Courses 2024 — 6,645 courses with title, skills, description, level, rating, and URL

For job and course matching, I built a TF-IDF index over combined text fields. For jobs, I concatenated the job title, skills description, and the first 300 characters of the full description. For courses, I combined the title, skills tags, and description.

jobs_clean['combined_text'] = (
    jobs_clean['title'] + ' ' +
    jobs_clean['skills_desc'] + ' ' +
    jobs_clean['description'].str[:300]
)

Then I fit a TfidfVectorizer with bigrams and 10,000 features:

jobs_vectorizer = TfidfVectorizer(
    max_features=10000,
    stop_words='english',
    ngram_range=(1, 2)
)
jobs_tfidf_matrix = jobs_vectorizer.fit_transform(jobs_clean['combined_text'])

At query time, the user's skill string gets transformed by the same vectorizer and compared against the full matrix using cosine similarity. The top-k results come back in milliseconds — no GPU needed, no network call.

I chose TF-IDF over dense vector search (FAISS + sentence embeddings) deliberately. Dense search needs an embedding model at query time, which adds latency and memory. TF-IDF is deterministic, fast, and reproducible — important when the whole point is offline-first operation.

The Inference Helper

Before building agents, I needed a clean wrapper around Gemma 4's generation. The model uses a specific chat format:

def ask_gemma(prompt, max_tokens=300, temperature=0.7):
    formatted = f"<bos><start_of_turn>user\n{prompt}<end_of_turn>\n<start_of_turn>model\n"

    inputs = tokenizer(
        formatted,
        return_tensors="pt",
        add_special_tokens=False
    ).to("cuda:0")

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_tokens,
            do_sample=True,
            temperature=temperature,
            top_p=0.9,
            repetition_penalty=1.3,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )

    input_len = inputs["input_ids"].shape[-1]
    response = tokenizer.decode(outputs[0][input_len:], skip_special_tokens=True)

    if "<end_of_turn>" in response:
        response = response.split("<end_of_turn>")[0]

    return response.strip()

A few things worth noting here:

add_special_tokens=False — because I'm manually prepending <bos> in the prompt string. If you let the tokenizer add it automatically as well, you get a duplicate BOS token which confuses the model.

repetition_penalty=1.3 — without this, the model loops. I found this out the hard way when my first test response was 200 repetitions of "matched matched matched".

Decoding only new tokens — outputs[0][input_len:] strips the input tokens from the output before decoding. Otherwise you get the full prompt echoed back before the response.

The Four Agents

Each agent is a focused prompt sent to ask_gemma. The agents run sequentially, not in parallel — this keeps memory usage flat and avoids context window issues.

Agent 1 — Skills Analyzer

Takes the raw resume text and returns a structured output in a fixed format:

TECHNICAL SKILLS: Python, NLP, LangChain, ...
SOFT SKILLS: Communication, Teaching, ...
EXPERIENCE: 5 years
LEVEL: mid
DOMAINS: Artificial Intelligence, NLP, Education

I enforce the format in the prompt rather than post-processing with regex. Gemma 4 follows structured output instructions reliably when you give it an exact template to fill.

Agent 2 — Career Path Advisor

Takes the extracted skills string and returns three career paths with job titles, required additional skills, USD salary ranges, and a growth potential score out of 10.

Agent 3 — Learning Plan Designer

Takes the skills and target role and returns a 3-month plan broken down by month — foundation topics in month 1, intermediate topics in month 2, advanced topics and portfolio projects in month 3.

Agent 4 — Resume and ATS Analyst

Takes the resume text and target role and returns an ATS score out of 100, three strengths, three improvement areas, missing keywords, and a suggested rewrite for the professional summary.

The skills string extracted by Agent 1 is passed directly into Agents 2 and 3, creating a lightweight chain without needing LangChain or CrewAI overhead.

The Gradio Interface

I used Gradio instead of Streamlit for one reason: on Kaggle, app.launch(share=True) generates a public ngrok URL in a single line. No tunnel setup, no separate process.

The interface has two inputs — resume text and target role — and six output tabs, one per agent plus job matches and course recommendations.

with gr.Blocks(title="GuidanceOS") as app:
    with gr.Row():
        with gr.Column(scale=1):
            resume_input = gr.Textbox(label="Resume Text", lines=14)
            role_input   = gr.Textbox(label="Target Role")
            submit_btn   = gr.Button("Analyze My Profile", variant="primary")
        with gr.Column(scale=2):
            with gr.Tab("Skills Analysis"):
                skills_out = gr.Textbox(lines=10)
            # ... five more tabs

app.launch(share=True)

I added gr.Progress() to the main function so the UI shows which agent is running instead of just freezing. Each agent call takes 30-90 seconds on T4 — the progress bar makes it feel responsive.

End-to-End Flow

When a user clicks Analyze:

Resume text → Agent 1 → structured skills profile
Skills string → TF-IDF search → top 5 jobs from 123K LinkedIn postings
Skills string → TF-IDF search → top 5 courses from 6.6K Coursera courses
Skills string → Agent 2 → three career paths with salaries
Skills string + target role → Agent 3 → 3-month learning roadmap
Resume text + target role → Agent 4 → ATS score and improvements
All outputs → six Gradio tabs

Total time: 3-5 minutes on a T4 GPU. All computation on-device. Zero external API calls.

What I Would Do Differently

A few things I'd change with more time:

Structured JSON output from agents. Right now the agents return free-form text. Enforcing JSON output would make the results easier to display in a proper UI — cards instead of plain text boxes.

FAISS for course search. TF-IDF misses semantic similarity — "data analysis" and "analytics" are treated as different terms. Sentence embeddings with FAISS would improve course matching quality significantly.

Session persistence with SQLite. The current setup doesn't remember previous conversations. Adding a lightweight SQLite store would let users build on previous sessions.

SHAP explainability. I had planned to add a SHAP chart showing which skills drove each job recommendation using a Random Forest trained on the jobs dataset. It didn't make the deadline but the data pipeline supports it cleanly.

Running It Yourself

The full notebook is on Kaggle:
kaggle.com/code/abbasi110/guidanceos-gemma4-offline-career-advisor

Source code on GitHub:
github.com/soohanAbbasi/GuidanceOS

You need a Kaggle account to run it. Add the gemma-4-e4b-it model and both datasets, set the accelerator to GPU T4 x2, and run all cells in order. The Gradio URL prints in the last cell.

That's the full build. If you have questions about any part of it — the quantization setup, the prompt templates, or the TF-IDF indexing — leave a comment and I'll answer.