DEV Community: Chauhan Balaji

I Built a Small LLM From Scratch — Here’s What I Finally Understood About Transformers

Chauhan Balaji — Wed, 21 Oct 2026 05:29:10 +0000

How a Tiny LLM Works: Building a Language Model From Scratch in PyTorch

Have you ever wondered what actually happens inside an LLM when you type:

"Hello, how are"

and it predicts:

"you"

We often hear words like Transformer, Attention, Tokens, RoPE, GQA, RMSNorm, and SwiGLU.

But what do these things actually do?

I recently worked through the implementation of a small language model from scratch, and I found that the easiest way to understand an LLM is to follow one piece of text through the entire model.

So in this post, we'll do exactly that.

By the end, you should understand the basic flow:

Text
 ↓
Tokens
 ↓
Embeddings
 ↓
Transformer
 ↓
Attention
 ↓
Feed Forward Network
 ↓
Prediction
 ↓
Next Token

Let's break it down step by step.

1. First: What Is a Language Model?

A language model learns to predict what comes next.

For example:

Input:
"The cat"

Prediction:
"sat"

Then:

"The cat sat"

Prediction:
"on"

Then:

"The cat sat on"

Prediction:
"the"

So at its simplest, a language model is doing:

Given the previous tokens, predict the next token.

That's the central idea behind autoregressive language models.

Our small model will learn this task using Shakespeare text.

2. Computers Don't Understand Text

Before the model can learn anything, we have a problem.

A neural network doesn't directly understand:

Hello

It needs numbers.

So we convert characters into IDs.

For example:

H → 20
e → 43
l → 50
l → 50
o → 53

So:

"Hello"

might become:

[20, 43, 50, 50, 53]

This is called tokenization.

In this particular project, we use a simple character-level tokenizer.

The basic idea is:

chars = sorted(set(text))

char_to_idx = {
    c: i for i, c in enumerate(chars)
}

idx_to_char = {
    i: c for i, c in enumerate(chars)
}

Then encoding becomes:

def encode(text):
    return [char_to_idx[c] for c in text]

And decoding:

def decode(ids):
    return "".join(idx_to_char[i] for i in ids)

So we now have:

Human language
     ↓
Characters
     ↓
Numbers

That's the first step.

3. How Does the Model Know What to Predict?

Suppose our text is:

"To be or not to be"

We can create training examples by shifting the sequence by one position.

Input:  To be o
Target: o be or

Why?

Because each position teaches the model what comes next.

For example:

Input seen        Target

T                  o
To                 [space]
To [space]         b
To [space] b       e

So the model repeatedly gets:

Previous characters → Next character

This is called next-token prediction.

The training data in our implementation creates these shifted input/target sequences automatically.

4. Numbers Go Into an Embedding Layer

We now have IDs such as:

[20, 43, 50, 50, 53]

But these IDs are just labels.

The number 50 isn't inherently more meaningful than 20.

So we convert each token ID into a vector.

This is called a token embedding.

Think of it like:

Token ID
   ↓
Embedding table
   ↓
Vector

For our model, each token becomes a vector of size 256.

So instead of:

we might have:

[0.12, -0.43, 0.87, ...]

with 256 values.

Now the neural network has a useful numerical representation to work with.

5. But How Does the Model Understand Relationships Between Tokens?

This is where the Transformer comes in.

A Transformer processes the token representations and allows tokens to interact with one another.

The most important component is:

Attention

Let's use a simple example:

"The dog chased the ball because it was moving."

What does "it" refer to?

A model needs to understand relationships between different words/tokens.

Attention helps the model decide:

Which other tokens should I pay attention to when processing this token?

6. Query, Key and Value

Attention uses three representations:

Q = Query
K = Key
V = Value

A simple way to think about them:

Query: What information am I looking for?
Key: What information do I contain?
Value: What information should I provide?

The model calculates how strongly a Query should attend to different Keys.

The basic attention calculation is:

Q × Kᵀ
   ↓
attention scores
   ↓
softmax
   ↓
attention weights
   ↓
weights × V
   ↓
attention output

In code, the core idea looks like:

scores = (q @ k.transpose(-2, -1)) * scale

weights = F.softmax(scores, dim=-1)

out = weights @ v

But there is an important problem.

During language generation, the model shouldn't be allowed to see future tokens.

7. Causal Attention: Don't Look Into the Future

Suppose the sequence is:

I love machine learning

When predicting "machine", the model shouldn't already know "learning".

So we use a causal mask.

It looks conceptually like this:

       T0 T1 T2 T3

T0     ✓  ✗  ✗  ✗
T1     ✓  ✓  ✗  ✗
T2     ✓  ✓  ✓  ✗
T3     ✓  ✓  ✓  ✓

✓ means the token can attend there.

✗ means it cannot.

So each token can see:

itself + everything before it

but never the future.

That's what makes next-token prediction possible.

The implementation applies a triangular causal mask before softmax.

8. Why Does the Model Need Positional Information?

Imagine these two sentences:

Dog bites man

and:

Man bites dog

The same words are present, but their positions are different.

A Transformer therefore needs positional information.

One modern approach is:

RoPE — Rotary Positional Embeddings

Instead of simply adding a position vector, RoPE rotates parts of the Query and Key representations.

The important intuition is:

Position 3 → Position 5
distance = 2

Position 7 → Position 9
distance = 2

The model can capture relationships based on relative position.

RoPE applies rotations to pairs of dimensions in the Query and Key vectors.

You don't need to memorize the rotation formula to understand the main idea:

RoPE gives attention information about token positions and relative distances.

9. Why Do We Need GQA?

Now we have another engineering problem.

Attention can become expensive, especially when models become large.

This is where Grouped Query Attention (GQA) can help.

Normally, we might have:

Q0 → K0, V0
Q1 → K1, V1
Q2 → K2, V2
Q3 → K3, V3
...

With GQA, multiple Query heads share Key and Value heads.

Our small model uses:

8 Query heads
2 KV heads

Conceptually:

Q0 ─┐
Q1 ─┤
Q2 ─┤──→ K0, V0
Q3 ─┘

Q4 ─┐
Q5 ─┤
Q6 ─┤──→ K1, V1
Q7 ─┘

So four Query heads share the same KV head.

Why is that useful?

Because fewer Key/Value heads means a smaller KV cache and less memory usage during inference.

The implementation expands the two KV heads to match the eight Query heads after applying RoPE.

The important lesson:

Modern LLM architecture isn't only about making models powerful. It's also about making them efficient.

10. What Happens After Attention?

Attention isn't the only part of a Transformer block.

After attention, we have a Feed-Forward Network (FFN).

In our model, we use:

SwiGLU

A simple older FFN might look like:

Input
  ↓
Linear
  ↓
ReLU
  ↓
Linear
  ↓
Output

SwiGLU uses a gating mechanism.

Conceptually:

              ┌──→ SiLU ──┐
Input ────────┤            × ──→ Output
              └──→ Up ─────┘

The code is essentially:

gate = F.silu(self.w_gate(x))
up = self.w_up(x)

output = self.w_down(gate * up)

The multiplication acts like a gate.

It allows the network to learn which information should pass through more strongly.

The implementation uses two parallel projections followed by element-wise multiplication and a final projection back to the model dimension.

11. Why RMSNorm?

You'll notice another component appearing before attention and the FFN:

RMSNorm

Neural networks can become difficult to train when activations become poorly scaled.

Normalization helps keep these values under control.

RMSNorm works by calculating the root mean square of the values and scaling them.

Conceptually:

Input
 ↓
Calculate RMS
 ↓
Normalize
 ↓
Learnable scale
 ↓
Output

A simplified implementation:

rms = torch.sqrt(
    x.pow(2).mean(dim=-1, keepdim=True) + eps
)

output = (x / rms) * weight

RMSNorm is simpler than LayerNorm because it doesn't center the values by subtracting the mean.

12. Residual Connections: Why Do We Add the Input Back?

Our Transformer block looks like:

Input
  │
  ├── RMSNorm
  ↓
Attention
  │
  ↓
+ ←──────── Input
  │
  ↓
  x'
  │
  ├── RMSNorm
  ↓
SwiGLU
  │
  ↓
+ ←──────── x'
  │
  ↓
Output

In code:

x = x + self.attention(
    self.attn_norm(x),
    rope_cos,
    rope_sin
)

x = x + self.ffn(
    self.ffn_norm(x)
)

The x + ... part is the residual connection.

Instead of forcing every layer to completely transform the representation, the layer can learn a useful change and add it to what was already there.

This pattern helps make deeper networks easier to train.

13. Now We Can Build the Whole Transformer

We can combine everything:

                Token IDs
                    ↓
              Token Embedding
                    ↓
          ┌─────────────────────┐
          │   Transformer       │
          │      Block           │
          │                     │
          │  RMSNorm             │
          │      ↓               │
          │  GQA + RoPE          │
          │      ↓               │
          │  Residual            │
          │      ↓               │
          │  RMSNorm             │
          │      ↓               │
          │  SwiGLU              │
          │      ↓               │
          │  Residual            │
          └─────────────────────┘
                    ↓
               × 4 Blocks
                    ↓
               Final RMSNorm
                    ↓
              Language Head
                    ↓
                 Logits
                    ↓
               Probabilities
                    ↓
              Next Character

Our implementation uses four Transformer blocks.

The model configuration includes:

Vocabulary:       65 characters
Embedding size:   256
Transformer blocks: 4
Query heads:      8
KV heads:         2
Context length:   256

The complete model connects token embeddings, Transformer blocks, final normalization and the language-model head.

14. How Does the Model Actually Learn?

Now we have a model.

But initially, it knows nothing.

So we train it.

The basic training loop is:

1. Get training data
        ↓
2. Forward pass
        ↓
3. Calculate loss
        ↓
4. Calculate gradients
        ↓
5. Update weights
        ↓
6. Repeat

In code:

xb, yb = get_batch(...)

logits, loss = model(xb, yb)

optimizer.zero_grad()

loss.backward()

optimizer.step()

Our training setup uses:

Batch size:     64
Learning rate:  3e-4
Optimizer:      AdamW
Training steps: 3000

Gradient clipping is also used:

torch.nn.utils.clip_grad_norm_(
    model.parameters(),
    max_norm=1.0
)

The purpose is to prevent very large gradients from destabilizing training.

15. What Is the Loss?

The model outputs scores called logits.

For our character vocabulary, the model produces a score for each possible character.

For example:

a → 1.2
b → -0.8
c → 2.4
d → 0.1
...

We convert these scores into probabilities.

Then we compare the predicted probability distribution with the correct next character.

This is where cross-entropy loss is used.

Simply put:

If the model gives a high probability to the correct answer, the loss is smaller.

If it gives a low probability to the correct answer, the loss is larger.

The model then uses the loss to calculate gradients and update its weights.

16. Training vs Validation

A model can memorize its training data.

So we don't only look at training loss.

We also evaluate on validation data.

Think of it like studying for an exam:

Training data
= Questions you practiced

Validation data
= New questions you haven't practiced

If training loss keeps decreasing but validation loss stops improving or gets worse, that can indicate overfitting.

During evaluation, we disable dropout and don't calculate gradients:

@torch.no_grad()
def estimate_loss():
    model.eval()
    ...
    model.train()

The implementation evaluates both training and validation splits to monitor the loss.

17. How Does Text Generation Work?

After training, we can finally generate text.

Suppose we give the model:

ROMEO:

The model predicts the next character.

Maybe:

ROMEO: W

Then we feed the new sequence back into the model:

ROMEO: W

It predicts another character:

ROMEO: Wh

Then:

ROMEO: What

And so on.

So generation is:

Prompt
 ↓
Predict next token
 ↓
Add token to prompt
 ↓
Predict again
 ↓
Add token
 ↓
Repeat

This is called autoregressive generation.

The implementation repeatedly takes the last-position logits, converts them to probabilities, samples the next token, and appends it to the sequence.

18. What Is Temperature?

Suppose the model predicts:

A → 70%
B → 20%
C → 8%
D → 2%

Should we always choose A?

Not necessarily.

Temperature controls how sharp or random the probability distribution becomes.

Low temperature

More predictable

Higher temperature

More varied

For example:

0.3 → conservative
0.8 → balanced
1.5 → more creative

So temperature doesn't change what the model learned.

It changes how we sample from what the model learned.

19. The Whole Process in One Picture

If all of this feels like a lot, remember just this:

                 TEXT
                   ↓
              Tokenization
                   ↓
               Token IDs
                   ↓
             Embeddings
                   ↓
        ┌───────────────────┐
        │ Transformer Block │
        │                   │
        │ RMSNorm           │
        │ ↓                 │
        │ Attention         │
        │ ↓                 │
        │ Residual          │
        │ ↓                 │
        │ RMSNorm           │
        │ ↓                 │
        │ SwiGLU            │
        │ ↓                 │
        │ Residual          │
        └───────────────────┘
                   ↓
             More Blocks
                   ↓
             Final Norm
                   ↓
                Logits
                   ↓
             Probabilities
                   ↓
          Predict Next Token
                   ↓
          Add Token to Input
                   ↓
                Repeat

That's the basic idea behind an autoregressive Transformer language model.

20. What I Think Is Most Important to Remember

If you're learning Transformers for the first time, don't try to memorize every equation immediately.

Start with these questions:

What is tokenization?

Converts text into numbers.

What are embeddings?

Turns token IDs into useful vectors.

What is attention?

Allows tokens to use information from other tokens.

What is causal masking?

Prevents the model from looking at future tokens.

What is RoPE?

Provides positional information through rotations of Query and Key representations.

What is GQA?

Allows multiple Query heads to share Key/Value heads, reducing KV-cache size.

What is SwiGLU?

A gated feed-forward network used inside the Transformer block.

What is RMSNorm?

Normalizes activations using their root mean square.

What is a residual connection?

Adds the previous representation back to the transformed representation.

What is training?

Adjusting the model's weights to reduce prediction error.

What is generation?

Repeatedly predicting and adding the next token.

If you understand these ten ideas, you already have a much stronger mental model of a Transformer.

21. The Most Useful Lesson I Got From Building It

Before working through this implementation, I knew many of these terms individually.

But knowing:

RoPE
GQA
RMSNorm
SwiGLU
Attention

isn't enough.

The important part is understanding how they work together.

A language model isn't one magical algorithm.

It's a collection of components, each solving a particular problem:

Tokenization
→ How do we represent text?

Embeddings
→ How do we represent tokens as vectors?

Attention
→ How do tokens interact?

Causal masking
→ How do we prevent cheating?

RoPE
→ How does the model know positions?

GQA
→ How can attention become more memory efficient?

SwiGLU
→ How does the network transform information?

RMSNorm
→ How do we keep activations stable?

Residual connections
→ How do we make deep networks easier to train?

Training
→ How does the model learn?

Generation
→ How does the model produce text?

Once you look at it this way, the architecture becomes much easier to understand.

Conclusion

The easiest way I found to understand an LLM was not to start with:

“Let's learn all the Transformer equations.”

Instead, I started with a much simpler question:

How does "Hello" eventually become the next predicted token?

Then I followed the data:

Hello
 ↓
Token IDs
 ↓
Embeddings
 ↓
Attention
 ↓
RoPE
 ↓
GQA
 ↓
SwiGLU
 ↓
Transformer blocks
 ↓
Logits
 ↓
Probability
 ↓
Next token

That made the architecture much less mysterious.

I'm still learning, but building a small model forced me to stop treating LLMs as a black box and start thinking about data flow, tensor shapes, architecture, training, and inference.

And that's probably the most useful advice I can give to someone learning AI:

Don't just learn what a component is. Learn what problem it solves and follow the data through it.

If you're learning Transformers too, I hope this makes the topic a little easier to understand.

Next step: take one of these components — attention, RoPE, GQA, or SwiGLU — and implement it yourself. That's where the concepts really start to stick.

To Rephrase or Not to Rephrase: Will AI Change Its Answer If I Ask Differently?

Chauhan Balaji — Fri, 09 Oct 2026 05:42:51 +0000

This is a submission for the Kaggle Benchmarking Challenge

To Rephrase or Not to Rephrase: Will AI Change Its Answer If I Ask Differently?

Kaggle Benchmarking Challenge submission

Benchmark & Leaderboard: Paraphrase Stability Benchmark
Task link: Paraphrase Stability Task
Notebook & per-cell records: Public notebook

What I Benchmarked

The idea is simple. Most leaderboards ask an AI a question once, grade it, and move on. But real users never prompt the same way twice. So I took 32 multiple-choice questions and asked each one in 5 semantically-equivalent phrasings (canonical, reworded, terse, verbose, options-shuffled), twice each.

That is 32 items x 5 phrasings x 2 repeats = 320 cells per model. Across 28 scored models that is 8,960 scored cells — plus 2 runs that errored mid-grid, which I report instead of hiding.

For example, the canonical version of one quantitative item looks like this:

A jacket is on sale for $80 after a 20% discount. What was the original price?

A. $96
B. $100
C. $104
D. $120

Select the correct answer and reply with just its letter (A, B, C, or D).

The terse version of the same cell changes only the instruction line:

A jacket is on sale for $80 after a 20% discount. What was the original price?

A. $96
B. $100
C. $104
D. $120

Answer:

A developer will immediately say $100. But will the model still say $100 when the stem is reworded, when the prompt is stripped to Answer:, or when the options are shuffled into a different order? And more importantly, what happens when the question is less obvious?

Scoring is deterministic — no LLM judge anywhere. A fixed parser extracts the chosen letter (bare letter, answer is X, or option-text fallback); unparsable replies count as wrong and are reported via parse-rate, never silently dropped. The headline metric is Stability-Adjusted Accuracy (SAA): an item scores 1 only if the model answered it correctly under all ten framings.

The Questions

Logic (8) — syllogisms and paradoxes, including the knights-and-knaves puzzle where A says: "We are both knaves."
Quantitative (8) — multi-step word problems like the bat-and-ball trap and percentage-discount reversals.
Knowledge (8) — verifiable facts (chemical symbols, capitals, dates).
Attention traps (8) — "How many animals did Moses take on the ark?" (It was Noah.)

Every answer key is recomputed programmatically before any model is called (bat-and-ball algebra, 1.5 * 0.5, day-of-week arithmetic). If the key mismatches, the harness refuses to run.

Models Tested

Picking the lineup was straightforward: I used Kaggle's Evaluate More Models to span the whole field.

First, the frontier flagships everyone benchmarks: GPT-6 Sol, GPT-5.6 Sol, Claude Opus 5.5.

Then the fast, cheap models I actually use day-to-day: Gemini 3.7 Flash, Gemini 3.1 Flash-Lite Preview, GPT-6 Luna. From my experience these are efficient — but does efficiency cost consistency?

Then open-weights and mid-tier models mostly out of curiosity: GLM-5, Qwen 3 Coder 480B, Gemma 4 26B A4B, Qwen 3 Next 80B Instruct.

I also tried to include the new Grok models, but I immediately hit coverage failures with Grok 4.20 (Non-Reasoning) and Grok 4.6, so they are not in the scored table (the harness's >=90% coverage assertion failed loudly, exactly as designed). Grok 4.20 Reasoning did finish — and we will talk about it.

Findings

First, the scoreboard. Only 4 of 28 models survived all ten framings on every item:

Model	SAA
GPT-5.6 Sol	1.00
GPT-6 Sol	1.00
Claude Opus 5.5	1.00
GLM-5	1.00
GPT-6 Luna	0.97
Gemini 3 Flash Preview	0.97
Claude Sonnet 4.6	0.97
Gemini 3.1 Flash-Lite Preview	0.97
Gemma 4 26B A4B	0.97
GPT-6 Astra	0.97
Claude Opus 4.8	0.97
GPT-6.1 Sol	0.96
Qwen 3 Coder 480B	0.94
Gemini 3.8 Flash	0.94
Gemini 3.7 Flash	0.94
Gemini 3.6 Flash	0.94
Claude Sonnet 5	0.93
Gemini 2.5 Pro	0.93
Qwen 3 Next 80B Instruct	0.91
Claude Opus 5	0.91
Gemini 3.5 Flash	0.90
Gemini 3.1 Pro Preview	0.90
GPT-5.4	0.88
Gemini 2.5 Flash	0.87
Claude Sonnet 5.5	0.85
Gemini 3.5 Flash-Lite	0.84
GPT-5.4 mini	0.72
Grok 4.20 Reasoning	0.66

Most models score ~99% on raw accuracy. The gap between 99% accuracy and 0.85 SAA is the entire point of this benchmark: the model knows the answer, then loses it the moment you ask differently.

When we look closer, the flips concentrate on the knights-and-knaves item (L3) and the 20%-discount item (Q2). The logic never changes between phrasings — only the words do:

canonical:  "On an island, knights always tell the truth and knaves always
             lie. You meet A, who says: 'We are both knaves.' What is A?"
reworded:   "Knights on this island always tell the truth; knaves always
             lie. A tells you: 'Both of us are knaves.' What is A?"

Same puzzle, same gold answer — and yet several models pick a different option under one of the two phrasings. A model that answers B, then C, then B again across repeats is sampling an answer, not knowing one. And to be honest, I am glad most models fell into the paradox. A small win for the author, who can still trick AI sometimes.

A reference run in detail: Gemini 3.7 Flash

Metric	Value
Raw accuracy	0.991 [95% CI 0.975, 1.000]
SAA	0.938 [95% CI 0.844, 1.000]
Flipped items	L3, Q2 (2 of 32)
canonical / reworded / verbose / shuffled	100% / 100% / 100% / 100%
terse	95.3%
Run instability (identical prompt, temp 0)	0.6%
Prompt instability (across phrasings)	4.7%
Position bias (shuffled, by gold letter)	A=1.00 B=1.00 C=1.00 D=1.00

Two things stand out. First, stripping the prompt to a bare Answer: hurt more than any rewording — the model needs a little conversational room to behave. Second:

The phrasing moves models far more than the backend moves itself.

Run instability (same prompt twice, temperature 0) was 0.6%; prompt instability was 4.7%. That gap is the whole argument in one line: phrasing sensitivity is a real model behavior, not API noise.

The "Newer Is Less Stable" Trap

This is the finding I did not expect. Within model families, stability degrades as generations get newer:

Claude Sonnet 4.6 -> 5 -> 5.5: 0.97 -> 0.93 -> 0.85
Gemini Flash-Lite 3.1 -> 3.5: 0.97 -> 0.84

We usually assume newer is strictly better. But as models are optimized for speed and single-shot benchmark accuracy, consistency under rephrasing can quietly regress. A leaderboard that asks each question once will never see this.

Two more surprises:

Open weights can be perfectly stable. GLM-5 ties GPT-6 Sol and Claude Opus 5.5 at 1.00 — stability is trained in, not bought.
Reasoning mode did not save Grok. Grok 4.20 Reasoning finished last at 0.66, and its non-reasoning sibling errored out entirely — so the clean reasoning-vs-non-reasoning ablation remains incomplete, and I would rather say so than paper over it.

What I Learned

When I designed the experiment, I worried the 32 questions would be too easy for today's models. Raw accuracy proved me right; SAA proved me wrong. Yes, I could improve the benchmark — 32 items is a sample, not a census, and the CIs are wide on purpose (cluster bootstrap over items, 2000 resamples). But even frontier models flip on self-referential logic the moment the wording shifts. AI can absolutely help with hard questions — but if a harmless rephrase changes the answer, blindly trusting the first response in production is a risk. Consistency is a capability too.

Methodology & Limitations

No LLM judge. Fixed letter-extraction parser; unparsable = wrong, reported via parse-rate.
CIs are cluster bootstraps over the 32 items — with 32 items, expect wide intervals; that honesty is the point.
SAA excludes incomplete items and says so, so flaky APIs cannot inflate scores.
Errored runs are reported, not dropped (Grok 4.20 Non-Reasoning, Grok 4.6).
MC-only, English-only. Stability under free-form generation is strictly harder; this is the controlled version of the question.
Don't rank two models on a 0.03 gap when their CIs overlap.

Every run keeps its full per-cell transcript, so any flip quoted here is auditable — no judge, no vibes, just letter extraction.

I Benchmarked 4 Frontier LLMs on Catching ML's "Silent Killers" — DeepSeek-R1 Missed the Most Basic Bug

Chauhan Balaji — Sat, 03 Oct 2026 05:21:13 +0000

Most public AI leaderboards test if a model can write code or pass a syntax check. But in real-world Machine Learning, the most dangerous code isn't syntactically broken—it's methodologically flawed. It passes unit tests, shows a green dashboard, and then dies silently in production.
For the Kaggle Benchmarking Challenge, I built "The Silent Killer": an adversarial evaluation harness designed to test if frontier LLMs can actually audit ML pipelines and catch fatal data science mistakes.

What task(s) did you run? I created a benchmark that feeds realistic, broken Heart Disease prediction pipelines to LLMs and asks them to identify the fatal flaw. I tested three specific "Silent Killers": Data Leakage: Fitting a StandardScaler on the entire dataset before calling train_test_split (the scaler "sees" the test set, biasing the score). The Wrong Metric: Using accuracy_score on a screening cohort that is 95% healthy and 5% sick (a model that just guesses "healthy" every time gets 95% accuracy). Target Leakage: Using number_of_cardiology_visits as a predictor (a proxy feature that only exists after a patient is already diagnosed). The Creative Approach (Dynamic Judge Rubric): The hardest part of benchmarking LLMs is that they give plausible-sounding, generic advice to get partial credit. A static rubric fails because models will just list every ML best practice they know. To solve this, I built a Dynamic Judge Rubric with a "No Misdiagnosis" Guard. The Judge LLM's grading criteria are assembled at runtime based on the specific bug in that row. If the model is reviewing the Data Leakage pipeline, but it complains about Class Imbalance instead, the Judge explicitly fails it.

# Criterion 3: distractor guard -- the anti-"pattern match" check
if rubric:
    criteria.append(
        "The response must NOT misdiagnose the flaw. It fails this check if it presents "
        "any of the following as THE fatal flaw instead of the flaw in criterion 1: "
        + "; ".join(rubric["distractors"])
        + ". Strictness: briefly listing such issues as secondary or minor observations is "
        "acceptable, but only if criterion 1 was satisfied."
    )

This separates true methodological comprehension from simple pattern-matching.

Which models did you run it against?
I used the Kaggle Model Proxy to test my harness against four frontier giants, chosen for their strong reasoning and coding capabilities:
Gemini 3.7 Flash (Fast, highly capable baseline)
Claude Sonnet 4.5 (Anthropic's flagship coding/reasoning model)
Grok 4.20 Reasoning (xAI's deep reasoning model)
DeepSeek-R1 (Famous for its deep, multi-step chain-of-thought reasoning)
What are the main insights?
The results were shocking. DeepSeek-R1, a model famous for its deep reasoning, completely missed the most fundamental ML bug of all time.
| Model | Data Leakage | Wrong Metric | Target Leakage | Overall Score |
|:------|:------------:|:------------:|:--------------:|:------------:|
| Gemini 3.7 Flash | ✅ Caught | ✅ Caught | ✅ Caught | 100% |
| Claude Sonnet 4.5 | ✅ Caught | ✅ Caught | ✅ Caught | 100% |
| Grok 4.20 Reasoning | ✅ Caught | ✅ Caught | ✅ Caught | 100% |
| DeepSeek-R1 | ❌ MISSED | ✅ Caught | ✅ Caught | 67% |
Where can we see it?
You can view the full methodology, fork the code, and add your own models to the live leaderboard via the Kaggle Model Proxy here:
👉 https://www.kaggle.com/code/chauhanbalaji/the-silent-killer-ml-data-leakage-metric-detect
Evaluating AI isn't about keyword matching. It's about designing adversarial, dynamic tests that can't be gamed. What's the worst "silent killer" bug you've seen an AI write? Let me know in the comments!