TL;DR
A leaderboard number is only as trustworthy as the gap between what a model trained on and what it was tested on. When test examples (or near-duplicates of them) leak into pretraining or fine-tuning data, the model memorizes answers instead of generalizing, and the reported score climbs for the wrong reason. This is benchmark contamination, also called train/test overlap or data leakage. It is common, often accidental, and frequently invisible in a self-reported number. This post explains why contamination inflates scores, three detection methods you can run yourself (n-gram overlap, canary strings, membership inference), a runnable Python n-gram checker, and a short checklist you can apply before you trust any benchmark claim.
What is benchmark contamination and why does it inflate scores?
Modern models are trained on web-scale corpora scraped from the open internet. Public benchmarks live on that same internet: on GitHub, in papers, on leaderboard repos, in blog posts that quote questions verbatim, and in Q&A sites where people discuss the exact items. When a benchmark's test questions and answers end up in the training mix, evaluation stops measuring reasoning and starts measuring recall.
The mechanism is simple. A benchmark is supposed to be a held-out sample. If the test examples were in the training data, the model can reproduce the answer from memory. Memorization looks exactly like competence on a single scored run, but it does not transfer to genuinely new inputs. The score goes up; the underlying capability does not.
Contamination comes in degrees. Verbatim contamination means the exact test string appears in training. Near-duplicate contamination means a paraphrase, a translated copy, or a reformatted version appears. Label leakage means the answer key or solution is present even if the question wording differs. Even partial exposure helps a model disproportionately on the memorized slice, which is enough to move a leaderboard when margins are a point or two.
Why can self-reported leaderboard scores be inflated?
Self-reported scores carry three structural risks that have nothing to do with bad faith.
First, the submitter controls the evaluation environment: the prompt template, the decoding settings, the number of few-shot examples, and sometimes the subset of items. None of those choices are contamination, but they compound with it.
Second, nobody fully audits the training corpus. For most open releases the pretraining data is described at a high level, not published item by item. A team can honestly say they did not intend to train on a benchmark while still having ingested it through a scraped mirror. Intent does not change the outcome.
Third, benchmarks age. A dataset released three years ago has had three years to be copied, quoted, reformatted, and re-uploaded. The older and more popular a benchmark is, the more likely its items are somewhere in a modern crawl. This is why a model can post a record score on a classic benchmark and a mediocre score on a freshly built private one.
None of this means every high score is fake. It means a score reported without a contamination check is an unverified claim, and the burden of evidence sits with whoever is pointing at the number.
How do you detect train/test overlap? Three methods
1. N-gram overlap
The most direct test: take each benchmark item, slide a window of n consecutive tokens or words across it, and check whether those n-grams appear in the training corpus (or in a proxy for it). High overlap at n of 8, 13, or 50 is strong evidence that the item, or a chunk of it, was seen during training. This is the method used by several major lab technical reports, which typically flag an item as contaminated when a sufficiently long n-gram matches. The strength of n-gram overlap is that it is cheap, interpretable, and does not need model internals. Its weakness is that it misses paraphrases and translations, so it gives a lower bound on contamination, never an upper bound.
2. Canary strings
A canary is a unique, random, unlikely-to-occur-naturally identifier that benchmark authors embed in their dataset specifically so they can later test whether it leaked. If a model can reproduce or recognize the canary GUID, the dataset was in its training data, full stop. BIG-bench popularized this with an embedded canary GUID. The limitation is that canaries only work if the benchmark author planted one and the dataset was consumed with the canary intact; stripped or reformatted copies defeat it. Still, when a canary test fires, it is close to conclusive.
3. Membership inference and behavioral tests
When you cannot inspect the training data at all, you probe the model's behavior. Membership inference asks whether the model treats a specific example as something it has seen before. Practical variants include: comparing perplexity on benchmark items versus closely matched held-out items (memorized items often have suspiciously low loss), the guided-prompting trick (does the model complete a test item far better when primed with the dataset name?), and option-order sensitivity (a model that memorized a multiple-choice answer is unusually robust to the correct answer's position while a reasoning model is not). These tests are noisier than n-gram or canary checks and need careful controls, but they are the only options for closed training data.
A runnable n-gram overlap check
Here is a compact, dependency-free n-gram overlap checker. Give it your benchmark items and any text you suspect may overlap with training (a crawl shard, a scraped page, a quoted dataset). It reports the fraction of each item's n-grams that also appear in the reference text.
from collections import Counter
def ngrams(text, n=13):
"""Word-level n-grams, lowercased and whitespace-normalized."""
toks = text.lower().split()
return [tuple(toks[i:i + n]) for i in range(len(toks) - n + 1)]
def overlap_score(item, reference_ngrams, n=13):
"""Fraction of the item's n-grams that appear in the reference set."""
item_grams = ngrams(item, n)
if not item_grams:
return 0.0
hits = sum(1 for g in item_grams if g in reference_ngrams)
return hits / len(item_grams)
def build_reference(corpus_texts, n=13):
"""Precompute the n-gram set for everything you can see of training data."""
ref = set()
for doc in corpus_texts:
ref.update(ngrams(doc, n))
return ref
if __name__ == "__main__":
# Stand-in for a crawl shard / scraped page you suspect overlaps training.
training_proxy = [
"the capital of the fictional country of zubrowka is lutz "
"and its currency is the klubeck used since the year 1932",
]
benchmark_items = [
"The capital of the fictional country of Zubrowka is Lutz "
"and its currency is the Klubeck used since the year 1932", # leaked
"Explain why a positive confidence interval overlap means "
"two leaderboard scores may not be distinguishable at all", # clean
]
N = 13
ref = build_reference(training_proxy, n=N)
for i, item in enumerate(benchmark_items):
s = overlap_score(item, ref, n=N)
flag = "CONTAMINATED" if s > 0.5 else "clean"
print(f"item {i}: {s:.0%} {N}-gram overlap -> {flag}")
Expected output:
item 0: 100% 13-gram overlap -> CONTAMINATED
item 1: 0% 13-gram overlap -> clean
Two practical notes. First, choose n deliberately: small n (3 to 5) produces false positives on common phrasing, large n (20 or more) misses chopped-up leaks, and 8 to 13 is a reasonable default for prose. Second, this is a lower bound. A 0 percent score proves only that there was no verbatim long-span match against the text you checked; paraphrase and translation slip right past it, which is why you pair it with behavioral tests.
A practical contamination checklist
Run this before you cite or compare any benchmark number.
- Is the benchmark newer than the model's data cutoff? A test built after training data was frozen cannot have leaked. Prefer recent or private benchmarks for headline comparisons.
- Did the authors plant a canary, and did the reporter check it? A passed canary test is cheap evidence and should be mentioned.
- Was an n-gram overlap report published against the training corpus (or a representative sample)? If the training data is open, this should exist. If it is not, note that the overlap is simply unknown.
- Is there a gap between the model's score on the popular benchmark and on a fresh, structurally similar one? A large drop on the new test is a classic contamination signature.
- Are decoding settings, prompt template, and few-shot count disclosed and held constant across the models being compared? Contamination aside, undisclosed harness choices make numbers incomparable.
- Is the score reported with uncertainty, and is the margin over the next model larger than that uncertainty? A contaminated slice often shows up as a margin that vanishes under resampling.
- Who ran the evaluation? Self-reported with no third-party reproduction is an unverified claim, not a result. Independent reproduction on held-out data is the gold standard.
If a claim fails several of these, the right posture is not "the model is cheating." It is "this number is unverified, and here is specifically what would make it trustworthy."
FAQ
Is benchmark contamination always intentional?
No, and usually it is not. The most common path is accidental ingestion through a web crawl, since benchmarks live on the same internet that pretraining corpora are scraped from. Accidental contamination inflates scores exactly as much as deliberate contamination, which is why detection matters more than assigning blame.
Does a high n-gram overlap score prove a model will fail on real tasks?
It proves the benchmark number is unreliable for that model, not that the model is incapable. The correct response is to re-evaluate on a clean, uncontaminated test and trust that number instead. Contamination invalidates a measurement; it does not by itself measure capability.
Why not just keep all benchmarks private?
Private benchmarks resist contamination but sacrifice reproducibility and community trust, since nobody else can inspect the items or rerun the evaluation. The common compromise is a public benchmark with a planted canary, a held-out private slice, and periodic refreshes so the test can be rebuilt once the old version has saturated the internet.
What n should I use for an overlap check?
For natural-language items, 8 to 13 words is a reasonable default. Go smaller and you flag ordinary phrasing as a match; go much larger and you miss leaks that were reformatted or chopped into pieces. Report the n you used, because the overlap fraction is meaningless without it.
Can I detect contamination without access to the training data?
Yes, but only with behavioral tests: perplexity gaps between benchmark items and matched held-out items, guided-prompting sensitivity, and answer-position robustness. These are noisier than n-gram or canary checks, so treat them as evidence that accumulates rather than a single decisive test.
Further reading
- When a 0.4-Point Lead Means Nothing: Reading Open-Model Leaderboard Margins Like a Statistician
- How Hugging Face Official Benchmark Leaderboards Actually Work: .eval_results YAML, the base_model Filter, and the 30% the Default View Hides
Top comments (3)
Vấn đề n-gram overlap thực sự là một cơn ác mộng khi đánh giá các mô hình ngôn ngữ lớn hiện nay. Mình từng gặp trường hợp một model đạt điểm cực cao trên MMLU nhưng khi chạy thử các task suy luận logic thực tế lại thất bại thảm hại, hóa ra dữ liệu benchmark đã bị leak vào tập training từ lâu. Việc sử dụng canary strings là một hướng đi rất đúng đắn để kiểm soát tình trạng này. Một kinh nghiệm nhỏ là khi làm membership testing, thay vì chỉ kiểm tra sự trùng khớp nguyên văn, bạn nên thử thay đổi cấu trúc câu hoặc dùng từ đồng nghĩa để xem mô hình có thực sự hiểu ngữ cảnh hay chỉ đang học thuộc lòng các mẫu câu có sẵn trong tập test.
Checklist item 4 (gap against a fresh benchmark) needs a second model or a second benchmark, and often you have neither. A cheaper version uses the n-gram checker you already wrote: split the benchmark into flagged and unflagged items and report accuracy on each, with the difference.
The catch is sample size. If 5% of 1,000 items are flagged, that is 50 items, and at 50% accuracy the 95% interval on the flagged group alone is about plus or minus 14 points. A 5-point "contamination effect" there is indistinguishable from noise, so the report should carry the interval or the test is only decoration.
And because the n-gram check is a lower bound (as you say), the unflagged group still contains paraphrased leaks. That biases the gap toward zero, so a small gap is weak evidence of a clean benchmark, while a large one is fairly strong evidence of contamination. Would you add a paraphrased-twin variant (same item, reworded) to the toolkit as a behavioural test that doesn't depend on seeing the corpus?
Strong writeup. There is a fourth category the checklist does not catch: leakage that survives a perfectly clean split, because the identifier lives in the representation rather than in the training overlap.
Concrete case, with numbers. Lin, Wu, Jung (arXiv:2606.06647, "The Identity Trap in EEG Foundation Models", Jun 2026) audit three pretrained EEG models. Under strict subject-disjoint cross-validation a high score is ambiguous: it can be a real clinical marker, or subject identity that correlates with the label. They diagnose it on the frozen representation, before fine-tuning. Frozen subject-variance is 13-89x a random null in 12/12 model x dataset pairs, and fine-tuning makes it worse (+10 to +63 pp). The dominance sits on a removable linear axis: erase it, and label decoding improves only where the label varies within subject (+6 to +12 pp). Aperiodic 1/f is one carrier (removing it drops the subject probe 9-19 pp on two of the three models). Their summary line: subject-disjoint cross-validation is necessary but not sufficient.
The lesson for your checklist: a clean split rules out the item leaking into training, but it does not rule out the model keying on an identifier the split kept intact. If your group id (speaker, subject, session, template) is confounded with your label, the number can be precise and wrong at the same time. Cheap fourth check: after the split, probe the frozen features for the group id, then measure how much label accuracy survives erasing that direction.
I keep a sourced set of cases where precision fails as validity rather than resolution here: dev.to/nuncius-ghost/chasing-a-num...