TL;DR. A public method called Model DNA — with a live tool — lets outsiders estimate whether an LLM was trained from scratch or derived from an open-weight base, using nothing but public artifacts: config.json, tokenizer.json, and embedding weights. This is a technical deep-dive that cites and reproduces that method in PyTorch, then analyzes where it holds up and where it breaks.
Key takeaways
- Three public signals — architecture config, tokenizer overlap, and embedding-space similarity via Linear CKA — combine to place a model on a lineage spectrum.
- One matching field is noise; five at once is a fingerprint. Provenance is a preponderance-of-evidence judgment, not a single test.
- The method's real strengths are reproducibility and rotation-invariant similarity; its real limits are the continued-pretraining gray zone, threshold sensitivity, and an embedding-only view.
- Fingerprinting reveals lineage, not intent. Building on an open-weight base is a legitimate, industry-standard practice; the output is a label, not an accusation.
Why this matters in 2026
Every few weeks a lab announces a "from-scratch, self-developed" foundation model. In mid-2026 those claims stopped being taken on faith. A Zhihu roundtable on the summer model wave and a thread with millions of views became the venue where "self-developed" claims were publicly stress-tested — and several were found more derivative than advertised (coverage).
Model DNA matters because it moved the argument from vibes to a reproducible procedure, and it has been run across major Korean foundation-model builders — among them LG, NAVER, Kakao, SKT, KT, NCSOFT, Upstage, and Motif. (This piece stays at the method level and assigns no verdict to any named company; per-model labels belong to the tool, not to a blog post.) What follows reproduces the procedure in PyTorch and evaluates it as a method, within the scope the source already made public.
The three signals (cited and reproduced)
The premise: no leaked internals required. Everything is read from a model's public files.
1. Architecture fingerprint — config.json
Compare the structural fields a lab chooses at design time: model_type, vocab_size, hidden_size, intermediate_size, num_hidden_layers, num_attention_heads, num_key_value_heads. Independently designed models rarely align on all of them at once.
import json
ARCH_FIELDS = [
"model_type", "vocab_size", "hidden_size", "intermediate_size",
"num_hidden_layers", "num_attention_heads", "num_key_value_heads",
"max_position_embeddings", "rope_theta",
]
def arch_match_count(cfg_a: dict, cfg_b: dict, fields=ARCH_FIELDS) -> int:
"""Number of structural fields that match simultaneously."""
return sum(1 for k in fields
if cfg_a.get(k) is not None and cfg_a.get(k) == cfg_b.get(k))
Reading rule (as the source puts it): a single coincidental field means nothing; five simultaneously is a fingerprint. How many to treat as a threshold depends on the diversity of your candidate base pool.
2. Tokenizer overlap — tokenizer.json
Two models trained truly independently almost never converge on the same vocabulary. Normalize shared tokens against the smaller vocabulary.
def tokenizer_overlap(vocab_a: dict, vocab_b: dict) -> float:
sa, sb = set(vocab_a), set(vocab_b)
return len(sa & sb) / min(len(sa), len(sb))
A supporting signal only — see Trap 2.
3. Embedding similarity — Linear CKA
The most robust signal compares representation geometry. Naive cosine comparison is fooled by rotation (Trap 1), so the method uses Linear CKA (Centered Kernel Alignment) — from Kornblith et al. (2019), Similarity of Neural Network Representations Revisited (ICML) — which is invariant to rotation, orthogonal transforms, and isotropic scaling.
For row-centered matrices X ∈ ℝ^{n×d1} and Y ∈ ℝ^{n×d2}:
CKA(X, Y) = ||Yᵀ X||²_F / ( ||Xᵀ X||_F · ||Yᵀ Y||_F )
Crucially, it is defined even when d1 ≠ d2, so models with different hidden sizes compare directly.
import torch
@torch.no_grad()
def linear_cka(X: torch.Tensor, Y: torch.Tensor) -> float:
# X:(n,d1), Y:(n,d2) — embeddings over the SAME token set (rows aligned)
X = X - X.mean(0, keepdim=True)
Y = Y - Y.mean(0, keepdim=True)
num = ((Y.t() @ X) ** 2).sum()
den = torch.sqrt(((X.t() @ X) ** 2).sum() * ((Y.t() @ Y) ** 2).sum())
return (num / den).clamp(0, 1).item()
Alignment is the catch. The two embedding matrices must index the same tokens. In practice you take the shared-token subset of the two tokenizers and gather those rows:
def aligned_embeddings(emb_a, vocab_a, emb_b, vocab_b):
shared = sorted(set(vocab_a) & set(vocab_b))
idx_a = torch.tensor([vocab_a[t] for t in shared])
idx_b = torch.tensor([vocab_b[t] for t in shared])
return emb_a[idx_a], emb_b[idx_b]
The genotype framework
The tool collapses the three signals into four labels — a clean way to read any result (Model Genome Korea):
| Genotype | Meaning |
|---|---|
| 🟢 Native | Self-designed architecture and from-scratch weights |
| 🔵 Adapted | Mostly original, one borrowed axis |
| 🟡 Mixed | Partial inheritance on both axes |
| 🔴 Ported | Exact foreign architecture match and inherited weights |
Two traps that produce confident wrong answers
Trap 1 — row-wise cosine similarity looks rigorous but isn't. It is fooled by rotation invariance: a genuinely derived model can be rotated to look "different," and a naive check clears it. That is exactly why CKA and config signals carry the weight.
Trap 2 — a shared tokenizer proves nothing alone. Tokenizer reuse is often a licensing or convenience decision. Treat overlap as supporting evidence, never a conclusion.
Deep analysis: how far can you trust it?
Strengths
- Reproducibility. All three signals compute from public artifacts in a few dozen lines. Claim and verification live on the same plane.
- Right invariance. Choosing Linear CKA is correct — it neutralizes the most common disguise (orthogonal transforms) that defeats cosine comparisons.
- Evidence fusion. Judging on the simultaneous agreement of three axes suppresses both false positives and false negatives.
Limits (must be acknowledged)
- Continued-pretraining gray zone. Embedding CKA identifies from-scratch training well but does not cleanly separate derivatives that keep a base's weights and train heavily on top. Here the verdict is probabilistic and config/tokenizer evidence dominates.
- Threshold sensitivity. "How many fields," "what CKA cutoff" depend on the candidate pool. Hard-coding constants makes conclusions wobble when the pool changes — which is why this write-up prescribes none.
- Embedding bias. Looking only at the embedding layer is cheap, but a model's "identity" also lives in mid and upper layers. A layer-wise CKA profile improves resolution in the gray zone.
- Alignment dependence. Few shared tokens (language- or domain-specific tokenizers) shrink the CKA sample and inflate variance. Report shared-token count alongside CKA.
Improvements worth adopting
- Extend single-layer embedding CKA to a layer-wise CKA curve (input → mid → output).
- Score against the entire candidate base pool and judge by relative rank, not an absolute cutoff.
- Report shared-token counts and bootstrap confidence intervals for statistical significance.
In short, Model DNA fuses the right signals under the right invariance — a solid starting point. It only avoids misjudgment when read as a spectrum with uncertainty, not a from-scratch/not binary.
What fingerprinting cannot tell you
- Lineage, not intent. It can show B shares structure with A; it cannot say whether that was disclosed, licensed, or hidden — ethics and paperwork, not linear algebra.
- "From scratch" is a spectrum, not a boolean. Data, init, architecture, and post-training each sit on a continuum of originality.
- Building on open weights is legitimate. The goal is transparency and accurate labeling, not accusation.
FAQ
Can you tell if an LLM was really trained from scratch?
Usually, with high probability. Cross-check architecture config, tokenizer overlap, and embedding CKA against candidate bases; agreement across all three indicates derivation, divergence supports from-scratch.
What is Model DNA / model provenance?
Estimating a model's origin — original vs. derived from an open-weight base — from public artifacts alone (config, tokenizer, weights), without training data or internal logs.
Is building on Llama, Qwen, or DeepSeek legitimate?
Yes. Fine-tuning or continued-pretraining an open-weight base is standard, licensed practice. Provenance tools report lineage, not misconduct.
How do you tell a fine-tuned model from a from-scratch one?
From-scratch models diverge on architecture and tokenizer and show low embedding CKA to any base. Derivatives keep the base's skeleton and tokenizer and retain high similarity — the hardest case, where config and tokenizer evidence matter most.
Why Linear CKA instead of cosine?
Because CKA is robust to rotation and scaling — the transforms a derived model uses to look "different."
Can models with different hidden sizes be compared?
Yes. CKA compares n×n Gram matrices, so d1 ≠ d2 is fine — as long as embedding rows are aligned to the same tokens.
Resources & related links
- Method under analysis: Model DNA — architecture & weight lineage
- Live tool (space): Model Genome Korea
- Announcement post: Architecture lineage of Korea's sovereign-AI models
- Academic basis: Kornblith, Norouzi, Lee, Hinton (2019), Similarity of Neural Network Representations Revisited, ICML.
- The debate: Zhihu roundtable "大模型卷一夏" · Zhihu thread · coverage
Provenance is becoming a norm, not a gotcha. The healthiest version is one where "we trained it from scratch" arrives with — or at least survives — the fingerprint. If you build models, publish the check yourself.
Top comments (0)