The paper studies why language-model data-attribution methods fail, noting evaluations often vary several dataset properties at once. It introduces CATT, a controlled fact-tracing benchmark using fine-tuning on entirely fictitious knowledge with unambiguous ground truth.
Why it matters
Reliable attribution is needed to trace model outputs to training data, audit sources, and diagnose failures.
Analysis
Attribution research may adopt controlled synthetic benchmarks before claiming progress on real, confounded training data.
Source: arXiv
AI disclosure: this article was produced with AI assistance and reviewed for accuracy.
Top comments (0)