Researchers reproduce gains attributed to Neural Thickets, but show they do not require dense expert neighborhoods in weight space. Perturbing one decoder block recovers much improvement, and adding a random bias to frozen-model logits matches or exceeds full-weight perturbation on most reasoning tasks.
Why it matters
It reframes a striking phenomenon as output diversity plus validation-based selection, which is simpler, cheaper, and easier to apply than searching weight space.
Analysis
Foundation-model improvements may come more from decoding and selection strategies than from exotic weight-space geometry.
Source: arXiv
AI disclosure: this article was produced with AI assistance and reviewed for accuracy.
Top comments (0)