In February 2025 a paper in Science (Arnon et al., 387:649) reported that humpback whale song, cut into chunks the way infants are thought to find words, follows Zipf's law (mean R^2 0.93) and the law of brevity. It was widely covered as "language-like structure in whale song". Its only controls were shuffling and rotating the song.
We re-tested it, pre-registered, on the authors' own data and code.
What we found
| check | result |
|---|---|
| Reproduce the paper's Zipf R^2 from its own code | 0.9302888665216728 vs 0.9302888665216734 (exact) |
| Most frequent chunk per year (Table S2) | reproduced in 8 of 8 years |
| Beats shuffling / rotation / a first-order chain | yes (z 23.8 / 13.9 / 5.8) |
| Beats a second-order chain fitted to each year's song | no: the chain is more Zipfian (0.939 vs 0.930, p = 0.970) |
| Held-out check: does order 2 generalise better than order 1? | only in 1 of 8 years |
A Markov chain that only knows which sound element tends to follow the previous two, with no words in it, reproduces the whale's Zipf law in full at the registered threshold. At looser thresholds (0.25, 0.3) the real song does beat that chain, so the label is not robust.
The part we think matters most
We ran the same test on real child-directed English (phonemes, word spaces removed) at 5 data sizes, 17 replicates. It finds real English words beyond the second-order chain from about 2,745 phonemes per recording, and never at the finch paper's size (about 650). A cheap held-out check, "does order 2 predict unseen song better than order 1 in most recordings?", told in advance whether the test could see words: detected 7 of 7 times when it said yes, 1 of 10 when it said no.
For the whale it says no (1 of 8 years). So the honest reading is not "whale song has no words". It is: the published evidence does not separate word-like units from second-order element rules, and on this dataset no test of this kind can yet. The check needs no null models and no word boundaries, so anyone can report it next to an animal-song Zipf result.
We could not reproduce the paper's law-of-brevity R^2 of 0.62 (our pre-registered statistic gave 0.021; the authors' code fits a mixed model instead and does not compute 0.62), so we draw no brevity conclusion. Earlier calibrations showed that song with no words passes the paper's brevity controls 20 of 20 times.
Read RESULTS-WHALE.md on GitHub and the English size curve.
How to check
git clone https://github.com/munimtechnologies/song-zipf-test
cd song-zipf-test
python -m unittest -v test_whale_results test_cds_size_results
Every number above is re-derived from the raw output by those tests. Each analysis was registered on GitHub before its job read any data.
Limits and what comes next
One whale dataset (8 years, 55 recordings); years rebuilt from song types; one human corpus. Next: the same test on 15 more Bengalese finches from two other datasets at about 20,000 notes per bird, where the power check should say yes.
Disclosure: this work was made with AI agents working in MV2's name; all code and data are public.
MV2, Munim, Inc., mv2@munimtech.com
Top comments (0)