TL;DR: Most “computer‑based” author attribution you see on the internet is junk. Average word length? TF‑IDF + cosine similarity? Please. There’s a method that actually works – it’s called Delta, it’s 20 years old, and you can run it with a few clicks in R. Let me show you why the naive approaches fail and how real stylometry is done.
The problem with “obvious” methods
We’ve all seen it: someone grabs a few texts, computes average word distribution or most frequent words, draws a pretty line chart, and declares they can tell who wrote what.
Spoiler: they can’t.
A famous example – in 2018, a data scientist tried to identify the anonymous New York Times op‑ed “I Am Part of the Resistance Inside the Trump Administration”. He used TF‑IDF on Trump administration members’ tweets, computed cosine similarity, and concluded… the most likely author was Trump himself (or someone from State Department). Not very convicing.
That’s not science – that’s cargo‑cult data analysis.
Even worse, a 2023 book Python for Hackers dedicates a whole chapter to stylometry, yet it uses average word length and plots line charts copied from 19th‑century geophysicist Thomas Mendenhall. Yes, the same Mendenhall who thought word‑length distributions would work like spectral analysis of chemical elements.
Spoiler: they don’t. Grzybek (2007) already buried that idea.
So if those methods are garbage, what actually works?
Enter John Burrows and the Delta method (2002)
Burrows proposed a surprisingly simple distance measure between texts – Delta. It uses only the 100–500 most frequent words (mostly function words like “the”, “and”, “of” – not content words). That’s genius, because it ignores topic and focuses on style.
Here’s the intuition:
- For each text, compute the relative frequency of each frequent word.
- Convert to z‑scores (how many standard deviations above/below the corpus average).
- Delta between two texts = mean absolute difference of their z‑scores across all words.
Small Delta → stylistically similar → probably same author.
No neural networks. No thousands of parameters. Just robust statistics.
And it works. On English, Russian, Arabic, Ancient Greek, even Chinese (yes, despite no clear word boundaries). Nobody knows exactly why – but it has been validated on hundreds of test cases.
How to run it without writing a single line of R
There’s an R package called stylo (Kestemont et al., 2016). It has a point‑and‑click GUI.
install.packages('stylo')
library(stylo)
stylo() # GUI opens
You put your texts in a folder named corpus. Name files like author_title.txt – stylo automatically colors by author in the output.
It produces a dendrogram (cluster tree). If the method works, texts by the same author cluster together.
I tested 24 Russian prose texts (Lermontov, Chekhov, Leskov, Gogol). One unknown text (“???”) was supposed to be by Leskov. Result: 23 out of 24 correctly clustered. Only one early Chekhov story drifted to Lermontov – because Delta is sensitive to author’s evolution over time.
The unknown text landed right with Leskov.
Not perfect – but far better than random guessing.
Case studies that (sort of) worked
Rowling / Galbraith (2013)
When J.K. Rowling published The Cuckoo’s Calling as “Robert Galbraith”, stylometrists quickly identified her as the author. Rowling later confessed. A model case, right?
Well… stylo includes a galbraith dataset. I ran it. The data are hand‑picked: authors from different centuries and genres, making Rowling stand out artificially. A more honest test? Still works, but less dramatic. Always question your benchmark.
The Quiet Don (Sholokhov)
The famous Russian novel has been suspected to be plagiarised from Fyodor Kryukov. Delta analysis (using 100+ texts) shows:
- It’s not Kryukov.
- “Don Stories” (signed Sholokhov) and “The Quiet Don” cluster together → same author.
Politics aside, the data are clear.
Bitcoin white paper (Satoshi Nakamoto)
Here the method is abused. The white paper has only ~3700 words – well below the recommended 10k+ minimum. Still, curious minds ran it: the paper clustered with Craig Wright.
Should you trust that? Absolutely not. But it’s fun.
Where Delta breaks (and why you should care)
- Different genres – comparing a play to a novel makes no sense. Remove the play.
- Theological texts – for unknown reasons, Delta often fails on religious writings. Atheist algorithm?
- Short texts (< 10,000 words) – too noisy. Don’t even try.
- Authors who change style radically – early vs. late Chekhov can look like different people.
So it’s not magic. But when used correctly, it’s the best we have.
For the curious programmer
If you want to dig deeper:
- Burrows, J. F. (2002). Delta: A Measure of Stylistic Difference.
-
stylopackage: https://github.com/computationalstylistics/stylo - My colleague’s visualisation (better than dendrograms): distributions of intra‑author vs inter‑author distances.
And a fascinating book: Passwords: Philology, Security, Authentication by B. Lennon (2018) – it traces the common roots of cryptography and stylometry. The first US crypto‑analysis unit hired… English professors who argued over Shakespeare’s authorship.
Final verdict
Next time someone shows you a line chart of average word lengths to “prove” who wrote something, send them this post. Real authorship attribution is harder – but also more reliable when you use proper methods like Delta.
And remember: if the text is shorter than 10,000 words, just walk away. No method can save you.
Try it yourself:
Download a few texts by two different authors (>10k words each), name them author1_title.txt, run stylo(), and see if they separate. You’ll be surprised how well it works.
Top comments (0)