Paul Graham posted a test this week that has nothing to do with sentence structure. Slop gives itself away, he said, when the diction doesn't match the idea, when something completely ordinary gets delivered with the excitement of someone announcing a discovery.
That's a different axis than everything else I've been reading about detection this month. Pangram scores a pattern: token by token, sentence by sentence, does the shape of this text statistically resemble a machine's output. Sloan, the human version I ran into on DEV.to, did the same thing by ear instead of by classifier, GPTZero as a second opinion. Both are measuring the same thing: surface. PG's test measures a relationship. What's actually being said, against how much weight the delivery is putting behind saying it.
Run my own flagged pieces through it and they'd pass clean. What got them flagged wasn't excitement outrunning substance, it was named data points and short paragraphs doing real argumentative work, plainly. Pangram's classifier and PG's ear would disagree with each other on the same writing. That's worth sitting with. The tool built to formalize the intuition doesn't actually agree with the intuition once you test them against the same text.
Pascal's comment on the last piece fits here too. Rephrasing his own English for flow, after 6 to 20 hours of writing the argument himself, isn't a case of ordinary ideas dressed up as brilliant ones. It's someone's real thinking, translated. Nothing in that process produces the register mismatch PG is describing. A classifier flagged it anyway, at 97% confidence, on a post from 2017.
The other thing PG's test doesn't need is a platform. Chris Best's whole pitch for shipping Pangram into Substack is that reader intuition doesn't scale, you need a tool doing this at the volume a feed operates at. PG is quietly arguing the opposite: the tell was always available to anyone reading carefully, you just have to know what you're listening for. That's closer to Josh Puckett's complaint in "In Defense of Writing," that readers already discount slop on sight, than it is to anything Substack shipped this week.
I don't think that makes the tool pointless. Most readers aren't reading carefully, most of the time, and a feed moving fast enough rewards that. But it does mean the two approaches are solving different versions of the problem. One scales an ear. The other replaces it.
I know which one caught real writing and called it fake, and I know which one would have let it through. Diction that matches the idea isn't something a classifier trained on hard negatives is built to notice. It's something you have to actually read for.
Top comments (0)