Why Detecting AI-Generated Text Is Harder Than You Think (And What I Built Anyway)

chen jiayan — Thu, 24 Sep 2026 02:16:42 +0000

The problem nobody talks about

Every "AI detector" landing page promises 99% accuracy. Then you paste in a
paragraph you actually wrote yourself and it flags you as ChatGPT. I kept
seeing this in the wild — students wrongly accused, editors discarding human
copy, and a pile of tools that were really just guessing.

So I went down the rabbit hole of how AI-content detection actually works,
built a tool to test the claims, and learned that the honest answer is far more
interesting than the marketing.

Why watermarking text is genuinely hard

Image models can embed an invisible statistical watermark (the SynthID-style
approach), and that's a real, checkable signal. Text is different. A model
generates tokens probabilistically; there's no natural place to hide a bit
string that survives copy-paste. Researchers have proposed watermarking the
logit distribution (green/red token lists), but it breaks under:

Paraphrasing (rewrite it and the signal is gone)
Short inputs (a 20-word sentence carries almost no statistical footprint)
Translation
Deliberate obfuscation

If a tool claims 100% accuracy on short text, it's lying. Anyone who's actually
benchmarked one knows it.

What the models actually look at

In practice detectors lean on a few weaker, statistical signals:

Perplexity — how "surprising" the word choices are. AI text tends to be low-perplexity, very smooth.
Burstiness — humans vary sentence length and rhythm wildly; LLMs are flatter.
N-gram / token-frequency patterns that leak the decoding strategy.

None of these is a watermark. They're probabilities, and they fail on edited,
mixed, or short content. That gap between "statistical likelihood" and
"provenance" is where most products quietly pretend to be something they aren't.

What I built

I got tired of the gap, so I built a detector that reports honestly — a
confidence score plus an explanation of which signals fired, rather than a
single fake certainty. You can try it here: https://detectaiwatermarks.com

The technical choices I'd highlight:


python
# A naive "is this AI?" check fails. A useful one returns evidence.
signals = {
    "perplexity": score_perplexity(text),
    "burstiness": sentence_variance(text),
    "watermark_scan": probe_known_watermark(text),  # often None, that's fine
}
# Verdict is calibrated per-length: a tweet != an essay.
verdict = calibrate(signals, length=len(text))

DEV Community: chen jiayan

Why Detecting AI-Generated Text Is Harder Than You Think (And What I Built Anyway)

The problem nobody talks about

Why watermarking text is genuinely hard

What the models actually look at

What I built