DEV Community

Miles W
Miles W

Posted on

Perplexity, Burstiness and the ESL False-Positive Problem: How Statistical AI Detectors Score Text

#machinelearning #nlp #ai #education

Most "AI detectors" are not magic classifiers that recognise a model's handwriting. Many of them reduce a document to a handful of statistics about how predictable and how uniform it is, then put a threshold on the result. That design has a well-documented failure mode: it penalises people who write in a second language.

This post walks through the main signals at a conceptual level, shows why simpler prose drifts toward the "AI" side of the threshold, and ends with what that means if you build or deploy these tools.

Signal 1: perplexity

Take a language model and feed it a text one token at a time. At each step the model assigns a probability to the token that actually comes next. Perplexity is, roughly, the exponentiated average negative log-probability across the sequence:

PPL(x) = exp( -(1/N) * Σ log p(x_i | x_<i) )
Enter fullscreen mode Exit fullscreen mode

Low perplexity means the model was rarely surprised by what came next.

Why does that correlate with machine text? Chat models decode toward high-probability continuations, so their output tends to sit in the low-perplexity region of the distribution. A detector exploiting this is essentially asking: does this look like something a model would have chosen?

Signal 2: burstiness

Perplexity is an average, and averages hide shape. Human writing tends to be uneven: a short punchy sentence, then a long one with a sub-clause, then a fragment. One way to capture that is to look at variance across sentences, whether of sentence length or of per-sentence perplexity.

Here is a deliberately toy illustration (not how any production detector works) using only sentence length:

import re, statistics

def sentence_lengths(text):
    sents = [s for s in re.split(r"[.!?]+\s*", text) if s.strip()]
    return [len(s.split()) for s in sents]

def burstiness(text):
    lengths = sentence_lengths(text)
    if len(lengths) < 2:
        return 0.0
    mean = statistics.mean(lengths)
    return statistics.pstdev(lengths) / mean  # coefficient of variation

even = ("Social media is important for young people. It has many advantages. "
        "It also has many disadvantages. Students should use it carefully.")
uneven = ("My cousin runs her whole bakery from Instagram, which still surprises "
          "our grandmother. Every morning. She posts before the ovens are even warm.")

print(burstiness(even), burstiness(uneven))
Enter fullscreen mode Exit fullscreen mode

The first passage has very even sentence lengths; the second swings around. A detector that weights low variance as "machine-like" will lean toward flagging the first.

Signal 3: stock patterns

Some detectors add lexical features: frequency of transitions like "Furthermore" or "In conclusion", tidy three-item lists, generic phrasing. These are cheap to compute and do correlate with chatbot output. They also correlate with how writing is taught.

Where ESL writing lands

Now consider a student writing carefully in their second language. Compared with a fluent native writer, they will often:

  • choose high-frequency words (lower surprisal per token → lower perplexity);
  • reuse a few trusted sentence templates (lower variance → lower burstiness);
  • use the formal connectors from the textbook (more stock-pattern hits).

All three signals move in the same direction. Nothing about honesty is being measured; the features simply describe "ordinary, careful prose", and careful learners produce exactly that.

The empirical result is stark. Liang et al. (2023) ran seven widely used AI-text detectors on 91 human-written TOEFL essays and 88 US eighth-grade essays. The detectors classified the US essays accurately, but the average false-positive rate on the TOEFL essays was about 61%. The TOEFL essays flagged by all seven detectors had significantly lower perplexity than the rest. And when the team simplified the vocabulary of the native essays, misclassification rose substantially. That comes close to a controlled demonstration that the feature, not the author, is driving the label.

Caveats the authors themselves give: it was a pilot study with relatively small samples, and most of the detectors tested in early 2023 used an older, smaller language model as their backbone. The tools have changed since; the dependence on predictability-style signals is the thing to keep watching.

Thresholds and base rates

Even a small false-positive rate matters at scale. Vanderbilt University, explaining in 2023 why it disabled the AI-detection feature of its commercial plagiarism checker, calculated that the vendor's own claimed 1% false-positive rate could have wrongly labelled around 750 of the 75,000 papers it submitted in 2022. If that rate is uneven across groups, the errors concentrate on the students the features disadvantage.

Disagreement between tools is another symptom. In the Stanford data, 97.8% of TOEFL essays were flagged by at least one detector, but only 19.8% by all seven: different thresholds and feature sets, very different verdicts.

If you build or deploy detectors

A few principles follow directly from the mechanism:

  1. Show evidence, not just a score. Sentence-level highlighting lets a reader see whether the "AI" signal is coming from a formulaic introduction or definition, a known false-positive zone.
  2. Never automate the decision. A statistical guess about style cannot establish authorship. Treat a flag as a reason to look closer and to talk to the writer, not as a conclusion.

Those are the principles we try to follow with the Coda One AI Detector. It is free with no daily limit and no signup, and it highlights the specific sentences that read as machine-written instead of handing you only a score. Like every detector, it is evidence, not proof.

Takeaway

Perplexity- and burstiness-style signals are useful descriptions of text. They are not descriptions of who wrote it. Any system that treats "predictable" as "machine-made" will systematically misread writers whose prose is predictable for perfectly human reasons, and language learners are the clearest example. Design the UI, the documentation and the workflow around that fact.

Sources

Top comments (0)