AI watermarking: a green/red token bias seeded by the prior word, a z-test detector, and why paraphrasing breaks it

The question I keep getting asked is "how do you prove a piece of text was written by a machine — without access to the machine?" AI watermarking is the surprisingly clever answer, and once I understood the trick I had to build it to believe it. The generator hides a signal in its output on purpose; a detector that knows only the rule (not the model, not any weights) reads it back with a confident statistical verdict. I built a demo where the green-list seeding, the green fraction, the z-score and p-value are all computed live in the browser — generate a passage with the watermark on or off, then let a model-blind detector call it. Here's how it works (following Kirchenbauer et al., 2023).

Split the vocabulary into a green list and a red list

The whole idea is to gently prefer half the vocabulary. Each step, hash the previous token to seed a PRNG, shuffle the vocabulary with it, and take the first half as the green list:

function greenList(prevToken){
  const rnd = mulberry32(hashStr(prevToken));   // seed from the previous word
  const arr = VOCAB.slice();
  for (let i = arr.length-1; i > 0; i--){        // seeded Fisher–Yates shuffle
    const j = Math.floor(rnd()*(i+1)); [arr[i],arr[j]] = [arr[j],arr[i]];
  }
  return new Set(arr.slice(0, Math.floor(arr.length * 0.5)));  // first half = green
}

Because the seed depends only on the previous token, the writer and the detector compute the exact same split at every position — no secret needs to be shared beyond the hashing rule. And because it changes every step, the split is invisible and can't be trivially stripped.

The watermark is a soft nudge, not a rule

Before sampling the next word, add a small constant δ to the score of every green candidate, then sample normally:

const scored = candidates.map(w => ({ w, s: baseScore(w) + (green.has(w) ? delta : 0) }));
return softmaxSample(scored);   // still random — δ only tilts the odds

Crucially it's soft. There are many green words, so the model keeps real choice and the text reads naturally — a reader sees nothing odd. Set δ=0 and you get ordinary, unwatermarked text; turn δ up and the watermark strengthens but the writing gets slightly more constrained. δ is the knob that trades detectability against quality.

The detector is a one-line z-test

A human writing has no idea which words are green, so each word is green with probability ~50% by pure chance. A watermarked model sails well above that. So the detector doesn't need to understand the text — it counts:

const T = scoredTokens, green = greenCount;
const z = (green - 0.5*T) / Math.sqrt(T * 0.5 * 0.5);   // the whole test
// human:       green ≈ 0.5·T  → z ≈ 0    → "human"
// watermarked: green ≫ 0.5·T  → z large  → "AI-written"

The z-score measures how many standard deviations above the expected count the real green count sits. A threshold of z > 4 gives a false-alarm rate around 1-in-30,000. Because z grows like √T, longer passages are far easier to call — a tweet may be too short even when watermarked, an essay is trivial. In the demo you can click any word to see the exact green/red split its predecessor seeded, and watch the count and verdict update.

Why paraphrasing breaks it

The signal lives in the exact chain of tokens, which makes it real but fragile. Swap a word for a synonym and you change both that word and the green list the next word is judged against — so a rewrite that ignores the green list drags the green fraction back toward 50% and the z-score below threshold. The demo's "paraphrase attack" does exactly this: the verdict collapses from "AI-written" to "looks human". Machine translation round-trips and heavy human editing do the same. There's no free lunch — robustness, quality and detectability trade off against each other.

Images do the analogous thing

There's no vocabulary for an image, so generators embed a faint secret ±1 pattern in the pixels and detect it by correlating against the same key — a watermarked image scores high, a clean one ≈ 0. Google's SynthID does a neural version that survives crops and compression far better than my toy, but the logic is identical, and heavy editing still erodes it. Watermarking is one layer of provenance alongside post-hoc detectors and content credentials — genuinely useful, not bulletproof.

Generate a passage, flip the watermark, and run the paraphrase attack yourself:
https://dev48v.infy.uk/ai/days/day46-ai-watermarking.html