DEV Community

Cover image for Why Your Profanity Filter Blocks Scunthorpe (And How to Fix It)
Tomer N
Tomer N

Posted on Originally published at Medium AI-assisted

Why Your Profanity Filter Blocks Scunthorpe (And How to Fix It)

In April 1996 a man named Doug Blackie tried to sign up for AOL. He typed his name, then his hometown, Scunthorpe, and the form wouldn't take it. After some back and forth with AOL's people in Dublin, the way he finally got in was to say he lived in "Sconthorpe".

You can probably see why. Four letters inside the town's name spell a word AOL's filter was told to refuse, and the filter never looked further than that. Thirty years later, it's still one of the easiest bugs to ship in a signup form, and it has a name: the Scunthorpe problem. (Drafted with AI help; every fact is sourced.)

I want to walk through why it keeps happening, and the four ways people usually write the check, with real code and what each one gets wrong.

It's not just one town

Once you start looking, the list is long. Wikipedia keeps a page of real cases, and a few of them stuck with me:

  • In 2004 Hotmail refused Craig Cockburn's surname, and at first told him to spell it C0ckburn, with a zero.
  • The Horniman Museum in London lost email in 2004 because filters read its name as "horny man".
  • In 2010 Xbox Live suspended a player for saying he lived in Fort Gay, West Virginia.
  • In 2018 a grocery store's cake form wouldn't print "summa cum laude", and the customer's cake came out with dashes where "cum" should have been.

Every one of these is the same mistake: a word was found inside a longer word, and nobody checked what the longer word was.

Why it happens

A filter has two jobs that pull against each other.

It has to catch the bad word when it's glued to something else: "dumbass", "shithead", a username like "xXfuckfaceXx". Those don't have spaces around the bad part, so the filter has to look inside words.

And it has to leave innocent words alone, which means not looking inside words. Scunthorpe, Cockburn, classic and cockpit all contain a bad word, and none of them is one.

You can't fully do both with a list of strings. Every fix below moves the line between the two, and you get to pick where.

To keep things concrete, all four versions use this short list:

const BAD = ['cunt', 'cock', 'ass', 'shit', 'penis', 'fuck', 'clit'];
Enter fullscreen mode Exit fullscreen mode

1. Look for the bad word anywhere

This is the version most of us write first, and it's what AOL had:

function findProfanity(text) {
  const t = text.toLowerCase();
  return BAD.find((word) => t.includes(word)) ?? null;
}
Enter fullscreen mode Exit fullscreen mode

It catches "dumbass", "shithead" and "xXfuckfaceXx". It also blocks Scunthorpe, Craig Cockburn, "a classic mistake" and "cockpit". It's thorough in both directions, which is the problem.

2. Only match whole words

The usual next step is a word boundary in a regular expression:

const WHOLE = new RegExp(`\\b(${BAD.join('|')})\\b`, 'i');

function findProfanity(text) {
  return WHOLE.exec(text)?.[1].toLowerCase() ?? null;
}
Enter fullscreen mode Exit fullscreen mode

Now Scunthorpe goes through, and so does everything else. "dumbass" isn't the word "ass", "shithead" isn't "shit", and a username has no word boundaries at all. On the eight texts in the grid further down, it blocked nothing innocent and caught none of the four rude ones.

That's fine for some products. If your users write sentences and mostly swear in whole words, this might be enough. For usernames it does almost nothing.

3. Strip an allow list, then look anywhere

So you go back to version 1 and add exceptions. The tempting way is to delete the allowed words from the text before checking:

const ALLOW = ['scunthorpe', 'cockburn', 'hancock', 'clitheroe', 'classic', 'cockpit', 'bass'];

function findProfanity(text) {
  let t = text.toLowerCase();
  for (const ok of ALLOW) t = t.replaceAll(ok, ' ');
  return BAD.find((word) => t.includes(word)) ?? null;
}
Enter fullscreen mode Exit fullscreen mode

Say "bass" is on there because a music forum's users kept getting blocked. Now look at "dumbass": the last four letters are "bass", they get deleted, and the insult goes through clean. Stripping substrings has the same bug as matching them, just pointed the other way.

4. Check word by word, with a dictionary

The version I'd actually ship splits the text into words first, and only looks inside a word when that word isn't a real one:

import { readFileSync } from 'node:fs';

const DICTIONARY = new Set(
  readFileSync('/usr/share/dict/words', 'utf8').toLowerCase().split('\n'),
);
const OURS = new Set(['scunthorpe', 'cockburn', 'hancock', 'clitheroe']);

function findProfanity(text) {
  for (const word of text.toLowerCase().match(/\p{L}+/gu) ?? []) {
    if (BAD.includes(word)) return word;
    if (DICTIONARY.has(word) || OURS.has(word)) continue;
    const hit = BAD.find((bad) => word.includes(bad));
    if (hit) return hit;
  }
  return null;
}
Enter fullscreen mode Exit fullscreen mode

Three things are going on here:

  • A word that's on the bad list by itself is blocked, even though "ass" and "cock" are also in the dictionary.
  • A real dictionary word is left alone, so "classic", "cockpit", "assassin", "cockatoo" and even "Penistone" pass without you listing them. macOS ships that file with 235,976 words, and Debian and Ubuntu install one with the wamerican package.
  • Anything else, like "dumbass" or "xxfuckfacexx", gets the substring check.

A grid of eight texts checked four ways: substring, whole words, stripped allow list and dictionary. The dictionary version blocks no innocent text and catches three of four rude ones
The four versions on the same eight texts, run on October 27, 2026

The dictionary doesn't know names, though. Scunthorpe, Cockburn, Hancock and Clitheroe aren't in it, which is why OURS exists. That's where you put the towns, surnames and product names your own users type. It's a much shorter list than an allow list for every English word, and it only grows when a real person hits it.

What none of these fix

Every version above missed "f.u.c.k off". Splitting on letters turns it into four single letters, and none of them is on the list. Disguises like dots, digits and look-alike letters from other alphabets are a whole separate problem, and a list of strings doesn't solve it on its own. You have to normalize the text first, and that's a post of its own.

The other gap is context. "Cock" is a rooster on a farming forum and an insult in game chat. No list can tell those apart, because the difference isn't in the word.

What I'd do

If I were adding a check to a signup form tomorrow:

  1. Start from version 4, with a real dictionary and a short list of your own names.
  2. Test it on text you know is clean: your users' towns, surnames and product names. Every block there is a bug.
  3. Don't reject when you're unsure. Hold the name for a quick human look instead, so someone from Clitheroe waits a minute instead of leaving.
  4. Give people a way to tell you. Every report is one more entry for OURS.

Doug Blackie's story is thirty years old, and the bug is still easy to write in an afternoon. The good news is the fix is too.

Thanks for reading, and good luck with your filter :)

I build Unprofane, a profanity filter API. Its Scunthorpe guide lists the innocent words filters trip on most.

Top comments (0)