Our HTML entity decoder is 27 lines, knows eight names, and will hand Postgres a NUL byte if you ask it to

Nakodo reads a lot of text it did not write: titles and descriptions from APIs, the
sitemaps and contact pages of brands' own sites, the bodies of inbound email. All of it arrives with HTML
entities in it, and none of it arrives as a DOM, so there is nothing to ask for textContent.

The whole decoder is 27 lines in src/lib/text.ts, with no imports, so anything can use it:

// The named entities that turn up in what we read: API snippets
// ("Tom's review"), page markup and email bodies. A non-breaking space
// becomes a plain one, since every reader wants text to match or show.
const ENTITIES: Record<string, string> = {
  amp: "&",
  quot: '"',
  apos: "'",
  lt: "<",
  gt: ">",
  nbsp: " ",
  commat: "@",
  period: ".",
};

export function decodeHtml(s: string): string {
  return s.replace(/&(?:#(\d+)|#x([0-9a-f]+)|([a-z]+));/gi, (entity, dec?: string, hex?: string, name?: string) => {
    if (name) return ENTITIES[name.toLowerCase()] ?? entity;
    const code = dec ? Number(dec) : parseInt(hex!, 16);
    return code <= 0x10ffff ? String.fromCodePoint(code) : entity;
  });
}

Eight names. Not the couple of thousand named character references the HTML standard defines. The six obvious
ones plus commat and period, which are there because text written to be read by a person and not by a parser
sometimes spells an address out that way. The rule for adding to the list is: it turned up in something we
actually read.

Decoding once is a decision

decodeHtml("<")   // "<"

Not "<". One replace pass, so an entity that decodes into the start of another entity stays as it is, which is
exactly what a browser does with that input: a page containing < shows you <. Two passes, or a
while loop until the string stops changing, would turn text that was escaped twice on purpose into markup. That
is also the whole of the sanitising story here: this function is not a sanitiser, it is only ever used on text
that is about to be stored and displayed as text, never inserted as HTML.

The rest of what it leaves alone:

decodeHtml("&copy; &mdash; &rsquo;")      // "&copy; &mdash; &rsquo;"   <- names we do not know
decodeHtml("&#39")                        // "&#39"                     <- no semicolon, no decode
decodeHtml("a & b")                       // "a & b"                    <- a bare ampersand is text
decodeHtml("&#99999999;")                 // "&#99999999;"              <- not a code point

The unknown-name case is the one with a visible cost. A page title containing ’ reaches a screen with
’ in it, because we never taught the table that name. The numeric form of the same character,
’, decodes fine. If I were fixing it today I would not import an entity table either: I would add the
dozen typographic names (copy, rsquo, lsquo, ldquo, rdquo, mdash, ndash, hellip, trade, reg,
deg, amp is already there) and keep the rule.

Three inputs it gets wrong

The code-point guard is code <= 0x10ffff, which is the right upper bound and the only check there is. Two kinds
of valid-looking number get through it and produce a string that is technically a string and practically a
problem.

A zero byte.

decodeHtml("x�y").length    // 3
JSON.stringify(decodeHtml("x�y"))   // '"x\u0000y"'

That is a real U+0000 in the middle of the text. JavaScript does not mind. Postgres does: a text value cannot
contain a zero byte, and the insert fails with an invalid-byte-sequence error rather than storing the rest. Our
decoded text goes on to be stored, so � on somebody's page is a write that fails for a reason nothing in the
stack trace would connect to an entity.

A lone surrogate.

const s = decodeHtml("�");
s.isWellFormed();                  // false
Buffer.from(s, "utf8");            // <Buffer ef bf bd>   <- the replacement character

String.fromCodePoint(0xD800) is perfectly legal and gives you half of a surrogate pair with no other half.
The string is not well formed UTF-16, so it cannot be encoded as UTF-8: anything that serialises it, including
every database driver and every HTTP body, quietly turns it into U+FFFD. JSON survives it, because
JSON.stringify writes the escape "\ud800" back out, which means a round trip through JSON preserves a value
that a round trip through a socket destroys. That is a lovely way to spend an afternoon, and 0xD800 to 0xDFFF
is three lines of guard:

const ok = code <= 0x10ffff && !(code >= 0xd800 && code <= 0xdfff) && code !== 0;

Names a browser would not accept. The gi flags make the name match case insensitive, and then
ENTITIES[name.toLowerCase()] looks it up:

decodeHtml("&NBSP;")   // " "   <- we decode it

A browser renders   as the literal text  , because HTML's named references are case sensitive apart
from a short legacy list (& and & both work;   does not). So on this one input we are more
generous than the thing we are imitating. It has never mattered and I would still rather the comment said so.

Where it runs

Eight call sites, and they are the four shapes of text that come in from outside: <loc> values pulled out of a
sitemap with a regex, titles and descriptions from APIs, the text of HTML pages we read, and the bodies of
inbound email. None of them is a place where a wrong decode would be loud. That is the argument for the tests
being the whole specification of this function, and for them being written as the inputs that actually arrived:

test("decodes the entities found in snippets, pages and emails", () => {
  assert.equal(decodeHtml("Tom's "review" & more"), `Tom's "review" & more`);
  assert.equal(decodeHtml("&lt;b&gt; &apos;x&apos;&nbsp;y"), "<b> 'x' y");
  assert.equal(decodeHtml("hello&#64;pub.com, hi&#x40;pub&period;com, hey&commat;pub.com"), "[e-mail], [e-mail], [e-mail]");
  assert.equal(decodeHtml("&#128512;"), "😀");
});

test("decodes once, and leaves anything it doesn't know", () => {
  assert.equal(decodeHtml("&amp;lt;"), "&lt;");
  assert.equal(decodeHtml("a & b &copy; &#99999999;"), "a & b &copy; &#99999999;");
});

Six assertions. The gap they describe is the one I went looking for: every case in there is a string somebody
wrote, and the three inputs that break it are strings a machine would write. A decoder's test file wants both
kinds, and mine has one.

The emoji assertion is in there for a reason that is not decoration, by the way: 😀 is above the
basic multilingual plane, so it only works because the function uses String.fromCodePoint and not
String.fromCharCode. Swap those two and every emoji in every title becomes a different character, silently,
and no test that stops at   would catch it.

If you want to see the text this runs on before it reaches it, the method page is the honest version:
nakodo.app/how-it-works says what gets read and what is kept. And the
guide on finding a creator's address (nakodo.app/guides/how-to-find-a-youtubers-email)
is the reader-facing half of why @ and . are in a table of eight.