Why Your JSON Fails to Parse: Zero-Width Spaces Copied from ChatGPT

brkn — Sat, 17 Oct 2026 09:28:26 +0000

You ask ChatGPT for a config snippet. You copy it, paste it into your editor, and compare it with the file that works. The two look identical, character for character. Then you run it.

In Node, a pasted file that starts with a byte order mark gives you this (the invisible character is written here as <U+FEFF>; your terminal prints nothing at that position):

SyntaxError: Unexpected token '<U+FEFF>', "<U+FEFF>{"name": "x"}" is not valid JSON

In Python, a zero-width space pasted into source code gives you this:

SyntaxError: invalid non-printable character U+200B

The BOM variant of the Python error is the same message with U+FEFF at the end. In both languages the message names a character you cannot see, and the line it points to looks fine. The string you pasted is not the string you think it is.

What is actually in the string

These are the code points that cause most of the trouble:

Code point	Name	Where it usually comes from
U+200B	Zero width space	Rendered web text, line-break hints, chat UIs
U+200C	Zero width non-joiner	Persian, Arabic and Indic text
U+200D	Zero width joiner	Emoji sequences, Indic scripts
U+2060	Word joiner	Typesetting, some rich-text editors
U+FEFF	Byte order mark / zero width no-break space	"UTF-8 with BOM" files, Windows tools, CSV exports
U+00A0	No-break space	HTML ` `, PDFs, rich-text copy
U+200E / U+200F	Left-to-right / right-to-left mark	Mixed-direction text
U+00AD	Soft hyphen	Hyphenated web pages and PDFs

Why do AI chat interfaces leave them behind? The usual explanation is that the chat UI builds its output incrementally and renders it as rich text, and some of that structure ends up in the clipboard when you copy. These are rendering and streaming artifacts, not watermarks. Nothing about them is a hidden signal, and they are not tied to the content of the answer. The same characters show up when you copy from PDFs, Slack, Notion and ordinary web pages.

One more thing worth knowing: a zero-width space inside a JSON string is valid JSON. JSON.parse accepts it. The failure is later, when config["name"] returns undefined because the key is really "name\u200B". A parse error only happens when the character lands outside a string, or when it is a BOM at the start.

How to see them

Shell:

cat -A file.json        # BOM shows as M-oM-;M-?, U+200B as M-bM-^@M-^K
xxd file.json | head    # a BOM is the bytes ef bb bf at offset 0
grep -nP '[\x{200B}\x{FEFF}\x{00A0}]' file   # needs a UTF-8 locale

If grep -P complains that a code point is too large, run it with LC_ALL=C.UTF-8.

Python:

s = open("file.json", encoding="utf-8").read()
print(repr(s))  # repr escapes U+200B as \u200b
print([hex(ord(c)) for c in s if ord(c) > 0x7e])

Node:

const s = require("fs").readFileSync("file.json", "utf8");
console.log([...s].map(c => c.codePointAt(0).toString(16)).filter(h => parseInt(h, 16) > 0x7e));

Editor and git:

VS Code: editor.renderControlCharacters draws control characters. For invisible Unicode, check the editor.unicodeHighlight.* settings. The "Highlight Bad Chars" and "Gremlins" extensions also mark them.
git diff shows a changed line where the old and new text look the same. Run git diff | cat -A and the difference appears as M- sequences.

The fix

Strip the characters, but handle two cases separately. NBSP should become a normal space, not disappear, or a\u00A0b turns into ab. And U+200D must survive when it joins emoji, because that is how a family emoji is built: person, ZWJ, person, ZWJ, child. Remove every ZWJ and the family falls apart into three separate emoji.

JavaScript, preserving ZWJ between emoji:

const KEEP = "\uE000"; // private-use placeholder, not present in normal text

function cleanInvisible(text) {
  return text
    // protect a ZWJ that sits between two emoji
    .replace(/(\p{Extended_Pictographic}\uFE0F?)\u200D(?=\p{Extended_Pictographic})/gu, `$1${KEEP}`)
    // remove zero-width chars, direction marks, word joiner, BOM, soft hyphen
    .replace(/[\u200B-\u200F\u2060\uFEFF\u00AD]/g, "")
    // NBSP becomes a regular space
    .replace(/\u00A0/g, " ")
    // restore the protected joiners
    .replaceAll(KEEP, "\u200D");
}

const family = "\u{1F468}\u200D\u{1F469}\u200D\u{1F467}";
console.log(cleanInvisible(family) === family);          // true
console.log(cleanInvisible("a\u200Bb\u00A0c\uFEFF"));    // "ab c"

Python's re has no \p{...}, so this version approximates "emoji" with two common ranges. Adjust them to your data:

import re

EMOJI = r"[\U0001F300-\U0001FAFF\u2600-\u27BF]"
KEEP = "\uE000"

def clean_invisible(s: str) -> str:
    s = re.sub(rf"({EMOJI}\uFE0F?)\u200D(?={EMOJI})", lambda m: m.group(1) + KEEP, s)
    s = re.sub("[\u200B-\u200F\u2060\uFEFF\u00AD]", "", s)
    s = s.replace("\u00A0", " ")
    return s.replace(KEEP, "\u200D")

Shell, using perl for the emoji-aware version and sed for the blunt one:

# perl: keeps ZWJ between emoji, NBSP -> space
perl -CSD -pe '
  s/(\p{Extended_Pictographic}\x{FE0F}?)\x{200D}(?=\p{Extended_Pictographic})/$1\x{E000}/g;
  s/[\x{200B}-\x{200F}\x{2060}\x{FEFF}\x{00AD}]//g;
  s/\x{00A0}/ /g;
  s/\x{E000}/\x{200D}/g;
' file.json > file.clean.json

# sed (GNU): byte-level, removes every ZWJ, so use it on code and config, not prose with emoji
LC_ALL=C sed -E 's/\xEF\xBB\xBF|\xE2\x80[\x8B-\x8F]|\xE2\x81\xA0|\xC2\xAD//g; s/\xC2\xA0/ /g' file.json

Where it bites hardest

JSON keys. The file parses, but obj.name is undefined because the key ends in U+200B.
.env files. A trailing U+200B on a value gives you an API key that is one character too long, and the provider answers with an auth error.
Shell commands. cd my\u2011dir, where the hyphen is a non-breaking hyphen (U+2011), fails with "No such file or directory" even though the name looks right.
YAML. Indentation made of NBSP is not indentation, so the parser reports a structure error on a line that looks aligned.
SQL. WHERE name = 'Ada' returns no rows when the stored value has a trailing U+200B or an NBSP.
URL slugs. A hidden character gets percent-encoded (U+200B becomes %E2%80%8B), so the link is a different URL from the one you meant.
CSV headers. A BOM glues itself to the first header, so row["id"] fails and the column is really named "\uFEFFid".

Prevent it

Paste with Ctrl/Cmd+Shift+V in editors that support paste-as-plain-text. This skips the rich-text clipboard layer, though characters already present in the plain-text layer still come through.
Add a pre-commit hook. This one skips U+200D so legitimate emoji sequences do not block a commit, and it reads the working tree, not the staged blobs:

#!/bin/sh
# .git/hooks/pre-commit
export LC_ALL=C.UTF-8
if git diff --cached --name-only --diff-filter=ACM -z \
  | xargs -r0 grep -InP '[\x{200B}\x{200C}\x{200E}\x{200F}\x{2060}\x{FEFF}\x{00AD}]'; then
  echo "Hidden Unicode characters found, commit aborted." >&2
  exit 1
fi

ESLint's no-irregular-whitespace covers U+00A0, U+FEFF and U+200B, but not U+200C, U+200D, U+200E, U+200F, U+2060 or U+00AD. It also ignores string literals by default (skipStrings is true), and strings are where pasted keys and values live.
For a one-off snippet, gpt-cleanup.com/zero-width-space-remover lists every hidden character with its code point before it cleans anything, and there is a Chrome/Edge extension for right-click cleaning. Everything runs in the browser.

TL;DR

Copied text can carry invisible characters (U+200B, U+FEFF, U+00A0 and others) that look like nothing and break parsers.
They are rendering and streaming artifacts, not watermarks.
Find them with cat -A, xxd, grep -P, or repr() and code-point listings in Python and Node.
Strip them with a regex, turn NBSP into a space, and keep ZWJ between emoji.