Invisible Characters Are Hiding in Your Copy-Pasted Text

You copy a line of text. You run it. It fails. You look again — and it looks identical to the version that worked.

Same characters, different result. What's going on?

Nine times out of ten, the culprit is an invisible character — a real Unicode code point that occupies space in your string but renders as nothing on screen.

Here's what they are, where they come from, and how to get rid of them.

What are invisible characters?

Invisible (or "zero-width") characters are real characters that take up no visible space. They're legitimately defined in Unicode, so your editor, language, and compiler treat them as ordinary input — you just can't see them.

The usual suspects:

U+200B  ZERO WIDTH SPACE
U+200C  ZERO WIDTH NON-JOINER (ZWNJ)
U+200D  ZERO WIDTH JOINER (ZWJ)
U+2060  WORD JOINER
U+FEFF  ZERO WIDTH NO-BREAK SPACE (also used as a byte-order mark)
U+00A0  NO-BREAK SPACE (not zero-width, but easy to miss)

There's also a family of bidirectional control characters (U+202A–U+202E, U+2066–U+2069) that change how text is displayed without changing the underlying bytes.

A quick way to spot one

Put two strings side by side and compare their lengths:

a = "hello"          # 5 characters
b = "he\u200bllo"    # looks like "hello" too, but...

print(len(a))  # 5
print(len(b))  # 6

If a comparison fails and you'd swear the strings are equal, count the characters.

Where do they come from?

You almost never type them on purpose. They sneak in when you copy and paste:

• Web pages — some sites inject zero-width characters for tracking, layout, or "copy protection."
• AI chat interfaces — text copied from LLMs often carries non-breaking spaces and other unusual Unicode, and zero-width characters show up too.
• PDFs and Word documents — soft hyphens, non-breaking spaces, and direction marks survive copy-paste.
• CMSs and form fields — rich-text editors love to add "helpful" invisible formatting.
• Deliberately — for watermarking, fingerprinting, or even attacks like Trojan Source (CVE-2021-42574), where invisible direction characters make source code look one way and compile another.

Why they cause real problems

• Code that won't compile — an identifier with a hidden zero-width character is a different identifier.
• Failing string comparisons — "abc" == "abc\u200b" is false.
• Broken search and replace — you can't find what you can't see.
• Git diffs that drive you crazy — a line that "didn't change" still shows up as modified.
• Weird SEO and form validation — invisible text is a classic spam signal, and a required field can be "filled" by characters no human typed.

How to find and remove them

Pick whatever fits your workflow:

1. A detector tool — paste the text and see every invisible character highlighted, with its code point.
2. Your editor — check for "show whitespace / render special characters." Zero-width characters are the tricky ones; they usually need a plugin or a hex view.
3. The command line — a quick check on Linux/macOS:

grep -P "[\x{200B}-\x{200D}\x{FEFF}]" file.txt

1. A script — once you know the code points, stripping them is one line:

import re
clean = re.sub(r"[\u200b\u200c\u200d\u2060\ufeff]", "", text)

The bottom line

If something looks right but behaves wrong, stop trusting your eyes and start counting characters. Invisible characters are the most common cause — and they're easy to remove once you can finally see them.

I built a small, free, browser-based tool for exactly this: it scans text for invisible and zero-width characters, shows you where they are, and lets you clean the text in one click. Everything runs locally in your browser — nothing is uploaded.

Try it here: https://ai-watermark-detector.com

If this was useful, tell me in the comments which invisible character caused you the most trouble.

Comments

[Harun - solo dev]

cg s, this is a massively underrated debugging trap. the trojan source (cve-2021-42574) mention is especially critical—bidirectional override characters are actively weaponized to make malicious code look benign during pr reviews.

one edge case where this absolutely wrecks havoc is in llm pipelines: when an ai chat interface injects a zero-width space (u+200b) into a generated variable name or json key, it silently breaks downstream linters, regex matchers, or strict schema validators.

curious about your tool's implementation: when stripping these characters, how do you handle the zero-width joiner (u+200d)? if you blindly regex it out, you completely break complex emoji sequences (like 👨‍👩‍👧‍👦 or flag emojis) that rely on it for proper rendering. do you have a context-aware parser or a safe-list to preserve legitimate zwj usage while nuking the malicious/formatting ones?

fantastic, privacy-first tool. keeping the sanitization entirely in the browser is exactly the right call for cleaning potentially sensitive code snippets. 🐯🔍