<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Text Repeaterr</title>
    <description>The latest articles on DEV Community by Text Repeaterr (@textrepeaterr).</description>
    <link>https://dev.to/textrepeaterr</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4234544%2F8a86f0b8-1bda-45d2-8eb4-293ff82e0eb8.png</url>
      <title>DEV Community: Text Repeaterr</title>
      <link>https://dev.to/textrepeaterr</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/textrepeaterr"/>
    <language>en</language>
    <item>
      <title>Why apps delete your "blank" message, and the Unicode characters that survive</title>
      <dc:creator>Text Repeaterr</dc:creator>
      <pubDate>Wed, 18 Nov 2026 14:11:10 +0000</pubDate>
      <link>https://dev.to/textrepeaterr/why-apps-delete-your-blank-message-and-the-unicode-characters-that-survive-5c00</link>
      <guid>https://dev.to/textrepeaterr/why-apps-delete-your-blank-message-and-the-unicode-characters-that-survive-5c00</guid>
      <description>&lt;p&gt;People ask for strange things: a WhatsApp message with nothing in it, an empty line in an Instagram bio, a game name that looks blank. Type a few spaces and the app refuses or removes them.&lt;/p&gt;

&lt;p&gt;The reason is simple. Almost every app trims input before it saves it. If nothing is left after the trim, the input counts as empty. The interesting part is what "nothing" means, because trim functions follow Unicode rules, and some characters that look blank are not whitespace by those rules.&lt;/p&gt;

&lt;h2&gt;
  
  
  What trim() actually removes
&lt;/h2&gt;

&lt;p&gt;In JavaScript, &lt;code&gt;String.prototype.trim()&lt;/code&gt; removes characters the spec calls WhiteSpace or LineTerminator. That covers every character in the Unicode "Zs" (space separator) category, plus tab, line breaks and U+FEFF.&lt;/p&gt;

&lt;p&gt;Python's &lt;code&gt;str.strip()&lt;/code&gt; with no arguments removes characters for which &lt;code&gt;str.isspace()&lt;/code&gt; is true. That is close to JavaScript, with one difference worth knowing: Python does &lt;strong&gt;not&lt;/strong&gt; treat U+FEFF as whitespace.&lt;/p&gt;

&lt;p&gt;Here is what each runtime does with some characters that look blank:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Character&lt;/th&gt;
&lt;th&gt;Code&lt;/th&gt;
&lt;th&gt;Category&lt;/th&gt;
&lt;th&gt;Removed by JS &lt;code&gt;trim()&lt;/code&gt;
&lt;/th&gt;
&lt;th&gt;Removed by Python &lt;code&gt;strip()&lt;/code&gt;
&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Space&lt;/td&gt;
&lt;td&gt;U+0020&lt;/td&gt;
&lt;td&gt;Zs&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;No-Break Space&lt;/td&gt;
&lt;td&gt;U+00A0&lt;/td&gt;
&lt;td&gt;Zs&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Figure Space&lt;/td&gt;
&lt;td&gt;U+2007&lt;/td&gt;
&lt;td&gt;Zs&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Narrow No-Break Space&lt;/td&gt;
&lt;td&gt;U+202F&lt;/td&gt;
&lt;td&gt;Zs&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Zero Width No-Break Space (BOM)&lt;/td&gt;
&lt;td&gt;U+FEFF&lt;/td&gt;
&lt;td&gt;Cf&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;no&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Zero Width Space&lt;/td&gt;
&lt;td&gt;U+200B&lt;/td&gt;
&lt;td&gt;Cf&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Word Joiner&lt;/td&gt;
&lt;td&gt;U+2060&lt;/td&gt;
&lt;td&gt;Cf&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hangul Filler&lt;/td&gt;
&lt;td&gt;U+3164&lt;/td&gt;
&lt;td&gt;Lo&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hangul Choseong Filler&lt;/td&gt;
&lt;td&gt;U+115F&lt;/td&gt;
&lt;td&gt;Lo&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Braille Pattern Blank&lt;/td&gt;
&lt;td&gt;U+2800&lt;/td&gt;
&lt;td&gt;So&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;You can check it yourself:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;cp&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mh"&gt;0x3164&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x2800&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x115F&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x200B&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0xA0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x202F&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0xFEFF&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;String&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fromCodePoint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cp&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;length&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="c1"&gt;// 3164 false true   &amp;lt;- not whitespace, survives trim&lt;/span&gt;
&lt;span class="c1"&gt;// 2800 false true&lt;/span&gt;
&lt;span class="c1"&gt;// 115f false true&lt;/span&gt;
&lt;span class="c1"&gt;// 200b false true&lt;/span&gt;
&lt;span class="c1"&gt;// a0   true  false  &amp;lt;- whitespace, trimmed away&lt;/span&gt;
&lt;span class="c1"&gt;// 202f true  false&lt;/span&gt;
&lt;span class="c1"&gt;// feff true  false&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;unicodedata&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="s1"&gt;'\u3164\u2800\u200B\u00A0\uFEFF'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s1"&gt;'U+{ord(c):04X}'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;unicodedata&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;name&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;unicodedata&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;category&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="s1"&gt;'survives strip:'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Why the survivors look blank
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Hangul Filler (U+3164)&lt;/strong&gt; is a letter (category Lo). In Korean text it holds the place of a missing part of a syllable block and is normally drawn with no ink. Because it is a letter, trim leaves it alone, and the app sees one character of content.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Braille Pattern Blank (U+2800)&lt;/strong&gt; is a symbol (category So): the braille cell with no dots raised. It renders as an empty space with real width.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Zero Width Space (U+200B)&lt;/strong&gt; also survives trim, but it has no width at all. Some apps strip format characters (category Cf) on top of trimming, or check for "visible" content, so a message made only of zero width spaces is often still rejected.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this looks like in practice
&lt;/h2&gt;

&lt;p&gt;I run a small set of browser text tools, and the &lt;a href="https://textrepeaterr.com/invisible-character/" rel="noopener noreferrer"&gt;invisible character page&lt;/a&gt; picks the character based on where you plan to paste it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;WhatsApp message, WhatsApp name, Discord, blank comment:&lt;/strong&gt; Hangul Filler (U+3164)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Instagram empty lines:&lt;/strong&gt; one Braille Pattern Blank (U+2800) per line&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In a test on 9 November 2026, six blank messages sent this way went through in WhatsApp. Apps change their input rules without notice, though, so the page has a "Didn't work? Try another" button that switches to the next candidate character.&lt;/p&gt;

&lt;h2&gt;
  
  
  If you build the input field
&lt;/h2&gt;

&lt;p&gt;If you want to reject input that only &lt;em&gt;looks&lt;/em&gt; empty, &lt;code&gt;trim()&lt;/code&gt; is not enough. A stricter check:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;looksBlank&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;/^&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;\&lt;span class="n"&gt;s&lt;/span&gt;\&lt;span class="n"&gt;u115F&lt;/span&gt;\&lt;span class="n"&gt;u1160&lt;/span&gt;\&lt;span class="n"&gt;u3164&lt;/span&gt;\&lt;span class="n"&gt;uFFA0&lt;/span&gt;\&lt;span class="n"&gt;u2800&lt;/span&gt;\&lt;span class="n"&gt;u200B&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;\&lt;span class="n"&gt;u200D&lt;/span&gt;\&lt;span class="n"&gt;u2060&lt;/span&gt;\&lt;span class="n"&gt;uFEFF&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;$&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;u&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="n"&gt;looksBlank&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'\u3164'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;           &lt;span class="c1"&gt;// true&lt;/span&gt;
&lt;span class="n"&gt;looksBlank&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'\u2800\n\u2800'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="c1"&gt;// true&lt;/span&gt;
&lt;span class="n"&gt;looksBlank&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;' \u200B '&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;         &lt;span class="c1"&gt;// true&lt;/span&gt;
&lt;span class="n"&gt;looksBlank&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'hi\u3164'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;         &lt;span class="c1"&gt;// false&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;U+1160 (Hangul Jungseong Filler) and U+FFA0 (Halfwidth Hangul Filler) are in there because they behave like U+3164.&lt;/p&gt;

&lt;p&gt;Two cautions:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Only use this to test whether a whole field is blank. Do not use it to &lt;em&gt;strip&lt;/em&gt; characters from real text: U+200C and U+200D are a normal part of emoji sequences and several scripts.&lt;/li&gt;
&lt;li&gt;Fillers are legitimate Korean characters. Blocking them inside text would break real input. Blocking a field made of nothing else is a product decision, not a correctness fix.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;trim()&lt;/code&gt; and &lt;code&gt;strip()&lt;/code&gt; remove Unicode whitespace, not "everything that looks empty".&lt;/li&gt;
&lt;li&gt;Hangul Filler and Braille Pattern Blank are not whitespace, so they survive and render blank.&lt;/li&gt;
&lt;li&gt;Zero width characters survive trim too, but many apps reject them separately.&lt;/li&gt;
&lt;li&gt;If blank-looking input is a problem in your app, test the whole value against an explicit list.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Have you seen an app that blocks Hangul Filler, or one that accepts zero width spaces? I'd like to know which.&lt;/p&gt;

</description>
      <category>unicode</category>
      <category>javascript</category>
      <category>python</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Invisible characters that break your code (and how to find them)</title>
      <dc:creator>Text Repeaterr</dc:creator>
      <pubDate>Mon, 09 Nov 2026 06:35:27 +0000</pubDate>
      <link>https://dev.to/textrepeaterr/invisible-characters-that-break-your-code-and-how-to-find-them-44el</link>
      <guid>https://dev.to/textrepeaterr/invisible-characters-that-break-your-code-and-how-to-find-them-44el</guid>
      <description>&lt;p&gt;You copy a value from a web page, a PDF or a chat message. It looks correct. But the password fails, &lt;code&gt;JSON.parse&lt;/code&gt; throws, or two strings that look identical are not equal.&lt;/p&gt;

&lt;p&gt;Often the cause is a character you cannot see. This post covers the usual suspects, why normal cleanup misses them, and how to find them in JavaScript, Python and the terminal.&lt;/p&gt;

&lt;h2&gt;
  
  
  The usual suspects
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Character&lt;/th&gt;
&lt;th&gt;Code&lt;/th&gt;
&lt;th&gt;Where it tends to come from&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Zero Width Space&lt;/td&gt;
&lt;td&gt;U+200B&lt;/td&gt;
&lt;td&gt;Web pages, chat apps, rich text editors&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Byte Order Mark (Zero Width No-Break Space)&lt;/td&gt;
&lt;td&gt;U+FEFF&lt;/td&gt;
&lt;td&gt;Files saved as "UTF-8 with BOM", some CSV exports&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;No-Break Space&lt;/td&gt;
&lt;td&gt;U+00A0&lt;/td&gt;
&lt;td&gt;Word processors, HTML &lt;code&gt;&amp;amp;nbsp;&lt;/code&gt;, PDFs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Narrow No-Break Space&lt;/td&gt;
&lt;td&gt;U+202F&lt;/td&gt;
&lt;td&gt;Formatted numbers and dates in some locales&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Soft Hyphen&lt;/td&gt;
&lt;td&gt;U+00AD&lt;/td&gt;
&lt;td&gt;Text copied from typeset documents&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Word Joiner&lt;/td&gt;
&lt;td&gt;U+2060&lt;/td&gt;
&lt;td&gt;Rich text and some keyboards&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Direction marks and overrides&lt;/td&gt;
&lt;td&gt;U+200E, U+200F, U+202A–U+202E, U+2066–U+2069&lt;/td&gt;
&lt;td&gt;Mixed left-to-right and right-to-left text&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Zero Width Joiner / Non-Joiner&lt;/td&gt;
&lt;td&gt;U+200D / U+200C&lt;/td&gt;
&lt;td&gt;Emoji sequences, Persian and Indic scripts&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Most of these are in the Unicode "Cf" (format) category. They have no width, so editors and browsers show nothing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why &lt;code&gt;trim()&lt;/code&gt; and &lt;code&gt;\s&lt;/code&gt; don't catch them
&lt;/h2&gt;

&lt;p&gt;This surprises people, so here it is in code:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;​&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;   &lt;span class="c1"&gt;// false: a zero width space is not whitespace&lt;/span&gt;
&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\s&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;﻿&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;   &lt;span class="c1"&gt;// true&lt;/span&gt;
&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\s&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;&amp;nbsp;&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;   &lt;span class="c1"&gt;// true&lt;/span&gt;

&lt;span class="nx"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;stringify&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt; &amp;nbsp;a​﻿&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;());&lt;/span&gt;
&lt;span class="c1"&gt;// "a​"  &amp;lt;- the zero width space is still there&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Python behaves similarly:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;​&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;isspace&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;   &lt;span class="c1"&gt;# False
&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;﻿&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;isspace&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;   &lt;span class="c1"&gt;# False
&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;&amp;nbsp;&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;isspace&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;   &lt;span class="c1"&gt;# True
&lt;/span&gt;
&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt; &amp;nbsp;a​ ﻿&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="c1"&gt;# 'a​ ﻿'  &amp;lt;- both survive
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So a value can pass your "trim and compare" step and still not match.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three bugs they cause
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. String comparison fails&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;abc&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;a​bc&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="c1"&gt;// false, though both print as abc&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is the classic "the password is correct but login fails" bug, and the reason a spreadsheet lookup won't match a value that looks identical.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. JSON won't parse&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A BOM at the start of a file breaks &lt;code&gt;JSON.parse&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;parse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;﻿{"a":1}&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="c1"&gt;// SyntaxError: Unexpected token '﻿', "﻿{"a":1}" is not valid JSON&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The error message even contains the character, which is why it looks like there's nothing wrong. In Python, open such files with &lt;code&gt;encoding='utf-8-sig'&lt;/code&gt; and the BOM is dropped for you.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Code that "looks fine" doesn't run&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A zero width space pasted into an identifier or a config key gives you an undefined variable or a missing setting, on a line that looks perfect.&lt;/p&gt;

&lt;h2&gt;
  
  
  Finding them in JavaScript
&lt;/h2&gt;

&lt;p&gt;List each hidden character with its code and position:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;hidden&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;[&lt;/span&gt;&lt;span class="sr"&gt;­​-‏‪-‮⁠-⁤⁦-⁩﻿&lt;/span&gt;&lt;span class="se"&gt;]&lt;/span&gt;&lt;span class="sr"&gt;/g&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;findHidden&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;[...&lt;/span&gt;&lt;span class="nx"&gt;text&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;matchAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;hidden&lt;/span&gt;&lt;span class="p"&gt;)].&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;m&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt;
    &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;U+&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nx"&gt;m&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;codePointAt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;toUpperCase&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;padStart&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;0&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt;
    &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt; at &lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nx"&gt;m&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;index&lt;/span&gt;
  &lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nf"&gt;findHidden&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;pass​word&amp;nbsp;and ﻿end&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="c1"&gt;// [ 'U+200B at 4', 'U+FEFF at 14' ]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No-break spaces are not in that pattern on purpose. They are visible as a gap, so it's usually better to turn them into normal spaces than to delete them:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;cleaned&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;text&lt;/span&gt;
  &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;[&lt;/span&gt;&lt;span class="sr"&gt;­​‎‏‪-‮⁠-⁤⁦-⁩﻿&lt;/span&gt;&lt;span class="se"&gt;]&lt;/span&gt;&lt;span class="sr"&gt;/g&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;''&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;[&lt;/span&gt;&lt;span class="sr"&gt;&amp;nbsp; &lt;/span&gt;&lt;span class="se"&gt;]&lt;/span&gt;&lt;span class="sr"&gt;/g&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt; &lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Finding them in Python
&lt;/h2&gt;

&lt;p&gt;The &lt;code&gt;unicodedata&lt;/code&gt; module names every character, which makes the output readable:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;unicodedata&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;find_hidden&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ch&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enumerate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;unicodedata&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;category&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ch&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Cf&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;ch&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;&amp;nbsp; &lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;U+&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;ord&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ch&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;04&lt;/span&gt;&lt;span class="n"&gt;X&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;unicodedata&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;name&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ch&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;?&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

&lt;span class="nf"&gt;find_hidden&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;pass​word&amp;nbsp;and ﻿end&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;# 4 U+200B ZERO WIDTH SPACE
# 9 U+00A0 NO-BREAK SPACE
# 14 U+FEFF ZERO WIDTH NO-BREAK SPACE
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Finding them in files
&lt;/h2&gt;

&lt;p&gt;With GNU grep and a UTF-8 locale:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-nP&lt;/span&gt; &lt;span class="s1"&gt;'[\x{200B}-\x{200F}\x{2060}\x{FEFF}]'&lt;/span&gt; file.txt
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;cat -A file.txt&lt;/code&gt; also shows them as byte sequences. A zero width space appears as &lt;code&gt;M-bM-^@M-^K&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Recent versions of VS Code highlight many invisible and confusable characters by default, so it's worth checking that the Unicode highlighting settings are on in your editor.&lt;/p&gt;

&lt;h2&gt;
  
  
  Don't delete ZWJ and ZWNJ blindly
&lt;/h2&gt;

&lt;p&gt;The zero width joiner (U+200D) is what turns three separate emoji into one family emoji. Strip it and 👨‍👩‍👧 falls apart into 👨👩👧. The zero width non-joiner (U+200C) is a normal part of written Persian and several Indian scripts.&lt;/p&gt;

&lt;p&gt;That's why the cleanup pattern above leaves U+200C and U+200D alone. Remove them only when you're cleaning code, passwords, IDs or spreadsheet keys, where they never belong.&lt;/p&gt;

&lt;h2&gt;
  
  
  A quick way to check by hand
&lt;/h2&gt;

&lt;p&gt;If you just want to paste a piece of text and see what's hiding in it, I built a free &lt;a href="https://textrepeaterr.com/invisible-character/" rel="noopener noreferrer"&gt;hidden character checker&lt;/a&gt;. It marks each character with its name and code and can remove them, with an option to keep emoji joiners. It runs in the browser, so the text isn't uploaded.&lt;/p&gt;

&lt;p&gt;Have you been bitten by one of these? I'd like to hear which character it was and where it came from.&lt;/p&gt;

</description>
      <category>debugging</category>
      <category>javascript</category>
      <category>python</category>
    </item>
  </channel>
</rss>
