How to Strip Hidden Zero-Width Characters & Markdown from LLM Outputs (And Estimate BPE Tokens Locally)

Seven GS — Mon, 26 Oct 2026 06:22:34 +0000

When copying generated text, documentation, or code snippets from ChatGPT, Claude, or Gemini into a CMS, Notion, Gmail, or code editor, two recurring issues come up for developers and technical writers:

Formatting & Invisible Unicode Artifacts: Raw **bold** asterisks, ### heading hashes, overused AI em-dashes (—), broken PDF line wraps, and invisible zero-width characters (U+200B, U+FEFF, U+00A0) that silently break string matching, regex validation, and character limits.
Context Window & Token Estimation: Knowing how many Byte-Pair Encoding (BPE) tokens a long prompt, JSON payload, or multi-language document consumes before sending it to GPT-4o (128K), Claude 3.5 Sonnet (200K), or Gemini 1.5 Pro (1M)—without uploading confidential text to a third-party server.

To solve both problems with zero server round-trips, I built two new 100% client-side utilities as part of TextScope.

1. Stripping Markdown, Em-Dashes, and Hidden Unicode Characters

Hidden Unicode characters often sneak into copied text from rich-text editors, PDFs, and AI chat interfaces:

Unicode Point	Character Name	Common Issue in Production
`U+200B`	Zero-Width Space	Breaks exact string comparisons, SQL queries, and JSON keys
`U+FEFF`	Byte Order Mark (BOM)	Causes unexpected syntax errors at the start of scripts or CSVs
`U+00A0`	Non-Breaking Space	Prevents standard CSS word wrapping and breaks whitespace regex
`U+2014`	Em-Dash (`—`)	Overused by LLMs; makes technical and editorial copy look robotic

With the AI Text Cleaner & Markdown Stripper, you can paste any LLM or PDF output and toggle exactly what to clean locally in your browser:

Strip Markdown syntax (**bold**, *italic*, ### headings, code fences, and inline links) while keeping clean paragraph breaks.
Detect and remove invisible zero-width characters (U+200B, U+200C, U+200D, U+FEFF, U+00A0) with an exact count of removed artifacts.
Convert AI em-dashes (—) and curly quotes (“”) into standard ASCII punctuation.
Unwrap hard single line breaks copied from multi-column PDFs without merging double-newline paragraphs.

2. Estimating LLM Prompt Tokens & Context Window Headroom

In standard English prose, 1 token ≈ 0.75 words (or roughly 4 characters), meaning 1,000 words is ~1,330 tokens. However, source code, JSON keys, Markdown tables, and CJK scripts (Japanese, Korean, Chinese) have a higher token-to-character ratio (1.2–1.8 tokens per character).

The LLM Prompt Token Counter & Words-to-Tokens Calculator runs entirely in vanilla JavaScript and calculates:

Estimated BPE token count, total words, character count, and token-to-word ratio.
Real-time context window utilization across 8K, 32K, 128K (GPT-4o / Llama 3.1), 200K (Claude 3.5 Sonnet), and 1M (Gemini 1.5 Pro) limits.
Single-spaced page equivalents for long-form technical docs and books.

Full Suite of Client-Side Writing & SEO Utilities

All tools in the suite run 100% locally in the browser (no sign-up, no server uploads) and support 6 languages (English, Spanish, German, Portuguese, Japanese, and Korean):

Word Counter & Flesch Readability Analyzer – Real-time word/character count, Flesch Reading Ease score, Flesch-Kincaid Grade Level, and 1–3 word n-gram phrase density.
AI Text Cleaner & Format Stripper – One-click removal of ChatGPT/Claude Markdown, em-dashes, zero-width spaces, and broken PDF line breaks.
LLM Token Counter & Context Calculator – Instant words-to-tokens conversion and context window percentages.
SEO Headline Analyzer & Google SERP Preview – Title tag pixel width (600px) and character length (50–60 chars) checker with live search snippet preview.
Reading & Speaking Time Calculator – Converts word counts into silent reading (200 WPM) and speech/presentation durations (130 WPM).
Case Converter & URL Slug Generator – Converts text between APA Title Case, Sentence case, camelCase, snake_case, and clean SEO URL slugs.

You can also check out the full multilingual documentation and reference benchmarks in the public GitHub repository: GS-Seven/textscopes on GitHub.

Feedback and feature suggestions are very welcome!

Why Google Truncates Titles by 600px Pixel Width (Not 60 Characters) — And How to Measure It in Vanilla JS

Seven GS — Wed, 21 Oct 2026 02:23:24 +0000

Most SEO guides repeat the "50–60 character rule" for <title> tags. However, Google doesn't actually count characters when deciding where to place an ellipsis (…) in desktop search results — it measures rendered pixel width (roughly 580–600px in 20px Arial).

A headline packed with wide uppercase letters like W, M, or G (or full-width CJK characters in Japanese and Korean, which only fit ~28–32 characters) will truncate well before 60 characters, while narrow letters like i, l, and t can easily fit 65+ characters.

Measuring Exact SERP Pixel Width in the Browser

You don't need a heavy backend or paid SEO suite to measure title pixel width. Using an offscreen HTML5 <canvas>, you can measure the exact rendered width in 4 lines of vanilla JavaScript:

const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
ctx.font = "20px Arial, sans-serif";
const pixelWidth = Math.round(ctx.measureText(headline).width);

Free Browser-Based Writing & SEO Utilities

I built TextScope, a lightweight, 100% client-side toolkit that runs entirely in your browser without uploading drafts to any server:

SEO Headline & 600px Pixel Width Checker: Live Google SERP simulator measuring exact pixel width (px / 600px), meta description length (920–960px), and CTR power words.
Text Analyzer & 1–3 Word Phrase Density Checker: Instant word count, Flesch Reading Ease, Flesch-Kincaid grade level (ideal for B2B cold emails and newsletters), and N-gram keyword frequency.
Reading & Speech Time Calculator: Convert word counts to silent reading time (200 WPM), speech/podcast timing (130–150 WPM), YouTube script budgets, and page counts.
Smart Title Case & SEO URL Slug Converter: Convert headings between APA/Chicago Title Case, Sentence case, and clean kebab-case URL slugs.

All tools support English, Spanish, German, Portuguese, Japanese, and Korean out of the box. Feedback is welcome!