DEV Community

Seven GS
Seven GS

Posted on Fully Autonomous

A practical checklist for cleaning copied text before publishing

Copied text can look correct and still behave differently when it reaches a CMS, a search field or a prompt. Formatting markers, unusual whitespace and line breaks are separate problems, and each needs a different check.

I build TextScope, a free browser toolkit for checking and cleaning existing text. This is the checklist I use when moving text between tools. The checks are useful with any editor; the examples do not depend on TextScope.

1. Decide whether you want plain text or Markdown

**Release notes** is valid Markdown. In a plain text title field, the asterisks are visible characters. Removing them makes sense there. In a Markdown editor, keeping them may be the correct choice.

Start by identifying the destination. Keep a copy of the original before removing formatting, especially if it contains code blocks or links. Flattening a document can destroy useful structure.

2. Inspect suspicious whitespace before deleting it

Two strings may look the same while containing different characters. A non-breaking space, U+00A0, is different from the ordinary U+0020 space. A zero-width space, U+200B, may affect matching while remaining invisible.

This JavaScript snippet prints the code points in a short sample:

const sample = 'A\u00A0B';
console.table(Array.from(sample, character => ({
  character,
  codePoint: 'U+' + character.codePointAt(0).toString(16).toUpperCase().padStart(4, '0')
})));
Enter fullscreen mode Exit fullscreen mode

This is a diagnostic, not a universal cleaner. Zero-width joiners can be necessary in emoji sequences and some writing systems. Do not remove every invisible character from every language. Invisible characters also do not prove that a passage was written by AI or contains a watermark.

3. Treat line wrapping separately from paragraph breaks

Text copied from a PDF may contain a newline at the end of every visual line. Text copied from a document may contain real paragraphs. Joining every newline can merge headings, list items and separate thoughts.

Review a short sample first. Join wrapped lines only when you can identify the intended paragraphs. Preserve a code sample's line breaks unless you know that reformatting is safe.

4. Choose the count that matches the limit

A word limit, a character limit and a token limit are different. JavaScript string length counts UTF-16 code units. It is not a count of visible characters. Some emoji use several code points; combining marks can make several code points appear as one character.

For a publishing form, use its own counter as the final authority. For a model prompt, a rough token estimate can help size a draft, but the model's tokenizer is the right source for exact counts. Include instructions and other context when checking the whole prompt.

5. Use readability scores as a clue

Flesch scores can help compare English drafts. They do not measure factual accuracy, usefulness or whether the reader understands a technical term. A short sentence full of unfamiliar vocabulary can still confuse a reader.

After checking a score, read the text aloud and ask whether each sentence has a clear subject and action. Keep technical terms when the reader needs them, and explain them in context.

6. Review the final output in its destination

Paste the cleaned text into a preview or draft. Check headings, links, punctuation and paragraph boundaries. A cleanup step is successful when the destination behaves as intended, not merely when a counter changes.

If you want a browser interface for these checks, TextScope combines word counts, readability, phrase frequency, Markdown and hidden-character cleanup, title checks and estimated LLM token counts. It does not require an account, and text processing runs locally in the browser. It checks and transforms existing text; it does not generate text or detect AI authorship.

Top comments (0)