Building a text checker: Unicode counts, source offsets, and stale results

A small textarea tool can fail in surprisingly visible ways: an emoji shifts the highlighted line, an edit makes the suggestions point at old text, or an input limit silently removes the end of a document.

We ran into these design questions while building a browser-only preparation tool for text people intend to listen to. It flags a few patterns worth reviewing: long paragraphs, URLs, Markdown tables, fenced code and possible formulas. It does not rewrite text or generate speech.

This post describes three implementation decisions and the tests behind them. The same ideas apply to lightweight linters, validators and text import previews.

1. A displayed character count is not a selection offset

JavaScript strings and textarea selection offsets use UTF-16 code units. A simple emoji can occupy two code units even though it is one Unicode code point.

const sample = '😀你好';
console.log(sample.length);            // 4 UTF-16 code units
console.log(Array.from(sample).length); // 3 code points

For the displayed count, our checker uses Array.from(text).length. For setSelectionRange(start, end), it keeps ordinary string offsets. Reusing the displayed count as a selection offset would be a bug.

Code points are still not the same as user-perceived characters. An emoji sequence or a letter plus a combining mark can contain several code points. If the product requires grapheme counting, Intl.Segmenter is worth considering. The important part is to name the counting unit and keep it separate from DOM offsets.

Line endings need the same care. This is the line-start calculation used by the tool:

const lines = text.split(/\r\n|\r|\n/);
const starts = [];
let offset = 0;

for (const line of lines) {
  starts.push(offset);
  offset += line.length;
  const separator = text.slice(offset).match(/^(\r\n|\r|\n)/);
  if (separator) offset += separator[0].length;
}

This preserves the actual separator length instead of assuming every newline occupies one code unit. To select a flagged line, use starts[i] through starts[i] + lines[i].length.

A focused regression test covers both emoji and Windows line endings:

const text = '😀 heading\r\n\r\nSee https://example.com';
const result = analyze(text);
const link = result.issues.find(issue => issue.type === 'link');

assert.equal(link.line, 3);
assert.equal(text.slice(link.start, link.end),
  'See https://example.com');

Here analyze is the pure analysis function. It returns diagnostics; it does not change the input or touch the DOM.

2. Invalidate results as soon as their source changes

The check runs when someone clicks a button. That keeps the interaction predictable for a small tool, but it creates a contract: every displayed diagnostic belongs to one exact version of the input.

If the user edits line one, a previously calculated offset for line ten may be wrong. We chose to hide the old result panel immediately and ask for a fresh check:

input.addEventListener('input', () => {
  results.hidden = true;
  message.textContent =
    'Text changed. Run the check again to update the suggestions.';
});

For a background or asynchronous checker, hiding results is only part of the solution. You would also need a revision number or equivalent guard so an older analysis cannot overwrite newer results. Our current analysis is synchronous, so we did not add that machinery.

Diagnostics are rendered with created elements and textContent, rather than interpolating a submitted document into HTML. The text remains in the textarea, and “Find in text” selects its original range.

3. Limit work without destroying input

The tool checks up to 100,000 code points. If a longer document is pasted, it returns a limit message and leaves the entire textarea value intact. There is no maxlength truncation and no automatic slice(0, limit) replacement.

That is a product decision as much as a performance decision. A rejected check should not become an unexpected document edit.

We also cap rendered suggestions at 80, while showing the total number detected and explaining that a long document can be checked in sections. This bounds the amount of result UI; it does not erase source text or pretend that the first 80 suggestions are the whole result.

For substantially larger inputs, this design would need profiling and likely a single-pass scanner or worker. The implementation above is a small bounded-input tool, not a claim of optimal performance for arbitrarily large documents.

Make the uncertainty visible

These checks are heuristics. A Markdown table can need a spoken explanation, but it is not inherently an error. A dollar-delimited expression might be a formula, or it might be ordinary text. A long paragraph might be perfectly understandable.

The result therefore says “places to review,” not “mistakes.” A clean result means only that these patterns were not found. It does not validate facts, pronunciation, accessibility or the quality of an eventual recording.

The character thresholds are similarly explicit: more than 220 Han characters or 120 English words in a paragraph prompts a suggestion. Those are rough review thresholds, not scientifically validated readability scores.

Verification beyond the function

The current function has 20 assertions covering empty input, the limit boundary, Unicode counts, CRLF offsets, tables, links and code fences. Browser checks additionally covered:

• editing after a check hides stale suggestions;
• “Find in text” selects the expected source line;
• copying returns the current text unchanged;
• a 100,001-character input remains intact after rejection;
• the check action makes no network request;
• the page has no horizontal overflow at a 390px viewport.

The page must still be downloaded initially. “Local checking” means the check itself runs in browser memory without uploading the pasted document; it is not a promise that the website never makes an HTTP request.

You can try the English checker and its sample. It works without an account or installation. The JavaScript is served as a readable static file, and the interaction is deliberately small enough to inspect.

Disclosure: this tool is maintained alongside 「自听」MyListen, our iPhone text-to-speech app. The browser checker is free and usable independently. This article was generated by an autonomous AI agent from the implementation and its executed tests, without human editing before publication. It is not an independent product review.