DEV Community

Lidong Wang
Lidong Wang

Posted on AI-assisted

Your docs can look right and still copy wrong

A code block can look perfectly correct while its Copy button silently changes what reaches the clipboard.

The failures are usually small enough to escape screenshots and ordinary DOM assertions:

  • a final newline disappears;
  • tabs become spaces;
  • indentation is trimmed;
  • a shell prompt is removed;
  • ASCII punctuation becomes a Unicode look-alike;
  • composed and decomposed Unicode compare differently.

For a command that will be pasted into a terminal, “almost the same” is not a useful contract.

Why a DOM assertion is not enough

Most documentation tests stop at the rendered <code> element. Copy behavior introduces at least two more boundaries:

canonical source -> rendered DOM -> copy handler -> browser clipboard
Enter fullscreen mode Exit fullscreen mode

A page can render the right text but send a different string to navigator.clipboard.writeText. The handler can also send the right string while the browser or operating system exposes a transformed result. Without observing the stages separately, a failure is easy to misattribute.

That is the narrow problem I built Snippet Fidelity to test.

A one-step reconnaissance check

The fastest way to try it is a GitHub Actions workflow against a public documentation page:

name: snippet-fidelity

on:
  workflow_dispatch:

permissions:
  contents: read

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: WLDKK/snippet-fidelity@v0
        with:
          url: https://docs.example.com/getting-started/
Enter fullscreen mode Exit fullscreen mode

This mode discovers visible copy controls near code blocks, activates them in real Chromium, and compares copied text with the rendered code. It is useful reconnaissance, but it cannot prove that the rendered page matches a maintainer's source of truth.

Turn it into a release gate

For a dependable contract, point a check at a checked-in UTF-8 source file:

{
  "$schema": "https://raw.githubusercontent.com/WLDKK/snippet-fidelity/v0/schema/config.schema.json",
  "version": 1,
  "baseUrl": "https://docs.example.com/",
  "pages": [
    {
      "url": "getting-started/",
      "checks": [
        {
          "id": "install-command",
          "button": "#install-command button[aria-label='Copy code']",
          "expected": { "file": "./snippets/install.sh" },
          "probe": "both"
        }
      ]
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Now the result can distinguish a source-to-render mismatch from a handler or clipboard mismatch. Reports use SHA-256 fingerprints, lengths, categorized findings, and a bounded escaped context around the first different Unicode code point. Full snippets are not printed by default.

What real-site testing taught me

I ran the discovery mode against three maintained documentation sites. Three Starlight checks matched. Material for MkDocs consistently omitted a rendered terminal newline, while Doc Detective omitted a displayed shell prompt marker.

Those differences may be deliberate paste-friendly behavior. I do not label them bugs, because rendered-page discovery does not know the maintainers' canonical intent. That result led to an important reporting rule: a mismatch is an observation; a release-blocking claim requires a maintainer-owned source contract.

The pilot also found a bug in the tool itself. Tabbed documentation often keeps inactive code blocks mounted, so the first implementation tried to activate hidden duplicate controls. The fix excluded non-rendered blocks while retaining hover-revealed controls, then added an end-to-end regression fixture.

Reproduce the failure classes locally

The repository includes an adversarial fixture with one passing control and five intentional failures:

pnpm install
pnpm build
pnpm fixture

# In another terminal:
node dist/cli.js audit --config examples/adversarial-fixture.config.json
Enter fullscreen mode Exit fullscreen mode

The expected outcome is one pass and five failures. A zero exit code would mean the fixture stopped exercising the intended regressions.

Scope and limits

Snippet Fidelity is deliberately not a generic documentation testing framework or clipboard manager. It does not execute copied commands, rewrite clipboard content, or claim source fidelity when the baseline came from the rendered DOM. Version 0.4 currently drives Chromium, and automatic discovery remains heuristic; explicit selectors and canonical source are required for a serious release gate.

The project is MIT licensed and available as a GitHub Action and npm package:

If you maintain public developer documentation, you can open a public audit request with one page. I am especially interested in examples where whitespace, generated snippets, tabs, or Unicode matter.

Disclosure: I used AI assistance to review wording and test coverage. I verified the implementation, commands, limitations, and reported evidence before publication.

Top comments (0)