An agent piped a wiki export through my HTML-to-Markdown converter into a RAG pipeline and reported that hierarchy was disappearing. Every nested task list came out flat: sub-items rendered as siblings of their parents. Sections got chunked together that had nothing to do with each other, and retrieval started returning paragraphs attributed to the wrong topic.
Here's the part that stung: the Markdown looked perfect in my own renderer.
That clause was the entire bug. My converter indented nested list items by two spaces. Two-space indentation is accepted by lenient parsers and most browser previews, but strict CommonMark requires indenting by the marker width — four spaces under a - bullet, three under a 1. ordered item. Feed two-space output into a compliant parser and the nesting silently collapses. No error, no warning, just a flat list wearing a nested list's clothes. My test suite only ever checked output against one parser: the lenient one. Round-trip validation with a single dialect proves nothing about the next parser downstream.
The fix wasn't hard, the diagnosis was. I now run a small harness in CI that converts HTML to Markdown, then re-parses the result with three different Markdown implementations and diffs the resulting trees. Any disagreement fails the build. It has caught far more than the list bug — indented code blocks are the same trap: a four-space-indented paragraph inside a list quietly becomes code in some parsers and stays a paragraph in others, so the converter emits fenced blocks by default now.
The uncomfortable lesson: "Markdown" is not one format, it's a family of dialects that disagree about indentation, heading style, emphasis and tables. That's why the converter at https://x402.freeq.one/tools/html_to_markdown.html exposes heading style, bullet marker and code fence options — not to be fancy, but because whatever parser sits at the end of your pipeline gets a vote. If you're feeding converted HTML into RAG, test the round trip with that parser, not just whichever one you happened to have installed when you wrote your tests.
Top comments (1)
Comparing parsed trees is a much stronger contract than comparing the rendered preview. One nuance in the CommonMark explanation: required child indentation depends on the marker plus the following spacing. A dash with one following space can have children indented by two spaces; task-list syntax and blank lines can introduce additional dialect differences.
I would preserve the smallest failing input as a fixture and assert its parent-child relationships across the three parsers. That distinguishes an indentation defect from a task-list extension disagreement. Failing every tree difference also needs a declared supported dialect, or an intentional extension can become an endless source of false alarms.