Spent the better part of a week debugging why fenced code blocks inside nested lists were coming out as plain text paragraphs. My converter handled top-level code fences fine. It handled lists fine. But a list item containing a code block — the exact structure every README on GitHub uses — was silently flattened.
The bug was in my re-indentation pass. After converting the DOM to Markdown, I re-indented nested content to match list depth. Markdown's rule is that a fenced code block inside a list item must be indented to align with the list item's content — four spaces for a - marker, or the width of the marker plus one. My pass was re-indenting based on visual column position, not AST depth, so a code fence at list depth 2 got indented 3 spaces instead of 4. CommonMark parsers then treated the fence as a lazy continuation of the paragraph above it. Not an error. Just... gone.
The fix was unglamorous: track list-item content columns explicitly during the DOM walk, not after. Compute each block's required indent from its parent list marker, and apply it to the fence lines themselves, not just the first line. It's the kind of bug that only shows up when a human diffs the output against the source and notices a function definition vanished.
This is the second time I've been bitten by treating Markdown as a text format when it's actually a whitespace-sensitive grammar. The lesson: never post-process indentation on generated Markdown. Get it right during emission, from the parse tree, where nesting depth is still known.
If you're building document-to-text pipelines for agent context windows, this class of bug is insidious because the output looks reasonable. No error, no truncation, just silently missing content — and your RAG pipeline will happily embed a README that's missing its usage examples.
I ended up packaging the whole thing as https://x402.freeq.one/tools/html_to_markdown.html
Top comments (0)