DEV Community

Cover image for Eliminate Redundant Markdown Parsing: Typically 2-10x Faster AI Streaming
king
king

Posted on

Eliminate Redundant Markdown Parsing: Typically 2-10x Faster AI Streaming

#ai #tooling #performance #opensource

Yesterday, I released incremark, a project I developed over the weekend. The real-world performance exceeded all expectations—typically delivering 2-10x speed improvements for AI streaming scenarios, with even greater gains for longer documents. While initially intended as an internal tool for my product, I realized that open-sourcing it could be a great direction to take.

The Problem

Every time an AI streams a new chunk of text, traditional markdown parsers re-parse the entire document from scratch—wasting CPU cycles on content that's already been rendered. Incremark solves this by parsing only what's new.

Benchmark Results: See It to Believe It

Short Markdown Document:

Longer Markdown Document:

Note: The performance improvement multiplier may vary between benchmark runs due to the chunking strategy. The demo page uses random chunk lengths: const chunks = content.match(/[\s\S]{1,20}/g) || []. This chunking affects stable block generation and better simulates real-world scenarios where a chunk may contain content from the previous or next chunk. Regardless of how the content is chunked, performance improvements are guaranteed. The demo website does not use any artificial chunking strategies to inflate results.

Live Demos:

For extremely long markdown documents, the performance gains are even more dramatic. A 20KB markdown benchmark achieves an incredible 46x speed improvement. The longer your content, the more dramatic the speedup—there's no theoretical upper limit.

Key Advantages

⚡ Typically 2-10x faster for AI streaming scenarios

🚀 Even greater speedup for longer documents (up to 46x tested)

🎯 Zero redundant parsing - each character is parsed only once

✨ Perfect for AI streaming - optimized for incremental updates

💪 Works with regular markdown too - not just for AI use cases

🔧 Framework support - React and Vue components included

Why Is It So Fast?

The Problem with Traditional Parsers

Anyone who has built an AI chat application knows that AI streaming output delivers content in small chunks to the frontend. After receiving each chunk, the entire markdown string must be fed to a markdown parser (whether it's remark, marked.js, or markdown-it). These parsers re-parse the entire markdown document every time, even the parts that have already been rendered and are stable. This creates significant performance waste.

Tools like vue-stream-markdown have made efforts at the rendering layer by rendering stable tokens as stable components and only updating unstable components, achieving smooth streaming output in the UI layer.

However, this still doesn't solve the fundamental performance issue: repeated parsing of markdown text. This is the real CPU performance monster. The longer the output document, the higher the performance waste.

Incremark's Core Performance Optimization

Beyond achieving smooth component reuse and updates at the UI rendering layer, incremark's key innovation is in markdown parsing: it only parses unstable markdown blocks and never re-parses stable ones. This reduces parsing complexity from O(n²) to O(n). In theory, the longer the output, the greater the performance improvement.

1. Incremental Parsing: From O(n²) to O(n)

Traditional parsers re-parse the entire document every time, causing parsing work to grow quadratically. Incremark's IncremarkParser class uses an incremental parsing strategy (see IncremarkParser.ts):

// Design Philosophy:
// 1. Maintain a text buffer to receive streaming input
// 2. Identify "stable boundaries" and mark completed blocks as 'completed'
// 3. For blocks currently being received, re-parse only that block's content
// 4. Complex nested nodes are treated as a whole until confirmed complete
Enter fullscreen mode Exit fullscreen mode

2. Intelligent Boundary Detection

The findStableBoundary() method in the append function is a key optimization:

append(chunk: string): IncrementalUpdate {
  this.buffer += chunk
  this.updateLines()

  const { line: stableBoundary, contextAtLine } = this.findStableBoundary()

  if (stableBoundary >= this.pendingStartLine && stableBoundary >= 0) {
    // Only parse newly completed blocks, never re-parse already completed content
    const stableText = this.lines.slice(this.pendingStartLine, stableBoundary + 1).join('\n')
    const ast = this.parse(stableText)
    // ...
  }
}
Enter fullscreen mode Exit fullscreen mode

3. State Management to Avoid Redundant Computation

The parser maintains several key states to eliminate duplicate work:

  • buffer: Accumulated unparsed content
  • completedBlocks: Array of completed blocks that are never re-parsed
  • lineOffsets: Line offset prefix sums, supporting O(1) line position calculations
  • context: Tracks nesting state of code blocks, lists, etc.

4. Incremental Line Update Optimization

The updateLines() method only processes new content, avoiding full split operations:

private updateLines(): void {
  // Find the last incomplete line (which may be continued by a new chunk)
  const lastLineStart = this.lineOffsets[prevLineCount - 1]
  const textFromLastLine = this.buffer.slice(lastLineStart)

  // Re-split only the last line and subsequent content
  const newLines = textFromLastLine.split('\n')
  // Only update the changed portions
}
Enter fullscreen mode Exit fullscreen mode

Performance Comparison

This design delivers exceptional results in real-world testing:

Document Size Traditional Parser (chars) Incremark (chars) Reduction
1KB 1,010,000 20,000 98%
5KB 25,050,000 100,000 99.6%
20KB 400,200,000 400,000 99.9%

The Key Invariant

Incremark's performance advantage stems from a key invariant: Once a block is marked as completed, it is never re-parsed. This ensures each character is parsed at most once, achieving O(n) time complexity.

🚀 Get Started Now

Stop wasting CPU cycles on redundant parsing. Try incremark today:

Quick Install 

npm install @incremark/core
# For React
npm install @incremark/react
# For Vue
npm install @incremark/vue
Enter fullscreen mode Exit fullscreen mode

Resources

Use Cases

Perfect for:

  • 🤖 AI chat applications with streaming responses
  • ✍️ Live markdown editors
  • 📝 Real-time collaborative documents
  • 📊 Streaming data dashboards with markdown content
  • 🎓 Interactive learning platforms

Whether you're building an AI interface or just want faster markdown rendering, incremark delivers the performance you need.

💬 Try It Out & Show Your Support

I'd love for you to try incremark in your projects and see the performance difference for yourself! The live demos are the best way to experience the speed improvements in action.

If you find incremark useful, please consider giving it a ⭐ star on GitHub—it really helps the project gain visibility and motivates me to keep improving it. Your feedback, issues, and contributions are also highly welcome!

Thank you for your interest in incremark! Let's make markdown rendering faster together. 🚀

Top comments (0)