DEV Community

Cover image for Processing Markdown in Python with mq
Takahiro Sato
Takahiro Sato

Posted on

Processing Markdown in Python with mq

#python #markdown #cli #ai

The markdown-query package brings the functionality of the mq Markdown processor to Python. mq uses a jq-like syntax to filter, transform, and extract data from Markdown documents.

Installation 

pip install markdown-query
Enter fullscreen mode Exit fullscreen mode

Basic Usage

The primary interface is the run() function, which takes three parameters: a query string, markdown content, and optional configuration.

import mq

markdown = """# Title

This is a paragraph.

## Section

- Item 1
- Item 2

``javascript
console.log("hello");
``
"""

# Extract all headings
result = mq.run("select(.h1 || .h2)", markdown, None)
print(result.values)
# ['# Title', '## Section']

# Get heading text without markdown formatting
result = mq.run("select(.h1 || .h2) | to_text()", markdown, None)
print(result.values)
# ['Title', 'Section']
Enter fullscreen mode Exit fullscreen mode

Working with Different Input Formats

The library supports multiple input formats through the Options class:

import mq

# Process HTML input
html = '<h1>Title</h1><p>Content</p><ul><li>Item</li></ul>'

options = mq.Options()
options.input_format = mq.InputFormat.HTML

result = mq.run(".h1 | upcase()", html, options)
print(result.values)
# ['# TITLE']
Enter fullscreen mode Exit fullscreen mode

Available input formats:

  • InputFormat.MARKDOWN (default)
  • InputFormat.HTML
  • InputFormat.MDX
  • InputFormat.TEXT

Query Examples

Extract Code Blocks 

# Get all code blocks
code_blocks = mq.run("select(.code)", markdown, None)

# Get code block content as text
code_text = mq.run("select(.code) | to_text()", markdown, None)
Enter fullscreen mode Exit fullscreen mode

Filter by Content 

# Find headings containing specific text
headings = mq.run('select(.h1 || .h2) | select(test("Section"))', markdown, None)

# Extract list items
items = mq.run(".[] | to_text()", markdown, None)
Enter fullscreen mode Exit fullscreen mode

Transform Content 

# Convert headings to uppercase
upper_headings = mq.run("select(.h1, .h2) | upcase()", markdown, None)

# Replace text in paragraphs
modified = mq.run('select(.paragraph) | gsub("paragraph"; "text")', markdown, None)
Enter fullscreen mode Exit fullscreen mode

Accessing Result Data

The run() function returns an MQResult object with a values list:

result = mq.run("select(.h1)", markdown, None)

# Access all values
for value in result.values:
    print(value)

# Access individual items
first_heading = result[0]
print(first_heading.text)           # Text content
print(first_heading.markdown_type)  # MarkdownType enum

# Iterate over results
for item in result:
    print(f"Type: {item.markdown_type}, Text: {item.text}")
Enter fullscreen mode Exit fullscreen mode

Integration with Other Tools

The library works well with other Python markdown processing tools:

from markitdown import MarkItDown
import mq

# Convert web pages to markdown, then process with mq
markitdown = MarkItDown()
result = markitdown.convert("https://example.com")

# Extract specific content
code_samples = mq.run(".code | to_text()", result.text_content, None)
all_links = mq.run(".link | to_html()", result.text_content, None)
Enter fullscreen mode Exit fullscreen mode

Configuration Options

The Options class provides additional configuration:

options = mq.Options()
options.input_format = mq.InputFormat.HTML
options.list_style = mq.ListStyle.PLUS      # Use + for lists
options.link_title_style = mq.TitleSurroundStyle.SINGLE
options.link_url_style = mq.UrlSurroundStyle.ANGLE

result = mq.run("select(.list)", content, options)
Enter fullscreen mode Exit fullscreen mode

Error Handling

Queries that fail will raise a PyRuntimeError:

try:
    result = mq.run("invalid_query", markdown, None)
except RuntimeError as e:
    print(f"Query failed: {e}")
Enter fullscreen mode Exit fullscreen mode

Performance

The library is built on Rust and compiled to native code, providing fast processing for large markdown files.

Resources

Support

The markdown-query package provides a straightforward way to apply mq's markdown processing capabilities in Python applications, from simple content extraction to complex document transformations.

Top comments (0)