DEV Community

Cover image for Extracting Markdown Headers and Links for LLMs with mq
Takahiro Sato
Takahiro Sato

Posted on • Edited on

Extracting Markdown Headers and Links for LLMs with mq

#cli #markdown #programming #llm

There may be times when you want to extract just the headers or links from Markdown files, rather than working with the entire document.
You can also use Large Language Models (LLMs) with only the extracted headers or links, instead of always providing the full document.

Here's how you could try this with mq, a jq-like CLI tool for Markdown:

mq command

Command

Note: The following command assumes you are running it in zsh.

mq '.h | let h = to_text() | s"- [${h}](${__FILE__})"' **/*.md
Enter fullscreen mode Exit fullscreen mode
  • .h selects all headers
  • let h = to_text() gets header text
  • s"- [${h}](${__FILE__})" outputs a Markdown link with the file path

Example

Suppose you have:

docs/intro.md

# Introduction
## Getting Started
Enter fullscreen mode Exit fullscreen mode

docs/usage.md

# Usage
## Advanced
Enter fullscreen mode Exit fullscreen mode

Run:

mq '.h | s"- [${self}](${__FILE__})"' docs/*.md
Enter fullscreen mode Exit fullscreen mode

Output:

- [# Introduction](docs/intro.md)
- [## Getting Started](docs/intro.md)
- [## Example Code Block](docs/intro.md)
- [# Usage](docs/usage.md)
- [## Advanced](docs/usage.md)
Enter fullscreen mode Exit fullscreen mode

Run:

mq '.h | let h = to_text() | s"- [${h}](${__FILE__})"' docs/*.md
Enter fullscreen mode Exit fullscreen mode

Output:

- [Introduction](docs/intro.md)
- [Getting Started](docs/intro.md)
- [Example Code Block](docs/intro.md)
- [Usage](docs/usage.md)
- [Advanced](docs/usage.md)
Enter fullscreen mode Exit fullscreen mode

You could use this list as a table of contents, or provide it to an LLM as a hint for summarization or exploration, instead of providing the full document.
This approach might be useful in some scenarios.

Install 

cargo install --git https://github.com/harehare/mq.git mq-cli
# or
brew install harehare/tap/mq
Enter fullscreen mode Exit fullscreen mode

Support

mq makes it easy to extract and format Markdown structure—such as headers and links—which may be helpful in some cases as an alternative to inputting the entire document.

Top comments (0)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.