DEV Community

Logiover
Logiover

Posted on

How to Scrape Website to Markdown Crawler for LLM & RAG — Free No-Code Guide (2026)

#website #webscraping #api #python

If you need to pull Website to Markdown Crawler for LLM & RAG data at scale and there's no clean official API, this guide shows how to export it to JSON, CSV or Excel in a couple of minutes — with no scraping code to maintain.

We'll use the Website to Markdown Crawler for LLM & RAG, a hosted scraper on the Apify platform. You only need a free Apify token — no Website to Markdown Crawler for LLM & RAG login, no proxies, no reverse-engineering.

What data you get

Each run returns structured records with fields like:

  • wordCount
  • url
  • title
  • metaDescription
  • h1
  • lang
  • canonical
  • crawledAt
  • text
  • markdown
  • html

Results are paginated automatically — you can pull thousands of rows per run and export them as JSON, CSV, Excel, JSONL or XML.

Option 1 — No code (Apify Console)

  1. Open the actor: https://apify.com/logiover/website-text-markdown-crawler
  2. Click Try for free.
  3. Leave the input empty for a broad sample, or set the fields below.
  4. Run it, then Export the dataset as CSV/JSON/Excel.

Option 2 — Node.js 

npm install website-text-markdown-crawler
Enter fullscreen mode Exit fullscreen mode 
const scrape = require("website-text-markdown-crawler");

(async () => {
  const items = await scrape({}, { token: process.env.APIFY_TOKEN });
  console.log(items.length, "results");
  console.log(items[0]);
})();
Enter fullscreen mode Exit fullscreen mode

Option 3 — Python 

pip install website-text-markdown-crawler
Enter fullscreen mode Exit fullscreen mode 
from website_text_markdown_crawler import scrape

items = scrape({}, token="YOUR_APIFY_TOKEN")
print(len(items), "results")
print(items[0])
Enter fullscreen mode Exit fullscreen mode

Grab a free token at https://console.apify.com/account/integrations.

Input options

Field Type What it does
startUrls array Start URLs
crawlScope string Crawl scope
maxPagesToCrawl integer Max pages to crawl
saveMarkdown boolean Save Markdown
saveText boolean Save plain text
saveHtml boolean Save HTML
maxConcurrency integer Max concurrency
proxyConfiguration object Proxy configuration

Common use cases

  • Build a fresh Website to Markdown Crawler for LLM & RAG dataset for analysis or dashboards
  • Feed Website to Markdown Crawler for LLM & RAG records into your own app, CRM or spreadsheet
  • Monitor changes on a schedule and get alerts via webhooks
  • Enrich leads or research with up-to-date Website to Markdown Crawler for LLM & RAG data

Automation & export

Because it runs on Apify, you can schedule it, trigger it via webhook, and pipe results into Google Sheets, S3, Zapier, Make or n8n. Every run's dataset is downloadable as CSV, JSON, JSONL, Excel or XML.

FAQ

Do I need an API key? Only a free Apify token. No Website to Markdown Crawler for LLM & RAG account needed.

How many results can I get? Thousands per run; raise the limit to pull more.

Is it an official Website to Markdown Crawler for LLM & RAG API? No — it's an unofficial, maintained alternative when there's no official or affordable API.

Can I export to CSV/Excel? Yes — JSON, CSV, JSONL, Excel and XML.

▶️ Run the Website to Markdown Crawler for LLM & RAG on Apify: https://apify.com/logiover/website-text-markdown-crawler

📚 Docs & code examples: https://github.com/logiover/website-text-markdown-crawler

🧰 More scrapers: https://apify.com/logiover

Top comments (0)