If you need to pull Sitemap to URL data at scale and there's no clean official API, this guide shows how to export it to JSON, CSV or Excel in a couple of minutes — with no scraping code to maintain.
We'll use the Sitemap to URL Crawler: RAG & AI Data Feeder, a hosted scraper on the Apify platform. You only need a free Apify token — no Sitemap to URL login, no proxies, no reverse-engineering.
What data you get
Each run returns structured records with fields like:
urllastmodchangefreqprioritysourceSitemap
Results are paginated automatically — you can pull thousands of rows per run and export them as JSON, CSV, Excel, JSONL or XML.
Option 1 — No code (Apify Console)
- Open the actor: https://apify.com/logiover/sitemap-to-url-crawler
- Click Try for free.
- Leave the input empty for a broad sample, or set the fields below.
- Run it, then Export the dataset as CSV/JSON/Excel.
Option 2 — Node.js
npm install sitemap-to-url-crawler
const scrape = require("sitemap-to-url-crawler");
(async () => {
const items = await scrape({}, { token: process.env.APIFY_TOKEN });
console.log(items.length, "results");
console.log(items[0]);
})();
Option 3 — Python
pip install sitemap-to-url-crawler
from sitemap_to_url_crawler import scrape
items = scrape({}, token="YOUR_APIFY_TOKEN")
print(len(items), "results")
print(items[0])
Grab a free token at https://console.apify.com/account/integrations.
Input options
| Field | Type | What it does |
|---|---|---|
startUrls |
array | Start URLs (Domain or Sitemap) |
maxUrls |
integer | Max URLs to Extract |
proxyConfiguration |
object | Proxy Configuration |
Common use cases
- Build a fresh Sitemap to URL dataset for analysis or dashboards
- Feed Sitemap to URL records into your own app, CRM or spreadsheet
- Monitor changes on a schedule and get alerts via webhooks
- Enrich leads or research with up-to-date Sitemap to URL data
Automation & export
Because it runs on Apify, you can schedule it, trigger it via webhook, and pipe results into Google Sheets, S3, Zapier, Make or n8n. Every run's dataset is downloadable as CSV, JSON, JSONL, Excel or XML.
FAQ
Do I need an API key? Only a free Apify token. No Sitemap to URL account needed.
How many results can I get? Thousands per run; raise the limit to pull more.
Is it an official Sitemap to URL API? No — it's an unofficial, maintained alternative when there's no official or affordable API.
Can I export to CSV/Excel? Yes — JSON, CSV, JSONL, Excel and XML.
▶️ Run the Sitemap to URL Crawler: RAG & AI Data Feeder on Apify: https://apify.com/logiover/sitemap-to-url-crawler
📚 Docs & code examples: https://github.com/logiover/sitemap-to-url-crawler
🧰 More scrapers: https://apify.com/logiover
Top comments (0)