Build an n8n Web Scraping Workflow Without Managing Infrastructure

Originally posted on the Scrapio blog — sharing here too.

n8n is one of the best workflow automation tools available — but its built-in HTTP nodes hit a wall fast when you're scraping real sites. JavaScript-rendered pages fail silently. Sites block datacenter IPs. Sessions don't persist.

Scrapio handles all of that behind a single API call, making it a natural backend for n8n scraping workflows.

What you'll need

• n8n (cloud or self-hosted)
• A Scrapio API key

Step 1 — Store your API key in n8n credentials

1. In n8n, go to Settings → Credentials → New Credential
2. Choose HTTP Request (generic)
3. Set Authentication to Header Auth
4. Name: Authorization, Value: Bearer sk-YOUR_KEY
5. Save as Scrapio API

Step 2 — Create the workflow

Add an HTTP Request node with these settings:

Field	Value
Method	POST
URL	https://api.scrapio.dev/v1/fetch
Authentication	Scrapio API (from step 1)
Body Content Type	JSON

Body:

{
  "url": "={{ $json.url }}",
  "output": ["markdown"]
}

The ={{ $json.url }} expression pulls the URL from whatever node feeds into this one — a Schedule trigger, a spreadsheet, a webhook, etc.

Step 3 — Parse the output

The response looks like:

{
  "request_id": "req_abc123",
  "mode": "inline",
  "status": "completed",
  "outputs": {
    "markdown": "# Page Title\n\nContent here..."
  },
  "usage": { "credits": 1 }
}

Add a Set node to extract {{ $json.outputs.markdown }} and pass it to downstream nodes — a Google Sheet, an AI summarizer, a database insert, whatever your workflow needs.

Example: daily competitor price monitor

A complete 4-node workflow:

1. Schedule Trigger — runs every day at 8am
2. HTTP Request (Scrapio) — fetches competitor pricing page as markdown
3. OpenAI node — extracts prices with a prompt: "From this markdown, return a JSON object with product names as keys and prices as values"
4. Google Sheets — appends the result to a tracking sheet

No servers. No Playwright. No proxy rotation. The entire workflow runs in n8n cloud and costs pennies per day.

Handling JavaScript-rendered pages

Add "render_js": true to the request body for SPAs and dynamic pages:

{
  "url": "={{ $json.url }}",
  "output": ["markdown"],
  "render_js": true
}

Handling multiple URLs

Feed a list of URLs from a Spreadsheet File or Airtable node, split with SplitInBatches, and pass each through the Scrapio HTTP Request node. n8n handles the loop; Scrapio handles the rendering.

Next steps

• See the Webpage to Markdown template for a copy-paste API recipe
• Read about scheduled scraping jobs for cron-based automation
• Check the Scrapio Fetch docs for all request options