DEV Community

Pirate Prentice
Pirate Prentice

Posted on

n8n HTTP Request Node: Advanced Patterns, Auth, Pagination, and Error Handling (Free Workflow JSON)

#n8n #automation #webdev #javascript

The n8n HTTP Request node is the most versatile node in n8n — it can call any REST API, webhook, or HTTP endpoint your workflow needs. But most tutorials only cover the basics. This guide goes deeper: authentication patterns, pagination, batching, retry logic, and a free workflow JSON that handles the tricky parts for you.

When to Use the HTTP Request Node

Use the HTTP Request node when:

  • The API you need doesn't have a dedicated n8n node
  • You need fine-grained control over headers, query params, or body format
  • You're calling internal services, custom APIs, or webhooks
  • The built-in node for a service doesn't expose the endpoint you need

Authentication Patterns

API Key (Header)

Most common. Set it in the node's Authentication dropdown → Header Auth, then add:

Name: Authorization
Value: Bearer {{ $env.MY_API_KEY }}
Enter fullscreen mode Exit fullscreen mode

Or use Predefined Credential Type if n8n already has it (GitHub, Stripe, Notion, etc.) — this auto-handles token refresh for OAuth providers.

Basic Auth

Select Basic Auth and enter username/password. n8n base64-encodes the header automatically.

OAuth2

Select OAuth2 and pick or create an OAuth2 credential. n8n handles token refresh automatically — no manual refresh logic needed.

Query Parameter Auth

Some APIs use ?api_key=xxx. Add it under Query Parameters:

Name: api_key
Value: {{ $env.MY_API_KEY }}
Enter fullscreen mode Exit fullscreen mode

Sending Different Body Types

Body Format When to Use
JSON Default for most REST APIs
Form Data (URL-encoded) Legacy APIs, OAuth token endpoints
Multipart Form Data File uploads
Raw / Binary Webhook payloads, binary streams
None GET/DELETE requests with no body

For JSON bodies, toggle Body → JSON and use n8n expressions:

{
  "user_id": "{{ $json.id }}",
  "email": "{{ $json.email }}",
  "created_at": "{{ $now.toISO() }}"
}
Enter fullscreen mode Exit fullscreen mode

Pagination Patterns

Offset Pagination

The most common pattern — loop until results dry up:

Schedule Trigger
  → Set (offset = 0, results = [])
  → HTTP Request (GET /items?limit=100&offset={{ $json.offset }})
  → IF (response.items.length === 100)
      TRUE → Set (offset += 100) → loop back to HTTP Request
      FALSE → done
Enter fullscreen mode Exit fullscreen mode

Cursor Pagination

For APIs that return a next_cursor or next_page_token:

Loop:
  HTTP Request (GET /items?cursor={{ $json.next_cursor || "" }})
  → IF ($json.next_cursor exists)
      TRUE → feed cursor back, loop
      FALSE → done
Enter fullscreen mode Exit fullscreen mode

Link Header Pagination (GitHub-style)

GitHub returns Link: <url>; rel="next" in response headers. Extract it in a Code node:

const linkHeader = $input.first().headers['link'] || '';
const match = linkHeader.match(/<([^>]+)>;\s*rel="next"/);
return [{ json: { nextUrl: match ? match[1] : null } }];
Enter fullscreen mode Exit fullscreen mode

Then loop while nextUrl is not null.

Error Handling

Retry on Failure

Enable Retry on Fail in the HTTP Request node options:

  • Max Tries: 3
  • Wait Between Tries: 1000ms

This handles transient network errors and 5xx responses automatically.

Handle Specific Status Codes

Enable Always Output Data to prevent the node from throwing on 4xx/5xx, then route based on status:

HTTP Request (Always Output Data ON)
  → IF ($json.statusCode >= 400)
      TRUE → Code (log error, notify Slack)
      FALSE → continue
Enter fullscreen mode Exit fullscreen mode

Rate Limiting

For APIs with rate limits, add a Wait node between batched requests:

Split in Batches (batch size: 10)
  → HTTP Request
  → Wait (1 second)
  → next batch
Enter fullscreen mode Exit fullscreen mode

Gotchas Table

Symptom Root Cause Fix
SSL certificate error Self-signed cert or expired cert Enable Ignore SSL Issues in node settings
401 on every request Token not refreshing Switch to Predefined Credential Type (handles OAuth2 refresh)
Body not sent on GET n8n strips body from GET by default Switch to POST, or use query params
JSON parse error API returned HTML error page Enable Always Output Data, log the raw response
Timeout on large responses Response too slow Increase Timeout in node options (default 10s)
CORS error Browser-based test hitting n8n cloud CORS is irrelevant server-side; happens only in browser previews
Missing headers in response n8n doesn't expose all headers by default Enable Include Response Headers and Status Code

Free Workflow JSON — Paginated API Fetcher

This workflow fetches all pages from an offset-paginated API and collects results:

{
  "name": "Paginated HTTP Fetcher",
  "nodes": [
    {
      "parameters": { "rule": { "interval": [{ "field": "hours", "hoursInterval": 24 }] } },
      "name": "Schedule Trigger",
      "type": "n8n-nodes-base.scheduleTrigger",
      "typeVersion": 1.2,
      "position": [240, 300]
    },
    {
      "parameters": {
        "assignments": {
          "assignments": [
            { "name": "offset", "value": 0, "type": "number" },
            { "name": "allResults", "value": "=[]", "type": "array" }
          ]
        }
      },
      "name": "Init",
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [460, 300]
    },
    {
      "parameters": {
        "url": "https://api.example.com/items",
        "authentication": "predefinedCredentialType",
        "sendQuery": true,
        "queryParameters": {
          "parameters": [
            { "name": "limit", "value": "100" },
            { "name": "offset", "value": "={{ $json.offset }}" }
          ]
        },
        "options": { "response": { "response": { "includeResponseHeadersAndStatus": true } } }
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [680, 300]
    },
    {
      "parameters": {
        "conditions": {
          "options": { "version": 2 },
          "conditions": [{ "leftValue": "={{ $json.items.length }}", "rightValue": 100, "operator": { "type": "number", "operation": "gte" } }]
        }
      },
      "name": "More Pages?",
      "type": "n8n-nodes-base.if",
      "typeVersion": 2.2,
      "position": [900, 300]
    }
  ],
  "connections": {
    "Schedule Trigger": { "main": [[{ "node": "Init", "type": "main", "index": 0 }]] },
    "Init": { "main": [[{ "node": "HTTP Request", "type": "main", "index": 0 }]] },
    "HTTP Request": { "main": [[{ "node": "More Pages?", "type": "main", "index": 0 }]] }
  }
}
Enter fullscreen mode Exit fullscreen mode

n8n Workflow Starter Pack ($29)

If you want production-ready workflows with retry logic, error routing, and monitoring already built in, the n8n Workflow Starter Pack includes a paginated HTTP fetcher, webhook handler, and 10+ other automation patterns.

Every workflow is documented, tested, and imports in under 2 minutes.

What APIs are you hitting with the HTTP Request node? Drop your use case in the comments — happy to share the right auth pattern or pagination approach.

Top comments (1)

Collapse
 
pirateprentice profile image
Pirate Prentice •

What APIs are you hitting with the HTTP Request node? REST, internal services, GraphQL over HTTP, or something more exotic? Drop your use case — happy to share the right auth or pagination pattern.