DEV Community

FIRAT
FIRAT

Posted on Originally published at kuponaut.com

How to give an AI agent live web search and scraping with Dumpling AI (MCP, n8n, credits explained)

An AI agent is only as current as the data it can reach. Dumpling AI puts web scraping, crawling, structured extraction, search and document tools behind one API key, and also exposes them as a hosted MCP server that Claude, Cursor and n8n can call directly. This walkthrough follows Dumpling AI's own documentation, read on 26 October 2026: creating the key, testing a scrape, extracting structured fields, wiring the MCP server into an agent, and working out what each call costs in credits. It ends with the prices shown on the pricing page that day.

When Dumpling AI is the right tool

The use case is an agent or automation that needs fresh information from the open web: reading a product page, pulling the text of an article, collecting a few pages from a documentation site, or turning a page into clean fields such as title and price. Dumpling AI handles the fetching, JavaScript rendering and cleanup, and returns Markdown, HTML, a screenshot or structured JSON.

Its documentation describes a Unified API that gives "one API key and one control surface for multiple managed providers", and names Firecrawl, Serper, Perplexity and DataForSEO among them. If you only scrape one site occasionally, a small script may be enough. Dumpling AI starts to pay off when several tools (scraping, search, document extraction, YouTube transcripts) would otherwise mean several accounts and several bills.

Step 1: create an API key and test in the Playground

Sign in to the dashboard at app.dumplingai.com and open the API Keys page to create a key. Store it like a password: every request sends it as a Bearer token in the Authorization header, together with Content-Type: application/json.

Before wiring anything into an agent, the getting-started page recommends the Direct Endpoint Playground in the app. Try one page you care about there first, so you can see the output format and the credit use before an agent starts making calls on its own.

Step 2: scrape a page into clean Markdown

The first call in the docs is the scrape endpoint, a POST /api/v1/scrape with a url in the body. Three optional settings matter most: format (markdown, html or screenshot; markdown is the default), cleaned (true by default, strips page clutter) and renderJs (true by default, so JavaScript-heavy pages load before they are read). The docs note that turning renderJs off can be faster for simple static pages.

curl -X POST https://app.dumplingai.com/api/v1/scrape \
  -H "Authorization: Bearer $DUMPLING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/pricing", "format": "markdown", "cleaned": true}'
Enter fullscreen mode Exit fullscreen mode

The response gives you the page title, the final URL after redirects, the content in the format you asked for and a metadata object. Each scrape costs 10 credits according to the endpoint reference. Responses also carry X-RateLimit-Limit and X-RateLimit-Remaining headers, which are worth logging once an agent runs unattended.

Step 3: crawl a small section of a site

When one page is not enough (a pricing page plus its FAQ, say), the crawl endpoint (/api/v1/crawl) follows links from a starting URL. Its limit defaults to 5 pages and depth to 2, and output can be markdown, text or raw.

Crawling is billed per page: the docs say it "uses 10 credits per crawled page". Keep limit and depth low until you know how many pages a site really has; the scraping tutorial's own advice is to "be deliberate about which pages you fetch to keep usage efficient".

Step 4: extract structured fields instead of raw text

If your agent needs specific values rather than a whole page, the extract endpoint (/api/v1/extract) takes a url and a schema, for example a title as a string and a price as a number, and returns a results object in that shape plus a temporary screenshot URL.

{
  "url": "https://example.com/product/123",
  "schema": {
    "title": "string",
    "price": "number",
    "inStock": "boolean"
  }
}
Enter fullscreen mode Exit fullscreen mode

Extraction is the expensive call: 250 credits per request in the endpoint reference, against 10 for a scrape. Content is capped at 100,000 tokens and the result is a single object, so a list of products needs separate handling. A practical pattern is to scrape first, check the Markdown, and use extract only for the pages where you really need typed fields.

Step 5: connect the MCP server to Claude, Cursor or n8n

Dumpling AI runs a hosted MCP server at mcp.dumplingai.com/mcp, with no installation required. You pass the same API key as an Authorization header with the value Bearer <your key>. The tutorial gives set-up steps for Claude Code (claude mcp add with the http transport), Claude Desktop (claude_desktop_config.json), Cursor (~/.cursor/mcp.json), Windsurf, and VS Code with Copilot.

claude mcp add --transport http dumplingai https://mcp.dumplingai.com/mcp \
  --header "Authorization: Bearer $DUMPLING_API_KEY"
Enter fullscreen mode Exit fullscreen mode

In n8n, the documented route is an AI Agent node plus an MCP Client Tool node, with credentials set to Connection Type "HTTP Streamable", Authentication "Header Auth" and Header Name Authorization. Once connected, the agent can call tools such as scrape, crawl, extract, search, search_news, get_youtube_transcript and doc_to_text by itself. That convenience is also the main cost risk, so give the agent a clear instruction about how many pages it may fetch per task.

Step 6: Make.com, HTTP nodes or the v2 Unified API

If you prefer fixed workflows to an agent, the getting-started page says the current Make.com module uses /api/v1 and needs only the API key, and that n8n and other automation tools can call the endpoints through ordinary HTTP nodes.

For technical users there is also an /api/v2 lane. Its run endpoint (POST /api/v2/run) takes a type (capability or endpoint), an id such as google_search or firecrawl.scrape, and an input object, so you can either use a standardised capability or call a specific upstream provider endpoint. The v2 reference we read does not list credit costs per capability, so test a few calls and check the usage and transactions pages in your account before scaling up.

What Dumpling AI costs (read 26 October 2026)

On 26 October 2026 the pricing page showed three plans billed annually, labelled "2 months free": Starter $40 a month (1,200,000 credits a year, 30 requests per minute), Pro $124 a month (4,200,000 credits a year, 60 requests per minute) and Business $249 a month (9,600,000 credits a year, 120 requests per minute). Monthly billing was recorded on 11 October 2026 at $49, $149 and $299. Plan credits do not roll over, and extra credits are sold as auto-recharge packs.

To put the credit figures in context using the endpoint costs above: at 10 credits per scrape, Starter's yearly 1,200,000 credits cover about 120,000 scrapes, or about 4,800 structured extractions at 250 credits each (our calculation).

A budget rule that has worked for us: decide the maximum credits per agent task (say 200, which is 20 scrapes or less than one extraction), put it in the system prompt, and check the transactions page after the first week.

Sources

Originally published on Kuponaut, where the version with the current partner offer and dated price record lives. Kuponaut is an affiliate of some of the tools it covers; this article contains no affiliate links.

Top comments (0)