Scraping API credits are not the same as requests

You pick a scraping API plan that says 1,000,000 credits, connect it to a job queue, and assume you can fetch something close to 1,000,000 pages. Then the first real target needs JavaScript rendering, a better proxy pool, or retry logic, and your budget model stops matching production.

A credit is a billing unit, not a request

Most scraping APIs sell the same basic shape: send a URL, get back HTML or extracted data, let the vendor deal with proxies, browsers, and anti-bot systems.

That hides an important detail. The API call is not always the thing you pay for. You pay for whatever the provider defines as a billable unit, and that unit can change based on options.

A static HTML fetch might cost 1 credit. The same URL with browser rendering might cost 5. A premium proxy might cost 10 or 25. A stealth proxy can cost much more. AI extraction may add another fixed charge.

That is not inherently bad. Running browsers and higher-quality proxies costs more. The problem is when the pricing page makes the large credit number easy to see and the multipliers easy to miss.

Here is the budgeting code I usually write before trusting any scraping plan:

const credits = 1_000_000;

const scenarios = [
  { name: "static HTML", creditsPerRequest: 1 },
  { name: "JS rendering", creditsPerRequest: 5 },
  { name: "premium proxy + JS", creditsPerRequest: 25 },
  { name: "stealth proxy", creditsPerRequest: 75 },
  { name: "JS + AI extraction", creditsPerRequest: 10 }
];

for (const s of scenarios) {
  console.log(
    `${s.name}: ${Math.floor(credits / s.creditsPerRequest).toLocaleString()} requests`
  );
}

Output:

static HTML: 1,000,000 requests
JS rendering: 200,000 requests
premium proxy + JS: 40,000 requests
stealth proxy: 13,333 requests
JS + AI extraction: 100,000 requests

Same plan. Very different workload.

The hard pages are the expensive pages

The multiplier usually appears exactly where your scraper starts failing.

A static request works fine until you hit a React app and get an empty shell:

<div id="root"></div>
<script src="/assets/app.83f1c.js"></script>

Your HTTP request returned 200 OK, but your parser finds no product price, no listing rows, and no useful text. You turn on browser rendering.

Then another site returns a CAPTCHA page:

<title>Just a moment...</title>
<p>Checking if the site connection is secure</p>

Your API call still may return 200 OK, but the content is not the target page. You turn on a better proxy tier or stealth mode.

Then the site randomizes markup, so your CSS selector breaks:

const price = $(".product-price").text(); // ""

You add AI extraction or a more flexible parser.

Each fix can be reasonable. Together, they can turn a cheap scrape into a much smaller quota than you expected.

Measure cost per successful extraction

Request cost is the wrong metric. Successful extraction cost is the useful one.

If you send 10,000 URLs and get 8,000 usable records, you should divide credits by 8,000, not by 10,000. Failed requests matter, but so do silent failures where the HTTP response looks fine and the data is wrong.

A simple harness can make this visible:

async function scrapeAndValidate(client, url) {
  const result = await client.scrape(url, {
    renderJs: true,
    extract: {
      title: "string",
      price: "string"
    }
  });

  if (!result.title || !result.price) {
    return {
      ok: false,
      reason: "missing_required_fields",
      statusCode: result.statusCode,
      sample: result.text?.slice(0, 200)
    };
  }

  return { ok: true, data: result };
}

Log these counts per provider and per option set:

urls_submitted=10000
http_200=9420
valid_records=8110
captcha_pages=620
empty_spa_shells=410
timeouts=180
credits_spent=50000
credits_per_valid_record=6.16

That last number is the one you should compare.

Some providers also avoid charging failed jobs, or cache repeated fetches for a period such as 24 hours. Those details can matter more than the headline quota if your pipeline retries aggressively or revisits the same URLs often.

Authenticated scraping changes the problem

Public pages mostly fail because of rendering, blocking, geo restrictions, or bad selectors. Authenticated pages add credential handling.

The quick version often looks like this:

await page.goto("https://example.com/login");
await page.fill("input[name=email]", process.env.LOGIN_EMAIL);
await page.fill("input[name=password]", process.env.LOGIN_PASSWORD);
await page.click("button[type=submit]");
await page.waitForURL("https://example.com/dashboard");

That works for a prototype. In production, it raises less fun questions:

• Where do the credentials live?
• Can the scraping provider see them?
• How do you rotate them?
• What happens with 2FA?
• Can you reuse a saved identity without passing raw cookies around?

If your scrape is really an authenticated action on a common site, Wire treats the login identity as part of the managed action instead of making every caller ship cookies or replay a login script.

You may not need that. For internal tools or one-off admin exports, a Playwright script with secrets loaded from your vault may be enough. For a multi-tenant app, credential storage and auditability become part of the scraping architecture, not an implementation detail.

What to compare before choosing a scraping API

For any scraping provider, ignore the biggest credit number until you answer these questions:

1. Does JavaScript rendering change the credit cost?
2. Do premium, residential, or stealth proxies change the cost?
3. Are failed requests charged?
4. What counts as failed: timeout only, non-2xx, CAPTCHA page, empty extraction?
5. Is there response caching, and does a cache hit cost credits?
6. Can you submit batches or async jobs, or do you need to build that queue?
7. Does structured extraction charge extra?
8. How do you handle logins without exposing credentials everywhere?
9. What are the concurrency limits, and do retries count against them?
10. What is the cost per valid record on your actual target sites?

Scraping APIs are useful because running your own browser fleet, proxy rotation, and anti-bot handling is annoying operational work. The tradeoff is that pricing models encode those same operational costs in ways that are easy to underestimate.

Before committing to a plan, run a representative 100 URL test set: static pages, JavaScript-heavy pages, blocked pages, repeated URLs, and authenticated pages if you have them. Log credits spent, valid records returned, and failure reasons. Then choose based on cost per valid record, not credits on the pricing page.