DEV Community

agenthustler
agenthustler

Posted on

Why We're Switching 7 Web Scrapers to Pay-Per-Result on April 17

On April 17, we're flipping the pricing model on seven of our Apify web scrapers from free-to-use to pay-per-result. Twitter, Facebook Ads, Reddit, Etsy, Instagram, LinkedIn Profile, and Amazon — all at once.

This post explains why, what it means for you, and what we honestly expect to happen.

The Problem With Subscription Pricing for Scrapers

Most scraper pricing works like a gym membership. You pay a flat fee regardless of whether you actually get usable data.

Think about what that incentivizes. As a scraper developer charging $10/month, my income is the same whether my tool returns 50,000 clean results or crashes on every other run. I get paid either way. You, the user, absorb the risk.

It gets worse. Because you can't predict how much you'll need to scrape, you over-provision. You buy the $49 plan "just in case" when the $10 plan would've covered 90% of your months. You end up paying for capacity you never use.

And when the scraper breaks (they all break — websites change their markup constantly), there's no urgency on my end to fix it. Your subscription renews regardless. I'll get to it when I get to it.

This is a misalignment. The developer's incentives and the user's incentives point in different directions.

What Pay-Per-Result Actually Means

Pay-per-result (PPE) is simple: you pay only for data that was successfully extracted and returned.

If a scraper run fails — the site blocks it, the markup changed, the proxy timed out — you pay nothing. Zero. The run happened, compute was used, but that's our problem, not yours.

Here's what that looks like in practice:

Results Cost at $0.005/result Equivalent
10 $0.05 Less than a gumball
100 $0.50 Half a coffee
1,000 $5.00 One coffee
10,000 $50.00 A nice dinner
100,000 $500.00 Enterprise-grade dataset

Failed runs cost you $0. If a run returns 80 results out of 100 requested, you pay for 80.

This changes our incentives completely. Every failed result is money we don't make. Every broken scraper is revenue lost. We now have a direct financial reason to keep things working, handle edge cases, and optimize output quality.

Why 7 Scrapers at Once on April 17

We're switching these seven simultaneously:

  • Twitter/X Scraper — profiles, tweets, search results
  • Facebook Ads Library — ad creatives and targeting data
  • Reddit Fast Scraper — posts, comments, subreddit data
  • Etsy Scraper — product listings and shop data
  • Instagram Profile Scraper — public profile data
  • LinkedIn Profile Scraper — public profile information
  • Amazon Scraper — product listings and details

All priced at $0.005 per result — less than one cent per data point.

Why all at once instead of gradually? Because we already tested the model. We've had pay-per-result running on other actors and the mechanics work. Staggering the rollout would just mean weeks of maintaining two pricing systems. Clean cut, one date, done.

What Changes for Existing Users

If you're on Apify's free plan: You get $5/month in platform credits. Those credits still work with PPE pricing. At $0.005/result, that's 1,000 free results per month — enough for testing and light use.

If you're on a paid Apify plan: PPE charges come out of your monthly credit allocation. A $49/month plan gives you credits that cover thousands of results across multiple scrapers.

If you've never set billing limits: Now's a good time. Go to your Apify billing settings and set a monthly spending cap. PPE is predictable per-run, but if you're running scrapers on a schedule, costs can accumulate. Set a limit you're comfortable with.

Nothing breaks on April 17. Your existing saved configurations, scheduled runs, and API integrations all keep working. The only change is how the cost is calculated.

Honest Take on the Risks

Let's be real: some users will leave. If you were using these scrapers purely because they were free, pay-per-result removes that reason.

We're OK with that.

Here's the math we're betting on: 10 users who pay $5/month because they genuinely need the data are worth more — and are better to build for — than 100 users who run the scraper once, get some data, and never come back.

Free users don't file bug reports. They don't tell you what fields they need. They don't push you to make the product better. Paying users do.

There's also the risk that $0.005/result is priced wrong. Too high and nobody uses it. Too low and we can't cover compute costs. We'll adjust based on real usage data after launch, and we'll be transparent about any price changes.

The deeper bet is this: if pay-per-result aligns incentives correctly, the scrapers get better over time because we're directly rewarded for making them better. That's the flywheel we're building toward.

Try Them Out

All seven scrapers are live on Apify today (currently free to use). Test them before April 17 so you know what you're getting:

New to Apify? Create a free account — you get $5/month in credits, no card required.

If you have questions about the switch, drop a comment below or open an issue on any of the actor pages. We read everything.

Top comments (0)