DEV Community: agenthustler

Pay-Per-Result APIs: Why the Industry Is Shifting Away from Monthly Subscriptions in 2026

agenthustler — Wed, 15 Apr 2026 04:43:35 +0000

Pay-Per-Result API Pricing Is Changing How Developers Buy Data

If you've ever signed up for a data API, you know the drill: pick a monthly tier, guess your usage, then either overpay for headroom you never touch or blow past quota on day 19. The pricing model is older than the web, and it's still everywhere.

A quieter shift has been happening on platforms where APIs and scrapers are sold as products: pay-per-result (PPE) billing. You don't pay for compute time, subscription tiers, or request counts. You pay per record you actually receive — a job listing, a profile, a review, a product.

Why Devs Prefer PPE

1. Cost maps to value. If a scraper returns 45,000 LinkedIn job postings, you pay for 45,000 rows. If it returns 42, you pay for 42. No idle months, no overage surprises.

2. Failure is free. Scraper broke because the target site shipped a redesign? You get zero results and pay zero. Try doing that with a $99/month subscription.

3. Budgeting is trivial. rows × unit_price is a spreadsheet cell, not a capacity-planning meeting. Finance teams love it.

4. No lock-in. Tier-based APIs punish you for leaving mid-cycle. PPE has no cycle.

Where You're Seeing This

Apify has been running a PPE model across its Actor marketplace, where individual scrapers set per-result prices (often $0.01–$0.10). You run a scraper, you get rows, you pay for rows. Billing is handled by the platform, so you're not chasing invoices.

I've been shipping actors there for a few months — LinkedIn Jobs, Indeed, IndieHackers, Instagram profile, Reddit — and what's become obvious is that PPE lets tiny niche scrapers exist at all. A dataset nobody needs enough to justify a $49/mo tier can still make sense at $0.02/result for the three people a week who want exactly that data.

Downsides, to Be Honest

Price discovery is harder. You don't know what a run will cost until you estimate row counts.
Spiky workloads can get expensive. If you need 2M rows once, a flat-tier API might be cheaper.
Per-result pricing rewards stable, well-defined schemas. Scrapers returning blobby HTML don't fit the model.

The Takeaway

If you're buying data in 2026, default to asking "what's the per-result price?" before "what's the monthly plan?" The per-unit number tells you more about the actual economics than any marketing page.

Check out the PPE actors I maintain at apify.com/cryptosignals — job market data, social profiles, review datasets. Pay for what you pull.

Tracking Amazon Price Drops at Scale — Build a Price Monitor with Python

agenthustler — Tue, 14 Apr 2026 10:13:40 +0000

A friend runs a small dropshipping store. Nothing fancy — 200 SKUs, mostly home goods, sourced from Amazon and marked up on their own Shopify. His biggest headache for years was margin drift: Amazon would quietly drop a price, he'd keep selling at the old price, and by the end of the month his "20% margin" products had actually been 3% losers for weeks.

He asked me for a cheap price monitor. I built it in an afternoon. Here's the whole thing.

The idea

For each SKU, pull the current Amazon price once every few hours. If it moved more than some threshold, fire an alert. Store the history so we can look at trends and answer questions like "which products are trending down this week?"

Stack:

Apify's Amazon Scraper for the actual price pulls ($0.005/result — basically free at this scale).
SQLite for history.
A tiny Python script on a $5 VPS for scheduling.
Slack webhook for alerts.

That's it. No Redis, no queue, no Docker.

Step 1 — Watchlist

Start with a CSV. Column 1 is the ASIN, column 2 is the price you're currently selling at.

import csv

def load_watchlist(path: str = "watchlist.csv") -> dict[str, float]:
    out = {}
    with open(path) as f:
        for row in csv.reader(f):
            out[row[0].strip()] = float(row[1])
    return out

Keep it boring. You'll thank yourself later when you're debugging at 2am.

Step 2 — Fetch current prices

import os, requests

APIFY_TOKEN = os.environ["APIFY_TOKEN"]
ACTOR = "cryptosignals~amazon-scraper"

def fetch_prices(asins: list[str]) -> list[dict]:
    run_input = {
        "asins": asins,
        "country": "US",
        "includeReviews": False,
    }
    r = requests.post(
        f"https://api.apify.com/v2/acts/{ACTOR}/run-sync-get-dataset-items",
        params={"token": APIFY_TOKEN},
        json=run_input,
        timeout=600,
    )
    r.raise_for_status()
    return r.json()

One call, all ASINs. For 200 products the actor finishes in about a minute. For thousands, chunk it — I found batches of 500 to be a sweet spot.

Step 3 — Store the history

import sqlite3
from datetime import datetime

DB = sqlite3.connect("prices.db")
DB.execute("""
CREATE TABLE IF NOT EXISTS price_history (
    asin TEXT,
    price REAL,
    currency TEXT,
    in_stock INTEGER,
    checked_at TEXT
)
""")
DB.execute("CREATE INDEX IF NOT EXISTS idx_asin_time ON price_history(asin, checked_at)")

def record(items: list[dict]) -> None:
    now = datetime.utcnow().isoformat()
    for it in items:
        DB.execute(
            "INSERT INTO price_history VALUES (?, ?, ?, ?, ?)",
            (
                it["asin"],
                float(it.get("price") or 0),
                it.get("currency", "USD"),
                1 if it.get("inStock") else 0,
                now,
            ),
        )
    DB.commit()

Never delete a row. Disk is cheap, history is priceless when you're trying to figure out if a price drop is a fluke or a trend.

Step 4 — Detect drops

def significant_drops(threshold_pct: float = 5.0) -> list[tuple]:
    rows = DB.execute("""
    SELECT asin, price, checked_at FROM price_history
    ORDER BY asin, checked_at DESC
    """).fetchall()

    latest = {}
    previous = {}
    for asin, price, _ in rows:
        if asin not in latest:
            latest[asin] = price
        elif asin not in previous:
            previous[asin] = price

    alerts = []
    for asin, cur in latest.items():
        prev = previous.get(asin)
        if not prev or prev == 0:
            continue
        pct = (cur - prev) / prev * 100
        if pct <= -threshold_pct:
            alerts.append((asin, prev, cur, pct))
    return alerts

5% is a reasonable default. Below that, most of the signal is noise — Amazon nudges prices by a few cents constantly.

Step 5 — Alert

import json

SLACK = os.environ["SLACK_WEBHOOK"]

def alert(drops: list[tuple]) -> None:
    if not drops:
        return
    lines = [
        f":chart_with_downwards_trend: *{a}* {p1:.2f} → {p2:.2f} ({pct:+.1f}%)"
        for a, p1, p2, pct in drops
    ]
    requests.post(SLACK, json={"text": "\n".join(lines)}, timeout=10)

Slack, Discord, email, SMS — it doesn't matter. What matters is that the alert lands somewhere you actually look.

Step 6 — Schedule

Cron is fine. Every four hours:

0 */4 * * * cd /opt/price-monitor && python3 run.py >> run.log 2>&1

run.py is the three-line composition of everything above:

from monitor import load_watchlist, fetch_prices, record, significant_drops, alert

items = fetch_prices(list(load_watchlist().keys()))
record(items)
alert(significant_drops(threshold_pct=5))

Use cases beyond dropshipping

Once the pipeline is running, other uses appear naturally:

Comparison shopping. Same ASIN across multiple country marketplaces — find arbitrage.
Restock alerts. Flip the in_stock flag check instead of price.
Deal blogs. If you run a content site, price drops become content. Low effort, good traffic.
Personal wishlists. The nerdiest one. I track 20 board games and get pinged when anything drops 10%+.

Cost check

At 200 products every 4 hours, that's 1,200 results per day × 30 days = 36,000 results/month. At $0.005/result the bill is $180/month — and my friend's monthly margin drift savings alone paid for it in the first week. For smaller lists (20–50 products) you're talking pennies.

If you want the actor: Amazon Scraper on Apify. It has a free tier so you can prove out the idea before wiring up cron.

The boring stack works. Go build.

How to Build a LinkedIn Talent Pipeline Scraper in 2026 (Without LinkedIn's API)

agenthustler — Tue, 14 Apr 2026 10:13:39 +0000

I spent the last two months helping a friend's recruiting agency move off a $4,000/month sourcing tool. The pitch was simple: they wanted to pull a few thousand LinkedIn profiles a week based on job titles, enrich them, score them, and feed the top matches into their ATS. LinkedIn's official API, as anyone who has tried it knows, is basically a locked door unless you're a Fortune 500 partner. So we went the scraper route — and it worked better than either of us expected.

Here's how I built it, what I learned, and the Python code you can steal.

Why not the official API?

LinkedIn's partner API (Talent Solutions) is gated behind sales calls, contracts, and minimums you don't want to see. For a small agency or a solo recruiter, it's not an option. The Sign In With LinkedIn OAuth endpoints only give you basic profile info for the user who logged in — not search, not lookups, not bulk data.

Everyone I know who scales LinkedIn sourcing does one of two things:

Pays a SaaS that scrapes on their behalf and wraps it in a UI.
Runs their own scraper.

Option 2 is cheaper, more flexible, and the data is yours to do with as you please.

The stack

I used Apify's LinkedIn Profile Scraper as the data layer. It handles the proxy rotation, fingerprinting, and retry logic that you really don't want to maintain yourself. I built the pipeline in plain Python — no framework, just requests, sqlite3, and a couple of CSV dumps.

Pricing was the other reason I went with this actor: $0.005 per result. For 5,000 profiles a week that's $25, which is absolutely nothing compared to the $4k/month the agency was paying before.

Step 1 — Pull the profiles

Let's say we want Senior Python Engineers in Berlin. I keep my seed list in a plain text file: one profile URL per line. You can generate that seed list from a LinkedIn search URL using the same actor, but for this article I'll assume you already have URLs.

import os
import requests

APIFY_TOKEN = os.environ["APIFY_TOKEN"]
ACTOR_ID = "cryptosignals~linkedin-profile-scraper"

def scrape_profiles(urls: list[str]) -> list[dict]:
    run_input = {
        "profileUrls": urls,
        "includeSkills": True,
        "includeExperience": True,
    }
    resp = requests.post(
        f"https://api.apify.com/v2/acts/{ACTOR_ID}/run-sync-get-dataset-items",
        params={"token": APIFY_TOKEN},
        json=run_input,
        timeout=600,
    )
    resp.raise_for_status()
    return resp.json()

run-sync-get-dataset-items blocks until the actor finishes and returns the dataset in one shot. For small batches that's the easiest pattern. For 5,000+ profiles, switch to the async runs endpoint and poll.

Step 2 — Store and deduplicate

Recruiters re-run the same searches every week. You do not want to re-scrape profiles you already have fresh data on, both for cost and politeness.

import sqlite3
from datetime import datetime, timedelta

DB = sqlite3.connect("talent.db")
DB.execute("""
CREATE TABLE IF NOT EXISTS profiles (
    url TEXT PRIMARY KEY,
    name TEXT,
    headline TEXT,
    location TEXT,
    skills TEXT,
    last_seen TEXT
)
""")

def needs_refresh(url: str, max_age_days: int = 14) -> bool:
    row = DB.execute(
        "SELECT last_seen FROM profiles WHERE url = ?", (url,)
    ).fetchone()
    if not row:
        return True
    last = datetime.fromisoformat(row[0])
    return datetime.utcnow() - last > timedelta(days=max_age_days)

def upsert(profile: dict) -> None:
    DB.execute("""
    INSERT INTO profiles(url, name, headline, location, skills, last_seen)
    VALUES (?, ?, ?, ?, ?, ?)
    ON CONFLICT(url) DO UPDATE SET
        name=excluded.name,
        headline=excluded.headline,
        location=excluded.location,
        skills=excluded.skills,
        last_seen=excluded.last_seen
    """, (
        profile["url"],
        profile.get("fullName"),
        profile.get("headline"),
        profile.get("location"),
        ",".join(profile.get("skills", [])),
        datetime.utcnow().isoformat(),
    ))
    DB.commit()

A 14-day TTL works well in practice. People don't update their headline every week.

Step 3 — Score candidates

This is where a pipeline stops being a scraper and starts being a tool. The agency cared about three signals: years in role, relevance of past companies, and whether the person was open to contract work (mentioned in the headline or about section).

def score(profile: dict) -> int:
    points = 0
    headline = (profile.get("headline") or "").lower()
    if "senior" in headline or "staff" in headline:
        points += 2
    if "open to" in headline or "contract" in headline:
        points += 3

    for exp in profile.get("experience", []):
        title = (exp.get("title") or "").lower()
        if "python" in title:
            points += 2
        if exp.get("company") in TARGET_COMPANIES:
            points += 5
    return points

Tune TARGET_COMPANIES to your industry. The agency keeps a list of ~60 companies whose alumni they love to source from; hitting one is a huge signal.

Step 4 — Ship to the ATS

Once you've ranked profiles, push the top N into wherever your team actually works. For the agency that meant a CSV drop into a shared folder, but a webhook into Greenhouse or Airtable works just as well.

import csv

def export_top(n: int = 50, path: str = "top_candidates.csv") -> None:
    rows = DB.execute(
        "SELECT url, name, headline, location FROM profiles ORDER BY last_seen DESC LIMIT ?",
        (n,),
    ).fetchall()
    with open(path, "w", newline="") as f:
        w = csv.writer(f)
        w.writerow(["url", "name", "headline", "location"])
        w.writerows(rows)

What I'd do differently

Two things bit me in the first month:

Don't scrape the same profile twice in one day. Even with proxy rotation, you're wasting money. The dedupe check above exists for a reason.
Rate yourself, not just the scraper. The actor handles its side. You should still cap your runs — I schedule one batch of 500 profiles every few hours rather than 5,000 in one shot. Smoother results, easier debugging.

Total cost

At $0.005/result, the agency's weekly 5,000-profile refresh costs $25. Add a few dollars for compute. Compared to the SaaS they cancelled, it pays for a nice dinner every week and then some.

If you want to skip the code and just try the actor, it's here: LinkedIn Profile Scraper on Apify. The input schema is documented and the free tier lets you run a few hundred profiles before you put a card down.

Happy sourcing.

I scraped 4,500 IndieHackers products - here's what the MRR data reveals

agenthustler — Tue, 14 Apr 2026 03:02:33 +0000

After spending a weekend building a scraper for IndieHackers' public product pages, I ended up with a dataset of 4,500 products — and the MRR numbers tell a very different story than the Twitter highlight reel.

I went in expecting to see a long tail of hobby projects and a handful of unicorns. What I actually found was a surprisingly healthy middle class of indie SaaS.

Here's what the data looks like after a few hours with pandas.

The numbers

Out of 4,500 products scraped, 1,544 (34%) self-report MRR on their profile. The rest either keep it private, haven't updated in forever, or are pre-revenue.

The MRR distribution for the reporters:

Bucket	Products	Share
$0 – $100	476	31%
$100 – $1K	447	29%
$1K – $10K	401	26%
$10K+	220	14%

The takeaway that surprised me: the $1K–$10K bracket is where most "successful" indie products actually live. Not the $100K MRR screenshots that go viral. A product doing $3,500/mo is doing better than ~85% of everything publicly listed.

Median MRR among revenue reporters: ~$750/mo. Not glamorous, but it's a real number, and it puts the "just quit your job" advice in perspective.

What's growing

Tagging each product by its stated category and doing a rough vertical split:

SaaS tools — still the biggest slice
Developer tools — overrepresented vs the broader market (no surprise given the audience)
AI / automation — the fastest-growing category in the listings; dozens of products added in the last 90 days alone
Productivity — steady but saturated
Marketing / growth — lots of entries, but most sit in the $0–$100 bucket

The AI bucket is the one worth watching. A year ago it was a sliver. Now it's pushing ~18% of new listings.

Who's behind them

Looking at the team size field:

Solo founders dominate the $1K–$10K bracket (62% of products in that range list a team of 1)
Products with 2+ founders skew toward the $10K+ bracket
Nobody in the $100K+ club is solo — every one has a team

That matches the intuition that revenue-per-founder has a ceiling, but the data is cleaner than I expected.

Filtering the data in Python

If you want to isolate, say, profitable AI products in the dataset, it's a one-liner with pandas:

import pandas as pd

df = pd.read_csv("indiehackers_products.csv")

ai_winners = df[
    (df["mrr"] > 1000) &
    (df["tags"].str.contains("ai", case=False, na=False))
]

print(ai_winners[["name", "mrr", "tags", "url"]].sort_values("mrr", ascending=False).head(20))

From there you can pivot on launch date, team size, or pricing model to find the patterns you care about.

How I'm using this

A few angles that turned out to be useful:

Competitor analysis — pick a vertical, sort by MRR, and you instantly see who the players are and roughly what they're pulling in.
Niche validation — if a category has 40 products but none above $500/mo, that's a signal (probably a bad one).
Founder outreach — if you sell to indie founders, this is basically a lead list with qualification data attached.
Pricing benchmarks — cross-referencing MRR bracket with listed price gives you a rough sense of how many paying customers each product has.

The dataset

I cleaned the full 4,500-row CSV up and put it on Payhip if you'd rather skip the scraping step: IndieHackers Products Dataset

Includes product name, URL, tags, MRR (where public), team size, launch date, and short description. Single CSV, no tracking, no subscription.

If you end up finding something interesting in the data, I'd genuinely like to hear about it — leave a comment and I'll follow up.

Web Scraping Pricing Comparison 2026: Bright Data vs Apify vs DIY

agenthustler — Sun, 12 Apr 2026 11:05:30 +0000

Web scraping costs range from $0 (DIY) to $10,000+/month (enterprise proxy networks). The right choice depends on your scale, technical resources, and how much maintenance you're willing to absorb.

This is a practical pricing comparison of the major web scraping approaches in 2026, with real numbers and clear recommendations for different use cases.

The Five Approaches

Every web scraping project falls into one of these categories:

Proxy providers (Bright Data, Oxylabs) — you write the scraper, they provide infrastructure
Managed platforms (Apify) — pre-built scrapers with compute and proxy included
Scraping APIs (ScraperAPI, ScrapingBee) — API call returns rendered HTML
Data APIs (Proxycurl, RapidAPI vendors) — API call returns structured data
DIY — your own servers, your own proxies, your own maintenance

Pricing Comparison Table

Provider	Model	Entry Price	Mid-Scale (100K pages/mo)	Large Scale (1M pages/mo)	Includes Proxies	Anti-Bot Handling
Bright Data	Per GB / per CPM	$2.80/GB residential	~$280-500/mo	~$2,000-4,000/mo	Yes	Yes (Web Unlocker)
Oxylabs	Per GB / per request	$3.00/GB residential	~$300-550/mo	~$2,200-4,500/mo	Yes	Yes (Web Unblocker)
Apify Platform	Per compute unit	$0.30/CU	~$25-80/mo	~$200-600/mo	Bundled in actors	Actor-dependent
Apify PPE Actors	Per result	$0.005-0.01/result	~$50-100/mo (10K results)	~$500-1,000/mo (100K results)	Included	Included
ScraperAPI	Per API call	$0.001/call (on plan)	~$100/mo	~$500/mo	Yes	Yes
ScrapingBee	Per credit	$0.0025/credit	~$100/mo	~$400-800/mo	Yes	Yes
DIY (VPS + free proxies)	Server costs	$5-20/mo VPS	~$20-50/mo	~$100-300/mo + your time	No (you manage)	No (you build)
DIY (VPS + paid proxies)	Server + proxy	$50+/mo	~$150-400/mo	~$1,000-3,000/mo	Purchased separately	You build

Prices as of April 2026. Actual costs vary by target site complexity and anti-bot measures.

Detailed Breakdown

Bright Data

Best for: Large-scale operations that need residential/mobile IPs and can write their own scrapers.

Bright Data is the largest proxy network (72M+ residential IPs). Their pricing model is primarily per-GB for proxy traffic, with Web Unlocker (anti-bot) charged per CPM (cost per thousand requests).

Residential proxies: $2.80/GB
Datacenter proxies: $0.60/GB
Web Unlocker: $2.50/CPM (per 1,000 successful requests)
Minimum commitment on some plans

Hidden costs: You still need to write and maintain the scraper code, handle parsing, manage retries, and deal with site changes. Developer time is the real cost.

Oxylabs

Similar to Bright Data in pricing and capability. Residential at $3.00/GB, datacenter at $0.70/GB. Their Web Unblocker competes directly with Bright Data's Web Unlocker.

Apify Platform

Best for: Teams that want pre-built scrapers without infrastructure management.

Apify's pricing is based on compute units (CUs). One CU = 1 GB RAM for 1 hour of compute. At $0.30/CU, the actual cost depends on the actor (scraper) efficiency.

Free tier includes $5/mo in platform credits — enough for small experiments.

Many Apify Store actors use pay-per-event (PPE) pricing, where you pay per result instead of per compute unit. This is simpler to predict:

LinkedIn Jobs scraper: $0.005-0.01 per job listing
Google Search scraper: $0.003-0.005 per result
E-commerce scrapers: $0.005-0.02 per product

PPE pricing includes proxy costs and anti-bot handling — no hidden fees.

ScraperAPI

Best for: Developers who want a simple API call that returns rendered HTML.

You send a URL, ScraperAPI returns the HTML after handling proxies, CAPTCHAs, and JavaScript rendering.

Hobby: $29/mo (100K API credits)
Startup: $99/mo (1M API credits)
Business: $249/mo (3M API credits)
1 API credit = 1 standard request; JavaScript rendering costs 5-10 credits

Trade-off: You still need to parse the HTML yourself. Good for simple pages, expensive for JavaScript-heavy sites.

ScrapingBee

Similar model to ScraperAPI. Pricing starts at $49/mo for 150K credits. JavaScript rendering costs 5 credits per request. Includes Google Search API at 25 credits per search.

DIY

Best for: Technical teams with specific requirements and tolerance for maintenance.

The upfront cost is low — a $10/mo VPS, Playwright or Puppeteer, and free rotating proxies (if you can find reliable ones). But the real costs are:

Developer time: 10-40 hours/month maintaining scrapers against site changes
Proxy costs: Free proxies are unreliable; paid residential proxies cost $2-5/GB
Anti-bot solutions: reCAPTCHA solving services ($1-3/1000 solves), browser fingerprint management
Infrastructure: Error handling, retry logic, job queues, data storage

Realistic all-in cost for a maintained DIY scraper at scale: $500-3,000/month including developer time (valued at $50/hr).

Decision Matrix

Use Case	Recommended	Why	Monthly Cost
Hobby/learning	DIY or Apify free tier	Learn the fundamentals, free or near-free	$0-5
Startup MVP	Apify PPE actors	Pay per result, no infrastructure, predictable costs	$10-100
Recruiting/HR data	Apify PPE (LinkedIn actors)	Purpose-built, handles LinkedIn anti-bot	$25-200
SEO/marketing data	ScraperAPI or Apify	Both handle Google well, choose by volume	$29-250
E-commerce monitoring	Bright Data or Apify	Bright Data for custom; Apify for pre-built	$100-1,000
Enterprise (1M+ pages)	Bright Data + custom scrapers	Best proxy network at scale, but need dev team	$2,000-10,000
One-time data collection	Apify PPE	No subscription, pay only for what you use	$5-50

What Changed in 2026

Several things shifted since 2024-2025:

Proxycurl shut down — The popular LinkedIn/company data API closed in 2025. Alternatives: Apify actors, Bright Data datasets, or DIY.
Anti-bot got harder — Cloudflare Turnstile, DataDome, and PerimeterX are on more sites. DIY scraping is increasingly expensive to maintain.
Pay-per-result pricing expanded — More platforms offer PPE/pay-per-result models, which shifts risk from the buyer to the provider.
AI assistants drive data demand — LLM-powered agents need structured data feeds, creating new demand for scraping infrastructure.

How to Calculate Your Actual Cost

Before choosing a provider, estimate your real needs:

# Quick cost estimator
pages_per_month = 100_000
avg_page_size_kb = 200

# Proxy provider (Bright Data)
data_gb = (pages_per_month * avg_page_size_kb) / 1_000_000
proxy_cost = data_gb * 2.80  # residential rate
print(f'Bright Data proxy only: ${proxy_cost:.0f}/mo')
print(f'+ Web Unlocker: ${(pages_per_month/1000) * 2.50:.0f}/mo')

# Apify PPE
ppe_cost = pages_per_month * 0.005  # low-end per result
print(f'Apify PPE: ${ppe_cost:.0f}/mo')

# ScraperAPI (Startup plan)
api_credits = pages_per_month * 1  # assuming no JS rendering
print(f'ScraperAPI: $99/mo (1M credits covers this)')

Bottom Line

For most developers and startups: Start with Apify's free tier or a PPE actor. You get structured data without managing proxies or infrastructure, and costs scale linearly with usage.

For scale (500K+ pages/month): Bright Data or Oxylabs give you the best proxy infrastructure, but budget for developer time to build and maintain scrapers.

For one-off projects: Pay-per-result pricing (Apify PPE actors) is almost always cheaper than a monthly subscription to any platform.

The biggest hidden cost in web scraping isn't the proxy bill — it's the engineering time to keep scrapers working as sites change. Factor that in before choosing DIY over a managed solution.

Prices verified as of April 2026. For the latest pricing, check each provider's website directly.

LinkedIn Jobs API for APAC Recruitment: Singapore, Indonesia, Malaysia Data Guide

agenthustler — Sun, 12 Apr 2026 11:04:07 +0000

If you're building recruitment tools, HR dashboards, or talent analytics for Southeast Asia, you already know: APAC hiring data is fundamentally different from Western markets.

LinkedIn is the dominant professional network in Singapore, Malaysia, and Indonesia — but accessing structured job data programmatically requires navigating API restrictions, regional data quirks, and multilingual listings.

This guide covers how to get clean, structured LinkedIn Jobs data for APAC recruitment, what fields matter, and the current options in 2026.

Why APAC Recruitment Data Is Different

Three things make APAC job data harder to work with than US/EU data:

1. Multilingual listings. A single Singapore job posting might mix English, Mandarin, and Malay. Indonesian listings blend Bahasa Indonesia with English technical terms. Your pipeline needs to handle this.

2. Regional job board fragmentation. LinkedIn dominates white-collar hiring, but JobStreet (Malaysia/Indonesia), MyCareersFuture (Singapore government portal), and Jobsdb (Hong Kong/Thailand) all carry listings that never appear on LinkedIn.

3. Visa and work permit requirements. Singapore's Employment Pass, Malaysia's DE Lex pass, Indonesia's KITAS — these aren't just nice-to-have fields. They determine whether a candidate can actually apply.

Factor	US/EU	APAC (SG/MY/ID)
Primary platform	LinkedIn	LinkedIn + JobStreet + government portals
Language	Single language	2-4 languages per listing
Work authorization	Binary (yes/no)	Pass-type specific (EP, S Pass, PEP, DP)
Salary disclosure	Common	Rare (salary ranges often hidden)
Hiring timeline	2-4 weeks	4-8 weeks (notice periods are longer)

Key Data Fields for APAC Recruitment

When scraping LinkedIn Jobs for APAC markets, these fields matter most:

job_location — Filter by city-level: "Singapore", "Kuala Lumpur", "Jakarta", "Bangsar South" (common KL tech hub)
description (full text) — Parse for visa sponsorship mentions, language requirements, and salary hints
seniority_level — APAC markets skew toward mid-senior roles on LinkedIn; junior roles are more common on local boards
company_size — Startups in Singapore (especially fintech) hire differently than MNCs
posted_date — Critical for tracking hiring velocity and seasonal patterns (Chinese New Year creates a predictable February dip)

How to Filter for Singapore, KL, and Jakarta

Using the LinkedIn Jobs Scraper on Apify, you can target APAC locations directly:

{
  "searchUrl": "https://www.linkedin.com/jobs/search/?location=Singapore&keywords=software+engineer",
  "maxItems": 100
}

For multi-city collection, run separate queries per location:

locations = [
    "Singapore",
    "Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia",
    "Jakarta, Jakarta, Indonesia",
    "Ho Chi Minh City, Vietnam"
]

for loc in locations:
    run_input = {
        "searchUrl": f"https://www.linkedin.com/jobs/search/?location={loc}&keywords=fintech",
        "maxItems": 200
    }

Tip: Use full location strings for Malaysia and Indonesia. "Kuala Lumpur" alone sometimes returns results from other countries.

Practical Example: Tracking Singapore Fintech Hiring

Singapore is the fintech capital of Southeast Asia. Here is how to track hiring trends:

Collect weekly snapshots of fintech job listings (keywords: "fintech", "digital bank", "payments", "blockchain")
Track volume over time — a spike in backend engineer listings often precedes a product launch
Monitor company-level patterns — are Grab, Sea Group, and DBS scaling specific teams?
Cross-reference with MAS announcements — new regulatory frameworks (like the 2025 Digital Payment Token rules) create predictable hiring waves

A dataset of 500-1000 Singapore fintech listings per month costs roughly $2.50-5.00 using pay-per-event pricing on Apify.

Comparison: LinkedIn Jobs Data Sources in 2026

Source	Status	APAC Coverage	Cost	Structured Data
LinkedIn Official API	Restricted (partners only)	Full	Free (if approved)	Yes
Proxycurl	Shut down (2025)	N/A	N/A	N/A
PhantomBuster	Active	Limited geo-targeting	$69+/mo	Partial
Apify LinkedIn Jobs Scraper	Active	Full city-level filtering	$0.005-0.01/result (PPE)	Yes (JSON)
DIY with Playwright	Active	Whatever you build	Server costs + maintenance	Whatever you build

The LinkedIn Official API requires a partnership agreement and is not available to most developers or startups. If you have access, use it — the data quality is unmatched.

For everyone else, a managed scraper with pay-per-result pricing is the most practical option. You get structured JSON output without maintaining proxy infrastructure or handling LinkedIn's anti-bot measures.

Getting Started

Create a free Apify account
Find the LinkedIn Jobs Scraper in the Store
Set your target location and keywords
Run and export to JSON, CSV, or connect directly to your pipeline via API

For APAC-specific recruitment analytics, combine LinkedIn data with local job boards for the most complete picture. LinkedIn captures 60-70% of white-collar listings in Singapore, but only 30-40% in Indonesia where JobStreet dominates.

Building recruitment tools for APAC? The structured data approach beats manual scraping every time — especially when you need consistent, repeatable data collection across multiple Southeast Asian markets.

How to Scrape PropertyGuru Data in Singapore: 2026 Guide

agenthustler — Sun, 12 Apr 2026 08:17:53 +0000

Singapore's property market moves fast. Whether you're an investor tracking price trends across districts, a real estate agent doing competitive research, or a developer building market analysis tools — you need structured property data.

PropertyGuru is Singapore's dominant property portal, with over 200,000 active listings covering HDB flats, condos, landed properties, and commercial spaces. It's the Zillow of Southeast Asia, and it holds a goldmine of data that's frustratingly hard to extract at scale.

What Data Is Available on PropertyGuru?

Each PropertyGuru listing contains rich structured data:

Price — asking price, PSF (per square foot), and historical price changes
Property details — size (sqft), bedrooms, bathrooms, floor level, tenure (freehold/leasehold/999-year)
Location — district number, street address, nearest MRT station and distance
Agent info — agent name, agency, contact details, number of active listings
Project details — developer name, TOP date, total units, facilities

For Singapore specifically, you'll want to understand the geographic classification:

CCR (Core Central Region) — Districts 9, 10, 11, Downtown Core. Orchard Road, Marina Bay. Premium pricing.
RCR (Rest of Central Region) — Districts like Queenstown, Toa Payoh, Geylang. Mid-tier pricing with growth potential.
OCR (Outside Central Region) — Jurong, Woodlands, Punggol. Mass market, HDB-heavy. This is where 80% of Singaporeans live.
URA Planning Areas — 55 distinct zones used by the Urban Redevelopment Authority for planning. Critical for understanding development potential and future MRT lines.

Understanding these regions is essential for any meaningful property analysis in Singapore.

Why People Need This Data

Property investors track PSF trends across districts to identify undervalued areas before en-bloc fever hits. When a new MRT line is announced, prices in surrounding districts shift — having historical data lets you model these patterns.

Real estate agents monitor competitor listings, pricing strategies, and agent market share. If a rival agency is dominating District 15 listings, you want to know.

PropTech developers build tools like mortgage calculators, investment dashboards, and rental yield estimators. All of these need fresh listing data as input.

Researchers and analysts study housing affordability, the HDB resale market, and the impact of cooling measures on transaction volumes.

Current Options (And Why They Fall Short)

Manual export: PropertyGuru lets you browse and filter, but there's no bulk export. Copy-pasting 50 listings is tedious. Doing 5,000 is impossible.

Existing Apify actors: As of early 2026, the PropertyGuru scrapers on Apify Store are deprecated and returning 0 results. The site has changed its structure, and nobody has updated these actors.

Enterprise solutions: Services like Bright Data offer pre-built datasets, but pricing starts at enterprise levels — overkill if you just need listing data for a specific district or property type.

Building Your Own PropertyGuru Scraper

Here's a basic approach using Python and requests:

import requests
from bs4 import BeautifulSoup
import json
import time

def scrape_propertyguru(district: int, listing_type: str = "sale", pages: int = 5):
    """Scrape PropertyGuru listings for a specific district."""
    results = []
    headers = {
        "User-Agent": "Mozilla/5.0 (compatible; research bot)"
    }

    for page in range(1, pages + 1):
        url = f"https://www.propertyguru.com.sg/property-for-{listing_type}/{district}"
        params = {"page": page}

        resp = requests.get(url, headers=headers, params=params)
        soup = BeautifulSoup(resp.text, "html.parser")

        # Extract listing cards - inspect the page for current selectors
        listings = soup.select("[data-listing-id]")

        for listing in listings:
            data = {
                "listing_id": listing.get("data-listing-id"),
                "title": listing.select_one(".listing-title"),
                "price": listing.select_one(".listing-price"),
                "size": listing.select_one(".listing-floorarea"),
                "district": district,
            }
            # Clean text from elements
            for key in ["title", "price", "size"]:
                if data[key]:
                    data[key] = data[key].get_text(strip=True)
            results.append(data)

        time.sleep(2)  # Be respectful with rate limiting

    return results

# Example: Scrape District 15 (East Coast/Katong) sale listings
listings = scrape_propertyguru(district=15, listing_type="sale", pages=3)
print(f"Found {len(listings)} listings in District 15")

Important notes:

Always check PropertyGuru's robots.txt and terms of service
Use reasonable rate limiting (2+ seconds between requests)
Selectors change frequently — you'll need to maintain your scraper
Consider using Playwright or Selenium if the site relies heavily on JavaScript rendering

Using Apify for Custom Tasks

If you don't want to maintain your own scraper infrastructure, you can build a custom Apify actor. Apify handles proxy rotation, scheduling, and storage — you just write the scraping logic.

For complementary location data, the Google Maps Scraper on Apify can help you enrich property listings with nearby amenities — schools, MRT stations, hawker centres, malls, and clinics. Proximity to amenities is one of the biggest price drivers in Singapore real estate.

For example, combine property listings with Google Maps data to answer: "Which District 19 condos are within 500m of an MRT station AND have 3+ schools nearby?"

Practical Tips for Singapore Property Data

HDB vs Private — These are fundamentally different markets. HDB resale is regulated by HDB with transaction data publicly available at data.gov.sg. Private property (condos, landed) is where scraped data adds the most value.
New launches vs Resale — New launch pricing comes from developers and is often only on PropertyGuru temporarily. Resale listings stay longer and have richer data.
PSF is king — Price per square foot is how Singaporeans compare properties. Always normalize by size.
MRT proximity matters enormously — Properties within 500m of an MRT station command a 10-15% premium. The Thomson-East Coast Line (TEL) completions through 2025-2026 are reshaping values in Districts 15 and 18.
Check URA caveats — Some listings are in areas zoned for future development or have plot ratio restrictions. Cross-reference with URA Master Plan data.

What's Next

The gap in the market for a reliable, maintained PropertyGuru scraper is real. The existing solutions are broken, and Singapore's property data needs are growing as more investors and PropTech startups enter the market.

If you build something useful, consider publishing it on the Apify Store — there's clear demand from the Singapore market, and the existing actors aren't delivering.

Have questions about scraping property data in Singapore? Drop a comment below.

How Singapore Recruiters Use LinkedIn Job Data to Beat the Talent Shortage in 2026

agenthustler — Sun, 12 Apr 2026 08:03:20 +0000

Singapore's job market is running hot — and recruiters are struggling to keep up.

With 77,700 open vacancies across the island and a talent shortage that shows no signs of easing, hiring teams are under pressure to fill roles faster, smarter, and cheaper. Meanwhile, 79% of Singapore recruiters now use AI tools in their workflow, according to recent industry surveys.

But here's what most recruiters miss: the richest source of hiring intelligence is already sitting in plain sight on LinkedIn. You're just not seeing all of it.

What LinkedIn Shows You vs. What the Data Actually Contains

LinkedIn's job search UI is built for job seekers, not hiring strategists. You get a search bar, some filters, and 25 results per page. That's fine if you're looking for a job. It's useless if you're trying to answer questions like:

Which companies in Tuas are ramping up semiconductor hiring this quarter?
What's the median salary range for DevOps engineers in Singapore vs. KL?
How many new fintech roles appeared in the CBD in the last 30 days?
Which recruitment agencies are posting the most jobs in your niche?

These are the questions that separate reactive recruiters from strategic ones. And the answers are all in LinkedIn's job data — you just need a way to extract it at scale.

Five Use Cases Singapore Recruiters Are Using Right Now

1. Mapping Competitor Hiring Patterns

When DBS, OCBC, or Grab suddenly posts 40 engineering roles in a week, that's a signal. It could mean a new product launch, an internal restructuring, or a team that just lost its lead. By tracking job posting velocity across target companies, you can anticipate talent movement before it happens.

A recruitment agency in Raffles Place told us they reduced time-to-fill by 30% simply by monitoring when competitors ramped up hiring — and reaching out to passive candidates at those companies before the poaching began.

2. Salary Benchmarking Across the Region

Singapore competes for talent with Hong Kong, Sydney, and increasingly with remote roles from US companies. Bulk job data lets you compare salary ranges across markets in real time — not from a report published six months ago.

Filter by job title, seniority, and location. Export to a spreadsheet. Build a salary matrix that's updated weekly, not annually.

3. Tracking Tech Talent Demand by Location

Singapore's tech talent isn't evenly distributed. One-North, Changi Business Park, and the CBD each have distinct hiring profiles. By geocoding job listings, you can map demand density — which matters when you're advising candidates on relocation or negotiating hybrid work arrangements.

Some agencies even track demand by MRT line to help candidates find roles with reasonable commutes. It sounds granular, but in a tight market, convenience sells.

4. Identifying Companies About to Hire (Before They Post)

If a company posts three senior roles in the same function within a week, they're likely about to post ten more. Early signal detection means you can reach out with candidates before the company even engages an agency — winning the mandate before the RFP.

5. Building Niche Talent Market Reports

The recruiters commanding premium fees in 2026 aren't just filling roles — they're selling intelligence. A monthly report on "AI/ML Hiring Trends in Singapore's Financial Sector" positions you as a strategic partner, not a CV-pusher.

Bulk job data makes these reports trivial to produce. Set up a weekly extraction, filter by industry and title, chart the trends, and send it to your clients. Most agencies charge SGD 2,000-5,000 per report.

How to Get LinkedIn Job Data at Scale

Manually copying job listings from LinkedIn is painfully slow, error-prone, and limited. That's why recruitment teams are turning to structured data extraction tools.

One popular option is the LinkedIn Jobs Scraper on Apify — a cloud-based tool that extracts job listings with all metadata: title, company, location, salary range, posting date, seniority level, and job description.

You don't need to install anything. Set your search criteria (keywords, location, date posted), run it, and download structured data in JSON, CSV, or Excel. Pricing is pay-per-result — roughly USD $1-3 per 1,000 job listings — which makes it accessible even for small agencies.

No coding required. The interface is point-and-click, and results typically arrive within minutes.

The ROI Math for Singapore Agencies

Let's keep this simple:

Cost: Extracting 10,000 Singapore job listings costs roughly SGD 5-15
Time saved: Manual research on the same data would take 40+ hours
Revenue opportunity: One successful placement from better market intelligence = SGD 10,000-30,000 in fees

The ROI isn't theoretical. It's arithmetic.

What This Means for Singapore's Recruitment Industry

The talent shortage isn't going away. MOM's latest projections show demand outpacing supply through 2027 in tech, healthcare, and green economy roles. The agencies that thrive won't be the ones working harder — they'll be the ones working with better data.

LinkedIn has the data. The tools to extract it are mature, affordable, and require no engineering team. The only question is whether you'll use them before your competitors do.

If you're a Singapore-based recruiter exploring data-driven hiring strategies, the LinkedIn Jobs Scraper is a good place to start. Pay only for results, no subscriptions required.

LinkedIn Profile Scraper API in 2026: 5 Alternatives After Proxycurl Shut Down

agenthustler — Sun, 12 Apr 2026 05:42:17 +0000

If you built anything on Proxycurl's LinkedIn Profile API, July 2025 was a rough month. LinkedIn's lawsuit forced them to shut down, leaving ~200K customers scrambling for alternatives. Your enrichment pipeline broke, your sales team lost real-time prospect data, and your recruiting tools went dark overnight.

I've spent the last few weeks testing every replacement I could find. Here's what actually works in 2026.

Who Needs LinkedIn Profile Data (And Why It Matters)

LinkedIn profile scraping isn't just a developer curiosity — it powers serious business workflows:

Sales teams enrich CRM leads with job titles, company names, and career history to personalize outreach
Recruiters pull candidate profiles at scale to build talent pipelines without manual copy-paste
Marketers build Ideal Customer Profiles (ICPs) by analyzing the demographics of existing customers
Researchers study labor market trends, skill demand shifts, and workforce mobility

Proxycurl gave all of these teams a clean REST API: send a LinkedIn URL, get back structured JSON with name, title, company, education, experience, skills, and more. Simple, fast, reliable — until it wasn't.

The 5 Best Alternatives in 2026

1. Apify LinkedIn Profile Scraper

Best for: Developers who want full control and pay-per-result pricing.

Apify runs actors (serverless scrapers) in the cloud. The LinkedIn Profile Scraper accepts a list of profile URLs and returns structured JSON with all the fields Proxycurl offered — name, headline, current position, experience history, education, skills, and location.

Pricing: Pay-per-event (~$0.035/profile on the Apify platform)
Fields: Full profile — name, headline, experience, education, skills, location, connections count
Rate limits: Controlled by your Apify plan (scales to thousands of profiles/day)
Legal stance: Runs on Apify's infrastructure; you control usage

2. Scrapingdog LinkedIn Scraper API

Best for: Teams that want a drop-in REST API with a free tier to test.

Scrapingdog offers a dedicated LinkedIn endpoint that returns profile data as JSON. Their free plan includes 1,000 credits — enough to validate your pipeline before committing.

Pricing: Free tier (1,000 credits), then from $40/mo
Fields: Name, headline, experience, education, location, about section
Rate limits: Based on plan tier
Legal stance: Proxy-based; they handle rotation and anti-bot

3. LinkdAPI

Best for: Event-driven architectures that need webhook delivery.

LinkdAPI differentiates with async webhook support — submit a profile URL, get results POSTed to your endpoint when ready. Useful if you're enriching leads in a queue-based system.

Pricing: From $49/mo for 500 profiles
Fields: Name, title, company, experience, education, certifications
Rate limits: 500-10,000 profiles/mo depending on plan
Legal stance: Operates as a data provider; ToS compliance is user's responsibility

4. Bright Data LinkedIn Dataset

Best for: Bulk analysis where you need thousands of profiles at once.

Bright Data offers pre-collected LinkedIn datasets and on-demand collection. Less "API" and more "data pipeline" — you define your target criteria, they deliver a dataset.

Pricing: Custom pricing; typically $0.01-0.05/record for bulk
Fields: Comprehensive — full profile, company data, skills endorsements
Rate limits: N/A (batch delivery)
Legal stance: They position themselves as compliant under data protection frameworks

5. PhantomBuster LinkedIn Profile Scraper

Best for: Non-technical teams that want a no-code solution.

PhantomBuster provides browser-based "Phantoms" that scrape LinkedIn using your own session cookie. It's the most accessible option for sales teams without developer support.

Pricing: From $69/mo for 500 automations
Fields: Name, headline, experience, education, connections
Rate limits: ~80 profiles/day to avoid LinkedIn detection
Legal stance: Uses your own LinkedIn session — risk is on you

Comparison Table

Feature	Apify	Scrapingdog	LinkdAPI	Bright Data	PhantomBuster
Free tier	Trial credits	1,000 free credits	No	No	14-day trial
Per-profile cost	~$0.035	~$0.04	~$0.10	~$0.01-0.05	~$0.14
Webhook support	Via integration	No	Yes (native)	No	Zapier
Full experience history	Yes	Yes	Yes	Yes	Yes
Skills & endorsements	Yes	Partial	Yes	Yes	No
Batch processing	Yes (built-in)	Sequential	Async queue	Native batch	Sequential
Self-hosted option	Yes (open source)	No	No	No	No

Migration Guide: Proxycurl to Apify in 10 Minutes

If you had Proxycurl code like this:

import requests

# Old Proxycurl code (no longer works)
resp = requests.get(
    "https://nubela.co/proxycurl/api/v2/linkedin",
    params={"url": "https://linkedin.com/in/williamhgates"},
    headers={"Authorization": "Bearer YOUR_PROXYCURL_KEY"}
)
profile = resp.json()
print(profile["full_name"], profile["headline"])

Here's the Apify equivalent:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("cryptosignals/linkedin-profile-scraper").call(
    run_input={
        "profileUrls": ["https://linkedin.com/in/williamhgates"]
    }
)

for profile in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(profile["name"], profile["headline"])
    print(f"Current: {profile.get('currentCompany', 'N/A')}")
    print(f"Experience: {len(profile.get('experience', []))} positions")
    print(f"Skills: {', '.join(profile.get('skills', [])[:5])}")

The key differences:

Async by default — Apify runs the scrape as a job, so you get richer data but wait a few seconds
Batch-native — pass 100 URLs in one call instead of looping
Dataset storage — results are stored and queryable, not just returned once

Install the client with pip install apify-client and grab your API token from Apify Console.

Which One Should You Pick?

Need a quick REST API with free testing? Start with Scrapingdog — their free tier lets you validate without a credit card.
Building a production enrichment pipeline? Go with Apify — pay-per-result scales better than monthly subscriptions, and the actor model lets you customize extraction.
Processing webhooks in an event-driven system? LinkdAPI is purpose-built for this.
Need 50K+ profiles for analysis? Bright Data bulk datasets are the most cost-effective at scale.
Non-technical team? PhantomBuster gets you started without writing code.

The Proxycurl shutdown was painful, but the ecosystem has matured. You actually have better options now than you did in 2024 — more flexible pricing, better batch support, and open-source alternatives you can self-host.

Building a LinkedIn data pipeline? I maintain the LinkedIn Profile Scraper on Apify — it's optimized for reliability and returns the same structured fields Proxycurl did.

Best Proxycurl Alternative in 2026: Apify LinkedIn Scrapers vs Scrapingdog vs LinkdAPI

agenthustler — Sun, 12 Apr 2026 05:05:39 +0000

Proxycurl was the go-to LinkedIn data API for recruiters, sales teams, and developers. At its peak, it served ~200K paying customers and pulled in $10M ARR. Then LinkedIn sued them in January 2025, and by July 2025, Proxycurl was dead.

If you relied on Proxycurl, you need a replacement. I tested the top alternatives so you don't have to.

What Happened to Proxycurl?

LinkedIn filed a lawsuit in January 2025 alleging unauthorized scraping and reselling of member data. Proxycurl fought it briefly, but by July 2025 they shut down completely. API endpoints went dark, documentation was pulled, and their website now shows a generic service discontinued page.

This left a massive gap. Proxycurl was clean, well-documented, and cheap. Finding something comparable is not trivial.

The Contenders

I evaluated four alternatives based on what matters most: reliability, pricing, data quality, and ease of migration.

Feature	Scrapingdog	LinkdAPI	BrightData	Apify LinkedIn Scrapers
Profile Data	Yes	Yes	Yes	Yes
Company Data	Yes	Yes	Yes	Yes
Job Listings	Limited	No	Yes	Yes
Pricing Model	Monthly plans	Monthly plans	Enterprise	Pay-per-event
Starting Price	$40/mo (5K credits)	$49/mo (2K lookups)	Custom (~$500+/mo)	~$0.005/result
Free Tier	1K credits	100 lookups	Trial only	Apify free tier
Rate Limits	10 req/s	5 req/s	Unlimited	Concurrent runs
API Format	REST JSON	REST JSON	REST JSON	REST JSON

Scrapingdog ($40-$200/mo)

Scrapingdog is a general-purpose scraping API with a LinkedIn endpoint. It works, but LinkedIn is not their focus — they cover 20+ sites. Their LinkedIn data sometimes has gaps in education history and skills sections.

Pros: Established company, good uptime, multi-site coverage.
Cons: LinkedIn data quality is inconsistent, monthly commitment, credits expire.

LinkdAPI ($49-$249/mo)

LinkdAPI is LinkedIn-focused, which means better data structure. Their profile endpoint returns clean JSON similar to Proxycurl. The downside is no job listing support, and the $49/mo minimum is steep if you are doing low-volume lookups.

Pros: Clean data format, LinkedIn-focused, Proxycurl-like response structure.
Cons: No job data, minimum $49/mo even for occasional use, 2K lookup cap on starter plan.

BrightData (Enterprise, $500+/mo)

BrightData is the enterprise option. If you are scraping millions of profiles for a sales intelligence platform, this is your pick. For everyone else, it is overkill. Pricing is opaque, onboarding involves a sales call, and the minimum commitment is high.

Pros: Scale, reliability, compliance team.
Cons: Expensive, slow onboarding, not suitable for startups or solo developers.

Apify LinkedIn Scrapers (Pay-Per-Event)

This is what I migrated to. Apify runs actors (cloud scrapers) that you call via API. Two actors cover the Proxycurl equivalent:

LinkedIn Jobs Scraper — public job listings with filters for title, location, and company
LinkedIn Profile and Company Scraper — profiles and company data for lead generation

The pricing model is pay-per-event (PPE): you pay per result returned, not per month. At $0.005-$0.01 per result, 10K profiles costs $50-$100 with no monthly commitment and no expiring credits.

Pros: No monthly fees, pay only for what you use, scales from 1 to 1M results, free tier available.
Cons: Slightly higher latency than direct APIs (runs are async), Apify platform learning curve.

Migration Guide: Proxycurl to Apify

Here is a working Python example that replaces a typical Proxycurl job lookup with the Apify LinkedIn Jobs Scraper:

import requests
import time

APIFY_TOKEN = "your_apify_token_here"  # get at apify.com/account#/integrations
ACTOR_ID = "cryptosignals/linkedin-jobs-scraper"

def scrape_linkedin_jobs(query, location="United States", max_results=25):
    # Start the actor run
    run_url = f"https://api.apify.com/v2/acts/{ACTOR_ID}/runs"
    payload = {
        "searchQueries": [query],
        "location": location,
        "maxResults": max_results
    }
    headers = {"Authorization": f"Bearer {APIFY_TOKEN}"}
    response = requests.post(run_url, json=payload, headers=headers)
    run_id = response.json()["data"]["id"]

    # Poll until finished
    status_url = f"https://api.apify.com/v2/actor-runs/{run_id}"
    while True:
        status = requests.get(status_url, headers=headers).json()
        if status["data"]["status"] in ("SUCCEEDED", "FAILED"):
            break
        time.sleep(5)

    # Fetch results
    dataset_id = status["data"]["defaultDatasetId"]
    results = requests.get(
        f"https://api.apify.com/v2/datasets/{dataset_id}/items",
        headers=headers
    ).json()
    return results

jobs = scrape_linkedin_jobs("python developer", location="Remote")
for job in jobs[:5]:
    print(f"{job.get('title')} at {job.get('company')}")

This pattern works the same way for profiles. Swap the actor ID to cryptosignals/linkedin-profile-scraper and adjust the input parameters. Check each actor page on Apify for the full input schema.

Which One Should You Pick?

Choose Scrapingdog if you need multi-site scraping and LinkedIn is just one of many sources.

Choose LinkdAPI if you only need profile data, want Proxycurl-like JSON responses, and don't mind monthly billing.

Choose BrightData if you are an enterprise doing millions of lookups and need a compliance team.

Choose Apify LinkedIn Scrapers if you want pay-per-use pricing, need jobs and profiles in one place, or you prefer controlling costs per request rather than per month.

For most developers migrating from Proxycurl, Apify makes the most sense financially, especially if your usage varies month to month. You are not locked into a $49/mo plan during months when you only need 200 lookups.

Final Thoughts

Proxycurl's shutdown was a wake-up call about depending on a single vendor for LinkedIn data. Whatever you choose, build your pipeline with swap-ability in mind. Abstract your data source behind an interface so the next time a provider goes down, migration is a config change, not a rewrite.

The LinkedIn data market is more fragmented now than in 2024, but that also means more options and more competitive pricing. For most use cases, pay-per-event pricing will save you money over monthly plans.

Indeed Job Data: 5 Real Use Cases for Recruiters, Analysts, and Developers

agenthustler — Sun, 12 Apr 2026 05:03:39 +0000

Indeed is the world's largest job board — 175 million monthly visitors, over 300 million job listings posted since launch, and coverage across 60+ countries. That's an enormous amount of structured data about who's hiring, what they're paying, and where the market is headed.

But accessing this data programmatically? That's where it gets tricky.

Indeed's official API is enterprise-only, expensive, and heavily restricted. Web scraping is fragile — layouts change, CAPTCHAs block you, and maintaining scrapers costs engineering time you could spend on actual analysis.

There's a better way.

Why Indeed Job Data Matters

Job postings are one of the most underrated data sources in business. Every listing is a buying signal — a company willing to spend money on talent. Aggregate enough of these signals, and you can:

Track which industries are growing or contracting
Identify companies scaling specific teams (engineering, sales, ops)
Benchmark salaries across roles, locations, and experience levels
Monitor competitor hiring strategies in real time

The problem was never why — it was how to get the data reliably.

5 Real Use Cases for Indeed Job Data

1. Recruitment Analytics

Staffing agencies and HR teams can monitor job posting volume by role, location, and company to spot demand trends before competitors do. If "AI Engineer" postings in Austin spike 40% month-over-month, that's actionable intelligence.

2. Lead Generation for B2B Sales

A company posting 15 new software engineering roles is likely buying tools, services, and infrastructure. Sales teams at SaaS companies, staffing firms, and consultancies use job posting data as a lead qualification signal — it's more reliable than intent data from ad platforms.

3. Salary Benchmarking

Compensation teams need market data. By extracting salary ranges from thousands of similar postings across geographies, you can build compensation benchmarks without paying $50K/year for a Radford subscription.

4. Labor Market Research

Economists, policy analysts, and investors track job posting data to understand economic health. Which sectors are hiring? Which are freezing? Indeed data provides ground-truth signals that lag less than government employment reports.

5. Custom Job Alert Pipelines

Build your own job monitoring system — filter by exact criteria (remote + Python + $150K+), deduplicate across sources, and push matches to Slack, email, or a dashboard. No more relying on Indeed's limited alert system.

How to Extract Indeed Jobs at Scale

The Indeed Jobs Scraper on Apify handles all the complexity — anti-bot measures, pagination, data normalization — so you get clean, structured JSON.

Here's how to run it with Python:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_API_TOKEN")

run_input = {
    "queries": ["python developer"],
    "location": "New York, NY",
    "maxResults": 100,
}

run = client.actor("cryptosignals/indeed-jobs-scraper").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"{item['title']} at {item['company']} — {item.get('salary', 'N/A')}")

Install the client with:

pip install apify-client

Sample Output

Each result comes back as structured JSON:

{
  "title": "Senior Python Developer",
  "company": "TechCorp Inc.",
  "location": "New York, NY",
  "salary": "$140,000 - $180,000 a year",
  "jobType": "Full-time",
  "description": "We are looking for an experienced Python developer...",
  "postedDate": "3 days ago",
  "url": "https://www.indeed.com/viewjob?jk=abc123",
  "companyRating": 4.2
}

You get the title, company, location, salary (when listed), job type, full description, posting date, direct URL, and company rating — all normalized and ready for analysis.

Pricing: $0.01 Per Result

Let's put this in perspective:

Method	Cost	Complexity
Indeed Official API	Enterprise pricing ($$$$)	Restricted access, approval required
Build your own scraper	Engineering time + proxies + maintenance	High — breaks regularly
Indeed Jobs Scraper on Apify	$0.01 per result	Zero — just call the API

For $1, you get 100 fully structured job listings. For $10, you get 1,000. That's less than the cost of a single proxy rotation on most scraping setups.

No infrastructure to manage. No CAPTCHAs to solve. No scrapers to maintain when Indeed changes their layout.

Get Started

Create a free Apify account
Open the Indeed Jobs Scraper
Enter your search queries and location
Hit Start and download your results as JSON, CSV, or Excel

You can also integrate it into any workflow via the Apify API — schedule daily runs, pipe results to a database, or trigger it from your own application.

Whether you're building a recruitment dashboard, qualifying B2B leads, or tracking the job market — Indeed data at $0.01/result removes the last excuse not to start.

Try the Indeed Jobs Scraper →

Google Maps API Too Expensive? 5 Cheaper Alternatives for Location Data in 2026

agenthustler — Sun, 12 Apr 2026 04:07:06 +0000

Google Maps API Pricing Is Crushing Small Projects

Google Maps Platform pricing has been a pain point since the 2018 overhaul. Here is what you pay today in 2026:

API Endpoint	Cost per 1,000 Requests	Free Tier
Geocoding	$5.00	40K/month ($200 credit)
Places Search	$32.00	~6,250/month
Places Details	$17.00	~11,764/month
Directions	$5.00-$10.00	40K/month
Street View	$7.00	~28,571/month

The $200 monthly credit sounds generous until you actually build something. A restaurant finder app making 50 Places Detail requests per user session burns through that credit in under 250 users. A real estate platform geocoding listings hits the wall even faster.

For businesses processing hundreds of thousands of requests, the bill can easily reach $2,000-$10,000/month. Here are the alternatives developers are actually switching to.

1. OpenStreetMap + Nominatim (Free, Self-Hosted)

OpenStreetMap is the Wikipedia of maps. Nominatim is its geocoding engine. Together, they replace Google's Geocoding and Places APIs for many use cases.

What you get:

Forward and reverse geocoding
Address search
POI data (restaurants, shops, landmarks)
Map tiles via services like MapTiler or Stadia Maps

Pricing: Free if self-hosted. Third-party hosted Nominatim (LocationIQ, Geocod.io) starts at $0.50/1,000 requests.

Pros:

Completely free at source
No vendor lock-in
Massive global coverage
Open data you can fork and customize

Cons:

Self-hosting requires a beefy server (64GB+ RAM for full planet import)
Data quality varies by region (excellent in Europe, patchy in parts of Asia/Africa)
No Street View equivalent
POI data is less comprehensive than Google for business listings

Best for: Projects where geocoding and basic POI search cover 80% of needs.

2. Scraping Google Maps Directly

If you specifically need Google Maps data (reviews, ratings, business hours, photos) but cannot afford the API pricing, web scraping is the most direct alternative.

Cloud scraping platforms like Apify offer Google Maps scrapers that extract business listings, reviews, and place details. You pay per result rather than per API call — typically $1-3 per 1,000 places, which is 10-30x cheaper than the official API.

What you can extract:

Business name, address, phone, website
Reviews and ratings
Opening hours
Photos
GPS coordinates

Pros:

10-30x cheaper than official API
Access to data Google does not expose via API (review text, photos)
Pay only for what you use
JSON/CSV output, easy to integrate

Cons:

Not officially sanctioned by Google
Data freshness depends on when you scrape
No real-time geocoding — this is for batch data collection
Google updates their frontend regularly, though cloud platforms handle the maintenance

Best for: Batch data collection of business listings, lead generation, market research.

3. HERE Maps API

HERE (formerly Nokia Maps) is the closest full-featured alternative to Google Maps Platform. They power the maps in Mercedes, BMW, and most automotive navigation systems.

Pricing:

Feature	HERE Cost / 1K	Google Cost / 1K
Geocoding	$0.90	$5.00
Places Search	$3.60	$32.00
Routing	$0.90	$5.00
Map Tiles	$0.72	$7.00

HERE offers a free tier of 250K transactions/month — significantly more generous than Google's $200 credit.

Pros:

Full-featured drop-in replacement for Google Maps
5-10x cheaper across the board
Generous free tier
Strong offline map support
Excellent in automotive and logistics

Cons:

Street View equivalent (HERE StreetLevel) has limited coverage
Smaller developer community than Google Maps
Places database not as comprehensive for small businesses
Migration requires code changes (different API structure)

Best for: Production applications that need a reliable, cheaper Google Maps replacement.

4. Mapbox

Mapbox powers maps for Meta, The New York Times, Instacart, and thousands of apps. Their strength is customizable, beautiful map rendering, but they also offer geocoding and routing.

Pricing:

100K free map loads/month
Geocoding: $0.75/1,000 requests (temporary) up to $0.60 at volume
Directions: $0.50-$2.00/1,000 requests
Custom map styles included

Pros:

Best-in-class map customization and design
Very competitive pricing
Excellent documentation and SDKs
Strong open-source tooling (Mapbox GL JS)

Cons:

Places/POI search is limited compared to Google
Geocoding quality slightly below Google in some regions
Pricing structure can be confusing with different product tiers
Recently made GL JS proprietary (v2+)

Best for: Consumer-facing apps where map appearance matters, or projects needing geocoding + custom maps.

5. LocationIQ + Overpass API (Budget Stack)

This is the budget developer's stack. LocationIQ provides hosted Nominatim geocoding at scale, while the Overpass API lets you query OpenStreetMap data directly for POIs.

Pricing:

LocationIQ: Free up to 5K requests/day, then $0.50/1,000
Overpass API: Free (rate limited on public servers)

Pros:

Cheapest option for geocoding at scale
Overpass queries are incredibly flexible (find all coffee shops within 500m of a point)
No credit card required to start
Built on open data

Cons:

Two separate services to integrate
Overpass has a learning curve (custom query language)
No routing or directions without adding another service
Business listing data is less complete than Google

Best for: Cost-conscious developers who need geocoding + POI search and are comfortable with some extra integration work.

Full Comparison Table

Solution	Geocoding Cost/1K	POI Data	Map Tiles	Real-Time	Setup
Google Maps	$5.00	Excellent	Yes	Yes	Easy
OpenStreetMap	Free (self-host)	Good	Via 3rd party	No	Hard
Google Maps Scrapers	$1-3 per 1K places	Excellent	No	No	Easy
HERE Maps	$0.90	Good	Yes	Yes	Medium
Mapbox	$0.75	Limited	Yes (best)	Yes	Easy
LocationIQ + Overpass	$0.50	Good	Via 3rd party	No	Medium

Which Alternative Is Right for You?

You need a full Google Maps replacement: Go with HERE Maps. It is the closest feature-for-feature alternative at 5-10x lower cost.

You need beautiful custom maps: Mapbox is unmatched for map rendering and design.

You need to scrape business data in bulk: Cloud scraping platforms give you Google Maps data at a fraction of the API cost, without the per-request pricing model.

You are building a side project on a budget: LocationIQ + OpenStreetMap covers geocoding and POIs for practically free.

You need everything for free: Self-host Nominatim and use Overpass, but budget time for setup and maintenance.

The $200 monthly Google Maps credit is enough for prototyping, but any app with real users will outgrow it quickly. Switching early saves both money and migration headaches later.

What are you using instead of Google Maps API? Share your stack in the comments.