DEV Community: Freshactors

How to scrape Microsoft Store app data in Python — ratings, reviews with helpful votes, any country (no API key)

Freshactors — Wed, 17 Jun 2026 17:56:02 +0000

Scraping the Microsoft Store is underrated: its public endpoints expose data Apple and Google don't — 7-day and 30-day rating windows (momentum!), and customer reviews with helpful-vote counts and the reviewer's device family. No API key, no login, no headless browser. In this tutorial we'll fetch app details for any country market, search the catalog, and mine reviews — as clean structured JSON in a few lines of Python.

The endpoints

The Store client runs on two public JSON services:

# details (BATCHED — many apps in one call), any market/language
GET https://displaycatalog.mp.microsoft.com/v7.0/products?bigIds={id1},{id2}&market=US&languages=en-us&fieldsTemplate=Details

# search (the Store's real consumer search) and reviews
GET https://storeedgefd.dsx.mp.microsoft.com/v9.0/pages/searchResults?query=netflix&market=US&locale=en-US&...
GET https://storeedgefd.dsx.mp.microsoft.com/v9.0/ratings/product/{id}?market=US&locale=en-US&pageSize=25&skipItems=0

App IDs are the 12-character codes starting with 9 in every Store URL — apps.microsoft.com/detail/9NKSQGP7F2NH is WhatsApp. (Watch out: XP... IDs from winget are classic desktop installers, not Store products — the catalog 404s on them.)

You can call these yourself — but then you own the quirks: the $type-heavy search envelope, humanized rating counts ("332K"), the skipItems review pagination, the ID-format trap, and fixing it the day Microsoft shifts a payload. A cleaner path: one actor, one stable schema, canary-monitored daily. Here's how with the Microsoft Store Scraper.

Step 1 — Install the Apify client

pip install apify-client
export APIFY_TOKEN="apify_api_xxx"

Step 2 — App details with rating windows

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("freshactors/microsoft-store-scraper").call(run_input={
    "mode": "details",
    "productIds": ["9NKSQGP7F2NH", "9NCBCSZSJRSB", "9WZDNCRFJ3TJ"],  # WhatsApp, Spotify, Netflix
})

for app in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(app["title"], app["rating"], app["rating30Days"])

A real record:

{
  "productId": "9NKSQGP7F2NH",
  "title": "WhatsApp",
  "publisher": "WhatsApp Inc.",
  "category": "Social",
  "rating":        { "average": 4.4, "count": 2294755 },
  "rating7Days":   { "average": 4.4, "count": 15549 },
  "rating30Days":  { "average": 4.4, "count": 68880 },
  "price": { "amount": 0, "currency": "USD", "isFree": true },
  "storeUrl": "https://apps.microsoft.com/detail/9NKSQGP7F2NH"
}

Comparing the windows to the all-time numbers tells you whether an app is improving or regressing right now — a signal the other stores simply don't publish. Want Germany instead? Add "market": "DE", "language": "de-de" and the descriptions, prices, and review pools localize.

Step 3 — Search the catalog

run = client.actor("freshactors/microsoft-store-scraper").call(run_input={
    "mode": "search",
    "searchTerms": ["pdf editor"],
    "maxSearchResults": 10,
})

Each row carries the product ID, rating, price, categories, content rating, and IAP flag — ready for a details follow-up on the IDs you care about.

Step 4 — Reviews with helpful votes

run = client.actor("freshactors/microsoft-store-scraper").call(run_input={
    "mode": "reviews",
    "productIds": ["9NKSQGP7F2NH"],
    "maxReviewsPerApp": 200,
})

Reviews paginate deep (WhatsApp has 55k+ on file) and each one includes helpfulPositive / helpfulNegative — i.e., community-validated complaints — plus deviceFamily and osVersion to split desktop vs Xbox feedback.

Pricing & reliability

Pay-per-result: app details $0.0025, search rows $0.001, reviews $0.0001 — no subscription, no per-run fee. A daily canary asserts the rating windows stay numeric, search coverage holds (Netflix must be findable), and review pagination advances — and we patch fast when Microsoft shifts an endpoint.

This completes the app-store set: same modes and schema family as our Apple App Store, Google Play, and VS Code Marketplace scrapers.

Questions or a field you're missing? Open an issue on the actor page — issues are answered fast.

How to scrape VS Code Marketplace extension stats in Python — installs, trending, reviews (no API key)

Freshactors — Wed, 17 Jun 2026 17:54:46 +0000

The VS Code Marketplace is the de-facto market map of developer tools — 80,000+ extensions with install counts, ratings, and trending scores. But there's no official stats dashboard and no documented bulk API. Here's the part most people miss: the Marketplace website itself runs on a public gallery JSON service that answers plain HTTP — no API key, no login, no headless browser. In this tutorial we'll pull extension stats, search a category sorted by installs, and fetch reviews — as clean structured JSON, in a few lines of Python.

The endpoint

The Marketplace's own search and listing pages call:

POST https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery
Content-Type: application/json
Accept: application/json;api-version=3.0-preview.1

with a JSON body of filter criteria (search text, exact extension name, or category) plus a flags bitmask that controls what comes back (statistics, versions, categories). The response carries installs, averagerating, ratingcount, and even trendingdaily/weekly/monthly velocity scores per extension. A separate public endpoint serves the newest reviews:

GET https://marketplace.visualstudio.com/_apis/public/gallery/publishers/{publisher}/extensions/{name}/reviews?count=50

So why not just requests.post() it yourself? You can — but then you own the contract: the right flag bits, the statistics-array parsing, the sort codes, the 100-per-page pagination, and fixing it the day Microsoft shifts the payload and your pipeline goes quietly empty. A cleaner path: call an actor that returns one stable schema and is canary-monitored daily. Here's how with the VS Code Marketplace Scraper.

Step 1 — Install the Apify client

pip install apify-client
export APIFY_TOKEN="apify_api_xxx"

(Token: Apify Console → Settings → Integrations.)

Step 2 — Get full stats for specific extensions

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("freshactors/vscode-marketplace-scraper").call(run_input={
    "mode": "details",
    "extensionIds": ["ms-python.python", "esbenp.prettier-vscode", "github.copilot"],
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["extensionId"], item["installs"], item["averageRating"], item["trendingMonthly"])

A real record looks like this:

{
  "extensionId": "ms-python.python",
  "name": "Python",
  "publisher": "ms-python",
  "publisherDisplayName": "Microsoft",
  "publisherVerified": true,
  "installs": 222361795,
  "averageRating": 4.2,
  "ratingCount": 630,
  "trendingDaily": 0.00215,
  "trendingWeekly": 0.41544,
  "trendingMonthly": 2.05958,
  "version": "2026.5.2026061001",
  "lastUpdated": "2026-06-10T11:13:22.933Z",
  "categories": ["Programming Languages", "Debuggers", "Data Science", "Machine Learning"],
  "repositoryUrl": "https://github.com/Microsoft/vscode-python",
  "marketplaceUrl": "https://marketplace.visualstudio.com/items?itemName=ms-python.python"
}

Step 3 — Map a category by install count

run = client.actor("freshactors/vscode-marketplace-scraper").call(run_input={
    "mode": "search",
    "category": "Data Science",
    "sortBy": "installs",
    "maxSearchResults": 50,
})

Fifty rows, biggest extensions first — instant competitive map of a niche. searchTerms works the same way for keywords (combine both to filter a category).

Step 4 — Reviews

run = client.actor("freshactors/vscode-marketplace-scraper").call(run_input={
    "mode": "reviews",
    "extensionIds": ["ms-python.python"],
    "maxReviewsPerExtension": 100,
})

Each review carries the rating, text, reviewer display name, extension version, and date. (The public endpoint serves the 100 most recent per extension — that's the endpoint's hard window, so the actor caps honestly.)

Pricing & reliability

Pay-per-result: extension details $0.0025, search rows $0.001, reviews $0.0001 — no subscription, no per-run fee. The actor is monitored by a daily canary (installs numeric, sorts working, reviews parsing) and patched fast when the gallery shifts — the listing shows a "last verified working" date that's actually real.

Three more app-store scrapers share the same shape if you need cross-store data: Apple App Store, Google Play, and Microsoft Store.

Questions or a field you're missing? Open an issue on the actor page — issues are answered fast.

How to scrape Personio career-portal jobs in Python — no API key

Freshactors — Thu, 11 Jun 2026 13:45:38 +0000

Personio is the ATS standard of the German-speaking SMB world — thousands of companies in Germany, Austria, and Switzerland run their careers page on a {tenant}.jobs.personio.de portal. Here's the part most people miss: every one of those portals serves a public XML feed of its published positions — no API key, no login, no headless browser. One GET returns the whole board with departments, seniority levels, and full descriptions. In this tutorial we'll pull a company's job board as clean structured JSON in a few lines of Python.

The endpoint

Every Personio career portal serves its feed at the explicit /xml path:

GET https://{tenant}.jobs.personio.de/xml
GET https://{tenant}.jobs.personio.de/xml?language=en   (optional localization)

The response is a <workzag-jobs> document with one <position> per job — id, office, department, name, employmentType, seniority, schedule, createdAt, and labeled description sections as CDATA HTML.

So why not just requests.get() it yourself? You can — but then you own the parser: handling CDATA sections, stripping the HTML, decoding entities, splitting multi-office strings, and fixing it the day the feed shape shifts and your pipeline goes quietly empty. A cleaner path: hand a list of tenants to an actor that returns one stable schema — the same schema as Greenhouse, Lever, Workable, SmartRecruiters, Recruitee, and Teamtailor. Here's how with the Personio Jobs Scraper.

Step 1 — Install the Apify client

pip install apify-client

Read your Apify API token (Console → Settings → Integrations) from an environment variable:

export APIFY_TOKEN="apify_api_xxx"

Step 2 — Run the actor with a list of tenants

companies accepts tenant subdomains (lanch) or {tenant}.jobs.personio.de URLs.

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "companies": ["teamative", "https://lanch.jobs.personio.de"],
    "includeDescription": True,
    "maxJobsPerCompany": 100,
}

run = client.actor("freshactors/personio-jobs-scraper").call(run_input=run_input)
print("Dataset id:", run["defaultDatasetId"])

Step 3 — Read the normalized output (departments + seniority included)

Every position comes back in the same shape, with null (never missing keys) where Personio's feed lacks a field:

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f'{item["company"]:<11} {item["title"]}  [{item.get("department") or "n/a"} / {item.get("seniority") or "n/a"}]')

A single record (a real one, from teamative's portal):

{
  "_type": "job",
  "_schemaVersion": "1.0",
  "_source": "personio",
  "company": "teamative",
  "jobId": "2623782",
  "title": "Initiativbewerbung (m/w/d)",
  "department": "Marketing",
  "seniority": "experienced",
  "location": "DE - Stuttgart",
  "allLocations": ["DE - Stuttgart"],
  "commitment": "Full-or-part-time",
  "url": "https://teamative.jobs.personio.de/job/2623782",
  "applyUrl": "https://teamative.jobs.personio.de/job/2623782",
  "postedAt": "2026-05-05T08:58:57.000Z",
  "descriptionText": "Über uns:\nteamative bietet Beratung, Entwicklung und... (labeled sections, clean text)",
  "_scrapedAt": "2026-06-10T12:34:05.149Z"
}

This is the same record shape our Greenhouse & Lever, Workable, SmartRecruiters, Recruitee, and Teamtailor scrapers emit — plus Personio's department and seniority, segmentation fields most ATS feeds don't expose.

Step 4 — Localized or lighter output

Want English titles/descriptions where the company maintains them? Pass a language code. Only need metadata for a hiring-signal dashboard? Drop the descriptions:

run_input = {
    "companies": ["teamative", "lanch"],
    "language": "en",               # localized where provided
    "includeDescription": False,    # smaller records; same cost & speed
    "maxJobsPerCompany": 100,
}

Prefer Node.js?

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('freshactors/personio-jobs-scraper').call({
    companies: ['teamative', 'https://lanch.jobs.personio.de'],
    includeDescription: true,
    maxJobsPerCompany: 100,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const job of items) console.log(`${job.company} — ${job.title} [${job.department ?? 'n/a'} / ${job.seniority ?? 'n/a'}]`);

What about cost?

Pay-per-event: $0.02 per company portal fetched and $0.0005 per job posting returned. So 5 companies returning 100 postings total is 5 × $0.02 + 100 × $0.0005 = $0.15 — departments, seniority, and full descriptions included. No subscription.

Why use the actor instead of the feed directly?

You can parse the XML yourself. The reason to use the actor is maintenance: it normalizes everything into one schema (shared with our five other ATS scrapers), handles CDATA/entities/multi-office strings, isolates per-company failures (an unknown tenant never kills your run), and is monitored by a daily canary — so a silent feed change doesn't quietly empty your pipeline.

The actor is here: Personio Jobs Scraper on Apify. Point it at your target companies and consume one normalized JSON feed.

Happy scraping.

How to scrape Teamtailor career-site jobs in Python — no API key

Freshactors — Thu, 11 Jun 2026 13:44:18 +0000

Teamtailor powers the career sites of thousands of companies — it's especially big among Nordic and European scale-ups (Polestar, Oatly, Instabee, and many more). Here's the part most people miss: every Teamtailor career site already serves a public JSON feed of its published jobs — no API key, no login, no headless browser. Even better, a single request returns the company's full published board, descriptions included. In this tutorial we'll pull a company's whole job board as clean structured JSON in a few lines of Python.

The endpoint

Every Teamtailor career site serves a machine-readable version of its jobs page at the .json suffix:

GET https://{company}.teamtailor.com/jobs.json

That returns a JSON Feed of every published posting — title, URL, publish date, the full description, and an embedded schema.org JobPosting with locations. One GET, the whole board. Custom-domain career sites (e.g. careers.oatly.com/jobs.json) serve the same feed.

(The bare /jobs URL also answers Accept: application/json content negotiation, but its CDN intermittently serves a cached HTML variant regardless of your headers — use the explicit .json suffix.)

So why not just requests.get() it yourself? You can — but then you own the parser: stripping the HTML descriptions, flattening the schema.org location objects, de-duplicating multi-location postings, and fixing it the day the feed shape shifts and your pipeline goes quietly empty. A cleaner path: hand a list of companies to an actor that returns one stable schema — the same schema as Greenhouse, Lever, Workable, SmartRecruiters, and Recruitee. Here's how with the Teamtailor Jobs Scraper.

Step 1 — Install the Apify client

pip install apify-client

Read your Apify API token (Console → Settings → Integrations) from an environment variable:

export APIFY_TOKEN="apify_api_xxx"

Step 2 — Run the actor with a list of companies

companies accepts subdomains (polestar), {name}.teamtailor.com URLs, or full career-site URLs — custom domains work too.

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "companies": ["polestar", "https://oatly.teamtailor.com"],
    "includeDescription": True,
    "maxJobsPerCompany": 100,
}

run = client.actor("freshactors/teamtailor-jobs-scraper").call(run_input=run_input)
print("Dataset id:", run["defaultDatasetId"])

Step 3 — Read the normalized output

Every posting comes back in the same shape, with null (never missing keys) where Teamtailor's feed lacks a field:

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f'{item["company"]:<10} {item["title"]}  ({item.get("location") or "n/a"})')

A single record (a real one, from Polestar's board):

{
  "_type": "job",
  "_schemaVersion": "1.0",
  "_source": "teamtailor",
  "company": "polestar",
  "jobId": "7610594",
  "title": "Customer Engagement Advisor | Denmark",
  "location": "Copenhagen, Nordics & UK, DK",
  "allLocations": ["Copenhagen, Nordics & UK, DK"],
  "country": "DK",
  "url": "https://polestar.teamtailor.com/jobs/7610594-customer-engagement-advisor-denmark",
  "applyUrl": "https://polestar.teamtailor.com/jobs/7610594-customer-engagement-advisor-denmark",
  "postedAt": "2026-04-21T14:35:06.000Z",
  "descriptionText": "Din mulighed... (full description as clean text)",
  "_scrapedAt": "2026-06-10T11:29:56.613Z"
}

This is the same record shape the Greenhouse & Lever, Workable, SmartRecruiters, and Recruitee scrapers emit, so Teamtailor companies drop into the same multi-ATS pipeline with zero special-casing. (Fields Teamtailor's public feed doesn't expose — department, commitment — come back as null, never missing.)

Step 4 — Lighter output

The descriptions arrive in the same request, so they're free. If you only need metadata (titles, locations, dates) — say, for a hiring-signal dashboard — set includeDescription: False to trim the payload:

run_input = {
    "companies": ["polestar", "instabee"],
    "includeDescription": False,   # smaller records; same cost & speed
    "maxJobsPerCompany": 100,
}

Prefer Node.js?

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('freshactors/teamtailor-jobs-scraper').call({
    companies: ['polestar', 'https://oatly.teamtailor.com'],
    includeDescription: true,
    maxJobsPerCompany: 100,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const job of items) console.log(`${job.company} — ${job.title} (${job.location ?? 'n/a'})`);

What about cost?

Pay-per-event: $0.02 per company career site fetched and $0.0005 per job posting returned. So 5 companies returning 100 postings total is 5 × $0.02 + 100 × $0.0005 = $0.15 — full descriptions included. No subscription.

Why use the actor instead of the feed directly?

You can call the feed yourself. The reason to use the actor is maintenance: it normalizes everything into one schema (shared with our four other ATS scrapers), cleans the HTML descriptions, handles custom domains and multi-location postings, isolates per-company failures (a dead board never kills your run), and is monitored by a daily canary — so a silent feed change doesn't quietly empty your pipeline.

The actor is here: Teamtailor Jobs Scraper on Apify. Point it at your target companies and consume one normalized JSON feed.

Happy scraping.

How to scrape SmartRecruiters jobs data with Python (no API key needed)

Freshactors — Wed, 03 Jun 2026 11:53:10 +0000

SmartRecruiters powers the careers sites of thousands of enterprises (Visa, Bosch, and many more), and every one of those boards is backed by a public Posting API — no API key, no login, no headless browser. In this tutorial we'll pull a company's full job board (titles, locations, employment type, remote flags, and the complete description for every role) as clean structured JSON, in a few lines of Python.

Why not just hit the API yourself?

You can — that's exactly what this actor does under the hood. SmartRecruiters exposes a two-tier public API:

List:   https://api.smartrecruiters.com/v1/companies/{company}/postings
Detail: https://api.smartrecruiters.com/v1/companies/{company}/postings/{id}

The list gives you rich metadata; the full job description lives in the detail endpoint, in HTML jobAd sections. So to get complete records you paginate the list, then fetch each posting's detail and stitch the sections together — and you maintain that two-step parser forever, including the day a field name quietly changes and your pipeline goes empty.

A cleaner path: hand a list of company identifiers to an actor that already does the pagination + detail-enrichment and returns one stable schema — the same schema as Greenhouse, Lever, and Workable. Here's how with the SmartRecruiters Jobs Scraper.

Step 1 — Install the Apify client

pip install apify-client

Read your Apify API token (Console → Settings → Integrations) from an environment variable:

export APIFY_TOKEN="apify_api_xxx"

Step 2 — Run the actor with a list of companies

companies accepts identifiers (visa) or board URLs (https://jobs.smartrecruiters.com/Visa). The identifier is the company segment in the jobs.smartrecruiters.com URL.

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "companies": ["visa", "https://jobs.smartrecruiters.com/BoschGroup"],
    "includeDescription": True,
    "maxJobsPerCompany": 500,
}

run = client.actor("freshactors/smartrecruiters-jobs-scraper").call(run_input=run_input)
print("Dataset id:", run["defaultDatasetId"])

Step 3 — Read the normalized output

Every posting comes back in the same shape, with null (never missing keys) where SmartRecruiters lacks a field:

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(
        f'{item["company"]:<10} '
        f'{item["title"]}  '
        f'({item.get("workplaceType") or "n/a"}, {item.get("location") or "n/a"})'
    )

A single record:

{
  "_type": "job",
  "_schemaVersion": "1.0",
  "_source": "smartrecruiters",
  "company": "visa",
  "jobId": "744000122509268",
  "title": "Sr. SW Engineer",
  "department": "Technology and Operations",
  "team": "Information Technology",
  "location": "Austin, TX, United States",
  "workplaceType": "hybrid",
  "commitment": "Full-time",
  "country": "US",
  "url": "https://jobs.smartrecruiters.com/visa/744000122509268",
  "applyUrl": "https://jobs.smartrecruiters.com/visa/744000122509268?oga=true",
  "postedAt": "2026-04-23T16:54:54.835Z",
  "descriptionText": "Company Description... Job Description... Qualifications...",
  "_scrapedAt": "2026-06-02T17:00:00.000Z"
}

This is the same record shape the Greenhouse & Lever and Workable scrapers emit — so SmartRecruiters companies drop into the same pipeline with zero special-casing.

Step 4 — Faster, lighter runs

The full description requires a detail call per posting. If you only need titles, departments, and locations — say, for a hiring-signal dashboard — turn it off for a list-only run (metadata + posting URL, one request per page):

run_input = {
    "companies": ["visa", "airbnb"],
    "includeDescription": False,   # list-only, no per-posting detail calls
    "maxJobsPerCompany": 200,
}

maxJobsPerCompany (1–5000) caps volume so a large employer doesn't dominate your run.

Prefer Node.js?

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('freshactors/smartrecruiters-jobs-scraper').call({
    companies: ['visa', 'https://jobs.smartrecruiters.com/BoschGroup'],
    includeDescription: true,
    maxJobsPerCompany: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const job of items) console.log(`${job.company} — ${job.title} (${job.workplaceType ?? 'n/a'})`);

What about cost?

Pay-per-event: $0.02 per company board fetched and $0.0005 per job posting returned. So 5 companies returning 100 postings total is 5 × $0.02 + 100 × $0.0005 = $0.15. No subscription.

Why use the actor instead of the API directly?

You can call the Posting API yourself. The reason to use the actor is maintenance: it paginates, enriches each posting with its description, normalizes everything into one schema (shared with Greenhouse/Lever/Workable), isolates per-company failures, and is monitored by a daily canary so a silent API change doesn't quietly empty your pipeline.

The actor is here: SmartRecruiters Jobs Scraper on Apify. Point it at your target companies and consume one normalized JSON feed.

Happy scraping.

How to scrape Recruitee jobs data (with salary) in Python — no API key

Freshactors — Tue, 02 Jun 2026 18:36:49 +0000

Recruitee powers the careers sites of thousands of companies (bunq, Channable, Vandebron, and many more), and every board is backed by a public Offers API — no API key, no login, no headless browser. The best part: a single request returns every offer with its full description, requirements, and — unusually for an ATS — salary. In this tutorial we'll pull a company's whole job board as clean structured JSON in a few lines of Python.

The endpoint

Recruitee exposes one endpoint per company subdomain:

https://{company}.recruitee.com/api/offers/

One GET returns the entire board — metadata, HTML description, requirements, and a salary object — in a single response. No detail call, no pagination. That's it.

So why not just requests.get() it yourself? You can — but then you own the parser: stripping the HTML, normalizing employment-type codes like fulltime_permanent, filtering out draft/closed offers, flattening the salary object, and fixing it the day a field name changes and your pipeline goes empty. A cleaner path: hand a list of company identifiers to an actor that returns one stable schema — the same schema as Greenhouse, Lever, Workable, and SmartRecruiters. Here's how with the Recruitee Jobs Scraper.

Step 1 — Install the Apify client

pip install apify-client

Read your Apify API token (Console → Settings → Integrations) from an environment variable:

export APIFY_TOKEN="apify_api_xxx"

Step 2 — Run the actor with a list of companies

companies accepts identifiers (bunq) or board URLs (https://bunq.recruitee.com). The identifier is the subdomain in the company's {name}.recruitee.com careers URL.

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "companies": ["bunq", "https://channable.recruitee.com"],
    "includeDescription": True,
    "maxJobsPerCompany": 500,
}

run = client.actor("freshactors/recruitee-jobs-scraper").call(run_input=run_input)
print("Dataset id:", run["defaultDatasetId"])

Step 3 — Read the normalized output (including salary)

Every offer comes back in the same shape, with null (never missing keys) where Recruitee lacks a field:

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    pay = item.get("salary")
    pay_str = f'{pay["min"]}–{pay["max"]} {pay["currency"]}' if pay else "n/a"
    print(f'{item["company"]:<12} {item["title"]}  ({item.get("location") or "n/a"})  pay: {pay_str}')

A single record:

{
  "_type": "job",
  "_schemaVersion": "1.0",
  "_source": "recruitee",
  "company": "bunq",
  "jobId": "2620732",
  "title": "Senior Backend Engineer",
  "department": "Engineering",
  "location": "Amsterdam, Netherlands",
  "workplaceType": "hybrid",
  "commitment": "Full-time",
  "country": "NL",
  "url": "https://careers.bunq.com/o/senior-backend-engineer",
  "applyUrl": "https://careers.bunq.com/o/senior-backend-engineer/c/new",
  "postedAt": "2026-05-29T09:45:21.000Z",
  "salary": { "min": 65000, "max": 90000, "period": "year", "currency": "EUR" },
  "descriptionText": "About the role... Requirements...",
  "_scrapedAt": "2026-06-02T18:00:00.000Z"
}

This is the same record shape the Greenhouse & Lever, Workable, and SmartRecruiters scrapers emit — plus a Recruitee-only salary object — so Recruitee companies drop into the same pipeline with zero special-casing.

Step 4 — Lighter output

The description + requirements arrive in the same request, so they're free. If you only need metadata (titles, departments, locations, salary) — say, for a comp-benchmarking or hiring-signal dashboard — set includeDescription: False to trim the payload:

run_input = {
    "companies": ["bunq", "vandebron"],
    "includeDescription": False,   # smaller records; same cost & speed
    "maxJobsPerCompany": 200,
}

maxJobsPerCompany (1–5000) caps volume so a large employer doesn't dominate your run.

Prefer Node.js?

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('freshactors/recruitee-jobs-scraper').call({
    companies: ['bunq', 'https://channable.recruitee.com'],
    includeDescription: true,
    maxJobsPerCompany: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const job of items) console.log(`${job.company} — ${job.title} (${job.salary ? job.salary.currency : 'no pay listed'})`);

What about cost?

Pay-per-event: $0.02 per company board fetched and $0.0005 per job posting returned. So 5 companies returning 100 postings total is 5 × $0.02 + 100 × $0.0005 = $0.15 — descriptions and salary included. No subscription.

Why use the actor instead of the API directly?

You can call the Offers API yourself. The reason to use the actor is maintenance: it filters to published offers, normalizes everything into one schema (shared with Greenhouse/Lever/Workable/SmartRecruiters), flattens the salary object, isolates per-company failures, and is monitored by a daily canary so a silent API change doesn't quietly empty your pipeline.

The actor is here: Recruitee Jobs Scraper on Apify. Point it at your target companies and consume one normalized JSON feed.

Happy scraping.

How to scrape Workable jobs data with Python (no API key needed)

Freshactors — Tue, 02 Jun 2026 17:23:50 +0000

Tens of thousands of companies run their careers page on Workable, and every one of those boards is backed by a public JSON endpoint — no API key, no login, no headless browser required. In this tutorial we'll pull a company's full job board (titles, locations, employment type, remote flags, and the complete description for every role) as clean structured JSON, in a few lines of Python.

Why not just `fetch()` the board yourself?

You can. Workable's public widget API lives at:

https://apply.workable.com/api/v1/widget/accounts/{shortcode}?details=true

Pass details=true and a single GET returns the entire board with full descriptions inline — which is genuinely convenient. The catch is the usual one for an undocumented endpoint: the moment Workable reshapes a field or the response, your parser silently returns nothing, and you find out when your dashboard goes blank. And if you're tracking many companies, you're now maintaining that parser forever.

A cleaner path: hand a list of company shortcodes to an actor that already normalizes the board into a stable schema — the same schema as Greenhouse and Lever — and just consume the output. Here's how with the Workable Jobs Scraper.

Step 1 — Install the Apify client

pip install apify-client

Grab your Apify API token from Settings → Integrations in the Apify Console, and read it from an environment variable so it never lands in source control:

export APIFY_TOKEN="apify_api_xxx"

Step 2 — Run the actor with a list of companies

The actor takes a companies array of Workable shortcodes (pearltalent) or board URLs (https://apply.workable.com/pearltalent/). The shortcode is the slug in the company's apply.workable.com careers URL.

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "companies": ["pearltalent", "https://apply.workable.com/walletconnect/"],
    "includeDescription": True,
    "maxJobsPerCompany": 500,
}

# Blocks until the run finishes, then returns run metadata.
run = client.actor("freshactors/workable-jobs-scraper").call(run_input=run_input)

print("Run status:", run["status"])
print("Dataset id:", run["defaultDatasetId"])

.call() is synchronous — it waits for the run to complete and hands you the run object, including the defaultDatasetId where results land.

Step 3 — Read the normalized output

Every posting comes back in the same shape. Fields Workable doesn't provide are null, never missing keys, so your downstream code can rely on the schema:

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(
        f'{item["company"]:<14} '
        f'{item["title"]}  '
        f'({item.get("workplaceType") or "n/a"}, {item.get("location") or "n/a"})'
    )

A single record looks like this:

{
  "_type": "job",
  "_schemaVersion": "1.0",
  "_source": "workable",
  "company": "walletconnect",
  "jobId": "2EB3D29E9B",
  "title": "Product Manager - Merchant Experience",
  "department": "Product",
  "team": null,
  "location": "United Kingdom",
  "allLocations": ["United Kingdom", "Portugal"],
  "workplaceType": "remote",
  "commitment": null,
  "country": "United Kingdom",
  "url": "https://apply.workable.com/j/2EB3D29E9B",
  "applyUrl": "https://apply.workable.com/j/2EB3D29E9B/apply",
  "postedAt": "2026-04-27",
  "updatedAt": null,
  "descriptionText": "About the role...",
  "_scrapedAt": "2026-06-02T16:40:46.908Z"
}

This is the same record shape the Greenhouse & Lever scraper emits — so if you're already pulling ATS data, Workable companies slot into the exact same pipeline with no special-casing.

Step 4 — A practical filter (remote roles, posted recently)

Say you only want remote roles posted in the last two weeks. With one schema, the filter is trivial:

from datetime import datetime, timedelta, timezone

cutoff = (datetime.now(timezone.utc) - timedelta(days=14)).date().isoformat()
remote_recent = []

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item.get("workplaceType") != "remote":
        continue
    if (item.get("postedAt") or "") >= cutoff:
        remote_recent.append(item)

print(f"{len(remote_recent)} remote roles posted in the last 14 days")

Step 5 — Lighter, faster runs

Two knobs control cost and speed:

includeDescription: false drops the full descriptionText from each record — handy when you only need titles, departments, and locations for a hiring-signal dashboard.
maxJobsPerCompany caps postings per company (1–5000), so a 1,000-role employer doesn't dominate your run.

run_input = {
    "companies": ["pearltalent", "gbg", "walletconnect"],
    "includeDescription": False,   # metadata only
    "maxJobsPerCompany": 200,
}

Prefer Node.js?

Same actor, same input, the JavaScript client:

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('freshactors/workable-jobs-scraper').call({
    companies: ['pearltalent', 'https://apply.workable.com/walletconnect/'],
    includeDescription: true,
    maxJobsPerCompany: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const job of items) {
    console.log(`${job.company} — ${job.title} (${job.workplaceType ?? 'n/a'})`);
}

What about cost?

It's pay-per-event: $0.02 per company board fetched and $0.0005 per job posting returned. So 5 companies returning 100 postings total is 5 × $0.02 + 100 × $0.0005 = $0.15. No subscription — you pay for what you pull, and one request returns the whole board including descriptions, so runs stay cheap.

Why use the actor instead of hitting the board directly?

You can curl that public endpoint yourself. The reason to use the actor is maintenance: it normalizes the board into a stable schema (shared with Greenhouse + Lever), isolates per-company failures so one dead board never kills the run, and is monitored by a daily canary so a silent change to Workable's response doesn't quietly empty your pipeline. That operational reliability is the whole point.

If you want a clean Workable jobs feed without owning a parser, the actor is here: Workable Jobs Scraper on Apify. Run it on a schedule, point it at your target companies, and consume one normalized JSON feed.

Happy scraping.

How to scrape Apple App Store data with Python (no API key needed)

Freshactors — Tue, 02 Jun 2026 12:17:37 +0000

Apple doesn't offer a clean public API for App Store details, search rankings, or reviews. If you've tried building your own scraper, you've probably discovered the annoying part: Apple's legacy review feed silently returns an empty 200 response when it's rate-limited, so your scraper happily reports "0 reviews" when there are thousands. Debugging that is a bad afternoon.

In this tutorial we'll skip that pain and pull structured App Store data in a few lines of Python — no Apple Developer account, no API key, no login. We'll use the App Store Scraper actor on Apify, which handles the throttling, retries, and pagination for you. The same approach works in Node if you prefer JavaScript.

What we'll build

Three small scripts, one per mode:

App details — full metadata for one or more apps.
Search — ranked results for a keyword (great for ASO).
Reviews — customer reviews with ratings, version, and helpful votes.

Prerequisites

Python 3.8+
A free Apify account and your API token (Settings → Integrations → API token)
The Apify client:

pip install apify-client

Set your token as an environment variable so it's not hardcoded:

export APIFY_TOKEN="your_token_here"

1. Scrape app details

You identify an app by its App ID — the number in its App Store URL. For apps.apple.com/us/app/instagram/id389801252, the App ID is 389801252.

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "mode": "details",
    "appIds": ["389801252", "324684580"],  # Instagram, WhatsApp
    "country": "us",
}

run = client.actor("freshactors/app-store-scraper").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"{item['title']} — {item['averageUserRating']}★ "
          f"({item['userRatingCount']} ratings), v{item['version']}")

A details record looks like this (trimmed):

{
  "title": "Instagram",
  "appId": "389801252",
  "bundleId": "com.burbn.instagram",
  "sellerName": "Instagram, Inc.",
  "averageUserRating": 4.7,
  "userRatingCount": 28910344,
  "version": "430.1.0",
  "price": 0,
  "formattedPrice": "Free",
  "primaryGenre": "Photo & Video",
  "contentRating": "12+",
  "currentVersionReleaseDate": "2026-05-28",
  "url": "https://apps.apple.com/us/app/id389801252"
}

Prefer bundle identifiers? Swap appIds for bundleIds in details mode:

run_input = {
    "mode": "details",
    "bundleIds": ["com.burbn.instagram"],
    "country": "us",
}

2. Search the App Store by keyword

Search returns ranked results carrying the same rich fields as details — perfect for keyword rank tracking or competitor discovery.

run_input = {
    "mode": "search",
    "searchTerms": ["habit tracker", "budget app"],
    "maxSearchResults": 25,
    "country": "us",
}

run = client.actor("freshactors/app-store-scraper").call(run_input=run_input)

for i, item in enumerate(client.dataset(run["defaultDatasetId"]).iterate_items(), 1):
    print(f"{i:>2}. {item['title']} — {item['sellerName']} "
          f"({item['averageUserRating']}★)")

Because each result includes userRatingCount and averageUserRating, you can rank competitors by rating volume or spot which apps dominate a keyword in seconds.

3. Scrape App Store reviews (the throttle-proof part)

This is where naive scrapers fall over. Set mode to reviews, pass appIds, and cap the volume with maxReviewsPerApp (up to 5,000). You can sort by mostRecent or mostHelpful.

run_input = {
    "mode": "reviews",
    "appIds": ["389801252"],
    "maxReviewsPerApp": 500,
    "reviewsSort": "mostRecent",
    "country": "us",
    "lang": "en-US",
}

run = client.actor("freshactors/app-store-scraper").call(run_input=run_input)

reviews = list(client.dataset(run["defaultDatasetId"]).iterate_items())
print(f"Fetched {len(reviews)} reviews")

# Quick rating distribution
from collections import Counter
dist = Counter(r["rating"] for r in reviews)
for stars in range(5, 0, -1):
    print(f"{stars}★: {dist.get(stars, 0)}")

Each review record:

{
  "_type": "review",
  "_schemaVersion": "1.0",
  "appId": "389801252",
  "reviewId": "11583920002",
  "country": "us",
  "rating": 1,
  "title": "Keeps crashing",
  "body": "Crashes on launch since the update on my iPad.",
  "userName": "mark_p",
  "appVersion": "430.1.0",
  "voteSum": 10,
  "voteCount": 12,
  "_scrapedAt": "2026-06-01T07:57:08.714Z"
}

Because each review carries appVersion, you can do something genuinely useful — detect a bad release:

from collections import defaultdict

by_version = defaultdict(list)
for r in reviews:
    by_version[r["appVersion"]].append(r["rating"])

for version, ratings in sorted(by_version.items()):
    avg = sum(ratings) / len(ratings)
    print(f"v{version}: {avg:.2f}★ over {len(ratings)} reviews")

If one version's average craters versus the previous one, you've found a regression before it sinks your overall rating.

A note on reliability

The reason this works without babysitting: the actor detects Apple's empty-but-200 throttle response and retries with backoff and proxy rotation, using the modern reviews API for deeper pagination where available and falling back to a hardened RSS path otherwise. Every record carries a _schemaVersion, and the actor is checked by a daily canary against known apps, so your ETL won't silently break when Apple changes something.

What does it cost?

Pricing is pay-per-event, so you only pay for what you get:

App details: $0.002 per app
Search results: $0.001 per app
Reviews: $0.0001 per review

So 500 reviews of one app costs about $0.05 in event fees. (Apify platform compute is billed separately, as with any actor.)

Doing it in Node instead

Same actor, same inputs — just the JS client:

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('freshactors/app-store-scraper').call({
  mode: 'reviews',
  appIds: ['389801252'],
  maxReviewsPerApp: 200,
  reviewsSort: 'mostHelpful',
  country: 'us',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Fetched ${items.length} reviews`);

Wrap-up

With one actor and a few lines of code you can pull App Store details, search rankings, and reviews as clean JSON — no API key, no login, and without writing your own throttle-handling logic. Point it at your own app to monitor release sentiment, at competitors for ASO research, or at a whole category for market analysis.

If you want to try it, the actor lives here: App Store Scraper on Apify. It runs in details, search, and reviews modes with the exact inputs shown above. Questions or edge cases? The Issues tab is monitored — keeping it working is the whole point.

How to scrape Redfin (US real estate) data with Node.js (no API key needed)

Freshactors — Mon, 01 Jun 2026 19:50:44 +0000

Redfin has some of the cleanest US real-estate data on the web — list prices, sold prices, beds/baths, square footage, lat/long, MLS IDs — but there's no public Redfin API you can just call. In this tutorial we'll pull structured Redfin listings as JSON using Node.js, without an API key, a login, or writing a single CSS selector.

We'll use a maintained Apify actor that handles the annoying parts (region resolution, Redfin's anti-bot "soft block" behavior, retries) so we can focus on the data.

Why not just `fetch()` Redfin yourself?

You can hit Redfin's internal GIS endpoint directly — for about a day. Two things bite you:

The soft block. Redfin's data API returns a 200 with an envelope that says errorMessage: "Success" even when it's blocking you as a bot. Your scraper "succeeds" and writes zero rows. You don't notice until your dashboard is empty.
Region IDs. Redfin doesn't take a city name — it takes an internal region_id and region_type. You have to resolve those first.

We'll offload both to the FreshActors Redfin Scraper, which parses the region straight from a Redfin URL and distinguishes a real zero-result from a block (retrying with backoff + IP rotation when it sees one).

Prerequisites

Node.js 18+
A free Apify account and your API token (Settings → Integrations)

Install the client:

npm install apify-client

Step 1 — Call the actor

The only required input is a Redfin URL. Paste a city/ZIP/county page from redfin.com and the actor reads the region from it.

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const input = {
  redfinUrls: ['https://www.redfin.com/city/30818/TX/Austin'],
  listingType: 'forSale',   // or 'sold'
  sort: 'price-desc',
  maxListings: 500,
};

// Run the actor and wait for it to finish.
const run = await client.actor('freshactors/redfin-scraper').call(input);

// Pull the results from the run's dataset.
const { items } = await client.dataset(run.defaultDatasetId).listItems();

console.log(`Got ${items.length} listings`);
console.log(items[0]);

That's the whole integration. No selectors, no proxy config, no Redfin region lookup.

Step 2 — What the output looks like

Each item is one property. A trimmed real record:

{
  "_type": "listing",
  "_schemaVersion": "1.0",
  "listingId": 216197035,
  "mlsId": "2116358131372161577",
  "mlsStatus": "Active",
  "address": "4602 Indian Wells Dr",
  "city": "Austin",
  "state": "TX",
  "zip": "78747",
  "price": 560000,
  "beds": 3,
  "baths": 2,
  "sqft": 2097,
  "pricePerSqFt": 267,
  "yearBuilt": 2019,
  "propertyType": "Single Family Residential",
  "latitude": 30.1327479,
  "longitude": -97.79,
  "daysOnMarket": 12,
  "url": "https://www.redfin.com/TX/Austin/4602-Indian-Wells-Dr-78747/home/31845362",
  "_scrapedAt": "2026-06-01T08:08:20.543Z"
}

You also get propertyId, unit, hoa, lotSize, soldDate, and listingType. The schema is versioned (_schemaVersion), so fields don't silently vanish between runs.

Step 3 — Do something useful with it

Let's find the cheapest-per-square-foot homes — a quick proxy for value hunting:

const deals = items
  .filter((h) => h.price && h.sqft)
  .map((h) => ({
    address: `${h.address}, ${h.city} ${h.state}`,
    price: h.price,
    ppsf: h.pricePerSqFt ?? Math.round(h.price / h.sqft),
    dom: h.daysOnMarket,
    url: h.url,
  }))
  .sort((a, b) => a.ppsf - b.ppsf)
  .slice(0, 10);

console.table(deals);

Combining a low pricePerSqFt with a high daysOnMarket is a classic motivated-seller filter.

Step 4 — Get sold comps (the "Redfin API" alternative people actually want)

For an AVM, a CMA tool, or a price-trend model, you want sold data. Flip one field:

const soldInput = {
  redfinUrls: ['https://www.redfin.com/zipcode/78701'],
  listingType: 'sold',           // sold homes
  sort: 'days-on-redfin-asc',    // most recent first
  maxListings: 300,
};

const soldRun = await client.actor('freshactors/redfin-scraper').call(soldInput);
const { items: sold } = await client.dataset(soldRun.defaultDatasetId).listItems();

const medianSoldPrice = (() => {
  const prices = sold.map((h) => h.price).filter(Boolean).sort((a, b) => a - b);
  const mid = Math.floor(prices.length / 2);
  return prices.length % 2 ? prices[mid] : (prices[mid - 1] + prices[mid]) / 2;
})();

console.log(`Median sold price in 78701: $${medianSoldPrice.toLocaleString()}`);

Sold records carry a soldDate, so you can bucket by month and chart how a ZIP's median price moves over time.

Step 5 — Scale it up (or down)

Pricing is pay-per-result: $0.002 per listing. So 100 listings cost $0.20 and a 1,000-row run is $2.00. Use maxListings to keep any run within budget, and pass several URLs in redfinUrls to cover multiple cities in one call (the cap applies across all of them).

For ongoing data, schedule the actor (Apify → Schedules) to run daily and append to a dataset — that's your always-fresh Redfin feed.

A note on reliability

The reason I reach for this actor over a hand-rolled script is the daily canary: it runs the scraper against live Redfin every day and flags field drift or block-pattern changes, so it gets patched before your pipeline returns garbage. With a DIY scraper, you're the canary — and you usually find out when a report is already empty.

Wrap-up

With ~15 lines of Node you can pull structured Redfin listings — for-sale or sold — without an API key, region lookups, or anti-bot wrestling. Point it at a URL, get clean JSON, and build your analysis on top.

Actor reference and full input docs: apify.com/freshactors/redfin-scraper.

Happy scraping. If Redfin changes something, open an issue on the actor — that's what keeps it fresh.

How to scrape Google Play data with Node.js (no API key needed)

Freshactors — Mon, 01 Jun 2026 19:31:35 +0000

Google Play has no official public API for app listings or reviews. So if you want app details, ratings, the ratings histogram, or customer reviews as clean JSON — for ASO tracking, review monitoring, an ML dataset, or market research — you either parse Google's deeply-nested, undocumented page payloads yourself, or you call something that already does.

This tutorial does the second thing. We'll pull Google Play data with Node.js in a few lines, without an API key, without proxies, and without writing a single HTML parser, by calling a hosted scraper actor on Apify. At the end I'll show the same thing for one-off runs you don't even need code for.

Why not just `fetch()` the Play Store?

You can fetch('https://play.google.com/store/apps/details?id=com.spotify.music') — but the useful data isn't in clean HTML. Google embeds it in AF_initDataCallback blocks as positional arrays with no field names, e.g. the exact rating lives at roughly data[1][2][51][0][1] and the ratings histogram at data[1][2][51][1]. Reviews come from a separate batchexecute RPC that returns a )]}'-prefixed blob and throttles aggressively. Those index paths also shift when Google reshapes the page, which is why hand-rolled scrapers quietly break.

Offloading the parsing + rate-limit handling to a maintained actor means you get stable, named fields like rating, ratingHistogram, installs, and developerResponse instead.

Setup

You'll need a free Apify account and your API token (Settings → Integrations). Then install the official client:

npm install apify-client

Set your token as an env var so it isn't hard-coded:

export APIFY_TOKEN="your_token_here"

1. Scrape app details

Here we fetch full details for two apps. The actor's input takes a mode, an array of appIds (package names — the id= part of a Play Store URL), and an optional country.

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('freshactors/google-play-scraper').call({
  mode: 'details',
  appIds: ['com.spotify.music', 'com.whatsapp'],
  country: 'us',
  lang: 'en',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

for (const app of items) {
  console.log(`${app.title} — ${app.rating}★ (${app.ratingCount.toLocaleString()} ratings), ${app.installs}`);
}

Each item is a clean object. An abridged record looks like this:

{
  "_type": "app_details",
  "appId": "com.spotify.music",
  "title": "Spotify: Music and Podcasts",
  "developer": "Spotify AB",
  "rating": 4.33,
  "ratingCount": 35677435,
  "ratingHistogram": { "1": 3695081, "2": 1079953, "3": 1332269, "4": 3058122, "5": 26511993 },
  "installs": "1,000,000,000+",
  "free": true,
  "currency": "USD",
  "genre": "Music & Audio",
  "contentRating": "Teen",
  "containsAds": true,
  "updated": "2026-05-26T14:23:49.000Z",
  "url": "https://play.google.com/store/apps/details?id=com.spotify.music"
}

Note the ratingHistogram — the per-star breakdown that's painful to extract by hand but is exactly what you need to compute, say, the share of 1★ reviews over time.

2. Scrape reviews (with pagination handled for you)

Switch mode to reviews and set maxReviewsPerApp. The actor paginates the reviews RPC, de-duplicates, and retries on throttling under the hood — you just say how many you want and how to sort them.

const reviewsRun = await client.actor('freshactors/google-play-scraper').call({
  mode: 'reviews',
  appIds: ['com.spotify.music'],
  maxReviewsPerApp: 500,
  reviewsSort: 'newest', // or 'mostHelpful' | 'rating'
  country: 'us',
});

const { items: reviews } = await client.dataset(reviewsRun.defaultDatasetId).listItems();

// Quick sentiment-ish slice: how many recent reviews are 1–2 stars?
const negative = reviews.filter((r) => r.rating <= 2);
console.log(`${negative.length}/${reviews.length} recent reviews are 1–2★`);

console.log(reviews[0]);

A review record:

{
  "_type": "review",
  "appId": "com.spotify.music",
  "reviewId": "7e1815f2-36d1-4187-a936-12f969747892",
  "userName": "Catherine Hempel",
  "rating": 5,
  "body": "The paid version is excellent but very expensive...",
  "thumbsUp": 14,
  "date": "2026-04-21T11:35:57.000Z",
  "appVersion": "9.1.40.1486",
  "developerResponse": null
}

The appVersion and developerResponse fields are handy: you can track whether complaints cluster around a specific release, and whether the developer actually replied.

3. Search the Play Store by keyword

If you don't know the package names yet — say you're researching a niche — use search mode. It searches Google Play for each term and returns full details for every matching app, tagged with the _searchTerm that surfaced it.

const searchRun = await client.actor('freshactors/google-play-scraper').call({
  mode: 'search',
  searchTerms: ['habit tracker', 'budget app'],
  maxSearchResults: 20,
  country: 'us',
});

const { items: found } = await client.dataset(searchRun.defaultDatasetId).listItems();

// Rank a niche by rating
found
  .sort((a, b) => (b.rating ?? 0) - (a.rating ?? 0))
  .slice(0, 5)
  .forEach((a) => console.log(`[${a._searchTerm}] ${a.title} — ${a.rating}★`));

That's an entire competitive map of a category — names, developers, ratings, install buckets, whether they're ad-supported — in one call.

Streaming large datasets

For big review pulls, don't load everything into memory. Iterate the dataset:

const dataset = client.dataset(reviewsRun.defaultDatasetId);
let offset = 0;
const limit = 1000;

while (true) {
  const { items } = await dataset.listItems({ offset, limit });
  if (items.length === 0) break;
  // process this page (write to DB, file, etc.)
  offset += items.length;
}

What does this cost?

Pricing is pay-per-event, so you pay for data returned, not server time:

App details: $0.002 per app → 100 apps = $0.20.
Reviews: $0.0001 per review → a 1,000-review pull = $0.10.

No subscription, no proxy bill. A daily competitor-tracking job over a few dozen apps costs cents.

No code? Run it from the UI

You don't strictly need the client. On the actor's page you can paste the same JSON input, click Start, and export the dataset as JSON, CSV, or Excel — or hit the dataset via its REST URL. The Node snippets above are just the programmable version of that.

Wrapping up

With one hosted actor you can pull Google Play app details, the ratings histogram, paginated reviews, and keyword search results as named JSON fields — no API key, no proxy setup, no positional-array archaeology. The actor is monitored by a daily canary and patched when Google changes its layout, so the pipeline you build today keeps returning data next month.

If you build something with it, or hit a package that behaves oddly, the Issues tab is the fastest way to reach a human.

How to scrape Greenhouse & Lever ATS jobs data with Python (no API key needed)

Freshactors — Mon, 01 Jun 2026 19:30:47 +0000

If you need live job-posting data — for sales intelligence, a niche job board, or labor-market research — two ATS platforms cover a huge slice of the market: Greenhouse and Lever. The good news: both expose public JSON boards, so you don't need an API key, an account, or a headless browser.

The annoying part: Greenhouse and Lever return completely different JSON shapes. Scrape both yourself and you end up maintaining two parsers that break independently. In this tutorial we'll skip that tax by calling a ready-made actor that normalizes both ATS into one schema — and you'll run it from Python in a few lines.

The problem with rolling your own

Let's be concrete. A raw Greenhouse board lives at:

https://boards-api.greenhouse.io/v1/boards/{token}/jobs

A Lever board lives at:

https://api.lever.co/v0/postings/{token}?mode=json

Different hosts, different field names, different location/department conventions, and different ways of expressing "remote". You can write adapters for both — but then you own them forever, including the day one of them quietly changes a field name and your pipeline silently goes empty.

A cleaner path: hand a list of company tokens to an actor that already maps both into a single record, and just consume the output. Here's how with the Greenhouse & Lever Jobs Scraper.

Step 1 — Install the Apify client

pip install apify-client

Grab your Apify API token from Settings → Integrations in the Apify Console. We'll read it from an environment variable so it never lands in source control:

export APIFY_TOKEN="apify_api_xxx"

Step 2 — Run the actor with a list of companies

The actor takes a companies array of bare tokens (gitlab) or full board URLs (https://jobs.lever.co/spotify). With ats: "auto", bare tokens are tried against Greenhouse first, then Lever; URLs are detected automatically.

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "companies": [
        "gitlab",                              # bare token (auto-detected)
        "https://jobs.lever.co/spotify",       # Lever URL
        "https://boards.greenhouse.io/airbnb", # Greenhouse URL
    ],
    "ats": "auto",
    "includeDescription": True,
    "maxJobsPerCompany": 500,
}

# Blocks until the run finishes, then returns run metadata.
run = client.actor("freshactors/greenhouse-lever-jobs-scraper").call(run_input=run_input)

print("Run status:", run["status"])
print("Dataset id:", run["defaultDatasetId"])

.call() is synchronous — it waits for the run to complete and hands you the run object, including the defaultDatasetId where results land.

Step 3 — Read the normalized output

Every record — Greenhouse or Lever — comes back in the same shape. ATS-specific gaps are null, never missing keys, so your downstream code can rely on the schema:

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(
        f'[{item["_source"]:<10}] '
        f'{item["company"]:<10} '
        f'{item["title"]}  '
        f'({item.get("workplaceType") or "n/a"}, {item.get("location") or "n/a"})'
    )

A single record looks like this:

{
  "_type": "job",
  "_schemaVersion": "1.0",
  "_source": "lever",
  "company": "spotify",
  "jobId": "1ff4a4e3-...",
  "title": "Account Executive - Backstage",
  "department": "Operations and Business Support",
  "team": "Platform",
  "location": "Toronto",
  "allLocations": ["Toronto"],
  "workplaceType": "hybrid",
  "commitment": "Permanent",
  "country": "CA",
  "url": "https://jobs.lever.co/spotify/1ff4a4e3-...",
  "applyUrl": "https://jobs.lever.co/spotify/1ff4a4e3-.../apply",
  "postedAt": "2026-03-12T17:10:21.350Z",
  "updatedAt": null,
  "descriptionText": "About the role...",
  "_scrapedAt": "2026-06-01T09:14:02.118Z"
}

Because both ATS share this schema, you never branch on _source to read a field — you only read it if you want to know where the record came from.

Step 4 — A practical filter (remote roles, posted recently)

Say you only care about remote engineering roles. With one schema, the filter is trivial:

from datetime import datetime, timedelta, timezone

cutoff = datetime.now(timezone.utc) - timedelta(days=14)
remote_recent = []

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item.get("workplaceType") != "remote":
        continue
    posted = item.get("postedAt")
    if posted and datetime.fromisoformat(posted.replace("Z", "+00:00")) >= cutoff:
        remote_recent.append(item)

print(f"{len(remote_recent)} remote roles posted in the last 14 days")

No Greenhouse-vs-Lever special-casing — workplaceType and postedAt mean the same thing in every record.

Step 5 — Lighter, faster runs

Two knobs control cost and speed:

includeDescription: false skips fetching full descriptionText — much faster when you only need titles, departments, and locations for, say, a hiring-signal dashboard.
maxJobsPerCompany caps postings per company (1–5000) so a 3,000-role employer doesn't dominate your run.

run_input = {
    "companies": ["gitlab", "spotify", "netflix", "airbnb"],
    "ats": "auto",
    "includeDescription": False,   # metadata only
    "maxJobsPerCompany": 200,
}

Prefer Node.js?

Same actor, same input, the JavaScript client:

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('freshactors/greenhouse-lever-jobs-scraper').call({
    companies: ['gitlab', 'https://jobs.lever.co/spotify'],
    ats: 'auto',
    includeDescription: true,
    maxJobsPerCompany: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const job of items) {
    console.log(`[${job._source}] ${job.company} — ${job.title}`);
}

What about cost?

Why use the actor instead of hitting the boards directly?

You can curl those public endpoints yourself. The reason to use the actor is maintenance: it normalizes both ATS into one schema, auto-detects Greenhouse vs Lever, isolates per-company failures, and is monitored by a daily canary so a silent ATS field change doesn't quietly empty your pipeline. That operational reliability is the whole point.

If you want to skip the two-parser tax, the actor is here: Greenhouse & Lever Jobs Scraper on Apify. Run it on a schedule, point it at your target companies, and consume one clean JSON feed.

Happy scraping.

How to scrape Shopify App Store data with Python (no API key needed)

Freshactors — Mon, 01 Jun 2026 19:28:30 +0000

If you build for the Shopify ecosystem — or just research it — you eventually want the App Store as data: app ratings, review counts, pricing, and the actual review text. The problem is there's no official public Shopify App Store API, and the listing pages are a moving target. Roll your own scraper and you'll spend more time chasing layout changes than analyzing data.

In this tutorial we'll pull Shopify App Store data three ways — app details, reviews, and catalog discovery — using a hosted actor and a few lines of Python. No API key for Shopify, no login, no headless-browser babysitting.

What we'll build

Fetch full details for a list of apps (rating, review count, pricing, developer).
Pull reviews for an app, including the reviewing store's country and the developer's reply.
Discover apps across the whole catalog by keyword — handy for competitor research.

We'll call the Shopify App Store Scraper actor on Apify via the official apify-client. The actor handles pagination, retries, and the brittle parts; you just read JSON.

Prerequisites

Python 3.8+
A free Apify account (for the API token)
The Apify client:

pip install apify-client

Grab your API token from Settings → Integrations in the Apify console. Keep it out of source control — read it from an environment variable.

export APIFY_TOKEN="apify_api_xxx"

1. Get app details

The actor's input is a small JSON object. Set mode to details and pass one or more appHandles — the handle is the slug at the end of an app URL (https://apps.shopify.com/klaviyo-email-marketing → klaviyo-email-marketing).

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "mode": "details",
    "appHandles": [
        "klaviyo-email-marketing",
        "judgeme-product-reviews",
    ],
}

# Run the actor and wait for it to finish
run = client.actor("freshactors/shopify-app-store-scraper").call(run_input=run_input)

# Read the results from the run's dataset
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"{item['title']:40}  ⭐ {item['rating']}  ({item['reviewCount']} reviews)  — {item['pricingSummary']}")

Each record looks like this:

{
  "_type": "app_details",
  "_schemaVersion": "1.0",
  "handle": "klaviyo-email-marketing",
  "url": "https://apps.shopify.com/klaviyo-email-marketing",
  "title": "Klaviyo: Email Marketing & SMS",
  "developer": "Klaviyo",
  "category": "DeveloperApplication",
  "rating": 4.6,
  "reviewCount": 2805,
  "pricingSummary": "Free to install",
  "builtForShopify": false,
  "icon": "https://cdn.shopify.com/app-store/listing_images/.../icon.png",
  "_scrapedAt": "2026-06-01T08:00:00.000Z"
}

The details come from the page's application/ld+json structured-data block, which is far more stable than scraping CSS classes — so this keeps working even when Shopify reshuffles the markup.

2. Pull reviews (with country and developer replies)

Switch mode to reviews. You can cap volume with maxReviewsPerApp and order results with reviewsSort (newest, highest_rating, or lowest_rating).

run_input = {
    "mode": "reviews",
    "appHandles": ["klaviyo-email-marketing"],
    "maxReviewsPerApp": 500,
    "reviewsSort": "lowest_rating",  # surface complaints first
}

run = client.actor("freshactors/shopify-app-store-scraper").call(run_input=run_input)

reviews = list(client.dataset(run["defaultDatasetId"]).iterate_items())
print(f"Fetched {len(reviews)} reviews")

# Quick complaint mining: show 1-star reviews and where they came from
for r in reviews:
    if r["rating"] <= 2:
        print(f"[{r['rating']}★ {r['country']}] {r['storeName']}: {r['body'][:120]}")

A review record carries the full text plus useful context:

{
  "_type": "review",
  "_schemaVersion": "1.0",
  "handle": "klaviyo-email-marketing",
  "reviewId": "2186287",
  "rating": 5,
  "storeName": "Bellaforma Jewelry",
  "country": "United States",
  "usedAppFor": "Over 1 year using the app",
  "date": "May 1, 2026",
  "body": "We've seen significant revenue growth through dialed-in flows...",
  "developerReply": null,
  "_scrapedAt": "2026-06-01T08:00:00.000Z"
}

Because the actor treats an empty-but-200 review page as a soft failure and retries with backoff, you won't get a misleading "0 reviews" when a request just hiccuped.

3. Discover apps across the catalog

Doing competitor research? discover mode walks Shopify's official catalog sitemap and returns full details for apps matching a keyword. Use maxApps to cap how many to enrich.

import pandas as pd

run_input = {
    "mode": "discover",
    "query": "email marketing",
    "maxApps": 50,
}

run = client.actor("freshactors/shopify-app-store-scraper").call(run_input=run_input)
apps = list(client.dataset(run["defaultDatasetId"]).iterate_items())

# Build a competitive landscape table, sorted by rating
df = pd.DataFrame(apps)[["title", "developer", "rating", "reviewCount", "pricingSummary"]]
df = df.sort_values("reviewCount", ascending=False)
print(df.head(10).to_string(index=False))

In a few seconds you have a ranked table of email-marketing apps — rating, review count, developer, and pricing — ready for a spreadsheet or a notebook. Leave query empty to enumerate the catalog from the top.

Tip: don't block on the run

.call() waits for the run to finish, which is fine for small jobs. For big pulls, start the run and stream results as they land:

# Fire and forget; poll later
started = client.actor("freshactors/shopify-app-store-scraper").start(run_input=run_input)
print("Run id:", started["id"])

You can also export the dataset directly to CSV/Excel/JSON from the Apify console, or schedule recurring runs (e.g. a daily review pull) with Apify Schedules.

What about cost?

The actor is pay-per-result, so a quick estimate: app details are $0.002 each, discovery results $0.001 each, and reviews $0.0001 each. Pulling details for 100 apps is about $0.20; 20,000 reviews is about $2.00. (Apify platform usage is billed separately.)

Wrapping up

With one hosted actor and the apify-client, you can turn the Shopify App Store into structured JSON — details, reviews, or a whole-catalog competitor map — without an API key or a brittle homegrown scraper. The actor is monitored by a daily canary, so when Shopify changes its pages it gets patched fast instead of silently dying on you.

If you want to try it, the actor lives here: Shopify App Store Scraper on Apify. Have a Shopify-data question or hit an edge case? Drop it in the comments — happy to help.

DEV Community: Freshactors

How to scrape Microsoft Store app data in Python — ratings, reviews with helpful votes, any country (no API key)

The endpoints

Step 1 — Install the Apify client

Step 2 — App details with rating windows

Step 3 — Search the catalog

Step 4 — Reviews with helpful votes

Pricing & reliability

How to scrape VS Code Marketplace extension stats in Python — installs, trending, reviews (no API key)

The endpoint

Step 1 — Install the Apify client

Step 2 — Get full stats for specific extensions

Step 3 — Map a category by install count

Step 4 — Reviews

Pricing & reliability

How to scrape Personio career-portal jobs in Python — no API key

The endpoint

Step 1 — Install the Apify client

Step 2 — Run the actor with a list of tenants

Step 3 — Read the normalized output (departments + seniority included)

Step 4 — Localized or lighter output

Prefer Node.js?

What about cost?

Why use the actor instead of the feed directly?

How to scrape Teamtailor career-site jobs in Python — no API key

The endpoint

Step 1 — Install the Apify client

Step 2 — Run the actor with a list of companies

Step 3 — Read the normalized output

Step 4 — Lighter output

Prefer Node.js?

What about cost?

Why use the actor instead of the feed directly?

How to scrape SmartRecruiters jobs data with Python (no API key needed)

Why not just hit the API yourself?

Step 1 — Install the Apify client

Step 2 — Run the actor with a list of companies

Step 3 — Read the normalized output

Step 4 — Faster, lighter runs

Prefer Node.js?

What about cost?

Why use the actor instead of the API directly?

How to scrape Recruitee jobs data (with salary) in Python — no API key

The endpoint

Step 1 — Install the Apify client

Step 2 — Run the actor with a list of companies

Step 3 — Read the normalized output (including salary)

Step 4 — Lighter output

Prefer Node.js?

What about cost?

Why use the actor instead of the API directly?

How to scrape Workable jobs data with Python (no API key needed)

Why not just fetch() the board yourself?

Step 1 — Install the Apify client

Step 2 — Run the actor with a list of companies

Step 3 — Read the normalized output

Step 4 — A practical filter (remote roles, posted recently)

Step 5 — Lighter, faster runs

Prefer Node.js?

What about cost?

Why use the actor instead of hitting the board directly?

How to scrape Apple App Store data with Python (no API key needed)

What we'll build

Prerequisites

1. Scrape app details

2. Search the App Store by keyword

3. Scrape App Store reviews (the throttle-proof part)

A note on reliability

What does it cost?

Doing it in Node instead

Wrap-up

How to scrape Redfin (US real estate) data with Node.js (no API key needed)

Why not just fetch() Redfin yourself?

Prerequisites

Step 1 — Call the actor

Step 2 — What the output looks like

Step 3 — Do something useful with it

Step 4 — Get sold comps (the "Redfin API" alternative people actually want)

Step 5 — Scale it up (or down)

A note on reliability

Why not just `fetch()` the board yourself?

Why not just `fetch()` Redfin yourself?

Why not just `fetch()` the Play Store?