DEV Community: Factden

How to Scrape Hotels.com Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 08:04:09 +0000

Hotels.com is part of Expedia Group, which is good news and a small headache. Good news: its reviews share the same backend as Expedia, Travelocity, Orbitz, and the rest, so one approach covers all of them. The headache: Hotels.com still uses its old /ho<id>/ URL format, which has to be resolved to the global property before you can pull anything.

This guide covers what you can pull from a Hotels.com review, why the DIY route is fiddly, working Python, a no-code shortcut, and a plain comparison. If you want the deeper reference, there is a full guide to scraping Hotels.com reviews too.

What you can pull from a review

Per review you can get:

Overall rating on the /10 scale plus a label, and category sub-ratings: Cleanliness, Service, Room comfort, Hotel condition, Amenities, Eco-friendliness
The full review text, detected language, and a translation flag
Stay context: check-in and check-out dates, travel companions, traveler category
Reviewer detail, verified flag, helpful votes, review photos
Owner responses, and a brandType field
An LLM-ready markdown block per review

Same clean, structured shape whether the review came from Hotels.com or any sibling brand.

Why scraping Hotels.com is hard

No public reviews API and no API key on offer. Scraping the public pages is the route.
The /ho<id>/ legacy id. A Hotels.com URL like hotels.com/ho119566/ does not carry the global property id directly; it has to be resolved first, or your request goes nowhere.
Reviews load dynamically, so a plain request returns a JavaScript shell, not the data.
Bot protection and rate limits on top.

So the work is resolving the legacy id, hitting the review endpoints, and staying unblocked, on repeat.

Three ways to get the data

	DIY Python	Hotels.com Reviews Scraper (actor)	Official API
Setup time	Hours to days	~30 seconds	Not available (no API key)
`/ho<id>/` resolution	You build it	Automatic	n/a
/10 ratings + 6 sub-ratings	Parse nested markup	Yes	n/a
Owner responses	Extra parsing	Yes	n/a
Cost	Proxies + eng time	Pay-per-result	n/a
Best for	One-off	Scheduled, at scale	Not an option

Option A: DIY in Python

A plain request to a Hotels.com page returns a shell, and the /ho<id>/ id is not the one the review endpoint expects:

import httpx

url = "https://www.hotels.com/ho119566/"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print("reviews" in r.text)   # usually False; ho-id must be resolved, reviews load separately

To get the data you resolve the /ho<id>/ to the global property id, call the review endpoints, and normalize the response. Fine for one hotel, several moving parts for a portfolio.

Option B: the no-code / API shortcut

When you want clean rows, the Hotels.com Reviews Scraper on Apify resolves the /ho<id>/ id for you and returns structured JSON. No login, no API key, no proxy setup.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/hotels-com-reviews-scraper").call(run_input={
    "hotelUrls": [
        "https://www.hotels.com/ho119566/",
        "242128"
    ],
    "maxReviewsPerHotel": 200,
    "sortBy": "newest",
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["overallRating"], review["reviewText"][:60])

Pass Hotels.com /ho<id>/ URLs or bare ids, and any other Expedia Group brand URL works too. sortBy takes newest, oldest, highestRating, or lowestRating; bound results with fromDate / toDate and minRating / maxRating.

What comes back: 27 fields per review

Group	Fields
Core	`reviewId`, `hotelId`, `hotelName`, `hotelUrl`, `source`, `brandType`, `submittedAt`
Rating	`overallRating` (/10), `ratingLabel`, `subRatings` (cleanliness, service, room comfort, condition, amenities, eco)
Text	`reviewText`, `language`, `isMachineTranslated`
Stay	`checkInDate`, `checkOutDate`, `travelCompanions`, `travelerCategories`, `roomTypeId`
Reviewer	`reviewerName`, `reviewerLocation`, `verified`, `isAnonymous`
Signals	`helpfulVotes`, `imagesCount`, `reviewPhotos`, `ownerResponse`
AI-ready	`markdownContent`

A trimmed sample row:

{
  "brandType": "Hotels.com",
  "hotelName": "The Venetian Resort",
  "overallRating": 9,
  "ratingLabel": "Wonderful",
  "subRatings": ["Cleanliness: 10", "Service: 9", "Hotel condition: 9", "Amenities: 10"],
  "reviewText": "Great suite, easy check-in, would stay again.",
  "travelerCategories": ["Couple"],
  "checkInDate": "2026-04-18",
  "checkOutDate": "2026-04-21",
  "language": "en",
  "helpfulVotes": 2
}

Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free Hotels.com sample (CSV/JSON) here: factden.com/sample-hotels. Load it into pandas and the /10 ratings and sub-ratings are ready to plot.

FAQ

Is scraping Hotels.com legal? The reviews are publicly visible. As with any scraping, check Hotels.com's Terms of Service and your local rules, and use the data responsibly.

Is there a Hotels.com API? No public reviews API and no key on offer. Scraping the public pages is the route.

What is the /ho<id>/ thing? Hotels.com uses a legacy property id in its URLs that does not match the global Expedia Group id. It has to be resolved first, which the actor does automatically.

Can I scrape Expedia and the other brands too? Yes. Hotels.com shares the Expedia Group backend, so the same actor handles Expedia, Travelocity, Orbitz, Wotif, CheapTickets, and ebookers, same schema, tagged by brandType.

How do I stop getting blocked? Residential proxies and slow pacing, or a tool that resolves the id and reads the review endpoints directly. Plain requests get a JavaScript shell.

Doing B2B software instead of hotels? See how to scrape G2 reviews.
Other FactDen scrapers: Expedia hotel reviews and Trip.com and Ctrip hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape Expedia Hotel Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 08:02:08 +0000

Here is the thing most people miss about scraping Expedia: Expedia, Hotels.com, Travelocity, Orbitz, Wotif, CheapTickets, and ebookers all run on the same backend. The same hotel shows up across all seven brands, and their reviews pool together. If you scrape only expedia.com, you leave most of the data on the table.

This guide covers what you can pull from an Expedia Group review, why the DIY route is fiddly, working Python, a no-code shortcut, and a plain comparison. If you want the deeper reference, there is a full guide to scraping Expedia reviews too.

What you can pull from a review

Per review you can get:

Overall rating on the /10 scale plus a label, and category sub-ratings: Cleanliness, Service, Room comfort, Hotel condition, Amenities, Eco-friendliness
The full review text, detected language, and a translation flag
Stay context: check-in and check-out dates, travel companions, traveler category
Reviewer detail, verified flag, helpful votes, review photos
Owner responses, and a brandType telling you which of the seven brands the review came from
An LLM-ready markdown block per review

The brandType field plus the shared backend are what let you pull every brand's reviews for a property in one pass.

Why scraping Expedia is hard

There is no public reviews API. Expedia Group does not offer one, so scraping the public pages is the route.
Seven brands, different URL formats. Six brands embed the global property id in a .h<id>. path; Hotels.com uses a legacy /ho<id>/ id that has to be resolved. A scraper built for one brand misses the rest.
Reviews are loaded dynamically, not sitting in the initial HTML, so a plain request gets you a shell.
Bot protection and rate limits on top.

So the work is resolving ids across brands, hitting the review endpoints, and staying unblocked, on repeat.

Three ways to get the data

	DIY Python	Expedia Reviews Scraper (actor)	Official API
Setup time	Hours to days	~30 seconds	Not available (no public reviews API)
All 7 Expedia Group brands	Build a scraper per brand	One run	n/a
/10 ratings + 6 sub-ratings	Parse nested markup	Yes	n/a
Owner responses	Extra parsing	Yes	n/a
Cost	Proxies + eng time	Pay-per-result	n/a
Best for	One-off	Scheduled, at scale	Not an option

Option A: DIY in Python

A plain request to a hotel page returns a JavaScript shell, not the reviews:

import httpx

url = "https://www.expedia.com/Las-Vegas-Hotels-Bellagio.h140596.Hotel-Information"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print("reviews" in r.text)   # usually False; reviews load via internal endpoints

To get the data you reverse-engineer the review endpoints, map each brand's URL format to the global property id (including the Hotels.com /ho<id>/ resolution), and normalize the responses. Fine for one hotel, a lot of moving parts for a portfolio.

Option B: the no-code / API shortcut

When you want clean rows, the Expedia Reviews Scraper on Apify takes any Expedia Group hotel URL (or a bare property id), resolves it, and returns structured JSON. No login, no proxy setup.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/expedia-hotel-reviews-scraper").call(run_input={
    "hotelUrls": [
        "https://www.expedia.com/Las-Vegas-Hotels-Bellagio.h140596.Hotel-Information",
        "140596",
        "https://www.hotels.com/ho115902/"
    ],
    "maxReviewsPerHotel": 200,
    "sortBy": "newest",
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["brandType"], review["overallRating"], review["reviewText"][:60])

Mix brands and bare ids freely. sortBy takes newest, oldest, highestRating, or lowestRating; bound results with fromDate / toDate and minRating / maxRating; and use reviewSources to keep only certain brands' reviews.

What comes back: 27 fields per review

Group	Fields
Core	`reviewId`, `hotelId`, `hotelName`, `hotelUrl`, `source`, `brandType`, `submittedAt`
Rating	`overallRating` (/10), `ratingLabel`, `subRatings` (cleanliness, service, room comfort, condition, amenities, eco)
Text	`reviewText`, `language`, `isMachineTranslated`
Stay	`checkInDate`, `checkOutDate`, `travelCompanions`, `travelerCategories`, `roomTypeId`
Reviewer	`reviewerName`, `reviewerLocation`, `verified`, `isAnonymous`
Signals	`helpfulVotes`, `imagesCount`, `reviewPhotos`, `ownerResponse`
AI-ready	`markdownContent`

A trimmed sample row:

{
  "brandType": "Expedia",
  "hotelName": "Bellagio",
  "overallRating": 10,
  "ratingLabel": "Exceptional",
  "subRatings": ["Cleanliness: 10", "Service: 10", "Hotel condition: 10", "Amenities: 10"],
  "reviewText": "Rooms were great. Buffet was awesome. Nice casino.",
  "travelerCategories": ["Friends"],
  "checkInDate": "2026-05-01",
  "checkOutDate": "2026-05-04",
  "language": "en",
  "helpfulVotes": 3
}

Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free Expedia sample (CSV/JSON) here: factden.com/sample-expedia. Load it into pandas and the /10 ratings and sub-ratings are ready to plot.

FAQ

Is scraping Expedia legal? The reviews are publicly visible. As with any scraping, check Expedia's Terms of Service and your local rules, and use the data responsibly.

Is there an official Expedia reviews API? No public one. Scraping the public pages is the route for most teams.

Can I scrape Hotels.com and the other brands too? Yes. All seven Expedia Group brands share the backend, so one actor covers Expedia, Hotels.com, Travelocity, Orbitz, Wotif, CheapTickets, and ebookers, with the same schema tagged by brandType.

Are ratings on the 5-star or 10-point scale? Expedia Group uses /10, and that is what the actor returns for overallRating and the sub-ratings.

How do I stop getting blocked? Residential proxies and slow pacing, or a tool that reads the review endpoints directly. Plain requests get a JavaScript shell, not the data.

Doing B2B software instead of hotels? See how to scrape G2 reviews.
Other FactDen scrapers: Hotels.com reviews and Trip.com and Ctrip hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape Trip.com and Ctrip Hotel Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 08:00:07 +0000

Most "Trip.com scrapers" only see half the picture. Trip.com is the international brand, but the same hotel usually has a much larger pool of reviews on its Chinese-domestic sibling, hotels.ctrip.com (携程). If you want honest guest sentiment for a property in Asia, you need both, and you need the Chinese ones translated.

This guide covers what you can pull from a Trip.com or Ctrip review, why the DIY route is a two-headed problem, working Python, a no-code shortcut, and a plain comparison. If you want the deeper reference, there is a full guide to scraping Trip.com and Ctrip reviews too.

What you can pull from a review

Per review you can get:

Overall rating and a label ("Outstanding"), plus sub-ratings: Cleanliness, Location, Service, Facilities
The review text, plus an English translation of Chinese reviews (reviewTextTranslated), and the detected language
Trip context: travel type (family, couple, business, solo), room name, check-in month
Reviewer detail: tier ("Review star"), lifetime review count, and for Ctrip the reviewer's Chinese province
Owner responses (text and date), useful counts, photo/video flags
An LLM-ready markdown block per review

The two things that separate a real dataset from a shallow one: the Chinese-domestic feed and its translation.

Why scraping Trip.com and Ctrip is hard

There is no public reviews API. Trip.com Group does not offer one, so scraping the public pages is the only route.
It is two systems, not one. trip.com (international) and hotels.ctrip.com / 携程 (Chinese) have different structures and languages. A scraper built for one usually misses the other, and the Chinese pool is often the larger one.
Chinese reviews need translation to be usable in an English pipeline, which is a whole extra step.
Bot protection and rate limits apply on both.

So the work is not parsing one page. It is handling both locales, translating, and staying unblocked, on repeat.

Three ways to get the data

	DIY Python	Trip.com & Ctrip actor	Official API
Setup time	Hours to days	~30 seconds	Not available (no public reviews API)
Both Trip.com + Ctrip feeds	Build two scrapers	One run	n/a
Chinese review translation	Add a translation step	Built in	n/a
Sub-ratings + owner responses	Parse nested markup	Yes	n/a
Cost	Proxies + eng time	Pay-per-result	n/a
Best for	One-off	Scheduled, at scale	Not an option

Option A: DIY in Python

A plain request to a hotel page tends to come back blocked or JavaScript-only:

import httpx

url = "https://www.trip.com/hotels/macau-hotel-detail-344983/galaxy-hotel/"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print(r.status_code)   # challenge / JS-rendered shell, reviews not in the HTML

Reviews load through internal endpoints, so you end up reverse-engineering those (per locale), then normalizing two different response shapes, then translating the Chinese text. Doable for one hotel, painful for a portfolio on a schedule.

Option B: the no-code / API shortcut

When you want clean rows, the Trip.com & Ctrip Reviews Scraper on Apify reads both feeds in a single run and hands back structured JSON with the Chinese reviews already translated. No login, no proxy setup.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/ctrip-trip-reviews-scraper").call(run_input={
    "startUrls": [
        "https://www.trip.com/hotels/macau-hotel-detail-344983/galaxy-hotel/",
        "https://hotels.ctrip.com/hotels/1286148.html"
    ],
    "maxReviewsPerHotel": 200,
    "sortBy": "mostRecent",
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["source"], review["overallRating"], review["travelType"])

Mix trip.com and hotels.ctrip.com URLs freely. sortBy takes mostRelevant, mostRecent, ratingHighToLow, or ratingLowToHigh, and you can bound results with fromDate / toDate and minRating / maxRating.

What comes back: 23 fields per review

Group	Fields
Core	`reviewId`, `hotelId`, `hotelName`, `hotelUrl`, `source` (trip.com / ctrip), `submittedAt`
Rating	`overallRating`, `ratingLabel`, `subRatings` (Cleanliness, Location, Service, Facilities)
Text	`reviewText`, `reviewTextTranslated`, `isMachineTranslated`, `language`
Reviewer	`reviewer` (name, tier, lifetime reviews, Chinese province for Ctrip)
Trip context	`travelType`, `roomName`, `checkInMonth`
Signals	`recommends`, `usefulCount`, `imagesCount`, `hasVideo`, `ownerResponse` (text, date)
AI-ready	`markdownContent`

A trimmed sample row (a Chinese review, translated):

{
  "source": "ctrip",
  "hotelName": "Galaxy Hotel",
  "overallRating": 5,
  "ratingLabel": "Outstanding",
  "subRatings": ["Cleanliness: 5.0", "Location: 5.0", "Service: 5.0", "Facilities: 5.0"],
  "language": "zh",
  "reviewText": "酒店很棒，服务一流，位置方便。",
  "reviewTextTranslated": "Great hotel, first-class service, convenient location.",
  "isMachineTranslated": true,
  "travelType": "Family",
  "reviewer": {"tier": "Rising review star", "lifetimeReviews": 4},
  "ownerResponse": {"text": "感謝您的正面評價...", "date": "2026-06-15"}
}

Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free Trip.com/Ctrip sample (CSV/JSON) here: factden.com/sample-trip. It includes both English and translated-Chinese reviews so you can see the shape.

FAQ

Is scraping Trip.com or Ctrip legal? The reviews are publicly visible. As with any scraping, check the sites' Terms of Service and your local rules, and use the data responsibly.

Is there an official Trip.com / Ctrip reviews API? No public one. Scraping the public pages is the only route for most teams.

Do I get the Chinese reviews in English? Yes. Ctrip reviews come with reviewTextTranslated and an isMachineTranslated flag, alongside the original.

Can I scrape both Trip.com and Ctrip at once? Yes, that is the point. Mix both URL types in one run and the output uses the same schema, tagged by source.

How do I stop getting blocked? Residential proxies and slow pacing, or a tool that reads the internal endpoints per locale. Plain requests will get challenged.

Doing B2B software instead of hotels? See how to scrape G2 reviews.
Other FactDen scrapers: Expedia hotel reviews and Indeed jobs and salary data.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape Indeed Jobs in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 07:56:17 +0000

Scraping Indeed sounds simple until you try it. There is no public API anymore, the site sits behind Cloudflare, and the salary you actually want arrives as a messy string like "$120,000 - $160,000 a year". This guide covers what you can pull from an Indeed job, why the DIY route is harder than it looks, working Python, a no-code shortcut, and a straight comparison so you can pick a lane. If you want the deeper reference, there is a full guide to scraping Indeed jobs too.

What you can pull from an Indeed job

Per listing you can get:

Title, company, and location (plus city, state, country code, and lat/long)
Salary, parsed into numbers: salaryMin, salaryMax, salaryPeriod, currency, not just the raw string
Job type, remote/hybrid flag, and the full description text
Benefits, date posted, easy-apply and urgent-hire flags
Company data on demand: rating, review count, industry, size, revenue, founded, website, and socials

Parsed salary and company enrichment are the two things people actually come for, and the two things a naive scrape gets wrong.

Why scraping Indeed is hard

Three reasons DIY is more work than it looks:

There is no official API. Indeed retired its public job-search API to new developers years ago, so there is no supported programmatic route. Scraping is the only option left.
Cloudflare and rate limits. A plain request to a search results page tends to return a 403 or a challenge. Hammer it and you get blocked fast.
Messy salary strings. Indeed shows salary as free text. Turning "$120,000 - $160,000 a year" into clean salaryMin/salaryMax/period reliably, across currencies and 40 countries, is fiddly.

So the work is not the parsing of one page. It is staying unblocked, normalizing salary, and pulling company data from separate pages, on repeat.

Three ways to get Indeed job data

	DIY Python	Indeed Jobs Scraper (actor)	Indeed Official API
Setup time	Hours to days	~30 seconds	Not available (retired)
Anti-bot handled	You (Cloudflare, blocks)	Built in	n/a
Salary as numbers	You parse the string	Yes: `salaryMin` / `salaryMax`	n/a
Company data	Separate scrape	Free, one toggle	n/a
Countries	You handle localization	40+	n/a
Cost	Proxies + eng time	Pay-per-result	n/a
Best for	One-off search	Scheduled, at scale	Not an option

The official-API column is the real story: since Indeed closed its API, a scraper (yours or a ready-made one) is the only way to get this data programmatically.

Option A: DIY in Python

See the wall first:

import httpx

url = "https://www.indeed.com/jobs?q=software+engineer&l=New+York"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print(r.status_code)   # often 403 / Cloudflare challenge

If you do get HTML back, the job cards parse roughly like this (selectors change often, so re-check against the live page):

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, "html.parser")
for card in soup.select("div.job_seen_beacon"):
    title = card.select_one("h2.jobTitle")
    comp  = card.select_one("[data-testid='company-name']")
    sal   = card.select_one("[data-testid='attribute_snippet_testid']")
    print(
        title.get_text(strip=True) if title else None,
        comp.get_text(strip=True) if comp else None,
        sal.get_text(strip=True) if sal else None,   # raw string, still needs parsing
    )

Then you still have to normalize that salary string into numbers, follow each job to its full description, and scrape the company page separately. For one search that is fine. For a pipeline, the upkeep adds up.

Option B: the no-code / API shortcut

When you just want clean rows, the Indeed Jobs Scraper on Apify talks to Indeed's structured endpoints (no fragile HTML path), handles the blocking, and returns parsed JSON. No login, no proxy setup.

From Python:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/indeed-jobs-scraper").call(run_input={
    "query": "software engineer",
    "location": "New York, NY",
    "country": "US",
    "maxItems": 200,
    "salaryMin": 120000,     # native salary-range filter
    "scrapeCompany": True,   # attach company profiles for free
})
for job in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(job["title"], job["company"], job["salaryMin"], job["salaryMax"])

Set country to any of 40+ codes (US, GB, IN, DE, SG, ...), filter by remote, datePosted, jobType, or experienceLevel, or pass startUrls / jobKeys to fetch specific listings.

What comes back: 26 structured fields per job

Group	Fields
Core	`title`, `company`, `location`, `city`, `state`, `countryCode`, `url`, `jobKey`
Salary	`salary` (raw), `salaryMin`, `salaryMax`, `salaryPeriod`, `currency`
Job detail	`jobType`, `remote`, `occupations`, `benefits`, `description`, `datePosted`, `isUrgentHire`, `easyApply`
Geo	`latitude`, `longitude`
Company (with `scrapeCompany`)	`companyRating`, `companyReviewCount`, `companyPageUrl` + a full company profile (industry, size, revenue, founded, website, socials)

A trimmed sample row:

{
  "title": "Senior Software Engineer",
  "company": "Plaid",
  "location": "New York, NY",
  "salary": "$120,000 - $160,000 a year",
  "salaryMin": 120000,
  "salaryMax": 160000,
  "salaryPeriod": "year",
  "currency": "USD",
  "jobType": ["Full-time"],
  "remote": "remote",
  "benefits": ["Health insurance", "401(k)"],
  "datePosted": "2026-06-10",
  "companyRating": 4.2,
  "url": "https://www.indeed.com/viewjob?jk=6d50b3ebeb3fb122",
  "jobKey": "6d50b3ebeb3fb122"
}

The numeric salary fields and the on-demand company profile are what make this usable for labor-market analysis without a cleanup pass. Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free Indeed jobs sample (CSV/JSON) here: factden.com/sample-indeed. Drop it into pandas and the parsed salary columns are ready to plot.

FAQ

Is scraping Indeed legal? The listings are publicly visible. As with any scraping, check Indeed's Terms of Service and your local rules (especially around personal data), and use the data responsibly.

Does Indeed have an API? Not a usable one. Indeed closed its public job-search API to new developers, which is exactly why people scrape the public pages now.

How do I stop getting blocked? Residential proxies, real headers, and slow pacing, or a tool that talks to Indeed's structured endpoints instead of scraping HTML. Plain requests to the search pages will get challenged.

Can I get salary as numbers? Yes. salaryMin, salaryMax, salaryPeriod, and currency are parsed from the raw string, so you can filter and aggregate directly.

Can I get company data too? Yes. Turn on scrapeCompany and each job carries the company's rating, size, industry, revenue, and profile links, no second scrape needed.

Which countries are supported? 40+, including US, UK, Canada, India, Germany, Singapore, Australia, and more, via the country code.

Doing competitive intelligence instead of hiring data? See how to scrape G2 reviews.
Other FactDen scrapers: G2 software reviews and Trip.com and Ctrip hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape G2 Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 07:52:34 +0000

If you have tried to scrape G2 reviews with a quick requests.get(), you already know how it goes: a 403, a CAPTCHA, or a blank page. G2 is one of the tougher public sites to pull at scale. But the data behind it (ratings, structured pros and cons, and which competitor a reviewer switched away from and why) is worth the trouble if you do any competitive intelligence.

This guide covers what a G2 review actually contains, why the naive approach fails, a Python path that works, a no-code shortcut, and a plain comparison so you can pick the right route. If you want a deeper field reference, there is a longer guide to scraping G2 reviews too.

What a G2 review actually contains

One G2 review holds more than it looks. Per review you can pull:

Overall rating (1 to 5) plus six sub-ratings (ease of use, ease of setup, quality of support, meets requirements, and more)
Structured pros, cons, and "problems solved" as separate fields, not one text blob
Switching data: the competitor the reviewer came from, and why they left (the field most people are actually after)
Reviewer context: industry, role, company size, country
Dates, verification, and the incentivized flag

The sub-ratings and switching fields are the reason to scrape G2 instead of skimming it by hand.

Why scraping G2 is hard: DataDome

G2 runs DataDome bot protection behind a Cloudflare CDN. In practice:

A plain requests or httpx GET comes back 403 almost immediately.
Swapping the User-Agent does nothing. DataDome fingerprints TLS, headers, and behavior.
Datacenter IPs get flagged fast. You need residential proxies plus a real browser fingerprint, or a scraping API that brings both.

You can absolutely scrape G2 yourself. The hard part is doing it reliably, at scale, without babysitting proxies. That maintenance tax is bigger than the parsing.

Three ways to get G2 review data

	DIY Python	G2 Reviews Scraper (actor)	G2 Official API
Setup time	Hours to days	~30 seconds	Weeks (sales + contract)
Anti-bot handled	You (proxies + browser)	Built in	n/a
Structured fields	You parse them	27, incl. switching data	Limited
Sub-ratings + "switched from"	Hard to parse	Yes, resolved to product names	No
Cost	Proxies + eng time	$0.004/row, ~1,250 free	Enterprise contract
Best for	One-off, few pages	Scheduled, at scale	Large enterprises

Option A: DIY in Python

See the wall for yourself first:

import httpx

url = "https://www.g2.com/products/slack/reviews"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print(r.status_code)   # 403 (DataDome)

Getting past it needs a residential proxy and a browser-grade fingerprint (a stealth headless browser, or a scraping API). Once you have the HTML, parsing review cards with BeautifulSoup looks roughly like this:

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, "html.parser")
for card in soup.select("[itemprop='review']"):
    rating = card.select_one("[itemprop='ratingValue']")
    title  = card.select_one("[itemprop='name']")
    body   = card.select_one("[itemprop='reviewBody']")
    print(
        rating.get("content") if rating else None,
        title.get_text(strip=True) if title else None,
        (body.get_text(strip=True) if body else "")[:120],
    )

Two things that will cost you time:

Selectors drift. G2 changes its markup, so re-check against the live page.
Sub-ratings and switching data sit in nested markup that is fiddly to parse consistently.

DIY is fine for a one-off across a handful of pages. For hundreds of products on a schedule, the proxy and anti-bot upkeep usually costs more than it saves.

Option B: the no-code / API shortcut

When you just want clean rows, the G2 Reviews Scraper on Apify handles the DataDome, proxy, and parsing layer and hands back structured JSON. No login, no proxy setup.

From Python, using the Apify client:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/g2-reviews-scraper").call(run_input={
    "mode": "reviews",
    "startUrls": ["https://www.g2.com/products/slack/reviews", "notion"],
    "maxReviewsPerProduct": 200,
    "sortReviews": "helpful",
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["overallRating"], review["reviewTitle"], review.get("previousCompetitors"))

Pass full product URLs or bare slugs (notion, slack). There is also a Products mode that finds competitor products by keyword before you pull their reviews.

What comes back: 27 structured fields

Group	Fields
Ratings	`overallRating`, `subRatings` (ease of use, setup, support, meets requirements)
Structured text	`pros`, `cons`, `problemsSolved`, `recommendations`, `reviewText`
Switching / battlecard	`didSwitchFromCompetitor`, `previousCompetitors`, `whySwitched`
Reviewer	`reviewerIndustry`, `reviewerRole`, `companySize`, `reviewerCountry`, `reviewerName`
Meta	`isIncentivized`, `helpfulVotes`, `submittedAt`, `reviewUrl`, `productSlug`, `productName`
AI-ready	`markdownContent` (a self-contained markdown block per review, for RAG / vector DBs)

A trimmed sample row, so you can see the shape:

{
  "productName": "Slack",
  "overallRating": 5,
  "subRatings": {"easeOfUse": 6, "easeOfSetup": 6, "qualityOfSupport": 5, "meetsRequirements": 6},
  "reviewTitle": "Runs our whole company",
  "pros": "Channels keep every project in one place.",
  "cons": "Notifications get noisy at scale.",
  "didSwitchFromCompetitor": true,
  "previousCompetitors": ["Microsoft Teams"],
  "whySwitched": "Better threads and a faster mobile app.",
  "companySize": "Mid-Market",
  "reviewerRole": "IT Administrator",
  "isIncentivized": false
}

The two fields most scrapers drop, previousCompetitors + whySwitched (resolved to real product names) and the per-dimension subRatings, are what make this useful for battlecards. Full field list and copy-paste snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data before running anything? There is a free G2 review sample (CSV/JSON) here: factden.com/sample (also mirrored on HuggingFace and Kaggle). Load it into pandas and the sub-ratings and switching fields show up straight away.

FAQ

Is scraping G2 reviews legal? The reviews are publicly available. As with any scraping, check G2's Terms of Service and your local rules (GDPR and similar for personal data), and use the data responsibly.

Does G2 have an API? Yes, but it is enterprise-tier: a sales call, a contract, and procurement. For most teams, scraping the public pages, or using a ready-made actor, is the faster route to the same public data.

How do I stop getting blocked? Residential proxies, a real browser fingerprint, and human-like pacing, or a tool that bundles all three. Datacenter IPs and plain requests will not survive DataDome.

How much does it cost? DIY costs proxies plus your time. The actor is pay-per-result at $0.004 per row, with about 1,250 rows free on Apify's $5 new-account credit.

Can I get the "switched from" competitor data? Yes. That is the previousCompetitors and whySwitched fields, resolved to product names. It is the reason most people scrape G2 in the first place.

Can I filter out incentivized (gift-card) reviews? Yes. Every row carries an isIncentivized flag, so you can recompute a rating on organic reviews only. G2's own UI will not let you do that.

I mapped switching and sub-ratings across 25 tools in the 2026 SaaS Battlecard Atlas (50,000 reviews).
Other FactDen scrapers: Indeed jobs and salary data and Trip.com and Ctrip hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

Found this useful? A ⭐ on the GitHub repo (https://github.com/factden/g2-reviews-scraper) helps other people find it.

I scraped 50,000 G2 reviews to map the 2026 SaaS Battlecard Atlas

Factden — Mon, 01 Jun 2026 10:30:32 +0000

The 2026 SaaS Battlecard Atlas: What 50,000 G2 Reviews Reveal About 25 of the Most-Used B2B Tools

A few weeks ago I got tired of guessing which B2B SaaS tools were actually worth recommending. G2 has the reviews, but reading 50,000 of them by hand isn't a great use of a weekend.

So I built a scraper. Then I ran it across 25 of the most-reviewed B2B tools on G2. Then I let pandas do the rest.

What came out surprised me in a few places. Here's the full breakdown.

TL;DR: 5 things B2B buyers should care about

Gong is the most-loved tool in the dataset. 4.88 stars overall, and the sub-ratings are all consistently high. Most "4.8 star" SaaS tools have one weak sub-rating they hide. Gong doesn't.
The biggest customer migration of 2026 is Mailchimp going to ActiveCampaign. 350 named-switching events from Mailchimp variants flowing into ActiveCampaign. Email marketing is consolidating, fast.
Apollo is winning the sales intelligence war. 88 customers named ZoomInfo (GTM Workspace) as their previous tool. That's the largest single sales-intel migration in the whole dataset.
UX is both the #1 thing customers praise AND the #1 thing they complain about. Mentioned in 49% of pros and 20% of cons. If you nail the UI, customers will love you for it. If you don't, they will not shut up about it.
Rippling, Deel, and Gong have the lowest "incentivization gaming" rates (3.5% to 11%). Meaning their high ratings are organic, not bought with gift cards. If you trust one tool's 4.8 stars, trust theirs.

How I built this dataset

I used g2-reviews-scraper, an Apify actor I built specifically for this. It extracts 32 structured fields per review including ratings, sub-ratings, switching data, structured pros/cons, demographics, and an LLM-ready markdown block. I wrote up the full method separately in how to scrape G2 reviews.

The numbers:

Detail	Value
Reviews analyzed	50,000
Products	25 (top-reviewed across 7 B2B categories)
Time range	June 2025 to May 2026 (last 12 months)
Fields per review	32
Reviewer demographics populated	99% country, 93% company size, 85% industry, 76% role
Named switching events	6,004
Customer quotes (whySwitched)	5,143

Product list (25 unique, vendor-deduped): Zoom Workplace, Google Workspace, Slack, Microsoft Teams, Miro, Salesforce Sales Cloud, HubSpot Sales Hub, Apollo.io, ZoomInfo (GTM Workspace), Salesloft, Gong, ActiveCampaign, Mailchimp Email, Hootsuite, monday.com, Asana, Trello, ClickUp, Smartsheet, Notion, Jira, Zendesk, Zoho Desk, Rippling, Deel.

1. What each tool is actually good at

This is the chart every B2B buyer should screenshot before their next vendor selection. Greener cells mean the tool scores better on that sub-rating.

A few things jumped out when I built this:

Rippling, Deel, and Gong dominate every column. It's rare to see consistency this clean. Most products win on one or two dimensions and lose on others. These three just win, everywhere.
Smartsheet and ActiveCampaign show up at the bottom across the board. Even their "overall" ratings hide weaker sub-rating performance. Buyers should not be fooled by the headline star count.
Quality of Support is the universal weakest sub-rating. Even the category leaders like Salesforce (6.14), HubSpot (5.96), and Zendesk (5.99) lag here. Don't expect great support from any of them; budget for your own enablement.
easeOfDoingBusinessWith has wide variance. That column captures procurement friction, contract negotiation, billing weirdness. If yours is a hidden buyer-pain category, you'd see it here.

Bottom line for buyers: don't trust the overall star. Look at the sub-rating that matters for your use case.

2. The 2026 Productivity Heroes

Five tools customers genuinely love. High overall rating combined with low "incentivization gaming" (meaning the ratings reflect organic praise, not paid reviews).

Rank	Product	Rating	Incentivization	Why it wins
1	Gong	4.88	11.4%	Revenue Operations leader. Highest sub-ratings across the board.
2	Rippling	4.84	3.5%	Payroll/HR with the cleanest organic praise rate in the dataset.
3	Deel	4.81	5.5%	Contractor payments. Strong on ease of doing business plus support (6.45).
4	monday.com	4.75	7.8%	Best PM score combined with low incentivization.
5	ZoomInfo (GTM Workspace)	4.74	8.2%	Sales intelligence. Surprisingly strong in a crowded category.

Here's the counter-intuitive part. The most-incentivized tools in the dataset (Mailchimp at 78%, Google Workspace at 71%, Trello at 65%) all rate between 4.5 and 4.8 stars. I expected the bought reviews to be inflating ratings dramatically.

They aren't. When you filter out the incentivized reviews, the ratings move by less than 0.1 star in our sample. The biggest shift I observed was 0.08 for Mailchimp Email. The signal is real even with the noise. (Section 7 has the full breakdown.)

3. Who's winning customers, and who's losing them

This was the chart I most wanted to build. The data comes from 6,004 named-switching events where a reviewer explicitly mentioned the tool they came from.

Green bars are vendors gaining customers (net positive switching flow). Red bars are vendors losing them. The annotation on the right shows each vendor's primary source (for winners) or destination (for losers).

Top 5 winners

Vendor	Net flow	Where they're gaining from
ActiveCampaign	+536	Mailchimp Email (251) and Mailchimp All-in-One Platform (99). Total dominance.
ClickUp	+219	Asana (69) and others. Leading PM consolidation.
Zoho Desk	+133	Freshdesk (80) and other help-desk tools.
Hootsuite	+107	Sprout Social (65). Surprising given Hootsuite's overall positioning.
HubSpot Sales	+101	Salesforce Sales (50). CRM mid-market shift.

Top 5 losers

Vendor	Net flow	Where they're losing to
Mailchimp Email	-215	Almost entirely to ActiveCampaign.
Trello	-137	Mostly to ClickUp.
Freshdesk	-108	Mostly to Zoho Desk.
Mailchimp Platform	-99	Also to ActiveCampaign.
ZoomInfo (GTM)	-88	All to Apollo.io.

The 5 biggest disruption stories

If you read nothing else from this report, these 5 named migrations are the story of B2B SaaS in 2026:

Email Marketing: Mailchimp Email to ActiveCampaign (251 switches). The single largest named migration in the dataset.
Sales Intelligence: ZoomInfo (GTM) to Apollo.io (88 switches). The rising challenger is winning.
Customer Support: Freshdesk to Zoho Desk (80 switches). Cost-conscious buyers are shifting.
Project Management: Asana to ClickUp (69 switches). ClickUp is eating Asana's mid-market.
Sales Engagement: Outreach to Salesloft (65 switches). Salesloft pulling ahead.

If you're currently using a "loser" vendor, look at where their customers are going. Those are the tools to evaluate. If you're on a "winner" vendor, you're aligned with where the market is moving.

4. What customers complain about most

What people say in the cons field, across all 50,000 reviews.

Top 5 universal complaints:

UX/UI (20.3% of reviews). The single most-complained-about issue across all B2B SaaS. Interfaces, intuitiveness, design. Customers notice when it's bad.
Learning curve (13.1%). Onboarding pain is universal.
Mobile (9.9%). Mobile apps consistently disappoint.
Performance (8.6%). Speed, lag, crashes.
Integrations (8.1%). Gaps in connecting to other tools.

For PMMs and sales enablement teams: these themes show up in 4,000 to 10,000 reviews each. That's a goldmine of objection-handling material. The actor exposes the structured cons field for any product, so you can mine these patterns for your own competitor set.

5. What customers actually praise

The other half of the story. What people say in the pros field, across all 50,000 reviews.

Top 5 universal pros:

UX/UI (49.4%). Also the most-praised dimension. Half of all customers compliment the interface.
Mobile (18.2%). When mobile works, customers notice.
Integrations (18.0%). Same theme as cons, but framed positively.
Customer support (7.6%). When it's good, customers are deeply grateful.
Customization (7.6%). Flexible products win loyalty.

The UX paradox: UX shows up in 49% of pros and 20% of cons. That's not a contradiction. That's the signal. UX is the single biggest lever in B2B SaaS satisfaction. Companies that get it right generate 2.4 times more positive mentions than the ones that don't.

6. The real customer voice (direct quotes)

These are pulled straight from the whySwitched field. Real customer language explaining their actual decisions.

"Gmail was much easier to administer than on-prem Exchange."
Mid-Market reviewer switching to Google Workspace

"Integrations and its customizable templates."
Asana reviewer who came from monday.com

"It has better project management features."
monday.com reviewer who came from Hootsuite

"For individual use Trello is faster."
Trello reviewer who came from Jira

"Better thread management and a more responsive mobile app."
Slack reviewer who came from Microsoft Teams

This is the customer language sales teams pay competitive intelligence firms thousands for. It's all in the raw data, queryable by competitor.

7. About those incentivized reviews

One important thing buyers should know: 25.5% of the G2 reviews in our sample are incentivized. The vendor provided a gift card, charitable donation, or similar perk in exchange for the review. This rate varies wildly by vendor, from 3.5% (Rippling) to 78% (Mailchimp).

Here's the good news: when you filter out the incentivized reviews, most products' ratings move by less than 0.1 star. The biggest shift I observed was 0.08 (Mailchimp Email). The rating gap between products is mostly genuine.

Here's the actionable news: G2 doesn't expose the "isIncentivized" flag as a filter in their own UI. The actor does. You can do this calibration yourself for any vendor by filtering out the incentivized reviews and seeing how the rating shifts. It's a one-click toggle in the actor's output.

This is exactly the kind of analysis competitive intelligence and procurement teams do. Now anyone can do it.

Want the dataset?

A balanced sample of 2,500 reviews (100 per product across all 25 tools) is available as a free CSV. All 32 fields populated. Leave an email and the download starts immediately on submit.

👉 Download the SaaS Battlecard Atlas dataset (free CSV)

For the complete dataset of your own competitor set, run the actor yourself.

Methodology

Scraper: factden/g2-reviews-scraper, an open Apify actor that extracts G2 reviews with 32 structured fields including switching data and LLM-ready markdown.
Sample selection: top-reviewed products by G2 review count, vendor-deduplicated, across 7 B2B categories. 25 products times 2,000 reviews each equals 50,000 reviews.
Date filter: 2025-06-01 to 2026-05-31 (recent customer voice).
Switching analysis: from the previousCompetitors field (12.0% population) and the per-product topCompetitors aggregate (250 ranked competitor relationships).
Theme mining: keyword-based pattern matching across pros/cons/whySwitched fields. Conservative, so false negatives are more likely than false positives.
Limitations: G2's data has known biases (US-skew, B2B-skew, incentivization variance). I've quantified incentivization rates per product; other biases remain as caveats.

Build your own Battlecard Atlas

Want to do this analysis on your specific competitor set? Run the G2 Reviews Scraper:

👉 Try the G2 Reviews Scraper free (first 1,250 reviews free on Apify's $5 monthly credit)

What you get out of one run:

All 32 fields per review (ratings, sub-ratings, structured pros/cons, switching history, demographics, LLM-ready markdown)
Pre-aggregated topCompetitors per product
isIncentivized flag for filtering
CSV / JSON / Excel export
Apify Schedules + Webhooks for ongoing monitoring

Built for: competitive intelligence analysts, RevOps teams, product marketers, sales enablement, AI/RAG engineers ingesting review data, and procurement teams doing due diligence. More structured-data actors at FactDen.

Working in other verticals? I built the same structured-data approach for Trip.com & Ctrip hotel reviews and Indeed jobs & salary data.

If you found something interesting in the data, or you want me to run this on a different competitor set, drop a comment or email me at hello@factden.com. The full dataset is a free CSV download. If you want to replicate this analysis for your own space, the actor is the fastest path. First 1,250 reviews are free.

DEV Community: Factden

How to Scrape Hotels.com Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull from a review

Why scraping Hotels.com is hard

Three ways to get the data

Option A: DIY in Python

Option B: the no-code / API shortcut

What comes back: 27 fields per review

Grab a free sample dataset

FAQ

Related

How to Scrape Expedia Hotel Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull from a review

Why scraping Expedia is hard

Three ways to get the data

Option A: DIY in Python

Option B: the no-code / API shortcut

What comes back: 27 fields per review

Grab a free sample dataset

FAQ

Related

How to Scrape Trip.com and Ctrip Hotel Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull from a review

Why scraping Trip.com and Ctrip is hard

Three ways to get the data

Option A: DIY in Python

Option B: the no-code / API shortcut

What comes back: 23 fields per review

Grab a free sample dataset

FAQ

Related

How to Scrape Indeed Jobs in 2026 (Python + a No-Code Shortcut)

What you can pull from an Indeed job

Why scraping Indeed is hard

Three ways to get Indeed job data

Option A: DIY in Python

Option B: the no-code / API shortcut

What comes back: 26 structured fields per job

Grab a free sample dataset

FAQ

Related

How to Scrape G2 Reviews in 2026 (Python + a No-Code Shortcut)

What a G2 review actually contains

Why scraping G2 is hard: DataDome

Three ways to get G2 review data

Option A: DIY in Python

Option B: the no-code / API shortcut

What comes back: 27 structured fields

Grab a free sample dataset

FAQ

Related

I scraped 50,000 G2 reviews to map the 2026 SaaS Battlecard Atlas

The 2026 SaaS Battlecard Atlas: What 50,000 G2 Reviews Reveal About 25 of the Most-Used B2B Tools

TL;DR: 5 things B2B buyers should care about

How I built this dataset

1. What each tool is actually good at

2. The 2026 Productivity Heroes

3. Who's winning customers, and who's losing them

Top 5 winners

Top 5 losers

The 5 biggest disruption stories

4. What customers complain about most

5. What customers actually praise

6. The real customer voice (direct quotes)

7. About those incentivized reviews

Want the dataset?

Methodology

Build your own Battlecard Atlas