DEV Community: Factden

How to Scrape India MCA Company and Director Contacts in 2026 (Python + a No-Code Shortcut)

Factden — Wed, 19 Aug 2026 05:23:42 +0000

If you sell into India, the contact that actually moves a deal is a decision maker, a company's director, not a generic info@ inbox. India's Ministry of Corporate Affairs (MCA) registry is where that lives: every registered company, its status, capital and filings, the full board, and on each director's DIN their personal email and phone. Getting it out cleanly, and matching a director to every company they sit on, is the hard part.

This guide shows both ways: a runnable Python recipe, and a no-code shortcut with the MCA India Company and Director Data Scraper that skips the hard parts entirely. Run it by industry and state to build a whole segment of B2B leads at once, then enrich the board with director contacts.

What you can pull

Company master for a CIN: legal name, status, company type and class, incorporation date, industry (NIC), state, authorised and paid-up capital, ROC, registered addresses, board size, and three years of filing history.
The full board for a company: every director with DIN, name, designation, and dates.
Director master for a DIN: status, date of birth, gender, nationality, and the full directorship network, every company they sit on with designation and dates.
Director contacts as an opt-in enrichment: the director's personal email and phone as registered with MCA on their DIN or DIR-3 KYC record.
Discovery by industry and state: pick a sector and a state and pull a whole segment of companies at once, not one CIN at a time.

Why scraping MCA directly is hard

The MCA portal does not hand you company and director records in simple HTML you can select with CSS. Most lookups sit behind a session and token flow, and the pieces you want, company master, the board, each director's master record, and the directorship graph, come from different calls that you then have to stitch into one clean row. Discovery is its own problem: finding every active company in an industry and state means understanding how a CIN prefix encodes industry and paginating the full result set. You can solve all of this, but it is real engineering, and it breaks whenever the registry reshapes a response.

That is the whole reason a maintained actor exists: it absorbs the token and lookup work and hands you a flat table.

DIY vs actor vs official data

	Write it yourself	MCA scraper (actor)	Official MCA portal
Setup	Token flow, lookups, response parsing	Paste a CIN, DIN, name, or pick industry and state	Manual per-company lookups
Company plus board plus directorship graph	You stitch several calls	Included, one clean row	One company at a time
Discovery by industry and state	You decode CIN prefixes	Included	Not offered as a bulk export
Director email and phone	Separate KYC lookup to solve	Opt-in enrichment	Not exposed in bulk
Cost model	Free but you build it	Free tier, then pay per result	Per-document fees, manual
Access	You own every breakage	Maintained for you	Their portal, no bulk export

There is no open MCA bulk API that hands you company and director records for a whole segment. The portal is built for one-company-at-a-time lookups. For open collection into your own pipeline, the choice is really "build and maintain it" or "call an actor and pay per run".

The no-code way (about a minute)

Create a free Apify account and open the actor page.
Click Try for free. The input is pre-filled with an example.
Pick a mode (Companies or Directors), then give it a CIN, DIN, name, or an industry and state, and click Start.
Download the results from the Output tab as JSON, CSV, or Excel.

To refresh a segment on a schedule, save your input as a Task and attach a Schedule. Each run appends a fresh snapshot, so a lead list stays current as new companies register.

Run it from Python

The actor runs on Apify, so you drive it with the Apify client. Install it with pip install apify-client, grab your token from Apify Settings, then:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

# Discover a whole segment: active IT companies in Maharashtra, with their boards.
run_input = {
    "mode": "company",
    "industries": ["Information and communication"],
    "states": ["Maharashtra"],
    "companyStatuses": ["Active"],
    "includeDirectors": True,
    "maxCompanies": 200,
}

run = client.actor("factden/mca-company-director-scraper").call(run_input=run_input)

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row)

To pull directors by name and enrich them with personal contacts instead, switch the mode and turn on the enrichment:

run_input = {
    "mode": "director",
    "directorNames": ["Nandan Nilekani"],
    "directorLeads": True,   # opt-in email + phone enrichment
    "includeCompanies": True,
    "maxDirectors": 25,
}

mode selects the run: company drives discovery by CIN, company name, or industry and state; director drives lookups by DIN or director name. Only the inputs the selected mode needs are used. directorLeads is the opt-in switch for personal email and phone.

What the output looks like

Trimmed from a real director run for a single director (contacts shown as dummy values here):

{
  "din": "02790976",
  "name": "NANDAN M. NILEKANI",
  "status": "Approved",
  "gender": "Male",
  "nationality": "Indian",
  "email": "nandan.nilekani@example.com",
  "phone": "+919876543210",
  "directorshipsCount": 5,
  "company": "INFOSYS LIMITED",
  "cin": "L85110KA1981PLC013115",
  "designation": "Director"
}

The fields you get depend on the mode you run:

Mode	What each row carries
`company`	One row per company: CIN, name, status, type, capital, industry, state, ROC, addresses, filings, board size
`company` + `includeDirectors`	The company plus its board, one director per row with designation and dates
`director`	One row per directorship: DIN, name, status, DOB, gender, and the company with designation and dates
`director` + `directorLeads`	The same, plus the director's personal email and phone (charged once per unique DIN, only when found)

Field names are documented on the listing and in the repo below rather than invented here, so what you build against matches what the actor emits.

Try it on a real dataset first

If you want to see the shape before running anything, there is a free sample MCA director dataset you can open in the browser, and the input, snippets, and field reference are on GitHub.

FAQ

Do I need an MCA login or API key?
No. There is no MCA portal login, no account, and no API key. You only need an Apify token to run the actor.

Can I search by company name, director name, or industry?
Yes. By exact CIN or DIN, by company or director name (a prefix match on the registered name), or by industry mapped to the NIC code, narrowed by state, status, type, and listing.

How are director contacts charged?
Once per unique director (DIN) and only when a real email or phone is found, never per row. A director who sits on five boards is billed once.

How do I keep a lead list current?
Save your input as a Task and attach a Schedule. Each run appends a fresh snapshot, so new registrations in your target segment show up over time.

Is it legal?
Company and director master data is public-record. The director email and phone enrichment returns personal data protected under India's DPDP Act, 2023, so you must have a lawful basis to obtain and process it. Use the output responsibly.

Building B2B lead and market data across sources? The same team maintains a G2 reviews scraper for software buyer intelligence and an Indeed jobs scraper for hiring and labor-market signals, so you can line up firmographics, buyer sentiment, and hiring in the same format.

Full write-up with more examples: how to scrape MCA company and director data, and the MCA data hub has the field reference and use cases.

How to Scrape Booking.com Hotels, Prices and Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Wed, 19 Aug 2026 05:21:30 +0000

Booking.com has the largest pool of hotel prices and guest reviews on the web, and no public data API to get any of it out. Its partner APIs are for accredited hotels managing their own listings, not for market research. So if you want to shop rates across a city, watch availability over time, or build a guest-sentiment dataset, you have to read the public pages yourself.

This guide covers what you can pull, why the DIY route is harder than it looks, working Python, a no-code shortcut, and a plain comparison. There is a fuller reference on how to scrape Booking.com, plus a step-by-step blog walkthrough.

What you can pull

Give it a city, an exact hotel, or a Booking.com link, and you get four clean, separate outputs:

Hotels (one row each): name, star class, guest score and review count, coordinates, address, currency, per-night and total price, availableRooms, and a scarcity message ("only 2 left").
Rooms and rates: one row per bookable room, with price per night, meal plan, refundable flag, free-cancellation deadline, max occupancy and room size.
Reviews (optional): score out of 10, positive and negative text, reviewer country, traveller type, stay date and nights, and the hotel's reply. Rating-only reviews are included and flagged.
Availability calendar (optional): one row per forward day, with the date, available or sold-out, and the minimum nightly price.

Every hotel and review row also carries a self-contained markdownContent block, ready to drop into a vector DB or an LLM prompt.

Why scraping Booking.com is hard

Three reasons the DIY route eats your week:

No single page has everything. Discovery, the price breakdown, the room list, reviews and the calendar all come from different places, each with its own parameters. Stitching them into one clean row per hotel is most of the work.
Prices are personalized and date-bound. The same hotel returns different rates by dates, party size, currency and device. You have to pin all of that or your numbers drift.
Volume and blocking. A city has hundreds of hotels; a hotel has thousands of reviews. Pulling that reliably, at rate, without getting blocked, is a full-time job on its own.

Three ways to get the data

DIY in Python - full control, but you own discovery, pagination, the price/room/review/calendar stitching, retries and blocking.
A no-code actor / API - you send input, it returns clean JSON. No parsing, no infra.
The official partner API - not an option unless you are an accredited Booking.com partner managing your own inventory.

Option A: DIY in Python

The honest starting point is discovery: find the hotels, then enrich each one. Expect to maintain the request parameters, pagination and retry logic yourself.

import httpx

# You resolve a destination, page through its hotels, then for each hotel
# fetch price + rooms, reviews and the calendar - each a separate call.
resp = httpx.get(
    "https://www.booking.com/searchresults.html",
    params={"ss": "Rome", "checkin": "2026-09-01", "checkout": "2026-09-02"},
    headers={"user-agent": "Mozilla/5.0"},
    timeout=30,
)
# ...then parse the results, follow each hotel, and merge prices, rooms,
# reviews and availability into one row. This is where the week goes.

That runs, but you still have to build the whole pipeline around it. When you want reliability at scale, the shortcut wins.

Option B: the no-code / API shortcut

The Booking.com Scraper on Apify takes input and returns clean JSON. Call it from Python with the Apify client:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("factden/booking-com-scraper").call(run_input={
    "searchLocation": ["Rome"],
    "includePrices": True,
    "includeReviews": True,
    "maxReviews": 50,
    "includeAvailabilityCalendar": True,
    "calendarDays": 30,
    "currency": "USD",
    "maxResults": 25,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["hotelName"], item.get("perNightPrice"), item.get("reviewScore"))

Swap searchLocation for searchTerms (a hotel name) or startUrls (Booking.com links) when you want specific properties. Prices, reviews and the calendar are independent toggles, so you pay only for what you pull.

What comes back

One flat row per hotel, plus separate reviews and calendar datasets. A trimmed hotel row:

{
  "hotelName": "The Savoy",
  "city": "London",
  "stars": 5,
  "reviewScore": 9.4,
  "reviewCount": 1446,
  "currency": "GBP",
  "perNightPrice": 733.33,
  "availableRooms": 7,
  "scarcityMessage": "We have 7 left",
  "rooms": [{ "roomName": "Superior Queen Room", "pricePerNight": 733.33, "refundable": false }]
}

Export as JSON, CSV, Excel or HTML, or pull it from the Apify API on a schedule.

Grab a free sample dataset

Want to see the shape before you run anything? Grab a free Booking.com sample as clean CSV.

FAQ

Does Booking.com have a public API? No. Its Connectivity and Demand APIs are for accredited partners managing their own listings, not for market research.

Can I discover every hotel in a city? Yes. Put a city, region or country in searchLocation and you get every hotel there, paginated and de-duplicated.

Does it get rating-only reviews? Yes, and there is no 500-review cap. Rating-only reviews (a score with no text) are included and flagged.

How much does it cost? Pay-per-event, no start fee: hotel $0.003, prices $0.005 per priced hotel, review $0.001, calendar day $0.0005. New Apify accounts get free monthly credit.

Building a travel-data pipeline? Pair this with the Google Hotels Scraper for the OTA rate ladder, or the Expedia Reviews Scraper. Full list at apify.com/factden.

How to Scrape Agoda Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Thu, 06 Aug 2026 05:16:25 +0000

Here is the thing most people hit when they try to scrape Agoda reviews: there is no public endpoint you can just call. Agoda's partner and affiliate APIs cover rates and availability only. None of them let you read a property's guest reviews. So the moment you want the actual words guests wrote, you are on your own.

And it gets trickier. The reviews you see on an Agoda hotel page are not in the HTML you get from a plain request. They load separately, after the page. On top of that, Agoda serves reviews in 33 languages, and by default they are fragmented across those languages, so a naive scrape gives you a lopsided slice of what guests actually said.

This post shows the honest Python starting point, why it stalls, and then a no-code shortcut that returns clean rows with one call. If you would rather read a longer writeup, there is a full how to scrape Agoda reviews guide on our site too.

What you can pull from a review

Every review row includes the core identity (hotelId, hotelName, hotelUrl, reviewId, source), the rating on Agoda's 0 to 10 score scale plus a word label (score, ratingText), the written content (title, text, and where available positives / negatives), the stay context (travelerType, roomType, checkInDate, checkOutDate, lengthOfStay), the reviewer basics (reviewerName, reviewerCountry), the date (reviewDate), any attached reviewPhotos, the hotel's ownerResponse if the property replied, and a markdownContent block that is ready to drop straight into an LLM prompt.

One honesty note: positives and negatives (the split liked / disliked fields) are usually empty for Agoda-native reviews. Agoda's own reviews come as one body of text. The separated liked / disliked structure mostly shows up on reviews sourced from Booking.com or Priceline, which you can optionally pull in. Do not build a pipeline that assumes every review has neat liked / disliked lists, because most will not.

Why scraping Agoda reviews is hard

There is no public reviews API. The official Agoda APIs are for rates and availability, so reviews are simply not on offer through a supported channel.
Reviews load dynamically. The hotel page HTML you get from a simple GET does not contain the structured review text, so parsing the page source gets you nothing useful.
Language fragmentation. With 33 review languages, a single-pass scrape tends to return whatever the page defaults to, not the full picture.
Two scoring worlds. Agoda uses a 0 to 10 score, not 1 to 5 stars, so any tooling that assumes a five-star scale will mangle your data.
Blocking and URL shape. Hotel URLs vary by region and Agoda actively guards against automated access. Handling that reliably is a project on its own.

Three ways to get the data

Approach	Setup	Handles blocking + dynamic load	Reviews coverage	Cost model
DIY Python	You write and maintain it	You build it yourself	Whatever you can reach	Your time + proxies
Agoda Hotel Reviews Scraper (Apify)	Paste a URL or ID	Yes, done for you	All reviews, all 33 languages, optional extra sources	Pay per review returned
Official Agoda API	Partner account	N/A	None, reviews are not offered	Not available

Option A: DIY in Python

Start where everyone starts, a request to the hotel page. The problem shows up immediately: the review text is not in the response body.

import httpx

url = "https://www.agoda.com/bayswater-inn-hotel/hotel/london-gb.html"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"}, timeout=30)

print(r.status_code)
print("reviewComment" in r.text)  # -> False, the reviews are not in the page HTML

The reviews arrive through a separate call the page makes after it loads, and that call is guarded. To do this properly you would reverse engineer that flow, rotate proxies, deal with region-specific URL shapes, page through results, normalize the 0 to 10 score, and keep it all working as the site changes. Doable, but it is a maintenance job, not a one-off script.

Option B: the no-code / API shortcut

If you would rather skip the maintenance, the Agoda Hotel Reviews Scraper on Apify does the fetching, paging, and normalizing for you. The actor handles blocking and URL resolution for you, so you just hand it hotels and get rows back.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("factden/agoda-hotel-reviews-scraper").call(run_input={
    "hotelUrls": ["https://www.agoda.com/bayswater-inn-hotel/hotel/london-gb.html", "11019"],
    "maxReviews": 500,
    "sortBy": "newest",
    "reviewSources": ["agoda"],
    "languages": [],
})

for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["score"], review["ratingText"], review["text"][:60])

A few things worth knowing:

You can paste full Agoda hotel URLs or bare numeric hotel IDs like "11019". Mix and match in the same list.
languages: [] means give me everything in one combined stream. Pass a subset of Agoda's 33 languages if you only want specific ones.
reviewSources: ["agoda"] keeps it Agoda-native. Add Booking.com or Priceline, or clear the list to pull all sources.
Filter with sortBy, fromDate / toDate, and minRating / maxRating (remember, that is on the 0 to 10 scale).
There are two datasets. The default Reviews dataset is one row per guest review. The Hotels dataset gives you one aggregate row per hotel with the overall score, review counts, per-source breakdown, and location.

Track reputation over time

Reviews are most useful as a trend, not a snapshot. Set fromDate to the day after your last pull and put the run on an Apify Schedule (say, every morning). Each run then grabs only the reviews posted since last time, so you build an incremental reputation feed for a property or a whole competitive set without re-pulling history.

What comes back: the fields per review

Group	Fields
Core	`hotelId`, `hotelName`, `hotelUrl`, `reviewId`, `source`
Rating	`score` (0 to 10), `ratingText`
Text	`title`, `text`, `positives`, `negatives`
Stay	`travelerType`, `roomType`, `checkInDate`, `checkOutDate`, `lengthOfStay`
Reviewer	`reviewerName`, `reviewerCountry`
Dates + owner	`reviewDate`, `ownerResponse`
Media	`reviewPhotos`
AI-ready	`markdownContent`

A trimmed sample row:

{
  "hotelId": 11019,
  "hotelName": "Park Avenue Bayswater Inn Hyde Park",
  "hotelUrl": "https://www.agoda.com/bayswater-inn-hotel/hotel/london-gb.html",
  "reviewId": "1149287470",
  "source": "Agoda",
  "title": "Above average",
  "text": "All in all good hotel, can't complain but wasn't the best.",
  "score": 6,
  "ratingText": "Good",
  "reviewDate": "2026-07-20",
  "reviewerName": "Tiffany",
  "reviewerCountry": "Australia",
  "travelerType": "Couple",
  "roomType": "Double Room",
  "checkInDate": "2026-07-03",
  "checkOutDate": "2026-07-07",
  "lengthOfStay": 4,
  "ownerResponse": null,
  "markdownContent": "# Park Avenue Bayswater Inn Hyde Park review (Agoda)\n\n**Score:** 6/10 - Good\n..."
}

Grab a free sample dataset

Want to see the real shape before running anything? Download the free sample dataset and inspect the exact fields, including a few reviews with photos and owner responses.

FAQ

Is there an official Agoda reviews API? No. Agoda's partner and affiliate APIs are for rates and availability. There is no supported way to read a property's guest reviews, which is the gap this actor fills, with no key and no partner account.

Do I need to find hotel IDs? No. Paste the normal Agoda hotel URL and you are done. If you happen to have the numeric ID, you can pass that instead, either works.

Can I get reviews in a specific language? Yes. Agoda carries 33 review languages. Leave languages empty for all of them in one stream, or pass the ones you want.

Why are positives and negatives empty on some reviews? Agoda-native reviews come as a single block of text, so the split liked / disliked fields are usually null. Those fields populate mainly on reviews sourced from Booking.com or Priceline.

How does pricing work? Pay per review returned, with no start fee, and nothing at all if a run returns zero reviews. Rates run from 4.00 USD per 1,000 reviews down to 2.00 USD per 1,000 at higher volume.

Can I monitor reviews on a schedule? Yes. Combine fromDate with an Apify Schedule to pull only new reviews each run and build an incremental reputation feed. Full walkthrough in the Agoda hotel reviews API listing and the docs repo.

Expedia Hotel Reviews Scraper if you need the same for Expedia.
Trip.com and Ctrip Reviews Scraper for Trip.com and Ctrip properties.
Doing B2B software instead of hotels? See how to scrape G2 reviews.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape Airbnb Prices, Availability and Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Wed, 05 Aug 2026 05:46:36 +0000

If you manage short-term rentals, the numbers you actually care about move every day: nightly prices, forward availability, occupancy, ADR, and RevPAR. A revenue manager does not pull those once. They pull them on a schedule, watch how a market drifts week over week, and price against it. Airbnb is where most of that signal lives, and getting it out cleanly is the hard part.

This guide shows both ways: a runnable Python recipe, and a no-code shortcut with the Airbnb data scraper that skips the hard parts entirely. Schedule it daily to track prices, availability and occupancy over time and feed your revenue-management pipeline, the same job you would otherwise pay AirDNA to do.

What you can pull

Listings for a whole market by location: nightly price, property details, coordinates, rating, and host, one clean row per listing.
The forward calendar for a listing: up to 12 months of availability and the real price by date, so you can see how a place is booking and what it charges night to night.
Occupancy, ADR and RevPAR for a listing, the core revenue-management signals, computed from the forward calendar.
A market report for a location: one aggregate KPI row (ADR, occupancy, RevPAR, revenue) plus the sampled listings, an AirDNA alternative you pay for only when you run it.
Reviews with sentiment: the full review history for a listing, with ratings and host replies.

Why scraping Airbnb directly is hard

Airbnb does not hand you prices and availability in simple HTML you can select with CSS. The data comes from Airbnb's internal endpoints, which expect requests to look a very particular way and quietly return nothing when they do not. Prices are quoted per date and per guest count, the forward calendar is paginated and keyed on internal listing tokens, and the occupancy math has to be derived from the calendar rather than read off a field. You can solve all of this, but it is real engineering, and it breaks whenever Airbnb reshapes a response.

That is the whole reason a maintained actor exists: it absorbs the endpoint work and hands you a flat table.

DIY vs actor vs AirDNA

	Write it yourself	Airbnb data scraper (actor)	AirDNA / official API
Setup	Endpoint work, proxies, response parsing	Paste a location or listing URL	Subscription onboarding
Prices and availability	You reverse-engineer it	Included, structured	Included, but locked to their dashboard
Occupancy, ADR, RevPAR	You derive it from the calendar	Computed for you	Included
Reviews with sentiment	Separate endpoint to solve	Included	Not the focus
Cost model	Free but you build it	Free tier, then pay per result	Fixed monthly subscription
Access	You own every breakage	Maintained for you	Their data, their platform

There is no open Airbnb API that lets you pull arbitrary listing prices, availability, and occupancy. AirDNA packages similar numbers, but as a fixed monthly subscription tied to their dashboard. For open collection into your own pipeline, scraping is the route, and the choice is really "build and maintain it", "rent AirDNA", or "call an actor and pay per run".

The no-code way (about a minute)

Create a free Apify account and open the actor page.
Click Try for free. The input is pre-filled with an example.
Pick a mode, then fill the one input it needs (a location, or a listing URL), then click Start.
Download the results from the Output tab as JSON, CSV, or Excel.

To run it every day, save your input as a Task and attach a Schedule. Each run appends a fresh snapshot, which is exactly the time series a revenue manager wants.

Run it from Python

The actor runs on Apify, so you drive it with the Apify client. Install it with pip install apify-client, grab your token from Apify Settings, then:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

# Market report for a location: one aggregate KPI row plus sampled listings.
run_input = {
    "mode": "market",
    "location": "Lisbon, Portugal",
    "sampleSize": 100,
    "marketMonths": 3,
    "currency": "USD",
}

run = client.actor("factden/airbnb-data-scraper").call(run_input=run_input)

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row)

To track one property's forward prices and occupancy instead, switch the mode and pass a listing URL:

run_input = {
    "mode": "occupancy",
    "startUrls": ["https://www.airbnb.com/rooms/39896685"],
    "months": 12,
    "currency": "USD",
}

The five modes are discover (listings by location), availability (12-month calendar and price by date), occupancy (occupancy, ADR, RevPAR), reviews (full review history with sentiment), and market (the aggregate KPI report). Only the one input the selected mode needs is used, location for discover and market, startUrls for availability, occupancy, and reviews.

What the output looks like

Trimmed from a real occupancy run for a single listing:

{
  "listingId": "39896685",
  "url": "https://www.airbnb.com/rooms/39896685",
  "currency": "USD",
  "months": 12,
  "adr": 168.4,
  "occupancy": 0.72,
  "revpar": 121.2,
  "availableNights": 84,
  "bookedNights": 216
}

The fields you get depend on the mode you run:

Mode	What each row carries
`discover`	One row per listing: nightly price, property details, coordinates, rating, host
`availability`	The forward calendar, with the real price for each available date (opt-in)
`occupancy`	Occupancy, ADR, RevPAR and the booked-vs-available night counts
`reviews`	Each review with its rating, text, host reply, and sentiment
`market`	One aggregate KPI row (ADR, occupancy, RevPAR, revenue) plus the sampled listings

Field names are documented on the listing and in the repo below rather than invented here, so what you build against matches what the actor emits.

Try it on a real dataset first

If you want to see the shape before running anything, there is a free sample Airbnb dataset you can open in the browser, and the input, snippets, and field reference are on GitHub.

FAQ

Do I need an Airbnb API key or login?
No. There is no key and no Airbnb login. You only need an Apify token to run the actor.

How do I track prices and occupancy over time?
Save your input as a Task and attach a daily Schedule. Each run appends a fresh snapshot, so you build a price-and-occupancy time series you can feed straight into a revenue-management model.

Is this really an AirDNA alternative?
For the core numbers, yes. You get ADR, occupancy, RevPAR, forward availability, and a market KPI report, but pay per run instead of a fixed monthly subscription, and the data lands in your own pipeline rather than a dashboard.

How far forward does availability go?
Up to 12 months of the forward calendar. Occupancy is most accurate for the near term, since months far out are still mostly unbooked, which is why the market report samples a shorter forward window.

Which currencies and languages does it support?
Prices can be returned in 20 currencies, and names, review text, and labels in several languages, so you can match a specific market.

Is it legal?
It collects publicly visible data. As with any scraping, use the output responsibly and follow the applicable terms and laws for your use case.

Working travel data across sites? The same team maintains a Google Hotels scraper for live hotel prices and the OTA rate ladder, and an Expedia hotel reviews scraper, so you can line up rates and reviews across Airbnb, Google Hotels, and Expedia in the same format.

Full write-up with more examples: how to scrape Airbnb data, and the Airbnb data hub has the field reference and use cases.

How to Scrape TripAdvisor Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Wed, 05 Aug 2026 05:46:01 +0000

Here is the thing most people hit when they try to scrape TripAdvisor reviews: the official TripAdvisor Content API returns only 3 reviews per location. Three. For a hotel with eight thousand reviews, that is not a dataset, that is a teaser. And it is gated behind an approval process, with no owner responses and no subratings.

This guide covers what you can pull from a TripAdvisor review, why the DIY route is fiddly, working Python, a no-code shortcut, and a plain comparison. If you want the deeper reference, there is a full guide to scraping TripAdvisor reviews too.

What you can pull from a review

Per review you can get:

The overall star rating (1 to 5), the review title, and the full review text with its language
Per-review subratings: cleanliness, service, value, location, rooms, sleep quality, where the reviewer left them
Dates: the published date and the travel date, plus helpful votes
Owner / management responses, with responder name and date
Reviewer profile: username, home location, contribution count
An LLM-ready markdown block per review

And this works across all three TripAdvisor place types: hotels, restaurants, and attractions. Every run also returns a place record with property details (rating, structured city ranking, price range, hotel class, amenities), TripAdvisor's official category subratings, and its own AI review summary.

Why scraping TripAdvisor is hard

The official API caps at 3 reviews. So for anything at scale, scraping the public data is the route, and there is no owner-response or subrating data in that API anyway.
Three different place types, three URL formats. Hotels use Hotel_Review, restaurants use Restaurant_Review, attractions use Attraction_Review. A scraper built for one type misses the others.
Reviews are paginated and loaded dynamically, not sitting in the initial HTML, so a plain request gets you a shell.
Bot protection and rate limits on top.

So the work is handling each place type, paging through every review, pulling the nested subratings and owner responses, and staying unblocked, on repeat.

Three ways to get the data

	DIY Python	TripAdvisor Hotel Reviews API (actor)	Official Content API
Reviews per location	All, if you build paging	All	Only 3
Setup time	Hours to days	~30 seconds	Approval required
Hotels + restaurants + attractions	Build per type	One run	Limited
Subratings + owner responses	Extra parsing	Yes	No
Property details + AI summary	Separate scrape	Same run	No
Cost	Proxies + eng time	Pay-per-result	Gated
Best for	One-off	Scheduled, at scale	Not much

Option A: DIY in Python

A plain request to a hotel page returns a JavaScript shell, not the reviews:

import httpx

url = "https://www.tripadvisor.com/Hotel_Review-g187497-d1465497-Reviews-W_Barcelona-Barcelona_Catalonia.html"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print("reviewText" in r.text)   # usually False; reviews load separately, paginated

To get the real data you page through every review, normalize hotels vs restaurants vs attractions, dig the subratings and owner responses out of nested structures, and keep requests unblocked. Fine for one place, a lot of moving parts for a portfolio or a whole city.

Option B: the no-code / API shortcut

When you want clean rows, the TripAdvisor Hotel Reviews API on Apify takes any TripAdvisor URL (or a bare location ID), and returns structured JSON: every review plus the place record. No login, no proxy setup, it handles the blocking for you.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/tripadvisor-hotel-reviews-api").call(run_input={
    "mode": "reviews",
    "startUrls": [
        {"url": "https://www.tripadvisor.com/Hotel_Review-g187497-d1465497-Reviews-W_Barcelona-Barcelona_Catalonia.html"}
    ],
    "locationIds": ["1465497"],
    "maxReviews": 500,
    "reviewLanguages": ["all"],
    "minRating": 1,
    "maxRating": 5,
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["rating"], review["title"], review["text"][:60])

Paste URLs or bare location IDs (the d-number in the URL), mix them freely. Set maxReviews high for all of them. Filter by reviewLanguages, bound the star range with minRating / maxRating, and window the dates with fromDate / toDate (both YYYY-MM-DD).

Don't know the places yet? Switch to Discover mode and search a whole city:

run = client.actor("factden/tripadvisor-hotel-reviews-api").call(run_input={
    "mode": "discover",
    "searchTerms": ["Barcelona"],
    "placeTypes": ["hotels"],
    "maxPlaces": 20,
    "discoveryDepth": "quick",
})

That returns a list of the city's places with full details. Take the returned location IDs back into Reviews mode to pull their reviews.

Track reputation over time

The fromDate filter is the one to know if you care about reputation monitoring. Set a Schedule in Apify, point fromDate at your last run, and each run brings back only the new reviews and rating changes since then. That is how teams track new reviews and ratings over time without re-pulling the whole history every time, one cheap incremental pull on a cron.

What comes back: 16 fields per review

Group	Fields
Core	`placeId`, `placeType`, `placeName`, `reviewId`, `url`
Rating	`rating` (1 to 5), `title`, `subratings` (cleanliness, service, value, location, rooms, sleep quality)
Text	`text`, `language`
Dates & signals	`publishedDate`, `travelDate`, `helpfulVotes`
Owner	`ownerResponse` (responder, text, date)
Reviewer	`user` (username, userLocation, contributions)
AI-ready	`markdownContent`

A trimmed sample row:

{
  "placeId": 1465497,
  "placeType": "hotel",
  "placeName": "W Barcelona",
  "reviewId": "1070232616",
  "rating": 5,
  "title": "Incredible stay",
  "text": "The rooftop and the service were exceptional...",
  "publishedDate": "2026-07-26T23:11:20-04:00",
  "travelDate": "2026-07",
  "helpfulVotes": 3,
  "subratings": [{ "name": "Service", "value": 5 }, { "name": "Location", "value": 5 }],
  "ownerResponse": { "responder": "GM W Barcelona", "text": "Thank you...", "publishedDate": "2026-07-28" },
  "user": { "username": "traveler_bcn", "userLocation": "London, UK", "contributions": 42 },
  "markdownContent": "# Incredible stay, W Barcelona\n**Rating:** 5/5 ..."
}

Every run also emits a place record per place: name, geo, overall rating, structured city ranking, price range, hotel class, amenities, TripAdvisor's official category subratings, and its AI review summary. Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free TripAdvisor sample (CSV/JSON) here: factden.com/sample-tripadvisor. Load it into pandas and the ratings and subratings are ready to plot.

FAQ

Is scraping TripAdvisor legal? The reviews are publicly visible. As with any scraping, check TripAdvisor's Terms of Service and your local rules (including GDPR/CCPA for the pseudonymous reviewer fields), and use the data responsibly.

Is there an official TripAdvisor reviews API? Yes, the Content API, but it is gated and returns only 3 reviews per location, with no owner responses or subratings. Scraping the public data is the practical route.

Can I scrape restaurants and attractions too, not just hotels? Yes. One actor covers hotels, restaurants, and attractions, with the same schema, tagged by placeType.

Are ratings on the 5-star scale? Yes, TripAdvisor uses 1 to 5 stars, and that is what rating returns, with per-review subratings on the same scale.

Can I monitor a place for new reviews on a schedule? Yes. Set a Schedule and use fromDate so each run only returns reviews posted since the last pull, ideal for ongoing reputation monitoring.

How do I stop getting blocked? The actor handles it for you and reads the data directly. Plain requests get a JavaScript shell, not the reviews.

Doing B2B software instead of hotels? See how to scrape G2 reviews.
Other FactDen hotel scrapers: Expedia hotel reviews and Trip.com and Ctrip hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape MakeMyTrip and Goibibo Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Wed, 05 Aug 2026 05:45:11 +0000

If you want honest guest sentiment for a hotel in India, one platform is never enough. MakeMyTrip and Goibibo are the country's two largest OTAs, both run by the same group, and the same property usually carries a different pool of reviews on each. A traveller who books on Goibibo often leaves a review Goibibo never shows on MakeMyTrip, and the other way round. To read a hotel's real reputation in the India travel market you need both feeds, side by side.

This guide covers what you can pull from a review, why the DIY route is a two-headed problem, working Python, a no-code shortcut, and a plain comparison. If you want the deeper reference, there is a full guide on how to scrape MakeMyTrip and Goibibo reviews too, plus a step-by-step blog walkthrough.

What you can pull from a review

Per review you can get:

Overall rating and the review title, plus the hotel's sub-ratings (Location, Cleanliness, Facilities, Food, Room, Value for Money, Child-friendliness) on the free hotel profile
The full review text, and the source brand so you always know whether a row came from MakeMyTrip or Goibibo
Trip context: travel type (couple, family, business, solo), room name, and check-in month
Reviewer detail: name (or an anonymity flag), and lifetime review count on Goibibo
Review photos with AI scene-tags (images as [{ url, aiTags }]), plus owner/management responses with a date
An LLM-ready markdown block per review (markdownContent)

The two things that separate a real dataset from a shallow one: the second brand's feed and the free hotel profile (star class, geo, sub-ratings) that ships with every run.

Why scraping MakeMyTrip and Goibibo is hard

There is no public reviews API. MakeMyTrip Group offers no public reviews or hotel-data API for either brand, so scraping the public pages is the only route.
It is two systems, not one. MakeMyTrip and Goibibo share one hotel identity but present reviews through different pages and structures. A scraper built for one usually misses the other, and the brand you skip is often where the newer reviews are.
The hotel profile lives apart from the reviews. Star class, property type, PIN code, geo-coordinates and the clubbed sub-ratings sit on a separate surface you have to fetch and join yourself.
Bot protection and rate limits apply on both brands.

So the work is not parsing one page. It is handling both brands, joining the hotel profile, and staying unblocked, on repeat.

Three ways to get the data

	DIY Python	MakeMyTrip & Goibibo actor	Official API
Setup time	Hours to days	~30 seconds	Not available (no public reviews API)
Both MakeMyTrip + Goibibo feeds	Build two scrapers	One run	n/a
Free hotel profile (stars, geo, sub-ratings)	Fetch and join yourself	Included free	n/a
Review photos + owner responses	Parse nested markup	Yes	n/a
Cost	Proxies + eng time	Pay-per-result	n/a
Best for	One-off	Scheduled, at scale	Not an option

Option A: DIY in Python

A plain request to a hotel page tends to come back blocked or JavaScript-only:

import httpx

url = "https://www.makemytrip.com/hotels/hotel-details/?hotelId=200703241029455940&city=CTGOI&country=IN"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print(r.status_code)   # challenge / JS-rendered shell, reviews not in the HTML

The reviews load through internal endpoints, so you end up reverse-engineering those (per brand), normalizing two different response shapes, and separately fetching the hotel profile to join on. Doable for one hotel, painful for a portfolio on a schedule.

Option B: the no-code / API shortcut

When you want clean rows, the MakeMyTrip & Goibibo Reviews Scraper on Apify reads both brands in a single run and hands back structured JSON, with the full hotel profile included free. No login, no proxy setup.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/makemytrip-scraper").call(run_input={
    "startUrls": [
        "https://www.makemytrip.com/hotels/hotel-details/?hotelId=200703241029455940&city=CTGOI&country=IN",
        "https://www.goibibo.com/hotels/hard-rock-goa-hotel-in-goa-6204281054243107966/"
    ],
    "reviewSource": "auto",
    "maxReviewsPerHotel": 200,
    "sortBy": "mostRecent",
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["source"], review["overallRating"], review["travelType"])

Mix MakeMyTrip and Goibibo URLs freely. reviewSource takes auto (match each link's own brand), makemytrip, goibibo, or both (collect both brands' reviews for the same hotel). sortBy accepts mostRelevant, mostRecent, helpful, positive, or negative, and you can bound results with fromDate / toDate and minRating / maxRating.

Because this is one actor with a lean input, you can schedule it to monitor ratings and new reviews across both OTAs over time: set sortBy: "mostRecent" and slide fromDate to your last run date, and each nightly pass pulls only what is new, for a few cents. That is the difference between a one-off pull and a live reputation feed for a whole portfolio.

What comes back: 19 fields per review

Group	Fields
Core	`reviewId`, `hotelId`, `hotelName`, `hotelUrl`, `source` (makemytrip / goibibo), `submittedAt`
Rating	`overallRating`, `title`
Text	`reviewText`
Reviewer	`reviewer` (name, isAnonymous, reviewsWritten for Goibibo)
Trip context	`travelType`, `roomName`, `checkInMonth`
Signals	`usefulCount`, `imagesCount`, `images` (url + aiTags), `ownerResponse` (text, date)
AI-ready	`markdownContent`
Meta	`extractedAt`

A trimmed sample row (a MakeMyTrip review):

{
  "reviewId": "01KXQ014AHMXEWYHEBNBX9ZSCY",
  "hotelName": "Caravela Beach Resort",
  "source": "makemytrip",
  "submittedAt": "2026-07-17",
  "checkInMonth": "2026-07",
  "reviewer": {"name": "Sumit Khare", "isAnonymous": false, "reviewsWritten": null},
  "travelType": "COUPLE",
  "roomName": "Deluxe Room Sea View",
  "overallRating": 5,
  "title": "Wonderful beachside stay",
  "reviewText": "Amazing experience. Great service, good food and a beautiful property...",
  "imagesCount": 3,
  "images": [{"url": "https://.../photo.jpg", "aiTags": ["palm_tree", "grass", "house"]}],
  "ownerResponse": null,
  "markdownContent": "# Caravela Beach Resort review (MakeMyTrip)\n\n**Rating:** 5/5..."
}

Every run also returns a free Hotels row per property: hotelStars, propertyType, hotelAddress, pinCode, geoLat, geoLong, overallRating, ratingLabel, reviewsCount, and the clubbed subRatings. Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free MakeMyTrip and Goibibo sample (CSV/JSON) here: factden.com/sample-makemytrip. It includes rows from both brands so you can see the shape before you run anything.

FAQ

Is scraping MakeMyTrip or Goibibo legal? The reviews are publicly visible. As with any scraping, check the sites' Terms of Service and your local rules (India's DPDP Act, GDPR, and so on), and use the data responsibly.

Is there an official MakeMyTrip / Goibibo reviews API? No public one. MakeMyTrip Group offers no public reviews or hotel-data API for either brand, so scraping the public pages is the only route for most teams.

Can I get both brands' reviews for the same hotel? Yes. The two platforms share one hotel identity, so set reviewSource: "both" and you get the MakeMyTrip and Goibibo review feeds for the same property, each row tagged by source.

Can I scrape both MakeMyTrip and Goibibo at once? Yes, that is the point. Mix both URL types in one run (or paste raw hotel IDs) and the output uses the same schema, tagged by source.

Can I track a hotel's rating over time? Yes. Schedule the actor with sortBy: "mostRecent" and a sliding fromDate, and each run pulls only new reviews across both OTAs, cheaply, so you build a rating and sentiment trend without re-scraping everything.

How do I stop getting blocked? Plain requests get challenged on both brands. This is the hard part of the job, and the actor handles it for you, no proxy or account setup required.

Doing hotels across more platforms? See how to scrape Trip.com and Ctrip reviews.
Other FactDen scrapers: Trip.com & Ctrip Reviews Scraper and Expedia hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape Google Hotels Prices and Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Wed, 22 Jul 2026 10:43:01 +0000

Google Hotels is one of the richest travel datasets on the public web. For any property it shows a live price, a ladder of booking sources (Booking.com, Agoda, Expedia, the hotel's own site, and a dozen more), per-room rates, star class, guest ratings, and reviews. If you run a hotel, resell rooms, build a travel app, or feed a pricing model, that is exactly the data you want. Getting it out cleanly is the hard part.

This guide shows both ways: a runnable Python recipe, and a no-code shortcut with the Google Hotels Scraper that skips the hard parts entirely.

What you can pull

Live prices for a stay, with the full OTA rate ladder per hotel (every booking source and its price plus a deep booking link).
Per-room rates where the source shares them, so you see the room-level breakdown, not just a single "from" price.
Hotel facts: name, star class, guest rating, review count, property type, coordinates, and a stable Google entity token.
Guest reviews with exact publish dates and owner responses, in a separate dataset.
A price window: track prices across a range of dates for the same set of hotels.

Why scraping Google Hotels directly is hard

Google Hotels does not render its prices in simple HTML you can select with CSS. The data comes from Google's internal travel endpoints, which expect a specific TLS fingerprint and reject plain HTTP clients, and the price ladder is spread across several nested response containers that change shape by property and by date. On top of that, the values are keyed on internal Google entity tokens, not on anything you would guess from a URL. You can solve all of this, but it is real engineering, and it breaks whenever Google reshapes a response.

That is the whole reason a maintained actor exists: it absorbs the endpoint work and hands you a flat table.

DIY vs actor vs official API

	Write it yourself	Google Hotels Scraper (actor)	Official Google API
Setup	TLS impersonation, proxies, response parsing	Paste a destination and dates	Partner onboarding required
Prices and OTA ladder	You reverse-engineer it	Included, structured	Not available for open scraping
Reviews with dates	Separate endpoint to solve	Included, exact dates and owner replies	Not available
Maintenance	You own every breakage	Maintained for you	Contractual
Access	Free but you build it	Free tier, then pay per result	Gated to hotel owners and OTAs

There is no public Google API that lets you pull arbitrary hotel prices and reviews. The Hotel APIs Google does publish are gated to hotel owners and booking partners for their own inventory. For open collection, scraping is the route, and the choice is really "build and maintain it" versus "call an actor".

The no-code way (about a minute)

Create a free Apify account and open the actor page.
Click Try for free. The input is pre-filled with an example search.
Set your destination and dates, or paste exact hotels, then click Start.
Download the results from the Output tab as JSON, CSV, or Excel.

You can search a destination the way you would type it on Google Hotels (hotels in New York City, Paris 5 star hotels, resorts in Bali), or add exact hotels one per line: a hotel name, a Google Hotels URL, a Google Maps link, or an entity token.

Run it from Python

The actor runs on Apify, so you drive it with the Apify client. Install it with pip install apify-client, grab your token from Apify Settings, then:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run_input = {
    "searchQueries": ["hotels in New York City"],
    "checkInDate": "2026-08-15",
    "checkOutDate": "2026-08-16",
    "includePrices": True,
    "includeReviews": True,
    "maxReviews": 20,
    "maxResults": 10,
    "currency": "USD",
}

run = client.actor("factden/google-hotels-scraper").call(run_input=run_input)

# Hotels dataset: one row per hotel, with the price ladder nested on the row
for hotel in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(hotel["name"], hotel["leadPrice"], hotel["currency"], hotel["vendorCount"], "sources")

Reviews land in a separate named dataset, so you can process prices and reviews independently.

What a hotel row looks like

Trimmed from a real run (The Marmara Park Avenue, New York):

{
  "name": "The Marmara Park Avenue",
  "starClass": 5,
  "rating": 4.2,
  "reviewCount": 1070,
  "currency": "USD",
  "leadPrice": 252,
  "leadPriceTotal": 252,
  "vendorCount": 13,
  "offers": [
    {
      "source": "Booking.com",
      "checkInDate": "2026-11-15",
      "checkOutDate": "2026-11-16",
      "roomName": "Deluxe Queen (DLXQ)",
      "perNight": 287,
      "total": 287,
      "currency": "USD",
      "official": false,
      "bookingLink": "https://www.google.com/aclk?..."
    }
  ]
}

The key fields on each hotel row:

Field	What it is
`name`, `starClass`, `propertyType`	Hotel identity and class
`rating`, `reviewCount`	Aggregate guest score and how many reviews
`leadPrice`, `leadPriceTotal`	Cheapest per-night and total for the stay
`vendorCount`	How many booking sources sell this hotel
`offers[]`	The full priced table: one row per source, date, and room, each with a booking link
`vendors[]`	Booking directory (source plus link, no prices)
`markdownContent`	A self-contained summary block, ready to drop into an LLM or RAG pipeline

Reviews carry publishedAt (the exact date, not "2 months ago"), reviewText, rating, and ownerResponse.

Try it on a real dataset first

If you want to see the shape before running anything, there is a free sample Google Hotels dataset you can open in the browser, and the input, snippets, and field reference are on GitHub.

FAQ

Do I need a Google API key?
No. There is no key and no Google login. You only need an Apify token to run the actor.

Can I get prices for many dates at once?
Yes. Set a price-window end date and each date's prices are added to the offers table for the same set of hotels, which is how you track rate movements over time.

Does it return per-room rates or just a "from" price?
Per-room rates when the booking source shares them with Google. Sources that only hand Google a single lead price show one row; the per-room breakdown for those lives only on the OTA's own site.

How are reviews dated?
With the exact publish date decoded to ISO-8601, plus the hotel's owner response when there is one. Most relayed feeds only give you a relative string like "3 weeks ago".

Is it legal?
It collects publicly visible data. As with any scraping, use the output responsibly and follow the applicable terms and laws for your use case.

What does it cost?
There is a free tier to try it, then you pay per result. Discovery is cheap, prices are an opt-in per hotel per date, and reviews are billed per review.

Working hotel data across sites? The same team maintains an Expedia hotel reviews scraper, a Hotels.com reviews scraper, and a Trip.com and Ctrip reviews scraper, so you can line up reviews across Google, Expedia, Hotels.com, and Trip.com in the same format.

Full write-up with more examples: how to scrape Google Hotels.

How to Scrape Hotels.com Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 08:04:09 +0000

Hotels.com is part of Expedia Group, which is good news and a small headache. Good news: its reviews share the same backend as Expedia, Travelocity, Orbitz, and the rest, so one approach covers all of them. The headache: Hotels.com still uses its old /ho<id>/ URL format, which has to be resolved to the global property before you can pull anything.

This guide covers what you can pull from a Hotels.com review, why the DIY route is fiddly, working Python, a no-code shortcut, and a plain comparison. If you want the deeper reference, there is a full guide to scraping Hotels.com reviews too.

What you can pull from a review

Per review you can get:

Overall rating on the /10 scale plus a label, and category sub-ratings: Cleanliness, Service, Room comfort, Hotel condition, Amenities, Eco-friendliness
The full review text, detected language, and a translation flag
Stay context: check-in and check-out dates, travel companions, traveler category
Reviewer detail, verified flag, helpful votes, review photos
Owner responses, and a brandType field
An LLM-ready markdown block per review

Same clean, structured shape whether the review came from Hotels.com or any sibling brand.

Why scraping Hotels.com is hard

No public reviews API and no API key on offer. Scraping the public pages is the route.
The /ho<id>/ legacy id. A Hotels.com URL like hotels.com/ho119566/ does not carry the global property id directly; it has to be resolved first, or your request goes nowhere.
Reviews load dynamically, so a plain request returns a JavaScript shell, not the data.
Bot protection and rate limits on top.

So the work is resolving the legacy id, hitting the review endpoints, and staying unblocked, on repeat.

Three ways to get the data

	DIY Python	Hotels.com Reviews Scraper (actor)	Official API
Setup time	Hours to days	~30 seconds	Not available (no API key)
`/ho<id>/` resolution	You build it	Automatic	n/a
/10 ratings + 6 sub-ratings	Parse nested markup	Yes	n/a
Owner responses	Extra parsing	Yes	n/a
Cost	Proxies + eng time	Pay-per-result	n/a
Best for	One-off	Scheduled, at scale	Not an option

Option A: DIY in Python

A plain request to a Hotels.com page returns a shell, and the /ho<id>/ id is not the one the review endpoint expects:

import httpx

url = "https://www.hotels.com/ho119566/"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print("reviews" in r.text)   # usually False; ho-id must be resolved, reviews load separately

To get the data you resolve the /ho<id>/ to the global property id, call the review endpoints, and normalize the response. Fine for one hotel, several moving parts for a portfolio.

Option B: the no-code / API shortcut

When you want clean rows, the Hotels.com Reviews Scraper on Apify resolves the /ho<id>/ id for you and returns structured JSON. No login, no API key, no proxy setup.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/hotels-com-reviews-scraper").call(run_input={
    "hotelUrls": [
        "https://www.hotels.com/ho119566/",
        "242128"
    ],
    "maxReviewsPerHotel": 200,
    "sortBy": "newest",
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["overallRating"], review["reviewText"][:60])

Pass Hotels.com /ho<id>/ URLs or bare ids, and any other Expedia Group brand URL works too. sortBy takes newest, oldest, highestRating, or lowestRating; bound results with fromDate / toDate and minRating / maxRating.

What comes back: 27 fields per review

Group	Fields
Core	`reviewId`, `hotelId`, `hotelName`, `hotelUrl`, `source`, `brandType`, `submittedAt`
Rating	`overallRating` (/10), `ratingLabel`, `subRatings` (cleanliness, service, room comfort, condition, amenities, eco)
Text	`reviewText`, `language`, `isMachineTranslated`
Stay	`checkInDate`, `checkOutDate`, `travelCompanions`, `travelerCategories`, `roomTypeId`
Reviewer	`reviewerName`, `reviewerLocation`, `verified`, `isAnonymous`
Signals	`helpfulVotes`, `imagesCount`, `reviewPhotos`, `ownerResponse`
AI-ready	`markdownContent`

A trimmed sample row:

{
  "brandType": "Hotels.com",
  "hotelName": "The Venetian Resort",
  "overallRating": 9,
  "ratingLabel": "Wonderful",
  "subRatings": ["Cleanliness: 10", "Service: 9", "Hotel condition: 9", "Amenities: 10"],
  "reviewText": "Great suite, easy check-in, would stay again.",
  "travelerCategories": ["Couple"],
  "checkInDate": "2026-04-18",
  "checkOutDate": "2026-04-21",
  "language": "en",
  "helpfulVotes": 2
}

Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free Hotels.com sample (CSV/JSON) here: factden.com/sample-hotels. Load it into pandas and the /10 ratings and sub-ratings are ready to plot.

FAQ

Is scraping Hotels.com legal? The reviews are publicly visible. As with any scraping, check Hotels.com's Terms of Service and your local rules, and use the data responsibly.

Is there a Hotels.com API? No public reviews API and no key on offer. Scraping the public pages is the route.

What is the /ho<id>/ thing? Hotels.com uses a legacy property id in its URLs that does not match the global Expedia Group id. It has to be resolved first, which the actor does automatically.

Can I scrape Expedia and the other brands too? Yes. Hotels.com shares the Expedia Group backend, so the same actor handles Expedia, Travelocity, Orbitz, Wotif, CheapTickets, and ebookers, same schema, tagged by brandType.

How do I stop getting blocked? Residential proxies and slow pacing, or a tool that resolves the id and reads the review endpoints directly. Plain requests get a JavaScript shell.

Doing B2B software instead of hotels? See how to scrape G2 reviews.
Other FactDen scrapers: Expedia hotel reviews and Trip.com and Ctrip hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape Expedia Hotel Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 08:02:08 +0000

Here is the thing most people miss about scraping Expedia: Expedia, Hotels.com, Travelocity, Orbitz, Wotif, CheapTickets, and ebookers all run on the same backend. The same hotel shows up across all seven brands, and their reviews pool together. If you scrape only expedia.com, you leave most of the data on the table.

This guide covers what you can pull from an Expedia Group review, why the DIY route is fiddly, working Python, a no-code shortcut, and a plain comparison. If you want the deeper reference, there is a full guide to scraping Expedia reviews too.

What you can pull from a review

Per review you can get:

Overall rating on the /10 scale plus a label, and category sub-ratings: Cleanliness, Service, Room comfort, Hotel condition, Amenities, Eco-friendliness
The full review text, detected language, and a translation flag
Stay context: check-in and check-out dates, travel companions, traveler category
Reviewer detail, verified flag, helpful votes, review photos
Owner responses, and a brandType telling you which of the seven brands the review came from
An LLM-ready markdown block per review

The brandType field plus the shared backend are what let you pull every brand's reviews for a property in one pass.

Why scraping Expedia is hard

There is no public reviews API. Expedia Group does not offer one, so scraping the public pages is the route.
Seven brands, different URL formats. Six brands embed the global property id in a .h<id>. path; Hotels.com uses a legacy /ho<id>/ id that has to be resolved. A scraper built for one brand misses the rest.
Reviews are loaded dynamically, not sitting in the initial HTML, so a plain request gets you a shell.
Bot protection and rate limits on top.

So the work is resolving ids across brands, hitting the review endpoints, and staying unblocked, on repeat.

Three ways to get the data

	DIY Python	Expedia Reviews Scraper (actor)	Official API
Setup time	Hours to days	~30 seconds	Not available (no public reviews API)
All 7 Expedia Group brands	Build a scraper per brand	One run	n/a
/10 ratings + 6 sub-ratings	Parse nested markup	Yes	n/a
Owner responses	Extra parsing	Yes	n/a
Cost	Proxies + eng time	Pay-per-result	n/a
Best for	One-off	Scheduled, at scale	Not an option

Option A: DIY in Python

A plain request to a hotel page returns a JavaScript shell, not the reviews:

import httpx

url = "https://www.expedia.com/Las-Vegas-Hotels-Bellagio.h140596.Hotel-Information"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print("reviews" in r.text)   # usually False; reviews load via internal endpoints

To get the data you reverse-engineer the review endpoints, map each brand's URL format to the global property id (including the Hotels.com /ho<id>/ resolution), and normalize the responses. Fine for one hotel, a lot of moving parts for a portfolio.

Option B: the no-code / API shortcut

When you want clean rows, the Expedia Reviews Scraper on Apify takes any Expedia Group hotel URL (or a bare property id), resolves it, and returns structured JSON. No login, no proxy setup.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/expedia-hotel-reviews-scraper").call(run_input={
    "hotelUrls": [
        "https://www.expedia.com/Las-Vegas-Hotels-Bellagio.h140596.Hotel-Information",
        "140596",
        "https://www.hotels.com/ho115902/"
    ],
    "maxReviewsPerHotel": 200,
    "sortBy": "newest",
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["brandType"], review["overallRating"], review["reviewText"][:60])

Mix brands and bare ids freely. sortBy takes newest, oldest, highestRating, or lowestRating; bound results with fromDate / toDate and minRating / maxRating; and use reviewSources to keep only certain brands' reviews.

What comes back: 27 fields per review

Group	Fields
Core	`reviewId`, `hotelId`, `hotelName`, `hotelUrl`, `source`, `brandType`, `submittedAt`
Rating	`overallRating` (/10), `ratingLabel`, `subRatings` (cleanliness, service, room comfort, condition, amenities, eco)
Text	`reviewText`, `language`, `isMachineTranslated`
Stay	`checkInDate`, `checkOutDate`, `travelCompanions`, `travelerCategories`, `roomTypeId`
Reviewer	`reviewerName`, `reviewerLocation`, `verified`, `isAnonymous`
Signals	`helpfulVotes`, `imagesCount`, `reviewPhotos`, `ownerResponse`
AI-ready	`markdownContent`

A trimmed sample row:

{
  "brandType": "Expedia",
  "hotelName": "Bellagio",
  "overallRating": 10,
  "ratingLabel": "Exceptional",
  "subRatings": ["Cleanliness: 10", "Service: 10", "Hotel condition: 10", "Amenities: 10"],
  "reviewText": "Rooms were great. Buffet was awesome. Nice casino.",
  "travelerCategories": ["Friends"],
  "checkInDate": "2026-05-01",
  "checkOutDate": "2026-05-04",
  "language": "en",
  "helpfulVotes": 3
}

Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free Expedia sample (CSV/JSON) here: factden.com/sample-expedia. Load it into pandas and the /10 ratings and sub-ratings are ready to plot.

FAQ

Is scraping Expedia legal? The reviews are publicly visible. As with any scraping, check Expedia's Terms of Service and your local rules, and use the data responsibly.

Is there an official Expedia reviews API? No public one. Scraping the public pages is the route for most teams.

Can I scrape Hotels.com and the other brands too? Yes. All seven Expedia Group brands share the backend, so one actor covers Expedia, Hotels.com, Travelocity, Orbitz, Wotif, CheapTickets, and ebookers, with the same schema tagged by brandType.

Are ratings on the 5-star or 10-point scale? Expedia Group uses /10, and that is what the actor returns for overallRating and the sub-ratings.

How do I stop getting blocked? Residential proxies and slow pacing, or a tool that reads the review endpoints directly. Plain requests get a JavaScript shell, not the data.

Doing B2B software instead of hotels? See how to scrape G2 reviews.
Other FactDen scrapers: Hotels.com reviews and Trip.com and Ctrip hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape Trip.com and Ctrip Hotel Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 08:00:07 +0000

Most "Trip.com scrapers" only see half the picture. Trip.com is the international brand, but the same hotel usually has a much larger pool of reviews on its Chinese-domestic sibling, hotels.ctrip.com (携程). If you want honest guest sentiment for a property in Asia, you need both, and you need the Chinese ones translated.

This guide covers what you can pull from a Trip.com or Ctrip review, why the DIY route is a two-headed problem, working Python, a no-code shortcut, and a plain comparison. If you want the deeper reference, there is a full guide to scraping Trip.com and Ctrip reviews too.

What you can pull from a review

Per review you can get:

Overall rating and a label ("Outstanding"), plus sub-ratings: Cleanliness, Location, Service, Facilities
The review text, plus an English translation of Chinese reviews (reviewTextTranslated), and the detected language
Trip context: travel type (family, couple, business, solo), room name, check-in month
Reviewer detail: tier ("Review star"), lifetime review count, and for Ctrip the reviewer's Chinese province
Owner responses (text and date), useful counts, photo/video flags
An LLM-ready markdown block per review

The two things that separate a real dataset from a shallow one: the Chinese-domestic feed and its translation.

Why scraping Trip.com and Ctrip is hard

There is no public reviews API. Trip.com Group does not offer one, so scraping the public pages is the only route.
It is two systems, not one. trip.com (international) and hotels.ctrip.com / 携程 (Chinese) have different structures and languages. A scraper built for one usually misses the other, and the Chinese pool is often the larger one.
Chinese reviews need translation to be usable in an English pipeline, which is a whole extra step.
Bot protection and rate limits apply on both.

So the work is not parsing one page. It is handling both locales, translating, and staying unblocked, on repeat.

Three ways to get the data

	DIY Python	Trip.com & Ctrip actor	Official API
Setup time	Hours to days	~30 seconds	Not available (no public reviews API)
Both Trip.com + Ctrip feeds	Build two scrapers	One run	n/a
Chinese review translation	Add a translation step	Built in	n/a
Sub-ratings + owner responses	Parse nested markup	Yes	n/a
Cost	Proxies + eng time	Pay-per-result	n/a
Best for	One-off	Scheduled, at scale	Not an option

Option A: DIY in Python

A plain request to a hotel page tends to come back blocked or JavaScript-only:

import httpx

url = "https://www.trip.com/hotels/macau-hotel-detail-344983/galaxy-hotel/"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print(r.status_code)   # challenge / JS-rendered shell, reviews not in the HTML

Reviews load through internal endpoints, so you end up reverse-engineering those (per locale), then normalizing two different response shapes, then translating the Chinese text. Doable for one hotel, painful for a portfolio on a schedule.

Option B: the no-code / API shortcut

When you want clean rows, the Trip.com & Ctrip Reviews Scraper on Apify reads both feeds in a single run and hands back structured JSON with the Chinese reviews already translated. No login, no proxy setup.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/ctrip-trip-reviews-scraper").call(run_input={
    "startUrls": [
        "https://www.trip.com/hotels/macau-hotel-detail-344983/galaxy-hotel/",
        "https://hotels.ctrip.com/hotels/1286148.html"
    ],
    "maxReviewsPerHotel": 200,
    "sortBy": "mostRecent",
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["source"], review["overallRating"], review["travelType"])

Mix trip.com and hotels.ctrip.com URLs freely. sortBy takes mostRelevant, mostRecent, ratingHighToLow, or ratingLowToHigh, and you can bound results with fromDate / toDate and minRating / maxRating.

What comes back: 23 fields per review

Group	Fields
Core	`reviewId`, `hotelId`, `hotelName`, `hotelUrl`, `source` (trip.com / ctrip), `submittedAt`
Rating	`overallRating`, `ratingLabel`, `subRatings` (Cleanliness, Location, Service, Facilities)
Text	`reviewText`, `reviewTextTranslated`, `isMachineTranslated`, `language`
Reviewer	`reviewer` (name, tier, lifetime reviews, Chinese province for Ctrip)
Trip context	`travelType`, `roomName`, `checkInMonth`
Signals	`recommends`, `usefulCount`, `imagesCount`, `hasVideo`, `ownerResponse` (text, date)
AI-ready	`markdownContent`

A trimmed sample row (a Chinese review, translated):

{
  "source": "ctrip",
  "hotelName": "Galaxy Hotel",
  "overallRating": 5,
  "ratingLabel": "Outstanding",
  "subRatings": ["Cleanliness: 5.0", "Location: 5.0", "Service: 5.0", "Facilities: 5.0"],
  "language": "zh",
  "reviewText": "酒店很棒，服务一流，位置方便。",
  "reviewTextTranslated": "Great hotel, first-class service, convenient location.",
  "isMachineTranslated": true,
  "travelType": "Family",
  "reviewer": {"tier": "Rising review star", "lifetimeReviews": 4},
  "ownerResponse": {"text": "感謝您的正面評價...", "date": "2026-06-15"}
}

Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free Trip.com/Ctrip sample (CSV/JSON) here: factden.com/sample-trip. It includes both English and translated-Chinese reviews so you can see the shape.

FAQ

Is scraping Trip.com or Ctrip legal? The reviews are publicly visible. As with any scraping, check the sites' Terms of Service and your local rules, and use the data responsibly.

Is there an official Trip.com / Ctrip reviews API? No public one. Scraping the public pages is the only route for most teams.

Do I get the Chinese reviews in English? Yes. Ctrip reviews come with reviewTextTranslated and an isMachineTranslated flag, alongside the original.

Can I scrape both Trip.com and Ctrip at once? Yes, that is the point. Mix both URL types in one run and the output uses the same schema, tagged by source.

How do I stop getting blocked? Residential proxies and slow pacing, or a tool that reads the internal endpoints per locale. Plain requests will get challenged.

Doing B2B software instead of hotels? See how to scrape G2 reviews.
Other FactDen scrapers: Expedia hotel reviews and Indeed jobs and salary data.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape Indeed Jobs in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 07:56:17 +0000

Scraping Indeed sounds simple until you try it. There is no public API anymore, the site sits behind Cloudflare, and the salary you actually want arrives as a messy string like "$120,000 - $160,000 a year". This guide covers what you can pull from an Indeed job, why the DIY route is harder than it looks, working Python, a no-code shortcut, and a straight comparison so you can pick a lane. If you want the deeper reference, there is a full guide to scraping Indeed jobs too.

What you can pull from an Indeed job

Per listing you can get:

Title, company, and location (plus city, state, country code, and lat/long)
Salary, parsed into numbers: salaryMin, salaryMax, salaryPeriod, currency, not just the raw string
Job type, remote/hybrid flag, and the full description text
Benefits, date posted, easy-apply and urgent-hire flags
Company data on demand: rating, review count, industry, size, revenue, founded, website, and socials

Parsed salary and company enrichment are the two things people actually come for, and the two things a naive scrape gets wrong.

Why scraping Indeed is hard

Three reasons DIY is more work than it looks:

There is no official API. Indeed retired its public job-search API to new developers years ago, so there is no supported programmatic route. Scraping is the only option left.
Cloudflare and rate limits. A plain request to a search results page tends to return a 403 or a challenge. Hammer it and you get blocked fast.
Messy salary strings. Indeed shows salary as free text. Turning "$120,000 - $160,000 a year" into clean salaryMin/salaryMax/period reliably, across currencies and 40 countries, is fiddly.

So the work is not the parsing of one page. It is staying unblocked, normalizing salary, and pulling company data from separate pages, on repeat.

Three ways to get Indeed job data

	DIY Python	Indeed Jobs Scraper (actor)	Indeed Official API
Setup time	Hours to days	~30 seconds	Not available (retired)
Anti-bot handled	You (Cloudflare, blocks)	Built in	n/a
Salary as numbers	You parse the string	Yes: `salaryMin` / `salaryMax`	n/a
Company data	Separate scrape	Free, one toggle	n/a
Countries	You handle localization	40+	n/a
Cost	Proxies + eng time	Pay-per-result	n/a
Best for	One-off search	Scheduled, at scale	Not an option

The official-API column is the real story: since Indeed closed its API, a scraper (yours or a ready-made one) is the only way to get this data programmatically.

Option A: DIY in Python

See the wall first:

import httpx

url = "https://www.indeed.com/jobs?q=software+engineer&l=New+York"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print(r.status_code)   # often 403 / Cloudflare challenge

If you do get HTML back, the job cards parse roughly like this (selectors change often, so re-check against the live page):

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, "html.parser")
for card in soup.select("div.job_seen_beacon"):
    title = card.select_one("h2.jobTitle")
    comp  = card.select_one("[data-testid='company-name']")
    sal   = card.select_one("[data-testid='attribute_snippet_testid']")
    print(
        title.get_text(strip=True) if title else None,
        comp.get_text(strip=True) if comp else None,
        sal.get_text(strip=True) if sal else None,   # raw string, still needs parsing
    )

Then you still have to normalize that salary string into numbers, follow each job to its full description, and scrape the company page separately. For one search that is fine. For a pipeline, the upkeep adds up.

Option B: the no-code / API shortcut

When you just want clean rows, the Indeed Jobs Scraper on Apify talks to Indeed's structured endpoints (no fragile HTML path), handles the blocking, and returns parsed JSON. No login, no proxy setup.

From Python:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/indeed-jobs-scraper").call(run_input={
    "query": "software engineer",
    "location": "New York, NY",
    "country": "US",
    "maxItems": 200,
    "salaryMin": 120000,     # native salary-range filter
    "scrapeCompany": True,   # attach company profiles for free
})
for job in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(job["title"], job["company"], job["salaryMin"], job["salaryMax"])

Set country to any of 40+ codes (US, GB, IN, DE, SG, ...), filter by remote, datePosted, jobType, or experienceLevel, or pass startUrls / jobKeys to fetch specific listings.

What comes back: 26 structured fields per job

Group	Fields
Core	`title`, `company`, `location`, `city`, `state`, `countryCode`, `url`, `jobKey`
Salary	`salary` (raw), `salaryMin`, `salaryMax`, `salaryPeriod`, `currency`
Job detail	`jobType`, `remote`, `occupations`, `benefits`, `description`, `datePosted`, `isUrgentHire`, `easyApply`
Geo	`latitude`, `longitude`
Company (with `scrapeCompany`)	`companyRating`, `companyReviewCount`, `companyPageUrl` + a full company profile (industry, size, revenue, founded, website, socials)

A trimmed sample row:

{
  "title": "Senior Software Engineer",
  "company": "Plaid",
  "location": "New York, NY",
  "salary": "$120,000 - $160,000 a year",
  "salaryMin": 120000,
  "salaryMax": 160000,
  "salaryPeriod": "year",
  "currency": "USD",
  "jobType": ["Full-time"],
  "remote": "remote",
  "benefits": ["Health insurance", "401(k)"],
  "datePosted": "2026-06-10",
  "companyRating": 4.2,
  "url": "https://www.indeed.com/viewjob?jk=6d50b3ebeb3fb122",
  "jobKey": "6d50b3ebeb3fb122"
}

The numeric salary fields and the on-demand company profile are what make this usable for labor-market analysis without a cleanup pass. Full field list and snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data first? There is a free Indeed jobs sample (CSV/JSON) here: factden.com/sample-indeed. Drop it into pandas and the parsed salary columns are ready to plot.

FAQ

Is scraping Indeed legal? The listings are publicly visible. As with any scraping, check Indeed's Terms of Service and your local rules (especially around personal data), and use the data responsibly.

Does Indeed have an API? Not a usable one. Indeed closed its public job-search API to new developers, which is exactly why people scrape the public pages now.

How do I stop getting blocked? Residential proxies, real headers, and slow pacing, or a tool that talks to Indeed's structured endpoints instead of scraping HTML. Plain requests to the search pages will get challenged.

Can I get salary as numbers? Yes. salaryMin, salaryMax, salaryPeriod, and currency are parsed from the raw string, so you can filter and aggregate directly.

Can I get company data too? Yes. Turn on scrapeCompany and each job carries the company's rating, size, industry, revenue, and profile links, no second scrape needed.

Which countries are supported? 40+, including US, UK, Canada, India, Germany, Singapore, Australia, and more, via the country code.

Doing competitive intelligence instead of hiring data? See how to scrape G2 reviews.
Other FactDen scrapers: G2 software reviews and Trip.com and Ctrip hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

How to Scrape G2 Reviews in 2026 (Python + a No-Code Shortcut)

Factden — Tue, 14 Jul 2026 07:52:34 +0000

If you have tried to scrape G2 reviews with a quick requests.get(), you already know how it goes: a 403, a CAPTCHA, or a blank page. G2 is one of the tougher public sites to pull at scale. But the data behind it (ratings, structured pros and cons, and which competitor a reviewer switched away from and why) is worth the trouble if you do any competitive intelligence.

This guide covers what a G2 review actually contains, why the naive approach fails, a Python path that works, a no-code shortcut, and a plain comparison so you can pick the right route. If you want a deeper field reference, there is a longer guide to scraping G2 reviews too.

What a G2 review actually contains

One G2 review holds more than it looks. Per review you can pull:

Overall rating (1 to 5) plus six sub-ratings (ease of use, ease of setup, quality of support, meets requirements, and more)
Structured pros, cons, and "problems solved" as separate fields, not one text blob
Switching data: the competitor the reviewer came from, and why they left (the field most people are actually after)
Reviewer context: industry, role, company size, country
Dates, verification, and the incentivized flag

The sub-ratings and switching fields are the reason to scrape G2 instead of skimming it by hand.

Why scraping G2 is hard: DataDome

G2 runs DataDome bot protection behind a Cloudflare CDN. In practice:

A plain requests or httpx GET comes back 403 almost immediately.
Swapping the User-Agent does nothing. DataDome fingerprints TLS, headers, and behavior.
Datacenter IPs get flagged fast. You need residential proxies plus a real browser fingerprint, or a scraping API that brings both.

You can absolutely scrape G2 yourself. The hard part is doing it reliably, at scale, without babysitting proxies. That maintenance tax is bigger than the parsing.

Three ways to get G2 review data

	DIY Python	G2 Reviews Scraper (actor)	G2 Official API
Setup time	Hours to days	~30 seconds	Weeks (sales + contract)
Anti-bot handled	You (proxies + browser)	Built in	n/a
Structured fields	You parse them	27, incl. switching data	Limited
Sub-ratings + "switched from"	Hard to parse	Yes, resolved to product names	No
Cost	Proxies + eng time	$0.004/row, ~1,250 free	Enterprise contract
Best for	One-off, few pages	Scheduled, at scale	Large enterprises

Option A: DIY in Python

See the wall for yourself first:

import httpx

url = "https://www.g2.com/products/slack/reviews"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"})
print(r.status_code)   # 403 (DataDome)

Getting past it needs a residential proxy and a browser-grade fingerprint (a stealth headless browser, or a scraping API). Once you have the HTML, parsing review cards with BeautifulSoup looks roughly like this:

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, "html.parser")
for card in soup.select("[itemprop='review']"):
    rating = card.select_one("[itemprop='ratingValue']")
    title  = card.select_one("[itemprop='name']")
    body   = card.select_one("[itemprop='reviewBody']")
    print(
        rating.get("content") if rating else None,
        title.get_text(strip=True) if title else None,
        (body.get_text(strip=True) if body else "")[:120],
    )

Two things that will cost you time:

Selectors drift. G2 changes its markup, so re-check against the live page.
Sub-ratings and switching data sit in nested markup that is fiddly to parse consistently.

DIY is fine for a one-off across a handful of pages. For hundreds of products on a schedule, the proxy and anti-bot upkeep usually costs more than it saves.

Option B: the no-code / API shortcut

When you just want clean rows, the G2 Reviews Scraper on Apify handles the DataDome, proxy, and parsing layer and hands back structured JSON. No login, no proxy setup.

From Python, using the Apify client:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("factden/g2-reviews-scraper").call(run_input={
    "mode": "reviews",
    "startUrls": ["https://www.g2.com/products/slack/reviews", "notion"],
    "maxReviewsPerProduct": 200,
    "sortReviews": "helpful",
})
for review in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(review["overallRating"], review["reviewTitle"], review.get("previousCompetitors"))

Pass full product URLs or bare slugs (notion, slack). There is also a Products mode that finds competitor products by keyword before you pull their reviews.

What comes back: 27 structured fields

Group	Fields
Ratings	`overallRating`, `subRatings` (ease of use, setup, support, meets requirements)
Structured text	`pros`, `cons`, `problemsSolved`, `recommendations`, `reviewText`
Switching / battlecard	`didSwitchFromCompetitor`, `previousCompetitors`, `whySwitched`
Reviewer	`reviewerIndustry`, `reviewerRole`, `companySize`, `reviewerCountry`, `reviewerName`
Meta	`isIncentivized`, `helpfulVotes`, `submittedAt`, `reviewUrl`, `productSlug`, `productName`
AI-ready	`markdownContent` (a self-contained markdown block per review, for RAG / vector DBs)

A trimmed sample row, so you can see the shape:

{
  "productName": "Slack",
  "overallRating": 5,
  "subRatings": {"easeOfUse": 6, "easeOfSetup": 6, "qualityOfSupport": 5, "meetsRequirements": 6},
  "reviewTitle": "Runs our whole company",
  "pros": "Channels keep every project in one place.",
  "cons": "Notifications get noisy at scale.",
  "didSwitchFromCompetitor": true,
  "previousCompetitors": ["Microsoft Teams"],
  "whySwitched": "Better threads and a faster mobile app.",
  "companySize": "Mid-Market",
  "reviewerRole": "IT Administrator",
  "isIncentivized": false
}

The two fields most scrapers drop, previousCompetitors + whySwitched (resolved to real product names) and the per-dimension subRatings, are what make this useful for battlecards. Full field list and copy-paste snippets are in the GitHub repo.

Grab a free sample dataset

Want to see the data before running anything? There is a free G2 review sample (CSV/JSON) here: factden.com/sample (also mirrored on HuggingFace and Kaggle). Load it into pandas and the sub-ratings and switching fields show up straight away.

FAQ

Is scraping G2 reviews legal? The reviews are publicly available. As with any scraping, check G2's Terms of Service and your local rules (GDPR and similar for personal data), and use the data responsibly.

Does G2 have an API? Yes, but it is enterprise-tier: a sales call, a contract, and procurement. For most teams, scraping the public pages, or using a ready-made actor, is the faster route to the same public data.

How do I stop getting blocked? Residential proxies, a real browser fingerprint, and human-like pacing, or a tool that bundles all three. Datacenter IPs and plain requests will not survive DataDome.

How much does it cost? DIY costs proxies plus your time. The actor is pay-per-result at $0.004 per row, with about 1,250 rows free on Apify's $5 new-account credit.

Can I get the "switched from" competitor data? Yes. That is the previousCompetitors and whySwitched fields, resolved to product names. It is the reason most people scrape G2 in the first place.

Can I filter out incentivized (gift-card) reviews? Yes. Every row carries an isIncentivized flag, so you can recompute a rating on organic reviews only. G2's own UI will not let you do that.

I mapped switching and sub-ratings across 25 tools in the 2026 SaaS Battlecard Atlas (50,000 reviews).
Other FactDen scrapers: Indeed jobs and salary data and Trip.com and Ctrip hotel reviews.

Questions, or a field you wish it extracted? Drop a comment.

Found this useful? A ⭐ on the GitHub repo (https://github.com/factden/g2-reviews-scraper) helps other people find it.

DEV Community: Factden

How to Scrape India MCA Company and Director Contacts in 2026 (Python + a No-Code Shortcut)

What you can pull

Why scraping MCA directly is hard

DIY vs actor vs official data

The no-code way (about a minute)

Run it from Python

What the output looks like

Try it on a real dataset first

FAQ

Related

How to Scrape Booking.com Hotels, Prices and Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull

Why scraping Booking.com is hard

Three ways to get the data

Option A: DIY in Python

Option B: the no-code / API shortcut

What comes back

Grab a free sample dataset

FAQ

Related

How to Scrape Agoda Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull from a review

Why scraping Agoda reviews is hard

Three ways to get the data

Option A: DIY in Python

Option B: the no-code / API shortcut

Track reputation over time

What comes back: the fields per review

Grab a free sample dataset

FAQ

Related

How to Scrape Airbnb Prices, Availability and Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull

Why scraping Airbnb directly is hard

DIY vs actor vs AirDNA

The no-code way (about a minute)

Run it from Python

What the output looks like

Try it on a real dataset first

FAQ

Related

How to Scrape TripAdvisor Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull from a review

Why scraping TripAdvisor is hard

Three ways to get the data

Option A: DIY in Python

Option B: the no-code / API shortcut

Track reputation over time

What comes back: 16 fields per review

Grab a free sample dataset

FAQ

Related

How to Scrape MakeMyTrip and Goibibo Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull from a review

Why scraping MakeMyTrip and Goibibo is hard

Three ways to get the data

Option A: DIY in Python

Option B: the no-code / API shortcut

What comes back: 19 fields per review

Grab a free sample dataset

FAQ

Related

How to Scrape Google Hotels Prices and Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull

Why scraping Google Hotels directly is hard

DIY vs actor vs official API

The no-code way (about a minute)

Run it from Python

What a hotel row looks like

Try it on a real dataset first

FAQ

Related

How to Scrape Hotels.com Reviews in 2026 (Python + a No-Code Shortcut)

What you can pull from a review

Why scraping Hotels.com is hard

Three ways to get the data

Option A: DIY in Python

Option B: the no-code / API shortcut

What comes back: 27 fields per review