I run a visa agency in Bali, and the other half of my week goes into Apify Actors. One of them reads Skyscanner: you give it a route and a date, it gives you one row per itinerary with the price, the seller behind that price, and the full list of every other seller with its own price, rating and booking link.
The seller list is the reason the Actor exists, and it is also where the data stops behaving like a price list. Below are 90 flight rows I can show you: 80 from my nightly health check on JFK → LHR (four runs, 10–14 November, 20 rows each) and 10 from a fresh DPS → SIN run today. Three things in them will break a naive pipeline.
20 rows were 8 planes
The 10 November run asked for 20 cheapest itineraries, JFK → LHR, 10 December. Deduplicate those 20 rows on departure time, arrival time and the operating airline and you are left with 8 physical departures. One of them shows up five times:
| row | flight number | marketing airline | operating | sellers | price |
|---|---|---|---|---|---|
| 1 | BA1515 | British Airways | American Airlines | 16 | $241.50 |
| 4 | AS6906 | Alaska Airlines | American Airlines | 6 | $241.50 |
| 7 | AY3774 | Finnair | American Airlines | 17 | $241.50 |
| 8 | IB4220 | Iberia | American Airlines | 20 | $241.50 |
| 13 | AA104 | American Airlines | American Airlines | 1 | $299.50 |
Same aircraft, same 21:15 push-back, same 09:15 arrival. Five marketing codes, so Skyscanner has five sellable itineraries, so my Actor emits five rows — that part is correct, the field legs[].segments[].operatingAirline tells you who actually flies it.
What I did not expect: the row carrying the operating carrier's own code is the most expensive of the five, by $58, and it is the only one of the five with a single seller. American sells its own seat at $299.50 direct; the same seat under a British Airways, Alaska, Finnair or Iberia number is on sixteen to twenty agency shelves starting at $241.50.
It is not a fluke of that date. The next night, 11 December, the same 21:15 metal came back under five codes at $298, $300.52, $307, $309.50 and $311.70 — a $13.70 spread on one seat, and the $309.50 is again the AA-coded row. On DPS → SIN the pattern repeats in miniature: Scoot's 16:25 departure is TR239 at $73 and SQ8519 at $107 — a Singapore Airlines code on a Scoot aircraft, 47% more for the same cabin.
So if you are counting "how many flights are there on this route", count departures, not rows:
def plane(r):
return (r["departAt"], r["arriveAt"],
tuple(s["operatingAirline"] for lg in r["legs"] for s in lg["segments"]))
planes = {}
for r in rows:
planes.setdefault(plane(r), []).append(r)
# JFK-LHR: 20 rows -> 8 planes. DPS-SIN: 10 rows -> 8 planes.
And if you are picking the cheapest seat, keep the duplicates — the cheap code is the point.
Twelve of twenty rows had the same price to the cent
In that same 10 November run, 12 of the 20 rows cost exactly $241.50, and all twelve name the same seller: Thrive Travel, Skyscanner rating 3.0 from 289 ratings. Those twelve rows are the three AA departures above (21:15, 10:10, 18:25) times four marketing codes. The airlines' own prices for the same three planes that night were $299.50 and $308.50.
One agency had put a single flat number on every variant of every AA departure that day, and because the Actor sorts by price, that number took the entire top of the result. Sorted "cheapest first", the first twelve rows told me about one seller's pricing policy, not about the route.
That is worth a guard in any monitoring job. If you track "cheapest fare on this route" day over day, you are partly tracking which agency is currently undercutting everyone with a flat rate — and whether it is still in business next week. The honest reference number is the airline's own price from the seller list, which the row hands you:
own = next((a["price"] for a in r["agents"] if a["isAirline"]), None)
The expensive end is one seller, and it barely moves
Every multi-seller row has a price ladder. The ladders in my 90 rows end in the same place. Of the 62 rows with more than one seller, 42 had the identical most expensive seller: BudgetAir.
Here is the full ladder of the cheapest DPS → SIN row — a 2h55m Scoot hop:
| seller | price | rating |
|---|---|---|
| Kiwi.com | $73.00 | 4.49 (5,514) |
| Scoot (airline) | $76.73 | 4.31 (80) |
| Super.com | $78.45 | 4.07 (1,521) |
| eDreams | $78.97 | 3.98 (2,320) |
| Booking.com | $80.00 | 4.91 (5,104) |
| … nine more between $80 and $109 | ||
| BudgetAir | $1,404.30 | 3.89 (956) |
Across the seven DPS rows BudgetAir appeared in, it quoted $1,403.20 – $1,435.30 while the row prices ran $73–$98. Across 39 JFK → LHR rows it quoted $1,638.60 – $1,836.00 while the rows ran $241.50–$493.08. Whatever that number is, it is almost flat per route and nearly independent of which flight it is attached to — on DPS → SIN it sits a median 12.7× above the second-highest seller on the same row.
The consequence is arithmetic. On that Scoot row the fifteen sellers average $173.33 and the median is $82.04: one entry doubles the mean. And "save 95% by booking with the cheapest seller" is technically what the row says ($73 vs $1,404.30) and tells the reader nothing.
Two defences, both one line. Use the median, never the mean, when you summarise a seller list. Or cap the list — agents is sorted cheapest-first and maxAgentsPerFlight slices from the front, so maxAgentsPerFlight: 10 keeps the real market and drops the tail. Capping by count is not the same as sanity-checking a price, so if you need the ladder intact, clip the outlier yourself at something like 4× the cheapest.
A third of the rows cannot be compared at all
Of my 90 rows, 28 carried exactly one seller. On JFK → LHR those sole sellers were American Airlines (14 rows), Delta (6), Virgin Atlantic (5) and Hop2 (1) — the airline itself, selling at its own price, no agency competing. On DPS → SIN the two single-seller rows both belong to Sky-tours, rating 2.07 from 344 ratings, at $107.
Which is the other thing to keep straight: rating is Skyscanner's rating of the seller, not of the airline or the flight. A 2.07 means people disliked the booking experience, and it sits in a row describing a Singapore Airlines-coded flight. Do not aggregate it into anything that looks like an airline quality score.
The price calendar answers a different question
The same Actor has a calendar mode: one row per departure day with the cheapest fare Skyscanner has on record and a price level. Fourteen days of DPS → SIN came back $62.37–$83.62, and not one day was labelled low — nine medium, five high. The banding is computed against a much longer window than the one you asked for, so in a two-week slice "no cheap days" may simply mean "you are looking at two weeks".
I also ran both modes at the cheapest day in the window as a cross-check. The calendar put 18 December at $69.76 and flagged it isCheapestDay. A live search of that exact date, twenty minutes earlier, found 139 itineraries and a cheapest of $73.00 — 4.6% above. The calendar is a date-picking signal, not a bookable price: pick the day from the calendar, then search that day.
Calling it, and what it costs
curl -X POST "https://api.apify.com/v2/acts/lergassy~skyscanner-flights-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H 'content-type: application/json' \
-d '{"origin":"DPS","destination":"SIN","departureDate":"+30",
"maxResultsPerSearch":10,"sortBy":"cheapest","currency":"USD","market":"US"}'
departureDate: "+30" means thirty days from today, which is what you want in a cron job — the date never goes stale. The 10-row run above took 25 seconds and cost $0.012 (one search event plus ten rows); the 14-day calendar cost $0.0152. The last row of every run is a free notice row that states what was delivered and charged, so filter on type != "notice" before you count anything. The Actor is Skyscanner Scraper.
Where it is the wrong tool: it does not book, it has no seat maps, baggage fees or fare rules, and prices move under you — two runs nine hours apart on the same date returned $300.52 and $300.54 with 14 and 15 sellers. If you need a stable price to quote a client, quote the airline's own price from the seller list and re-read it the hour you send the invoice.
Top comments (0)