DEV Community: Matvey

46 Runs, One Route, the Same Cheapest Flight Every Time. The Two Sources Never Once Agreed on Its Price

Matvey — Sat, 07 Nov 2026 13:14:14 +0000

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them does something that sounds trivial and is not: it runs the same flight search on Google Flights and KAYAK at once, puts both answers in one dataset, and adds a row saying which source was cheaper and by how much.

That verdict row is the product. It is also the part I trusted least, so I let my own health check hammer it: the same input every day since 6 October — DPS → SIN, one way, 15 March 2027, economy, 30 rows per source. 46 successful runs between 6 October and 6 November. This is what a month of that looks like.

Both sources agree on the flight. Neither agrees on the price

In 46 of 46 runs the cheapest row from Google Flights and the cheapest row from KAYAK are the same physical flight: Jetstar JQ86, departing 14:55, landing 18:00, A321neo. No matching heuristics needed — same flight number, same minute.

In 46 of 46 runs the two prices differ. Not once in a month did they land on the same number for that seat.

The asymmetry in how they differ is the interesting part. Google's price for JQ86 across the month took three distinct values:

Google Flights, JQ86	runs
$60	23
$59	1
$65	22

One step, on 27 October, between a run at 00:10 and a run at 20:07. Otherwise flat for three weeks at a time.

KAYAK's price for the same flight, same runs, took eleven distinct values between $57 and $76 — a 33% spread — and moved inside the hour:

	20:07	20:09
27 Oct	$67	$63

	20:10	20:27
31 Oct	$61	$68

	20:05	20:11
5 Nov	$68	$61

So the verdict flipped: Google cheaper in 25 runs, KAYAK in 21. A coin toss, and the coin is KAYAK's, because Google's side of it barely moved.

The seller column explains all of it

KAYAK is a metasearch engine, so its price is not the airline's price — it is whichever agency KAYAK had surfaced at the moment the run stopped reading. The row says who:

seller on the JQ86 row	prices seen
Jetstar (direct)	$57, $61, $63
Booking.com	$62, $63, $67, $68
Gotogate	$68
Kiwi.com	$69, $74, $76
CheapOair	$73
Justfly.com	$75, $76
Priceline	$79, $83, $84, $85

Google's rows carry no seller at all — seller was null on all 29 Google rows of the latest run, because Google Flights hands off to the airline or an agency after the click rather than quoting one on the results page.

Part of KAYAK's spread is my own fault, and I wrote that up separately: KAYAK's search is a poll that fills in with sellers while you read it, and a row cap that is satisfied early stops the read early, so price is the minimum of a partially arrived set. With the cap fixed at 30 for all 46 runs, the remaining movement is sellers rotating in and out of the cheapest slot.

Either way the practical rule is the same, and it is not "KAYAK is dearer":

The gap between two metasearch engines is mostly a seller-rotation gap, not a fare gap. A single run tells you which source was cheaper in that minute. If you are publishing "source X is 8% cheaper on this route", you need a schedule, not a run.

And across the rest of the board, the direction reverses

Match every flight that both sources returned in the same run, on flight numbers plus departure minute, taking each source's cheapest row per flight. Excluding JQ86, that is 530 matched pairs over the month:

KAYAK cheaper on 443
Google cheaper on 26
identical on 61 (11.5%)
median gap −3.1% in KAYAK's favour

So KAYAK is systematically a few percent cheaper on the body of the result set, and still loses the headline on the single cheapest flight more than half the time. If your pipeline reads only the floor — the cheapest row, the verdict's cheaperSource — you are sampling the one row in the dataset with the most variance in it. Cheap insurance: compute your own median over matched flights alongside the verdict.

Three joins that look right and are not

Join on flight numbers and the departure minute, nothing else. Not on aircraft: KL836 in the latest run is Boeing 777 to Google and Boeing 777-300ER to KAYAK, same seat, same price. Not on departAt as a string either — Google sends 2027-03-15T14:55, KAYAK sends 2027-03-15T14:55:00. Slice to 16 characters first. (The Actor's own merge mode uses exactly that key.)

Expect the same flight twice from KAYAK. Over the 46 runs KAYAK sent 43 duplicate rows — same flight, different seller, different price — and Google sent none. Today's control run has JQ86 twice: $61 from Jetstar and $67 from Booking.com. Deduplicate keeping the minimum, not the first row. And do not use sellersForThisFlight as a seller count while you do it: on those duplicate rows it reads 1.

Codeshares do not merge, by design. QF5659 appears in KAYAK's rows in all 46 runs and in Google's in none — because Google folds it into JQ86's legs[].operatedBy: "Qantas QF5659". Same A321neo, same 14:55 departure. As QF5659 it was priced $83 to $116; as JQ86, $61 to $68. A "cheapest flight per aircraft" report has to resolve that itself; no source will do it for you.

One more thing worth knowing before you write a dashboard: the two sources are not looking at the same shelf. Per run the overlap between Google's ~28 flights and KAYAK's ~29 was on average 12.5 flights, ranging 8 to 16. Each source's top-30 is its own window on the day, ranked by its own idea of "best".

The merge mode, and where it helps

Set outputMode to byFlight and one row carries every source's price for that flight. LHR → CDG on 4 December, Google Flights plus Momondo, 8 rows each → 12 flight rows, 4 of them merged:

{
  "type": "flight",
  "airline": "British Airways", "flightNumbers": ["BA310"],
  "departAt": "2026-12-04T13:20",
  "price": 111, "currency": "EUR", "cheapestSource": "google-flights",
  "priceBySource": { "google-flights": 111, "momondo": 132 },
  "sellerBySource": { "google-flights": null, "momondo": "eDreams" },
  "priceGap": 21
}

Momondo priced all four shared BA flights at exactly €132 against Google's €111 — a flat €21, which is a seller margin, not four independent fares. The other 8 of 12 rows came from one source only.

Calling it

curl -X POST "https://api.apify.com/v2/acts/lergassy~flight-price-comparison/run-sync-get-dataset-items?token=APIFY_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"origin":"DPS","destination":"SIN","departureDate":"2027-03-15",
       "sources":["google-flights","kayak"],"maxResultsPerSource":5,"currency":"USD"}'

Today's control run of exactly that, from a clean account: 28.6 seconds, 9 flight rows plus a free verdict row. The verdict read {"google-flights": 65, "kayak": 61} — KAYAK cheaper, the other way round from yesterday's run, which is the whole point of this article.

Cost: the parent run charged $0, and the two source Actors it started on the account charged $0.005 and $0.011. That is how Flight Price Comparison is wired — each source is its own Actor run on your own account, billing its own rows, with no comparison fee on top. The side effect is a concurrency ceiling: four sources plus the parent is five concurrent runs, the entire Apify Free-plan allowance, so the Actor retries a source that gets refused (8 attempts, 15 seconds apart) instead of dropping it. Run times over the 46 health checks: median 40 s, range 12–126 s.

A failing source writes a free error row and the others still return, so a run is never all-or-nothing.

When this is the wrong tool

It needs an origin, a destination and a date — there is no browse mode, so "cheapest week in March from Bali" is a different job. Fares are point-of-sale dependent, and these reads are anonymous and currency-forced, so your own booking screen may differ. Round-trip rows are the total for both directions, and departAt/arriveAt span the whole trip — subtract them and you get a holiday, not a flight; use legs or the outbound*/inbound* columns.

And treat gapPercent with care when the sources are shaped differently. On a STN → DUB test the verdict reported Ryanair cheaper by 177% — true in that both numbers are the cheapest row of their source, misleading in that Ryanair's row is the cheapest one-way fare of the day excluding bags while KAYAK's was a priced round trip for two adults. The verdict compares the cheapest rows it was given. It does not promise they are the same product; on a single-source pair like Google plus KAYAK they usually are, and I checked that 46 times before saying so.

Trump Posted 206 Times on 3 November. My Daily Read Saw 28 of Them

Matvey — Sat, 07 Nov 2026 05:14:52 +0000

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them reads Truth Social: you hand it usernames or post links and it returns one row per post — text, exact publish time, likes, ReTruths, replies count, media files, link preview card, quoted post, author. No account, no cookies, no login. Truth Social runs a Mastodon-compatible API, and three of its endpoints answer a logged-out visitor: account lookup, an account's statuses, and a single status.

I have been running a small health check on it twice a day since 5 October, which accidentally left me a logged corpus: 169 runs, 4,475 rows, 1,000 distinct posts, some of them read 53 times. Today I added one deliberate pull — 250 posts of @realdonaldtrump, 18 seconds, $1.00 on the Free plan — and the two together say something I had wrong in my own documentation.

The posting rate is not a rate

My health check asks for the 20 newest posts, twice a day. For most of October that was comfortable: 12 to 27 new posts a day from @realdonaldtrump, so a daily read caught almost everything.

Then the 250-post pull came back with this day distribution:

2026-11-07    3
2026-11-06    7
2026-11-05    4
2026-11-04    9
2026-11-03  206   <-
2026-11-02   21   (partial - the 250 cap cut here)

206 posts on 3 November — the day before the US midterms. 141 of them carry a link card to swampthevoteusa.com, state-by-state endorsement lists: "TO THE GREAT PEOPLE OF KENTUCKY, GET OUT AND VOTE FOR: – Andy Barr for U.S. Senate…". The median gap between two of those 141 posts is 9 seconds, and 119 of the 140 gaps are under 30 seconds. One 15-minute window, 13:15–13:30 UTC, holds 52 posts.

So I simulated what a schedule would have delivered that day, replaying the real timestamps against a read that returns the 20 newest posts:

Schedule	Reads in the day	Posts delivered of 206
every 15 min	97	138 (67%)
hourly	25	79 (38%)
every 3 h	9	58 (28%)
daily	2	20 (10%)

A 15-minute monitor — the cadence I recommend in my own README — would have missed a third of election eve. Not because of a block, a timeout or a bug: because 20 is smaller than 52.

The fix is not a faster schedule, it is a bigger bite. maxPostsPerUser costs nothing when nothing new was posted, because my Actor charges per row and rows it has already delivered under an "only new posts" memory are free. Setting it to 200 on a 15-minute schedule costs exactly the same as 20 on 364 quiet days a year, and on the 365th it is the difference between a feed and a sample. I have raised my own default guidance accordingly.

`postedAfter` does not define your window — the cap does

I asked for postedAfter: "7 days" with maxPostsPerUser: 250. What came back spans 4.2 days, from 2 November 23:08 to 7 November 03:12. The timeline is read newest-first; the cap hit before the date did, and the oldest row sits mid-burst on 2 November.

The row count alone cannot tell you this happened — 250 is just 250. What tells you is comparing the oldest createdAt to the date you asked for. If they disagree, your window is truncated and you have no idea what is behind it. Any date-bounded pull needs that assertion:

oldest = min(r["createdAt"] for r in posts)
assert oldest <= wanted_from, f"capped at {oldest}, asked for {wanted_from}"

Catch a post in the first minute and its engagement is a rounding error

The upside of a 15-minute schedule is latency. The cost is that every engagement number on a fresh post is close to meaningless, and my logged corpus happens to measure exactly how close.

Twelve posts were first read within six hours of publication and read again just past the 24-hour mark. Likes at first read, as a share of the 24-hour value:

Age at first read	Likes	% of the 24 h value	Account
6 seconds	2	0.0%	realDonaldTrump
1.3 min	183	1.9%	realDonaldTrump
2.5 min	216	2.0%	realDonaldTrump
23 min	2,592	21.6%	realDonaldTrump
35 min	556	21.8%	WhiteHouse
1.7 h	8,256	49.0%	realDonaldTrump
2.2 h	3,494	48.5%	realDonaldTrump
3.0 h	6,034	58.1%	realDonaldTrump
4.7 h	1,424	70.7%	WhiteHouse
5.2 h	1,675	71.5%	WhiteHouse

The curve is monotone across two accounts and three orders of magnitude of audience. Half the likes arrive in the first two hours; a post caught at one minute old shows you 2% of them. The 6-second row is the one worth remembering: likesCount: 2 on a post that finished the day at 13,707.

Practically: a monitoring pipeline must treat the first row as text and timestamp only. If you want engagement, re-read the post once a day later. The same post asked for twice is returned and billed once per run, so a second pass costs one row, and the exact createdAt makes the age arithmetic trivial.

One more wrinkle: counts are not monotone. Across 3,086 consecutive re-reads of the same post, 57 (1.8%) came back with fewer likes than the previous read — people unlike things. The largest drop I logged was 5. So max() across reads, not last(), if you want a high-water mark.

Follower counts, unlike Telegram's rounded 6.3M, arrive exact to the digit, which makes a daily series readable. @realdonaldtrump over my 20 logged days: 13,094,290 → 13,173,870, +79,580. The interesting rows are the outliers — −121 on 1 November (a net loss, so the field really is a live count and not a counter) and +21,193 on 4 November, the day after the 206-post burst, against a ~1,500/day baseline.

A third of the sample had no text — and then none of it did

My Actor's text field is plain text with the HTML stripped. In the October corpus, 190 of 539 @realdonaldtrump posts (35.3%) had an empty text. All 190 had media: 103 single videos, 79 single images, 8 image sets. A keyword filter or an LLM classifier reading text sees nothing on a third of his posts.

Quote posts make it worse. Of 65 quotes in that sample, 45 had an empty quotedText — the quoted post was itself a bare video or picture — and 40 of those also had no text of their own. The row is structurally complete and semantically empty; the only thing to read is a .mp4 URL.

Then the election-week pull came back with just 12 of 250 (4.8%) empty, median text length 1,116 characters, and 164 of 250 carrying a link card. Same account, same field, same parser. So the useful number is not "35%" or "5%" — it is don't branch on text being present. Check mediaCount and quotedMediaUrls before you decide a post said nothing, and expect the ratio to move with whatever the account is doing that week.

What the site will not give you without a login

Measured on 1 October across the public API surface, and still true: /api/v2/search, /api/v1/trends, the trending-truths feed, reblogged_by, favourited_by and the reply tree all answer 401 to a guest. /api/v1/statuses/<id>/context is hard-blocked by the WAF every time. The hashtag timeline is the sneaky one — it answers HTTP 200 with [], so a naive client reports "no posts about that tag" instead of "you are not allowed to ask".

So: no search, no hashtag feeds, no trends, no comment threads. You bring the usernames. Each post row carries repliesCount, which tells you a discussion exists without letting you read it. A handful of accounts also hide themselves from logged-out visitors — @DanScavino's lookup returns 401 — and those come back as a free error row rather than taking the run down.

Transport is the other half. Datacenter addresses get Cloudflare's block page on 100% of requests. On residential addresses the verdict is per-address and sticky: roughly a quarter are challenged and stay challenged. The surprise was the TLS fingerprint — measuring 4 fingerprints × 8 sessions × 7 endpoints, the Firefox and Chrome 151 fingerprints passed while plain Chrome and Firefox 144 were challenged from a good address. So the client keeps whatever session last worked and, on a challenge, rotates both the address and the fingerprint with a back-off. One more server-side quirk: exclude_reblogs is accepted and ignored, so ReTruth filtering has to happen on your side.

Calling it

{
  "usernames": ["realDonaldTrump", "WhiteHouse"],
  "maxPostsPerUser": 200,
  "postedAfter": "2 days",
  "onlyNewRowsStore": "trump-watch"
}

A real row from today's run, trimmed:

{
  "type": "post",
  "id": "117552785929148974",
  "url": "https://truthsocial.com/@realDonaldTrump/117552785929148974",
  "createdAt": "2026-11-03T13:39:24.203Z",
  "username": "realDonaldTrump",
  "authorFollowers": 13175051,
  "text": "TO THE GREAT PEOPLE OF KENTUCKY, GET OUT AND VOTE FOR:\n\n– Andy Barr for U.S. Senate\n…",
  "isRepost": false, "isQuote": false,
  "repliesCount": 111, "repostsCount": 1421, "likesCount": 5092,
  "mediaTypes": ["image"],
  "linkUrl": "https://swampthevoteusa.com/kentucky/",
  "linkDomain": "swampthevoteusa.com"
}

From Python, run and read in one call:

import requests
rows = requests.post(
    "https://api.apify.com/v2/acts/lergassy~truth-social-scraper/run-sync-get-dataset-items",
    params={"token": "apify_api_..."},
    json={"usernames": ["realDonaldTrump"], "postedAfter": "1 day", "maxPostsPerUser": 200},
    timeout=300,
).json()
posts = [r for r in rows if r.get("type") == "post"]
textless = [p for p in posts if not p["text"].strip()]
print(len(posts), "posts;", len(textless), "with no text;",
      min(p["createdAt"] for p in posts), "oldest")

The Actor is Truth Social Scraper. It bills per post row and per profile row, no start fee; error rows, posts cut by your date or ReTruth filters, duplicates and posts already delivered under an only-new memory are free — which is why raising maxPostsPerUser to 200 on a schedule is not the expensive mistake it looks like.

When it is the wrong tool. If you need the comments under a post, a search across Truth Social, or trending topics, stop here — none of that is public, and no amount of proxy budget changes it. If you want sentiment on a video-only post, this gives you the file URL and nothing else; the transcription is a separate job. And if you are building a market signal, remember what the 6-second row says: you can have the text fast or the engagement right, and on the first read you do not get both.

797 of 1,071 TikTok Shop Reviews Came With No Text At All. The 1-Star Ones Talk Twice as Often

Matvey — Fri, 06 Nov 2026 21:13:57 +0000

I run a visa agency in Bali. Scrapers on Apify are the other half of my week, and one of them does a narrow job: every buyer review of a TikTok Shop product in the US or the UK — stars, text, date, photos, the variant bought, the reviewer.

The reason people buy review data is text: the complaints, the feature requests, the sentence that explains why 4.5 stars is not 4.9. So before writing another word about "AI review analysis", I counted how much text is actually in there. On the product my own health check hits every night, the answer is a quarter of it.

Two samples, the same 72%

The test product is a US supplement listing, 1729448464509734958 — Toplux Magnesium Complex. On 8 October TikTok reported 170,737 reviews on it; today it reports 180,293, so about 330 arrive a day.

Sample one: a single run on 8 October, reviewsNewerThan: "3 days", cap 5,000. It returned 1,071 reviews in 46 seconds, and reviewText was an empty string on 797 of them — 74.4%.

Sample two, in case that was one strange weekend: twelve nightly 100-row runs between 13 October and today, deduplicated by reviewId into 964 unique reviews. Empty text on 686 — 71.2%.

Together, 2,035 unique reviews, 1,483 of them with nothing written in them: 72.9%. The 274 texts in sample one add up to 3,087 words. That is a long blog post, extracted from 1,071 paid rows.

Nothing is broken: TikTok Shop asks for a star rating in one tap after delivery and treats the text box as optional. But if your pipeline is rows → LLM → themes, you are paying for four rows to get one sentence, and you should know that going in.

The complaints talk, the praise does not

Here is the part that changes an analysis. Share of reviews carrying any text, by star rating:

stars	sample 1 (1,071)	sample 2 (964)
5★	197 of 848 — 23%	186 of 739 — 25%
4★	20 of 79 — 25%	16 of 79 — 20%
3★	11 of 45 — 24%	14 of 38 — 37%
2★	4 of 20 — 20%	6 of 17 — 35%
1★	42 of 79 — 53%	56 of 91 — 62%

An angry buyer is two to three times more likely to type something than a happy one. The middle of the scale is noisy because the counts are small, but both ends replicate across two samples a month apart.

The consequence is worth a sticky note. In sample one the mean rating over all 1,071 rows is 4.491; over the 274 rows that have text it is 4.190. Drop the silent rows — exactly what a text-mining pipeline does — and the product loses 0.3 stars before any model sees it.

So compute the distribution from every row, and read the text knowing it over-represents the 1-star tail by design. If you want complaints and nothing else, ask for them instead of filtering a full export: reviewsFilter: "1" walks the 1-star filter directly, and on a UK lash-extension product it returned all 26 one-star reviews in a single API call, 17 with text (65% — same pattern, other market).

What the text is when there is text

Of the 274 texts in sample one:

65 are three words or fewer. Real examples: Great product, Excelente 👌, received, 9.5/1, Muy bueno.
9 contain no Latin letters at all — 👍👍👍👍👍, ♥️♥️, 10/10, and one Arabic line that translates to "please stop sending me the magnesium".
45 are 20 words or more — the useful tail, which includes an 83-word four-star review admitting the buyer has not tried the product and is rating the other reviews.
76 are in Spanish. There is no language field on the row — TikTok does not send one — so detect it yourself before you average sentiment across a mixed bag.
5 used TikTok's prompted template (Effectiveness: Good, Ease of Use: Easy, Packaging: Very good). I expected these structured lines to dominate; they are 2% of the texts. Don't build a parser for them.

And photos are rarer than the Store screenshots suggest: 55 of 1,071 rows have at least one image (5%), and 12 of those 55 have a photo and no text, which is the only case where reviewImages is the whole review.

Three fields that carried no information, and two that do

Measured over all 2,035 unique rows:

verifiedPurchase was true on 2,032. Three false. It is not a filter, it is a constant.
incentivized was false on every single row. Don't build a "paid review share" chart on it without checking your own data first.
variant was "Default" on all 2,035 — one SKU. On the UK lash product the same field is real (60&80D 10-20mm-3D Effect Lash) and gives you complaints per variant: the field is only as informative as the listing.

The two that do work, with a catch each. reviewerId is a stable numeric ID — 987 distinct reviewers in sample one, 39 of them reviewing twice — but it is null on 47 rows (4.4%), so your group-by needs a fallback bucket. And reviewerName is masked by TikTok before it reaches me: J**, A**a, 💋**🧸. Sample one has 900 distinct masked names for 987 reviewers, and ** appears 8 times. Never deduplicate reviewers by name.

Timestamps, at least, are honest: reviewedAt is a real millisecond-precision UTC instant on all 1,071 rows, not a "4 months ago" string.

The ceiling nobody mentions: 5,000 per filter

This is the limit to plan around. One sort order plus one filter stops at about 5,000 reviews: page 51 of 100 answers has_more: false. Asking deeper gets you nothing.

To go past it, the Actor reads the star breakdown out of the first answer and walks the combinations — each star filter, the other sort order for buckets over 5,000, the photos-only and verified-only filters — dropping duplicate reviewIds. That is how "25,000 reviews per product" is reached, and 25,000 is the hard cap per run.

On a product with 180,293 reviews, that is 14% at the absolute best, and it is not a random 14%. The honest ways to use this:

Newest-first with a date window. reviewsNewerThan: "1 day" on this product returned 351 reviews in 27 seconds — a complete, unbiased slice of one day. Schedule it daily and you build the full history going forward instead of fighting for the past.
One star rating at a time, when the bucket is small enough to finish (all 26 one-star UK reviews, one call).
Treat a single giant history dump as a sample, and say so in the report you build on it.

A smaller listing has no such problem: under 5,000 the first walk gets everything, and the run statistics report TikTok's own total per product so you can check.

How it gets them, in case you are building your own

POST /api/shop/pdp_desktop/get_product_reviews only answers a request signed by TikTok's own page scripts. Plain HTTP returns {"code":10000} on every route I tried — direct, residential proxy, a commercial unblocker, the mobile twin of the page — and headless Chromium is refused as well (code 100000). What works is a headful browser on a virtual display: open one product page of the market, wait for the security scripts, then ask the API for any product ID in that market. page_size: 100 is accepted — 1,071 reviews took 13 calls and one browser launch.

Two quirks. An empty page that arrives without has_more: false is a hiccup, not the end of the list — ask again and it comes back. And geography is per market: US reviews answer from Apify's own datacenter address, a UK product needs a UK residential one, and a UK product asked from a US page returns zero reviews.

Calling it

{
  "productUrls": ["https://shop.tiktok.com/us/pdp/1729448464509734958"],
  "maxReviewsPerProduct": 2000,
  "reviewsNewerThan": "1 day",
  "reviewsSort": "newest",
  "reviewsFilter": "all"
}

A real row from today's run, trimmed:

{
  "type": "review",
  "reviewId": "7705163491157870350",
  "productId": "1729448464509734958",
  "productTitle": "Toplux Magnesium Complex 8 Essential Magnesium Supplement",
  "country": "US", "reviewRating": 2,
  "reviewText": "Lo acato de enpesar a usted y muy bueno",
  "reviewedAt": "2026-11-06T20:49:58.096Z",
  "reviewerName": "💋**🧸", "reviewerId": "6979971148214518789",
  "verifiedPurchase": true, "incentivized": false,
  "variant": "Default", "skuId": "1729448464509800494",
  "reviewImageCount": 1, "reviewCountry": "US",
  "scrapedAt": "2026-11-06T20:52:22.707Z"
}

Two stars, text in Spanish that reads like praise, one photo. That row contains most of this article.

From Python, run it and measure what I have been measuring:

import requests, collections
rows = requests.post(
    "https://api.apify.com/v2/acts/lergassy~tiktok-shop-reviews-scraper/run-sync-get-dataset-items",
    params={"token": "apify_api_..."},
    json={"productUrls": ["https://shop.tiktok.com/us/pdp/1729448464509734958"],
          "maxReviewsPerProduct": 300, "reviewsNewerThan": "2 days"},
    timeout=900,
).json()
r = [x for x in rows if x.get("type") == "review"]          # skip free notice rows
empty = sum(1 for x in r if not (x.get("reviewText") or "").strip())
print(len(r), empty, collections.Counter(x["reviewRating"] for x in r))

The Actor is TikTok Shop Reviews Scraper: $0.0015 per review (down to $0.00105 on higher Apify plans) plus $0.005 per run start, no login, no cookies. Today's control run with "Max cost per run" set to $0.05 delivered 33 reviews in 21.6 seconds and charged $0.04455, and 11 of those 33 rows had text — the same share I started with. Notice and error rows are free.

When it is the wrong tool. Only US and UK products serve the review list on the web — a link from any other market gets a free notice row explaining that. If you need price, stock, sales or shop stats too, reviews alone will not give them to you; that is a job for TikTok Shop Scraper. There are no seller replies and no Q&A, because the public page has none. And if your deliverable is prose for a model to read, budget by words, not by rows: on this listing, 1,000 paid reviews buy about 2,900 words over 256 texts, 60 of them three words or fewer.

1,084 Mercado Libre Cards Told Me How Many Were Sold. Between Them They Used Ten Numbers

Matvey — Fri, 06 Nov 2026 13:15:46 +0000

I run a visa agency in Bali. The half of my week that has nothing to do with visas goes into scrapers on Apify, and one of them reads Mercado Libre product cards — the search results page of Latin America's biggest marketplace, which is really 18 separate storefronts on 18 domains and currencies: mercadolibre.com.mx, .com.ar, .cl, .com.co, mercadolivre.com.br and a dozen smaller ones.

Because the Actor runs a health check on itself every night, I ended up with something more useful than a one-off measurement: 57 successful runs since 9 October, 1,144 product rows, and 47 of those runs are the same query — audifonos bluetooth on mercadolibre.com.mx, 20 rows a time — spread over 22 distinct days. That is a panel, not a snapshot. So this week I pointed the analysis at my own history instead of at the site.

The field that fell apart first was the one everybody wants.

`soldQuantity` is a bucket wearing a number's clothes

Units sold is the reason people scrape marketplace search in the first place. Price you can get anywhere; "how many did this seller move" is the signal.

Across 1,144 rows, soldQuantity was filled on 1,084. Those 1,084 rows contained exactly ten distinct values:

1000    293 rows        10000   273        100     148
5000     98             50000    81        250000   71
500      53             100000   35        50       21
25       11             (null)   60

That is the whole vocabulary. No 1,240. No 37. No 118,000. Mercado Libre's card never prints a sold count — it prints a rung on a ladder: +1000 vendidos, +250mil vendidos. My parser reads either that text or the sold_quantity number in the tracking JSON the page embeds for its own analytics, and both are already rounded down to the same rungs. The rounding is theirs, not mine.

So "sold" here means "at least". A listing in the 1000 bucket sold somewhere between 1,000 and 4,999 units, and the row cannot tell you where.

It is worse than lossy, because the rungs are not even monotone against the other popularity field on the same card. Grouping every row that had both soldQuantity and reviewCount:

bucket	rows	review count: min / median / max
1000	173	6 / 326 / 8,040
5000	68	432 / 1,709 / 18,262
10000	232	1,159 / 5,066 / 69,436
50000	72	3,252 / 18,676 / 67,793
100000	21	24,872 / 68,323 / 69,436
250000	65	6,909 / 8,669 / 79,514

The 250000 bucket has a lower median review count than the 50000 bucket — 8,669 against 18,676. Partly that is cheap generic earbuds that sell in volume and get reviewed rarely; partly it is that sold counts attach to the catalogue entry while reviews attach to whatever listing is winning the buy box today. Either way, if you were planning to rank sellers by soldQuantity, you are ranking by a ten-value enum with ties everywhere.

Over three weeks, price moves and the bucket does not

Here is the part that only a panel shows. Of the 157 distinct listings that passed through those 22 days of one search, 26 appeared on five or more days. For each of them I diffed price and sold bucket across the whole window:

price changed at least once on 22 of 26
soldQuantity changed at least once on 2 of 26

One listing — an authorised-reseller AirPods 5 at MXN 2,999 — was in the top 20 on 21 of the 22 days. Its price never moved by a peso. Its bucket went 500 → 1000 once, on day two, and then sat still for three weeks. That single step is the only sales information those 21 rows contain.

Meanwhile the prices are genuinely alive. A Lenovo RS110 open-ear set ran from MXN 196.42 to 453.37 over 14 days — 2.31x. Median swing across the 26 was 1.05x, so most of the movement is a few percent of jitter, but the tail is real and it is where a repricing tool earns its keep. Conclusion for anyone scheduling this kind of scraper: run it daily for price, and treat the sold bucket as a quarterly signal, because that is the resolution it actually has.

Two layouts of one listing, two different prices

While diffing day over day I hit 17 listings (of 340) that appeared sometimes with a variationsText line — Disponible en 4 colores — and sometimes without. Mercado Libre renders a colour family as either a single-variant card or a picker card, apparently at random per request.

For 13 of the 17 the price was the same either way. For one it was not, and it flipped back and forth for three weeks at a fixed position 7 in the results:

2026-10-20   price 182.71   originalPrice null   variationsText "Disponible en 4 colores"
2026-10-22   price 101.36   originalPrice 192.16 variationsText null
2026-10-23   price 182.65   originalPrice null   variationsText "Disponible en 4 colores"
2026-10-26   price 102.78   originalPrice 194.85 variationsText null

Same itemId, same slot, 1.8x apart, and the discount only exists in one of the two layouts. The picker card quotes the family's base price with no old price; the single-variant card quotes one discounted colour. Neither is wrong, and no field on the row tells you which reality you got — except variationsText being filled, which is why I now keep that field even though it looks like decoration.

Which brings me to a bug of my own, found in the same pass. That field read "Disponible en 4 colores4" on 79 of 247 rows that had it: the compacted block carries the colour count twice, once in the sentence and once as a screen-reader pill, and .text() glues them. Same family of mistake as the shipping label I had already de-duplicated months earlier ("Envío gratisEnvío gratis"). Fixed by dropping a trailing repeat of a number already in the string; control run today, same query, 20 rows: 3 rows with variations, 0 duplicated. Build 0.1.47.

Positions 1–4 and 13–16 are not results

isSponsored was true on 410 of 1,144 rows — 36%. The placement is not random: of the 160 rows that sat in positions 1, 2 or 3, 155 were ads. The counts cluster at positions 1–4 and again at 13–16, which is exactly where Mercado Libre drops its two ad blocks into a 20-card page.

So an unfiltered top-20 export is roughly a third paid placement, and your "bestsellers for audifonos bluetooth" list starts with four of them. includeSponsored: false drops them. I leave the default on, because for ad-intelligence work those rows are the data.

Two more things the card does not have, measured over the same 1,144 rows: brand is null on 873 (76%) and seller on 421 — Mercado Libre simply does not print either on most search cards. And 320 rows carried a star rating with no review count, because the compacted review widget shows 4.8 alone in some layouts and 4.8 (65161) in others. For a complete record you need the product page, which is a separate, more expensive pass in this Actor for exactly that reason.

Finally, do not read totalResults as a count. It was 9999 on 1,054 of 1,144 rows — the ceiling, not a total. Narrow the query and it becomes honest: cable hdmi 8k 3 metros returned 321 and 322 on two consecutive runs.

Calling it

{
  "country": "MX",
  "searchTerms": ["audifonos bluetooth"],
  "maxItems": 20,
  "includeSponsored": false,
  "sort": "relevance"
}

A real row from today's control run, trimmed:

{
  "type": "product",
  "title": "Auriculares Bluetooth Anker Soundcore P40i 2024, color lila, verde oscuro y claro",
  "url": "https://articulo.mercadolibre.com.mx/MLM-5259417430-_JM",
  "itemId": "MLM5259417430", "productId": null,
  "siteId": "MLM", "country": "MX", "currency": "MXN",
  "price": 799.01, "originalPrice": 1499, "discountPercent": 46,
  "installmentsText": "3 meses sin intereses de $266.34",
  "soldQuantity": 50, "rating": 4.8, "reviewCount": null,
  "seller": "Soundcore", "isOfficialStore": true,
  "freeShipping": false, "fullShipping": true, "shippingText": "Llega gratis hoy",
  "isSponsored": true, "position": 1, "page": 1,
  "brand": null, "condition": null, "variationsText": null,
  "domainId": "MLM-HEADPHONES", "categoryId": "MLM5072",
  "totalResults": 9999, "scrapedAt": "2026-11-06T13:13:33.917Z"
}

Note isOfficialStore: true on a row that is also an ad, and reviewCount: null sitting next to rating: 4.8 — the two traps from above, both in the first row of an ordinary run.

From Python, run and read in one call:

import requests, collections
rows = requests.post(
    "https://api.apify.com/v2/acts/lergassy~mercadolibre-products-scraper/run-sync-get-dataset-items",
    params={"token": "apify_api_..."},
    json={"country": "AR", "searchTerms": ["zapatillas running"], "condition": "new",
          "sort": "price_asc", "maxItems": 40},
    timeout=600,
).json()
items = [r for r in rows if r.get("type") == "product"]
print(collections.Counter(r["soldQuantity"] for r in items))   # expect a handful of rungs

The Actor is MercadoLibre Products Scraper. It bills per product row with no start fee, and the nightly 20-row run I have been quoting costs $0.045 and takes 40 to 56 seconds, most of which is the headless browser and the residential proxy that search pages need. Reviews, the deals page and official-store pages come from the same Actor over plain HTTP, no proxy, which is why a review row is priced at $0.001 against $0.0025 for a product card.

When it is the wrong tool. If you need a real sold count, a stock level or a brand on every row, search cards cannot give it to you — open product pages and pay for the heavier pass. If you need questions and answers, nothing public has them; Mercado Libre shows those only to signed-in users and this Actor never signs in. If you want market share for a category, remember totalResults caps at 9,999 and the deepest the site will page is a few thousand cards, so you are sampling, not counting. And if you are writing your own: keep concurrency at three. At five and above, Mercado Libre stops answering with products and starts answering with a challenge page, which looks exactly like an empty category until you log the HTML.

168 of 169 Paid 'description' Fields Were Identical to the Free 'summary'. The Real Text Was 17x Longer

Matvey — Fri, 06 Nov 2026 05:17:19 +0000

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them reads Eventbrite search pages: you hand it a search URL or a city, it pages through the results and returns one row per event — name, dates with the time zone, venue with coordinates, category, tags, image. Switch on the detail pass and it also opens each event page for the ticket price range, the organizer and the description.

That detail pass costs the buyer a second charge per event. I spent today measuring what it actually buys, on two search pages: Bali, Indonesia and Berlin, Germany, a hundred-event cap each.

Bali returned 69 events across 4 pages in 28.0 seconds. Berlin returned 100 in 22.2 seconds. Then I diffed two fields that should never be the same — summary, which comes free from the search listing, and description, which the buyer pays for.

They were identical on 168 of 169 rows. The 169th had no description at all.

The meta tag is not the description

Eventbrite event pages carry JSON-LD, the schema.org/Event block search engines read. It has offers with lowPrice and highPrice, an organizer, start and end timestamps with the UTC offset — all genuinely useful, none of it on the search page. It also has a description. I read that field, shipped it, and called it "full description" in my own documentation.

It is the meta description. Capped at 140 characters, built from the first sentences of the listing — which is exactly what the search page already hands over as summary, for free.

Across those 69 Bali events, description averaged 108 characters. Three rows sat exactly at 140, the cap. The field was not wrong, it was just a duplicate the buyer was paying me to deliver twice.

The text the organizer actually wrote is in the same HTML I had already downloaded, a few kilobytes further down, inside the Next.js payload:

const m = html.match(/<script id="__NEXT_DATA__" type="application\/json">([\s\S]*?)<\/script>/);
const ctx = JSON.parse(m[1]).props.pageProps.context;
const mods = ctx.structuredContent.modules;   // [{type:'text', text:'<p>…'}, {type:'image', url:…}]

structuredContent.modules is the event body as the organizer built it — text blocks, images, the occasional video embed. Joining the text blocks and stripping tags gives the real thing. I refetched all 69 event pages and measured both sources on the same bytes:

	JSON-LD `description`	`structuredContent` text
Pages that had it	69 / 69	69 / 69
Average length	108 chars	1,841 chars
Longest	140	12,764
Rows where it was longer	—	67 of 69
Rows with nothing at all	0	2

Seventeen times more text, median ratio 7.8x, from a request I was already paying for. The two empty ones are honest: those organizers wrote no body copy, so the Actor falls back to the meta description rather than returning null.

The control run after the fix, same search, 20 events: description identical to summary on 0 of 20 rows, median length 647, longest 11,838. Prices and currency still filled on 20 of 20 — I did not trade one field for another.

While I was in there, the same payload gave up eleven more fields that cost nothing extra, because they ride on the page already being fetched: durationMinutes (filled on 69 of 69), salesStatus, refundPolicy, ageRestriction (9 of 69), doorTime, parking, organizerEvents, organizerFollowers, organizerVerified, descriptionBlocks, and nextSessionAt.

`currency` is the organizer's currency, not the country's

Now the finding I cannot fix, only document.

All 69 Bali rows carry country: "ID". Their prices came back in eight currencies:

USD 44   AUD 10   GBP 8   EUR 3   SGD 1   MYR 1   NZD 1   HKD 1

Zero in rupiah. A dive trip in North Bali is priced 5299.5 HKD. A free Bitcoin meetup in Canggu is priced 0 GBP. A bracelet workshop in Bali is served from eventbrite.ch at 7.49 EUR. Those 69 events arrived across eight different Eventbrite domains — .com, .com.au, .co.uk, .sg, .ch, .fr, .ie, .hk.

This is not a parsing artifact. I checked both independent sources on the page — the JSON-LD priceCurrency and basicInfo.currency in the Next.js payload — and they agree on 69 of 69 rows. Eventbrite publishes the price in the currency of the organizer's account. A yoga teacher who registered in Sydney sells a Canggu sound bath in Australian dollars, and the site never converts.

Berlin is why nobody notices: 95 of its 100 rows are EUR, because Berlin events are mostly run by Berlin organizers. Bali is a destination — its events are sold by Australians, Brits, Singaporeans — so the same behaviour becomes obvious. If your search covers a tourist region, sorting those rows by priceMin sorts by nothing at all, and averaging them produces a number with no unit.

Two more things to distrust in the price fields. priceMin equals priceMax on 49 of 69 rows, which usually means one ticket type, not a one-price event. And priceMax is whatever the most expensive offer on the page is: the Bali Venture Capital World Summit reports 168.94 to 57662.15 EUR, a 341x spread, because the top of that range is a sponsorship package nobody attends as a guest.

A series row is one date out of hundreds

Eventbrite recurring events are a single search result. My rows flag them — isSeries, seriesId, occurrencesInSeries — and on the Bali page 36 of 69 rows were series, covering 1,568 dates between them. The biggest single row: 421 occurrences. A sound bath studio: 283.

So "69 events in Bali" and "69 things happening in Bali" are very different counts, and a weekly calendar built from the row's one date will keep rediscovering the same workshop. The new nextSessionAt confirms what that one date means — on all 8 series rows in my control run it matched startsAt exactly, i.e. it is the next session on sale. The other 282 are not in the row, and the search page does not have them.

Berlin again shows the contrast: 7 series rows out of 100, 14 occurrences total.

Two smaller traps from the same two runs. organizerName is null on 11 of 69 Bali rows — null in both page sources, because those organizers never set a public name; you get organizerUrl with a numeric id and nothing else. And one organizer, Eurasia Research, owns 9 of the 69 rows with near-identical academic conferences, so an unfiltered "events in Bali" list is 13% one seller. Date range of that list: 6 November 2026 to 29 October 2030. Only 40 of 69 fall in the current month.

Calling it

{
  "startUrls": ["https://www.eventbrite.com/d/indonesia--bali/all-events/"],
  "maxEvents": 100,
  "includeDetails": true
}

A real row from today's run, trimmed:

{
  "type": "event",
  "eventId": "1991054633005",
  "name": "Chakras sound sanctuary - SOUND HEALING sessions | Sound bath Bali",
  "startsAt": "2026-11-06T14:00:00",
  "timezone": "Asia/Makassar",
  "isSeries": true,
  "occurrencesInSeries": 283,
  "nextSessionAt": "2026-11-06T14:00:00+08",
  "venueName": "ADDA YOGA CANGGU - Meditation | Breath | Sound",
  "city": "Kec. Kuta Utara", "country": "ID",
  "latitude": -8.643315, "longitude": 115.1427276,
  "category": "Religion & Spirituality",
  "priceMin": 29.89, "priceMax": 29.89, "currency": "AUD", "isFree": false,
  "organizerName": "ADDA YOGA", "organizerEvents": 700, "organizerFollowers": 19,
  "durationMinutes": 90, "refundPolicy": "custom", "salesStatus": "sales_ended",
  "descriptionBlocks": 11, "urgency": ["salesEndSoon"]
}

Note salesStatus: "sales_ended" on an event the search page listed as upcoming — 3 of 69 Bali rows were sold out or closed. That field did not exist this morning.

From Python, run and read in one call:

import requests
rows = requests.post(
    "https://api.apify.com/v2/acts/lergassy~eventbrite-scraper/run-sync-get-dataset-items",
    params={"token": "apify_api_..."},
    json={"location": "Berlin, Germany", "dates": "this_weekend", "maxEvents": 200},
    timeout=600,
).json()
events = [r for r in rows if r.get("type") == "event"]
print(len(events), "events;", len({r["currency"] for r in events if r.get("currency")}), "currencies")

The Actor is Eventbrite Scraper. It bills per event row, with a second charge only if you ask for the detail pass; error rows are free and there is no start fee. The detail pass is also what the time goes on: the same Bali search finished in 8.2 seconds with details off and 28.0 seconds with them on, because that is 69 extra page loads.

When it is the wrong tool. There is no search across all of Eventbrite from the API side — you bring a search URL or a city, and Eventbrite's own search decides what exists. Past events are gone; search pages show upcoming only, so a history has to be accumulated by running on a schedule. Attendee lists, ticket inventory and sales numbers are not public and never will be. And one operational detail worth knowing: Eventbrite's WAF answers a flagged search request with HTTP 405, not 403 or 429 — from my own server the search page returns 405 while individual event pages return 200 all day. The Actor climbs a proxy ladder when that happens; if you are writing your own, treat a 405 on a GET as a bot check rather than a method error, which is what I spent an afternoon assuming.

677 of 785 Telegram Reaction Chips Counted Zero. The Custom Emoji and the Paid Stars Were Fine

Matvey — Thu, 05 Nov 2026 21:19:34 +0000

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them reads public Telegram channels: you hand it a list of usernames, it returns one row per post — text, date, views, reactions, media, forwards, with the channel card on every row. No account, no API key, no phone number. It reads t.me/s/<channel>, the public web preview any visitor sees, 20 posts per request, paging backwards with ?before=<message id>.

I spent today measuring it against 12 channels — 200 posts each, 2,400 posts total, 1 to 4.6 seconds per channel. The headline number is not a feature. It is a bug I shipped and did not notice for six weeks.

Three ways to draw a reaction, one of which broke everything

Telegram puts reaction chips under a post. In the HTML they come in three shapes, and I only tested against two of them.

A custom emoji, the kind channels with a Premium boost use:

<span class="tgme_reaction"><tg-emoji emoji-id="5265077361648368841"></tg-emoji>54.7K</span>

A paid star reaction:

<span class="tgme_reaction"><i class="icon icon-telegram-stars"></i>6.3K</span>

And the ordinary one — a plain unicode emoji:

<span class="tgme_reaction"><i class="emoji" style="background-image:url('...')"><b>👍</b></i>101</span>

My parser read the count by stripping tags from the chip and handing the rest to a number parser anchored at the start of the string. For the first two shapes the emoji element is empty, so stripping leaves 54.7K and 6.3K and everything works. For the third, the emoji is in the HTML as text, inside <b>. Stripping leaves 👍 101. My parser wanted a digit at position zero, found an emoji, returned null, and I defaulted null to 0.

So the row said {"emoji": "👍", "count": 0}. The emoji was right. The number was sitting two characters away.

The fix is to read the tail after the last closed emoji element instead of the whole chip:

const tail = inner.replace(/^[\s\S]*<\/(?:i|tg-emoji)>/, '');
const count = parseCount(tail.replace(/<[^>]*>/g, ' ').trim())
    ?? parseCount(inner.replace(/<[^>]*>/g, ' ').trim());

I saved one page per channel to disk and parsed the same bytes with both versions, so nothing could drift between runs:

Channel	Chips on the page	Counted zero before	`reactionsTotal` before → after
bloomberg	215	215	0 → 11,790
tginfo	232	217	40 → 15,867
binance_announcements	119	119	0 → 6,311
Bybit_Announcements	57	47	99 → 1,232
cointelegraph	57	57	0 → 823
varlamov	23	22	1 → 2,159
durov	82	0	2,483,290 → unchanged
total	785	677

Durov's channel is the reason I missed it. His reactions are almost all custom emoji and paid stars — 882 custom and 199 paid out of 1,156 chips — so the one channel in my demo input, the one my daily health check runs, was the one shape that worked. Across all 2,400 posts after the fix: 7,969 chips, zero of them counting zero, and not one chip lost a number it already had.

What `reactionsTotal` still is not

Having the counts back does not make the sum a measure of humans.

Paid stars are currency, not people. 429 of those 7,969 chips are star reactions, and they add up to 8,014,263 — 29% of Durov's entire reaction total. The record holder is t.me/durov/342: reactionsTotal 3,042,500, of which 2,490,000 is a single paid-star chip. One wallet can outvote a city. If you are ranking posts by engagement, sum reactions while skipping entries flagged paid: true.

Custom emoji are opaque. 1,142 chips come back as custom:5265077361648368841, because Telegram's web preview ships an id and no picture of the feeling. On Durov's channel that is 67% of the reaction total. You can count them; you cannot say what they mean, and you cannot compare them across channels.

Zero means two different things. 5 of my 12 channels — @telegram, @meduzalive, @bbcrussian, @whale_alert, @breakingmash — returned reactionsTotal: 0 on all 200 posts, and the markup has no chips at all. Reactions are switched off. That is not "nobody reacted", and if you divide by it you will get a very confident answer about nothing.

`views` is a running counter, and my own history proves it

My health check has been running this Actor on @durov and @telegram daily for a month, which accidentally gave me a longitudinal series on single posts. t.me/durov/552, published 11 October 2026, read 28 times:

2026-10-21  1,250,000 views   29,850 reactions
2026-10-24  1,390,000          31,960
2026-10-28  1,610,000          35,550
2026-11-01  1,820,000          38,180
2026-11-05  2,020,000          43,240

+770,000 views in 15 days on a post that was already ten days old. @telegram's post 460, published 26 August, went from 1,680,000 on 6 October to 2,340,000 today across 40 reads — +39% on a post nobody was promoting.

Meanwhile the denominator moves the other way. @telegram's channelSubscribers in those same 40 reads: 9,390,000 → 9,180,000, falling monotonically, −210,000 in a month. @durov: 10,300,000 → 10,100,000.

My row carries viewsPerSubscriber, which I added because buyers asked for it. For durov/552 it read 0.121 on 21 October and 0.200 today — +65% with no new behaviour from anyone, just a numerator that only rises and a denominator that only falls. The field is only comparable between posts read on the same day.

The practical consequence is that sorting by views sorts by age. @telegram's 200 most recent posts, ranked by views:

8,250,000   2024-08-25   t.me/telegram/329
7,350,000   2024-09-09   t.me/telegram/333
6,390,000   2024-04-29   t.me/telegram/303
...
2,340,000   2026-08-26   t.me/telegram/460   <- the newest post in the sample

All ten of the top ten are from 2023–2024. Of the top 50 by views, 41 are in the older half of the sample. Median views by age on that channel: 1,280,000 for posts 30–90 days old, 2,050,000 at 90–365 days, 3,220,000 past a year. Nothing is broken; views simply never stop accruing. If you want "best post", divide by age or compare posts within a week of each other.

One more precision trap: Telegram serves views as text like 5.88M, 6.3M, 22.3K. Of 2,393 posts with a view count, 2,152 carry three significant digits, 216 carry two, and 25 carry one. All 399 posts above a million are multiples of 10,000. There were 1,309 distinct view values among 2,393 posts. A post reading 6,300,000 is somewhere in a ±50,000 band, and no scraper can do better, because the number is rounded before it reaches the page.

`postId` is not a post count — it is a message counter, and albums eat it

Tempting to compute "posts per week" from the id range. Don't. Taking 200 posts per channel and comparing the id span to the number of rows:

Channel	Rows	Id span	Ids missing	Multi-photo posts (photos)
whale_alert	200	200	0	0
bloomberg	200	201	1	—
breakingmash	200	297	97	15 (52)
cointelegraph	200	286	86	77 (155)
varlamov	200	286	86	17 (92)
durov	200	278	78	6 (45)

Most of the gap is albums: a grouped set of ten photos is ten messages with ten ids that render as one post. On Cointelegraph, 77 album posts carry 155 photos — roughly 78 swallowed ids against 86 missing. The remainder is deletions. So a gap in postId tells you nothing on its own, and videoUrl is worth checking too: 398 of my posts had a video, 386 gave a thumbnail, 335 gave a direct .mp4.

Every failure answers HTTP 200

I probed 18 usernames. All 18 returned 200. Six returned zero posts, for four different reasons — which my Actor used to report with one generic message. It now names them:

coindesk — 11,198 subscribers, and t.me/coindesk shows a "Preview channel" button pointing at /s/coindesk, which 302s straight back to t.me/coindesk. The page advertises a preview that does not exist.
reuters, bbcnews — a bare contact page with a "Send Message" button. Telegram serves byte-identical pages (~9.6 KB) for a real user account and for a name nobody has claimed, so no scraper can tell you which.
tginfochat, ru_python — groups, not channels. No preview, ever.
theblock__, 12345 — t.me redirects these to telegram.org. Probing told me why: a name ending in _ or containing no letter is not a username Telegram accepts. Those are now rejected before the request, as a free error row.

Worth a confession: my own input form suggested https://t.me/s/bbcnews as an example. That is a contact page. Fixed today, along with the above.

Calling it

{
  "channels": ["durov", "https://t.me/s/bloomberg"],
  "maxPostsPerChannel": 200,
  "searchQuery": "",
  "postsNewerThan": "2026-10-01"
}

A real row, trimmed:

{
  "type": "post",
  "channel": "bloomberg",
  "url": "https://t.me/bloomberg/3225",
  "date": "2026-09-22T03:26:17+00:00",
  "views": 64800,
  "reactionsTotal": 765,
  "reactions": [{"emoji":"❤","count":214},{"emoji":"🔥","count":165},{"emoji":"👍","count":160}],
  "channelTitle": "Bloomberg",
  "channelSubscribers": 186000,
  "viewsPerSubscriber": 0.348,
  "scrapedAt": "2026-11-05T20:52:07.411Z"
}

From Python, run and read in one call:

import requests
TOKEN = "apify_api_..."
rows = requests.post(
    "https://api.apify.com/v2/acts/lergassy~telegram-channel-scraper/run-sync-get-dataset-items",
    params={"token": TOKEN},
    json={"channels": ["bloomberg", "tginfo"], "maxPostsPerChannel": 100},
    timeout=300,
).json()
posts = [r for r in rows if r.get("type") == "post"]
human = sum(x["count"] for p in posts for x in p["reactions"] if not x.get("paid"))
print(len(posts), "posts;", human, "unpaid reactions")

The Actor is Telegram Channel Scraper. It bills per post row at $0.0002, with no start fee; error rows and posts cut by your date or minimum-views filter are free, which is why I can afford to argue with an input list.

When it is the wrong tool. There is no search across all of Telegram — you bring the channel names, and searchQuery searches inside a channel, using Telegram's own channel search. Private channels and groups are out, permanently: no web preview, nothing to read. Discussion comments under posts are not in the preview either. And if what you actually want is per-channel reach and posting rhythm rather than raw rows, I keep that in a separate Actor, Telegram Channel Analytics — partly because computing it from views requires knowing everything above, and I would rather do that arithmetic once than explain a 65% jump in engagement that was really just a month passing.

One Live Palantir Posting Is Dated December 2009. 535 of My 2,207 Lever Jobs Are Over a Year Old

Matvey — Thu, 05 Nov 2026 13:14:51 +0000

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of the dullest ones reads company job boards: you hand it a list of companies on Lever, it returns one row per open job. No login, no browser — Lever publishes every board as JSON at api.lever.co/v0/postings/<slug>, and one request returns the whole board with descriptions in it.

The reason people buy that is the daily feed: pull the boards of 40 companies you care about every morning, keep what is new, push it to Slack or a sheet. Which means everything depends on one field — the posting's date. So I measured it across 12 boards and 2,207 open postings, and the date turned out to be the least trustworthy thing in the row.

The field, and what it actually is

Lever's posting object carries exactly one timestamp: createdAt, in milliseconds. There is no updatedAt, no publishedAt, no "last refreshed" in the public board API. My Actor passes it through as postedAt, unchanged — I checked five Spotify rows against the raw board and they match to the millisecond:

2026-11-04T16:12:53.423Z  ==  2026-11-04T16:12:53.423Z
2026-10-23T21:51:17.203Z  ==  2026-10-23T21:51:17.203Z

So postedAt is not "when this job was advertised". It is when the requisition record was first created inside the company's Lever account. If a company reopens the same req every quarter, or imported its history when it migrated to Lever, that date is archaeology.

The clearest example on Palantir's board, which has 309 open postings:

2009-12-05T00:00:00Z   Forward Deployed Software Engineer   London, United Kingdom
2013-01-31T00:00:00Z   Product Designer                     London, United Kingdom
2013-03-08T00:00:00Z   Deployment Strategist                London, United Kingdom
2014-01-23T00:00:00Z   Forward Deployed Software Engineer   New York, NY

That first one is live right now — jobs.lever.co/palantir/5168e8fd-… answers 200 with a 730 KB page and a working apply button — and it is dated 16.9 years ago. Note the timestamps: exactly midnight UTC, four of them. Six of my 2,207 postings have a midnight-exact createdAt, all from 2009–2019; everything created from 2016 on has a real time-of-day. Midnight is the fingerprint of an import, not of someone pressing Publish.

The distribution, not the outlier

One ancient row is a curiosity. The distribution is the problem. All 2,207 open postings, age by createdAt, measured 5 November 2026:

Age	Postings
created in the last 24 h	31
last 7 days	84
last 30 days	423
older than 180 days	843
older than 365 days	535

Median age: 103 days. Per board, it is wildly uneven:

Board	Open jobs	Median age	Oldest
veeva	873	197 d	7.5 y
shieldai	655	57 d	2.3 y
palantir	309	496 d	16.9 y
kpler	110	31 d	7.5 y
spotify	86	44 d	1.0 y
matchgroup	59	56 d	3.0 y
ro	54	177 d	6.5 y
wealthfront	23	86 d	3.1 y
outreach	22	55 d	2.1 y
neon	8	920 d	4.0 y

Palantir by creation year: 121 postings from 2026, 65 from 2025, 44 from 2024, 30 from 2022 — 61% of its live board predates this year. Neon's eight postings have a median age of two and a half years.

The cap hides it, and that is how I missed it for weeks

My Actor sorts a board newest-first before applying maxJobsPerCompany, which is the right default — if you cap, you want the fresh end. But it means a cap is also a filter on honesty. Same three boards, newest 20 vs the rest:

Board	Whole board (median age)	Newest 20	Everything after row 20
palantir	496 d	13 d	505 d
spotify	44 d	12 d	51 d
outreach	55 d	45 d	565 d

My own health check runs the default demo input — spotify, palantir, outreach, 20 per board — and for weeks it reported rows with a median age of 13 to 22 days. The board underneath it is a median of 496 days old. Nothing was broken; I was looking at the top of a sorted list and calling it a market.

The flip side is worth knowing too: those boards are stable. Twenty-five runs of those three boards between 23 October and 4 November produced 1,191 job rows and 100 distinct job IDs — a board API hands you the same set every time, unlike a relevance-ranked job search, where I have measured consecutive pulls sharing 0 of 25 results. So the ID set is a key you can diff against yesterday's. That is the only honest way to get "new since yesterday" out of Lever: keep your own history of IDs. postedWithin: "day" filters on createdAt, and across my sample it would have kept 31 postings out of 2,207 — correct per the field, and not the same question as "what appeared on this board today".

Getting the slug right is most of the work

There is no directory of Lever boards. You supply the companies, as slugs (spotify) or careers URLs (https://jobs.lever.co/palantir). I tried guessing 199 slugs from company names:

19 answered.
180 returned HTTP 404 — those companies are on Greenhouse, Ashby, Workable, their own CMS, or use a slug that isn't their name.
Of the 19, 7 returned 200 with an empty array: kraken, whoop, trinet, clari, voleon, velocityglobal, carbonhealth. The board exists; nobody is hiring through it. Kraken has open roles — just not here.

Those two failures look identical downstream ("no jobs for this company") and mean opposite things. My Actor separates them: an unreadable slug gets its own free row with type: "error" saying so, and a run where the filters emptied everything gets a free row naming the filter to widen. Free as in not billed — you pay per job row only. Practical rule: open the company's careers page, click a job, and read the link. lever.co/<slug> in the URL is your answer.

Three fields I would not trust without looking

employmentType. Lever's underlying field is commitment, free text the company types. My 2,207 postings contain 21 distinct spellings, including Full-Time, Full Time Employee, Full-time, Temp Full-time, Full-time: Remote, International EOR, CLT and Scholarship. The Actor lowercases and strips punctuation, which collapses 21 to 18 — fulltime 1,373 rows, fulltimeemployee 558. Same thing, two keys. If you filter on equality you will lose 558 of those 1,931 full-time jobs.

Salary. Lever has a structured salaryRange, and 597 of 2,207 postings fill it — 27%, which sounds usable until you see where it comes from: 538 of those 597 are one company (Shield AI). Spotify, Palantir, Veeva, Wealthfront, Ro and Neon publish zero structured ranges between them. Publishing a range is a company policy, not a market average, so cross-company salary stats built on this field describe whoever opted in. The Actor's workaround is to read the range out of the posting text when the field is empty and flag which it was: across 100 distinct rows, 12 had salarySource: "board-field" and 25 had "posting-text". Two thirds of the salary I got was written in prose.

emails. Contact extraction from postings sounds like recruiter leads. On those same 100 rows, 38 carried an email — and all 38 were Palantir, all of them the identical accommodations@palantir.com from the legal boilerplate at the bottom of every posting. It is a correct extraction of a useless address. Dedupe per company before you call anything a lead.

One field held up better than I expected: workplaceType. Lever lets a company leave it unspecified, and my Actor maps unspecified to null — but 0 of 2,207 postings were unspecified. All of them say onsite, hybrid or remote. The catch is concentration again: 828 rows say remote and 721 of them are Veeva.

Calling it

Input, as plain as it gets:

{
  "companies": ["spotify", "https://jobs.lever.co/palantir"],
  "titleKeywords": ["engineer"],
  "locations": ["London", "Stockholm"],
  "maxJobsPerCompany": 100
}

A real row, trimmed:

{
  "type": "job",
  "source": "lever",
  "board": "spotify",
  "title": "Staff Engineer – Identity and Access Management User Platform",
  "location": "Stockholm",
  "allLocations": ["Stockholm", "London"],
  "countryCode": "SE",
  "department": "Platform",
  "employmentType": "permanent",
  "workplaceType": "remote",
  "postedAt": "2026-11-04T16:12:53.423Z",
  "company": "Spotify",
  "url": "https://jobs.lever.co/spotify/d86e7feb-…",
  "applyUrl": "https://jobs.lever.co/spotify/d86e7feb-…/apply",
  "salaryMin": null, "salarySource": null,
  "emails": []
}

Note location is the first of allLocations, not the only one — 280 of my 2,207 postings list more than one city (33 of Spotify's 86 do). Location filters match against the whole list, so filtering for London keeps a job whose primary city is Stockholm. That is intentional and worth remembering when you count jobs per city.

From Python, run and read in one go:

import requests
TOKEN = "apify_api_..."
r = requests.post(
    "https://api.apify.com/v2/acts/lergassy~lever-jobs-scraper/run-sync-get-dataset-items",
    params={"token": TOKEN},
    json={"companies": ["spotify", "outreach"], "maxJobsPerCompany": 50},
    timeout=300,
).json()
jobs = [x for x in r if x.get("type") == "job"]
print(len(jobs), "jobs;", sum(1 for j in jobs if j["salarySource"]), "with a salary")

The Actor is here: Lever Jobs Scraper. It bills per job row, so filters cost you nothing and error rows are free.

When it is the wrong tool. There is no keyword search across all of Lever — you bring the company list, and if you don't have one, this Actor cannot find it for you. For market-wide searching I use a different one of mine, Jobs API, which searches Indeed and LinkedIn as well as reading boards. And if your product needs "posted today" to be literally true, no Lever scraper can give you that, mine included: the only timestamp in the source is the day somebody created a database record, and sometimes that was 2009.

224 'New' Remote Jobs in Twelve Days. Thirty-One Were Posted in the Last 24 Hours

Matvey — Thu, 05 Nov 2026 05:13:50 +0000

I run a visa agency in Bali, and the other half of my week goes into Apify Actors. One of them reads Indeed's remote listings — the postings the employer actually flagged Remote, not the ones with "remote" in the title. It gets used for the boring, useful thing: a daily feed of fresh remote roles in a category.

A daily feed sounds like a solved problem. Pull the top 25 per keyword every morning, keep the job IDs you have not seen, email the rest. I had that running against my own Actor as a health check, so I had two weeks of identical runs sitting in storage. When I finally went through them, the "unseen IDs" rule turned out to be measuring something other than new jobs.

The runs

Twenty-four runs, 22 October to 4 November, byte-identical input every time:

{
  "keywords": ["customer support", "software engineer"],
  "country": "us",
  "maxJobsPerQuery": 25,
  "datePosted": "any",
  "sortBy": "relevance",
  "includeContacts": true
}

Fifty rows in 23 of the 24 runs — 25 per keyword — and 48 once, when Indeed returned only 23 engineers. Median wall time 6.1 seconds (3.6 to 16.0) for 50 jobs with full descriptions, because Indeed's search endpoint hands over the whole posting in the search response; there is no per-job page fetch to pay for.

1,198 job rows in total. 373 distinct job IDs.

That ratio is the whole article. Twenty-four pulls of "the top 25" produced 373 different jobs, not 25 jobs seen 24 times.

The top 25 is not a top 25

Here is the overlap between consecutive software engineer pulls, counting shared job IDs out of 25:

Pair	Gap	Shared
31 Oct 20:11 → 20:32	21 min	25 / 25
1 Nov 20:10 → 20:32	22 min	25 / 25
2 Nov 20:10 → 20:13	3 min	25 / 25
3 Nov 20:12 → 20:18	6.5 min	0 / 25
4 Nov 20:12 → 20:14	2 min	22 / 25

Same keyword, same filter, same country, minutes apart. Sometimes the identical page; once, 0 of 25 in six and a half minutes. Across all 23 consecutive software engineer pairs the overlap ranged from 0 to 25, with eight pairs at 6 or fewer.

Across the whole fortnight, counting how many of the 24 runs each job appeared in:

Appeared in	Jobs
exactly 1 run	165
2–3 runs	104
4–11 runs	93
12 or more	11

One job survived 22 of 24 runs (a Software Engineer role at Trillium Engineering, posted 26 August). Forty-four percent of everything I ever saw showed up once and never again.

This is not jobs appearing and expiring. expired was false on all 373 rows, and the two near-disjoint runs are not split by freshness in any clean way — the 20:12 set had a median posting age of 98 days, the 20:18 set 40 days, and both contained postings from the same week. What varies is which slice of a much larger result set Indeed's relevance ranking decides to put on page one for a given request. The filter is deterministic; the ordering is not.

So: do not treat position in a relevance-sorted job search as a stable key for anything. Not "the market shrank", not "this role was pulled", not "we are now ranking above competitor X". Twenty-five is a sample, drawn fresh each time.

Deduplicating by ID does not give you new jobs

Take one run per day, keep a running set of IDs, count the ones you have never seen — the classic feed rule. Then check how many of those were actually posted in the previous 24 hours:

Day	New IDs	Posted in last 24 h
23 Oct	26	3
24 Oct	23	2
26 Oct	14	1
27 Oct	9	3
28 Oct	6	3
29 Oct	40	4
30 Oct	24	1
31 Oct	16	1
1 Nov	18	2
2 Nov	4	0
3 Nov	32	6
4 Nov	12	5
Total	224	31

Eighty-six percent of what a dedupe-by-ID feed would have called "new remote jobs" were postings that already existed and simply had not surfaced yet. If you send that to subscribers daily, the complaint arrives in about a week: I applied to this one already.

The fix is on the input side, not in your dedupe logic. Ask Indeed for freshness instead of inferring it:

{"keywords": ["customer support"], "country": "us", "datePosted": "day", "maxJobsPerQuery": 100}

and then still filter on postedAt yourself, because datePosted is Indeed's idea of recent, not yours. Keep the ID set as a safety net against duplicates, never as the definition of "new".

While you are at it, ignore listedAt. It is Indeed's own indexing timestamp and it does not behave: the gap between postedAt and listedAt was exactly zero on 238 of 373 rows, ran up to 2,400 days on the oldest posting, and went negative on three — one row claims it was posted on 2026‑02‑12 and listed on 2025‑03‑19. postedAt is the date to trust.

"Remote" does not mean "location-free"

Every one of the 373 rows carried isRemote: true, workplaceType: "remote", and the string Remote among Indeed's own attributes. The Remote filter is honest. But location:

What `location` contains	Jobs
a city, often with a ZIP — `Boston, MA`, `Salt Lake City, UT`, `New Haven, CT 06510`	196
the literal string `Remote`	159
a state or area name — `Minnesota`, `Puerto Rico`	18

A remote job with a ZIP code is not a contradiction and it is not a bug in the data. It is the employer saying remote, but hire near this office or remote, but in this state for tax reasons. More than half the results were that kind of job. A real row:

{
  "id": "6f72db136ac34058",
  "title": "Software Engineer II, Engineering",
  "company": "Ellevation Education",
  "location": "Boston, MA",
  "region": "MA",
  "countryCode": "US",
  "isRemote": true,
  "workplaceType": "remote",
  "employmentType": "fulltime",
  "salaryMin": 125000,
  "salaryMax": 145000,
  "salaryCurrency": "USD",
  "salaryPeriod": "year",
  "salaryEstimated": false,
  "postedAt": "2026-10-20T05:00:00.000Z",
  "expired": false
}

If your product promises work from anywhere, filter to location == "Remote" and accept that you just threw away 57% of the results. If it promises remote roles for people in Ohio, do the opposite. Either way the decision has to be explicit — isRemote will not make it for you.

Salary: two currencies of time in one column

204 of 373 rows carried a salary. The periods:

`salaryPeriod`	Rows
`year`	110
`hour`	91
`week`	2
`month`	1

Average the midpoints of salaryMin/salaryMax without looking at salaryPeriod and you get $73,309. Annualise first — hourly × 2,080, weekly × 52, monthly × 12 — and the same rows give $101,956, median $95,000. The naive number is not merely wrong, it is wrong in the direction that makes remote work look underpaid, because hourly postings are concentrated in customer support.

MULT = {"year": 1, "hour": 2080, "week": 52, "month": 12, "day": 260}

def annual(row):
    lo, hi, per = row.get("salaryMin"), row.get("salaryMax"), row.get("salaryPeriod")
    if lo is None or per not in MULT:
        return None
    return (lo + (hi or lo)) / 2 * MULT[per]

One more thing in that column: 18 of the 204 values are not round numbers. 99591.140625, 85849.921875, 24.040000915527344. Those are 32-bit floats somewhere upstream of me, round-tripped into JSON. Round them before display — and do not read 99591.14 as a precisely negotiated figure.

A field I expected to matter and did not: salaryEstimated was false on all 204 rows. Indeed's own estimate bands did not appear once in this sample, so every salary here came from the employer. Do not build logic that depends on the estimate being there.

Age, and what `datePosted: "any"` really means

Median posting age in the sample: 30 days. But 86 of 373 were older than 90 days, 25 were older than a year, and the oldest — a Customer Experience Team Member at LuckyGunner.com — was posted on 9 October 2019, seven years ago, still not flagged expired, still landing on page one of a relevance search. "Relevance" has no opinion about age unless you give it one.

Calling it

curl -X POST "https://api.apify.com/v2/acts/lergassy~remote-jobs-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords":["customer support"],"country":"us","datePosted":"day","maxJobsPerQuery":25}'

Fifty rows with full descriptions cost $0.04 on the Free plan ($0.0008 per job, cheaper on paid plans) and come back in about six seconds, so the sync endpoint is fine for feed-sized pulls. The Actor is Remote Jobs Scraper; if you also want on-site and hybrid roles or company career sites, the same engine without the forced Remote filter is Jobs API.

When this is the wrong tool

If you need the complete set of remote jobs for a keyword, no top-N search will give it to you, and my 373-from-24-pulls number is the evidence. Widen maxJobsPerQuery and split the query by state or job type rather than pulling the same page repeatedly — repetition buys you a random new slice, which is a terrible way to pay for coverage.

If you need a market-size figure — "how many remote support roles exist in Canada" — this is the wrong instrument entirely. Search result counts on Indeed are not a census, and a 25-row sample redrawn every call is not a trend line.

Where it does earn its keep is the per-posting layer: full description, apply link, employer website and size, salary as numbers with an explicit period, and the recruiter's email when the posting contains one — 35 of my 373 rows had an email, 9 had a phone — in one request per 25 jobs, with no login and no browser. Just do not ask page one to tell you what changed since yesterday. Ask postedAt.

One MP4, 27 Runs, 15 Different Transcripts. Lower-Case Them and 26 Are the Same 213 Words

Matvey — Wed, 04 Nov 2026 21:12:37 +0000

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them turns video into text — paste a YouTube, TikTok, Instagram or Vimeo link, or a direct MP4, and get the spoken words back with each passage tied to the second it was said.

Like every Actor I publish, it has a scheduled check: once a day, send one fixed file and make sure data comes out. Boring, cheap, and it has now produced something more useful than the alarm it was built for — a month of identical requests against the same audio, 5 October to 4 November 2026. Twenty-seven runs, one 60.01-second MP4, quality: fast (Whisper large v3 turbo), language left to auto-detect and detected as English every time.

Twenty-seven identical requests produced fifteen different transcript strings.

If you are building anything that compares transcripts — change detection on a weekly webinar, a cache keyed on output, a regression test that asserts on text — that number is the one that will hurt you. The good news is the shape of the difference, because it is not what it looks like.

What actually moved

The file is a passage from Pride and Prejudice read aloud three times over, which turns out to be an accidentally excellent test fixture: each run transcribes the same two sentences three times, so you get within-run evidence for free.

	across 27 runs
Distinct transcript strings	15
`wordCount` values	211, 213, 214, 215, 216
`charCount` values	1158, 1159, 1160, 1161, 1162, 1163, 1164
`segmentCount` values	9, 13, 14, 15, 16, 17, 18, 19
`durationSeconds`	60.01 — all 27
`language`	English — all 27

This is not a build drifting under me. The Actor was rebuilt through the month (0.1.1 up to 0.1.17), but seven of these runs are the identical build 0.1.17, and on their own they produce four different word counts (211, 214, 215, 216) and three different segment counts (9, 13, 18). Nothing in the request pins the sampling either — the recogniser call sends no temperature and no seed, so there is no determinism to be had and no bug to fix.

Now normalise. Two steps:

import re

def norm(t):
    t = t.lower()
    t = re.sub(r"[^a-z0-9' ]+", " ", t)      # 1: drop punctuation and hyphens
    t = re.sub(r"\s+", " ", t).strip()
    return t.replace("some one", "someone")  # 2: one compound word

Step 1 alone takes 15 distinct strings down to 8. Step 1 plus step 2 takes them to 2 — and the second of those is a single run. Twenty-six of the 27 transcripts are byte-identical once you lower-case them and collapse one compound: 213 words, every time.

So the honest word-error rate on this file is not "15 variants out of 27". It is one run out of 27.

The three coin flips

Counting all 81 occurrences (three repeats × 27 runs), the model never changes what it heard. It changes how it writes it down:

Choice	Count A	Count B
`someone`	38	`some one` 43
`little known`	79	`little-known` 2
`neighborhood, this truth`	65	`neighborhood. This truth` 16

And — this is the part I did not expect — all 27 runs are internally inconsistent. Not one run punctuated the three identical repeats the same way. A typical row: "the rightful property of some one or other of their daughters" in the first repeat, "someone" in the second, "some one" again in the third. Same voice, same words, same request, eleven seconds of compute, three different answers inside one string.

Which explains the word counts completely. The Actor computes them the dumbest possible way, and should:

'wordCount': len(text.split()),
'charCount': len(text),

some one is two tokens, someone is one. Three occurrences per file means wordCount can legally land anywhere in a four-wide band without a single syllable being misheard. The 211 at the bottom of the range is the one run that wrote little-known twice — a hyphen eating two spaces.

The one run that was actually different

For completeness, the outlier: 20 October, 19 segments, 212 normalised words against everyone else's 213. Two genuine recognition errors, both in the later repeats of the same sentence:

repeat two: ...must be in want of a wife. However, however little known... — a doubled word
repeat three: ...in the minds of the surrounding families is considered the rightful property... — the words that he are gone

Three words wrong out of 213, in 1 of 27 runs. That is the real failure mode to budget for, and it is a different thing from the fourteen cosmetic variants that an exact-match test would also flag.

What I changed in my own pipelines

Never diff raw transcripts. If your job is "has this recording changed", compare normalised text, not the field as delivered. Raw comparison on this file reports a change 14 times out of 27 and is wrong every time.

Never key a cache or a dedup on transcript equality. You will store fifteen copies of one answer. Hash the normalised form, or better, hash the source URL plus durationSeconds — which was 60.01 on all 27 runs and is the field the billing agrees with.

Never treat wordCount as an identity or a volume metric. It is a token count over a string whose tokenisation is unstable. For cost, durationMinutes is the field that matters: all 27 runs billed audio-minute × 1 + file-processed × 1 = $0.011, because billing is per started minute of audio and the audio never changed.

In tests, assert with a tolerance. Mine now compares normalised word sequences and allows a small edit distance — on this fixture, the threshold that passes all 27 runs and would still catch a real break is three words in 213.

If you need the spelling of names and jargon to be stable, that is what vocabularyHint is for — it is the only lever here that actually constrains the output, and it constrains vocabulary, not punctuation.

Calling it

curl -X POST "https://api.apify.com/v2/acts/lergassy~video-to-text/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://example.com/clip.mp4"],"quality":"fast","includeSegments":true}'

Today's scheduled run of exactly that shape: 10.1 seconds, one row, $0.011, on 1 GB of memory. The row carries transcript, wordCount, charCount, segmentCount, durationSeconds, language, model, and segments like these:

[
 {"start": 0,     "duration": 4.56, "text": "It is a truth universally acknowledged, that a single man in possession of a good fortune,"},
 {"start": 4.94,  "duration": 1.36, "text": "must be in want of a wife."},
 {"start": 6.62,  "duration": 4.04, "text": "However little known the feelings or views of such a man may be on his first entering"}
]

A social video link costs more than a file, and it is worth knowing why before you plan a batch. A 152-second Instagram reel in the same series billed 3 audio-minutes + 1 file + 1 video-link = $0.119 and took 20 seconds: the $0.09 is the audio being pulled out of a video page through a residential exit node, and a direct file link never pays it. The Actor is Video to Text.

One error row from the same month explains the charge model better than I can: budget-reached, "This file would be charged $0.091 but only $0.060 remain within the run budget." Cap a run with "Max cost per run" and the cap is counted in dollars against the whole row — file plus minutes plus video-link fee — before the download starts. Error rows are free and carry status: "error", so check status before reading transcript.

When this is the wrong tool

If you need a transcript that is byte-stable across runs, no hosted recogniser will give you one — store the audio and the run ID and re-read the row you kept. If you need per-word timings, these segments are phrase-level: nine of them for a minute of continuous reading, in the most common case. And if the words are already sitting in a YouTube caption track, don't pay to re-recognise them. Where this earns its keep is everything with no caption track at all — meetings, lectures, voice notes, downloaded MP4s — at nine-tenths of a cent a minute, with fifteen different strings that all mean the same 213 words.

The Free Press Gave Me 9% of a Paywalled Post. Lenny's Gave 88%. Both Rows Carry the Same Flag

Matvey — Wed, 04 Nov 2026 13:20:35 +0000

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them reads Substack. Substack serves its own archive pages as JSON, so a publication's whole back catalogue is a few HTTP calls away with no browser and no login — which makes it a tempting source for anything content-shaped: competitor tracking, topic research, RAG corpora.

This week I went to do the obvious thing with it. Pull a few newsletters, pull the body text, and compute something boring like words per like. The arithmetic fell apart on the first sort, and the reason is a single integer field that means two different things depending on who published the post.

The run

Four publications, 25 newest posts each, full text switched on:

{
  "publications": [
    "https://www.lennysnewsletter.com",
    "noahpinion.substack.com",
    "https://www.astralcodexten.com",
    "https://www.thefp.com"
  ],
  "maxPostsPerPublication": 25,
  "includeContent": true
}

100 post rows, 4 publication rows, 7 m 50 s, charged $0.09025. The split: 51 free posts, 49 paid, though that split is wildly uneven by publication — 22 of The Free Press's 25 newest posts are paywalled, against 1 of 25 at Astral Codex Ten.

Every row carries Substack's own wordCount (its wordcount field, verbatim) and, with the switch on, contentHtml plus a plain-text contentText. I counted the words in contentText myself with len(re.findall(r"[A-Za-z0-9'-]+", text)) and divided.

Free posts: the number is honest


Free posts measured	51
Median `my count / wordCount`	1.005
Minimum	0.981
Maximum	1.298

Median of 1.005 on 51 posts is as close to "the same number" as two different tokenisers get. The 1.298 outlier is interesting though, and it is not a scraping failure.

That post is an Astral Codex Ten book review of Lee Kuan Yew's memoirs: wordCount 5,683, my count 7,375. Its HTML contains 72 <div class="footnote"> blocks. Strip those and re-count: 5,652 against a claimed 5,683 — a 0.5% gap, which is just my regex and Substack's disagreeing about hyphens.

So: Substack's wordcount excludes footnotes. Seventeen of my 51 free posts read more than 2% long, and sixteen of those were footnote-carrying Astral Codex Ten and Noahpinion pieces. "Storm Of Steel" (36 footnotes): 7,559 claimed, 8,625 counted, 7,551 with footnotes removed. "Why Donald Trump won in 2024" (34 footnotes): 12,818 claimed, 13,454 counted, 12,769 without. If you are building a reading-time estimate or chunking for RAG, that is a real 8–30% of the text living outside the official length.

Paid posts: the number describes a document you don't have

Paid posts come back with every piece of metadata intact — title, date, likes, comments, restacks, wordCount — and the body you get is the publisher's free preview, correctly flagged as contentIsPreview: true and, sensibly, not charged for.

What no field tells you is how much of the post that preview is. On my 49 paid posts:


Median `my count / wordCount`	0.344
Words `wordCount` promised, in total	134,493
Words actually delivered	65,845

Half the corpus, and the halving is not uniform. Grouped by publication it stops looking like a platform behaviour and starts looking like an editorial decision:

Publication	paid posts	median share delivered	range	claimed → delivered
Lenny's Newsletter	19	70.2%	52.5–87.5%	74,441 → 51,857
Noahpinion	7	34.4%	20.3–42.1%	25,420 → 8,367
The Free Press	22	15.7%	9.0–76.8%	34,612 → 5,621
Astral Codex Ten	1	0%	—	20 → 0

Concretely: How to turn your AI into a world-class design partner claims 4,604 words at Lenny's and handed me 4,028 of them. Meet the 'Election Defenders' Preparing for a Contested Election claims 1,751 words at The Free Press and handed me 167. Both rows say isPaid: true, contentIsPreview: true. Nothing in the data distinguishes "we show you most of it" from "we show you the first paragraph."

Two things make this easy to miss:

The previews do not look truncated. Only 2 of 49 paid bodies end in an ellipsis, and both are Free Press posts. Lenny's previews just stop — my sample's cut on How to tell if a startup has a moat lands mid-argument on "...but many still fail to become good businesses." If your pipeline's heuristic for "is this complete" is a trailing …, it will pass 47 of 49 truncated documents.

contentIsPreview is not a paywall flag. One paid post, Astral Codex Ten's Hidden Open Thread 454.5, returned contentHtml: null — nothing at all. That makes contentIsPreview false, because there is no preview to flag. Filter on isPaid, not on the preview flag, or that row quietly joins your "full text" bucket with an empty body.

The practical rule I now use: wordCount is a property of the post; contentText is a property of what Substack is willing to show an anonymous reader. Never divide one by the other across a mixed free/paid sample, and never put paid bodies in the same corpus as free ones without recording the share:

import re
WORD = re.compile(r"[A-Za-z0-9'\-]+")

for r in (x for x in items if x["type"] == "post"):
    got = len(WORD.findall(r.get("contentText") or ""))
    share = got / r["wordCount"] if r.get("wordCount") else None
    r["textShare"] = share          # 1.0-ish = whole post; 0.09 = first paragraph
    r["usableText"] = (not r["isPaid"]) and got > 0

Two smaller traps in the same dataset

The publication row's url is whatever you typed. I passed noahpinion.substack.com; the publication row came back with url: "https://noahpinion.substack.com" while all 25 of its post URLs are on www.noahpinion.blog. Join posts to publications on the publication name or on customDomain, never on the host of url. (The Free Press is the same trap from the other end: its subdomain is bariweiss, not thefp — the brand was renamed, the subdomain was not.)

subscriberTier is empty more often than you'd guess. Substack publishes a band, not a count — "Millions of subscribers" for Lenny's, "Hundreds of thousands" for Noahpinion — and subscribersOrderOfMagnitude turns that into 1000000 and 100000 respectively. But Astral Codex Ten and The Free Press returned null for both fields. Two of four. If you are sizing a niche, expect a coverage hole, not a number for every row; plans was filled for all four ($10/month and $100/year at three of them, $20/$200 at Lenny's).

Calling it

curl -X POST "https://api.apify.com/v2/acts/lergassy~substack-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"publications":["noahpinion.substack.com"],"maxPostsPerPublication":2}'

That exact call, this morning: 22 seconds, 4 rows, publication ×1 + post ×2 = $0.0029. The last row has type: "notice" and is a free plain-English note about what you were charged — it is not data, filter it out. The Actor is Substack Scraper.

One operational note from the logs: a single request to noahpinion.substack.com died with tls handshake eof and succeeded on retry from a different address. *.substack.com is pickier than the custom domains — it answers 403 to a Chrome-impersonating client from every address I have, while a plain client gets a 200 and the redirect. If you write your own Substack client and it works on your laptop but 403s in CI, that asymmetry is what you are hitting, and the fix is less browser-like, not more.

When this is the wrong tool

If you need the body of paid posts, nothing here helps and nothing should — the median paid preview in my sample was a third of the article, and no proxy changes that. If you need subscriber counts rather than bands, Substack does not publish them. And if you want comments or Notes, this reads posts and publications only. Where it does earn its keep is the metadata layer: 100 posts across four publications with likes, restacks, word counts, authors, tags and paywall status, in under eight minutes for nine cents — the archive endpoint hands all of that over in one call per 20 posts, whether you ask for the text or not.

I Averaged All 31 Reviews 2GIS Handed Me and Got 4.36. The Page Says 4.6

Matvey — Wed, 04 Nov 2026 05:14:20 +0000

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them reads 2GIS — the business directory that covers Russia, Kazakhstan, the UAE, Uzbekistan, Czechia, Chile and a dozen more countries, and that most Western scraping stacks have never heard of. It is a Google Maps substitute in places where Google Maps is thin, and unlike Maps it hands out e-mails.

This week I went to do the boring thing everybody does with review data: pull a category, pull the reviews, recompute the average, see who is really good. It did not reconcile, and the reason is a field pair that looks redundant and is not.

The two counters

Every place row comes back with both of these:

"rating": 4.6, "reviewCount": 30, "ratingCount": 41

reviewCount is 2GIS's general_review_count. ratingCount is general_review_count_with_stars. The second one is always the larger of the two, because on 2GIS you can tap stars without writing anything — and the reviews feed only serves the ones with text.

On a 40-row pull of кофейня in Novosibirsk (438 results exist; I took the first 40):


Places where `ratingCount` > `reviewCount`	40 of 40
Sum of `reviewCount`	34,509
Sum of `ratingCount`	72,030
Readable share of the vote base	47.9%
Median readable share per place	61.1%
Worst place	19.4%
Best place	87.7%

So across this sample, slightly more than half of the opinions that move a 2GIS star rating have no text attached and are not in any feed you can page through. That is not a scraper limitation I can fix with a better proxy; the endpoint does not carry them.

Then the arithmetic stopped working

To check whether the readable half is at least a fair sample, I took the smallest place in the set — Breakfast house, Красный проспект 31, 30 text reviews against 41 star votes — and pulled every review it has.

{ "placeUrls": ["70000001117226436"], "includeReviews": true, "maxReviewsPerPlace": 500 }

31 reviews came back (the counter said 30 — it lags; one arrived that morning). All 31 had text. 26 of 31 carried an owner reply, 18 carried photos, and they spanned 25 August to 4 November 2026.

The star distribution of those 31: 24 fives, 1 four, 1 three, 3 twos, 2 ones. Sum 135, mean 4.355.

The page says 4.6.

Before blaming the ten invisible voters, I checked whether they could close the gap. Give every one of the ten unseen votes a perfect 5:

(135 + 50) / 41 = 4.512

Still not 4.6. The displayed rating is not the arithmetic mean of all 41 votes under any assignment of the ten I cannot see — the maximum reachable value is 4.51. 2GIS is weighting something: recency, reviewer history, something it does not publish. I cannot tell you which, and neither can anyone reading the API. What I can tell you is that it is settled, not a rounding artefact.

Practical consequence: do not recompute the rating from scraped reviews and present it as the rating. They are two different numbers measuring overlapping but unequal populations, and on a real place they came out 0.25 stars apart in my sample — in the direction that makes the business look worse than its page does.

If you want a defensible metric from review text, compute it and name it for what it is — "mean of 31 readable reviews" — and keep rating beside it as the platform's own figure.

The other number that is not about this place

Chains make the same mistake available in a second flavour. Every row carries orgRating and orgBranchCount next to rating, and orgRating is the brand's score across all its branches.

Of my 40 coffee shops, 24 were branches of a multi-location brand. In 21 of those 24 the brand score was higher than the branch's own, mean gap +0.29, worst case +0.9: Куликов on улица Кирова sits at 3.8 with 1,002 reviews while the 41-branch brand averages 4.7.

The sample also contained the same brand several times over, which makes it very visible:

Brand	branch ratings in the sample	`orgRating`
Академия Кофе	4.6, 4.5, 4.5, 4.5	4.7
Skuratov Coffee	4.5, 5.0, 5.0, 4.7	4.8
Brewsell	4.1, 4.4, 4.6	4.7
Чашка кофе	3.4, 3.8	4.3

Pick the wrong column and every branch of Чашка кофе scores 4.3, including the one the neighbourhood rates 3.4. Sort a lead list by orgRating and you have sorted brands, not locations.

Running it

The Actor is plain HTTP against the two public JSON APIs the 2GIS web app uses — no browser, no login, no API key of your own. 2GIS Places Scraper, one row per place, reviews as an optional second row type.

curl -X POST "https://api.apify.com/v2/acts/lergassy~2gis-places-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries":["кофейня"],"cities":["Новосибирск"],"maxPlacesPerQuery":2,"includeReviews":true,"maxReviewsPerPlace":3}'

That exact call, this morning: 8 rows in 6.6 s, charged place ×2 and review ×6 = $0.0082.

Reading it back, with the two traps handled:

rows = [r for r in items if r["type"] != "notice"]   # the free summary row
places  = [r for r in rows if r["type"] == "place"]
reviews = [r for r in rows if r["type"] == "review"]

for p in places:
    readable = p["reviewCount"] / p["ratingCount"]    # how much of the rating you can read
    mine = [r["rating"] for r in reviews if r["placeId"] == p["id"]]
    print(p["name"], p["rating"], round(readable, 2),
          round(sum(mine) / len(mine), 2) if mine else None,
          "chain" if (p["orgBranchCount"] or 1) > 1 else "solo")

Three small things that bite if you skip them:

The last row of every run has type: "notice" and is a free plain-English summary of what you were charged. It is not data. Filter it.
Reviews are a separate row type in the same dataset, joined by placeId, not nested inside the place. Export to CSV without splitting by type and you get a sheet with two different column sets stacked.
emails was filled on 18 of my 40 places, phones on 36, websites on 34, socials on 36. For contact data that is a good hit rate by directory standards, but it is a directory, not a verifier — the e-mail is whatever the business typed into its listing.

One thing that genuinely needs a proxy

While writing this I fired the same reviews endpoint straight from a plain datacenter host with curl, to grab a raw response for the article. It never answered — still hanging after 120 seconds, when the identical request through the Actor on Apify's datacenter proxy had returned in under seven. The catalog API does this too; it is why the Actor retries twice and then falls back to a residential address instead of simply timing out. If you build your own 2GIS client and it "works on my laptop, hangs in CI", that is the thing you are hitting, not rate limiting.

When not to use it

2GIS is organised by city, not by country — Москва, Алматы, Dubai work, Russia does not, and an uncovered city comes back as one free error row. Reviews are $1 per 1,000 on top of $1.50 per 1,000 places, so turning them on across a 5,000-place sweep is a real bill; sample instead. And if the number you actually need is the vote count behind the star rating rather than the readable reviews, take ratingCount and stop — the other 52% is not coming.

Nine of Twelve Outbound Links Came From Accounts Pinterest Made Itself. Their Profiles Report 1 Monthly Views

Matvey — Tue, 03 Nov 2026 21:16:49 +0000

I run a visa agency in Bali, and the other half of my week goes into Apify Actors. The Pinterest one answers a question marketing people ask constantly: for this niche, which websites is Pinterest actually sending clicks to? One run, group the rows by domain, done.

On 3 November 2026 I ran it on minimalist kitchen and the answer came back as a list of websites I had never heard of — pinned by accounts with zero followers, zero pins and a name that starts with preview_.

The run

Pinterest search, pins scope, 20 pins, engagement switch on, 3 November 2026, no login. Pinterest Scraper on a plain Apify account.


Rows	20 of 20 asked for, zero errors
Run time	10.8 s
Pins with an outbound link	12 of 20
…of those, pinned by a `preview_` account	9 of 12
Pinners with a `preview_` username	9 of 20
`title` filled	18 of 20
`saves` filled	0 of 20
`reactions` filled	2 of 20
Charged	`pin ×20` = $0.03, `pin-details ×0`

Nine of the twenty results were pinned by preview_roomhint_com, preview_linbiepham_com, preview_miscaluxe_com, preview_elodiestyling_com, preview_prettynestdecor_de, preview_wishonbrithday_com, preview_38f117427e82883f95d280 and preview_bd3b790782095ce502293e. Every single one of those nine pins carried a title and an outbound link. The eleven ordinary accounts in the same twenty produced three links between them.

What a preview account looks like from the profile endpoint

So I fed six of those usernames back in as profile inputs. Six profile rows, 2.1 s, $0.012:

field	all six
`fullName`	`Pinterest Preview <the username's tail>`
`followers`	0, 0, 0, 0, 2, 3
`pinCount`	0 on five of six (26 on the sixth)
`boardCount`	1
`websiteUrl`	`null`
`monthlyViews`	−1
`imageUrl`	`s.pinimg.com/images/user/default_280.png`
`createdAt`	15–21 October 2026

Two things in there are worth stopping on.

monthlyViews is −1, on all six. The field comes from Pinterest's profile_reach, and −1 is not a number of views — it is a sentinel meaning "not computed for this account". If you average that column across a sample, every preview account drags your mean down by one view and silently turns into a real data point. I treat anything below zero as missing now, and so should any aggregation you write on top of these rows.

websiteUrl is null — on accounts whose entire purpose is to represent a website. The domain is in the username (preview_miscaluxe_com → miscaluxe.com) and in each pin's link, and nowhere in the field named for it. Two of the nine are named with hex instead of a domain (preview_bd3b790782095ce502293e), and those still link out to ordinary sites — homestorageorganizer.com and cozykitchenandliving.com — so the hex is an opaque id for a domain, not the absence of one.

These are not dead accounts, either. I checked every domain the nine pins pointed at: roomhint.com, linbiepham.com, elodiestyling.com, prettynestdecor.de, miscaluxe.com, wishonbrithday.com, cozykitchenandliving.com — 7 of 7 answered 200 with real titles ("RoomHint | Storage, Organization & Room Ideas That Work", "Misca Luxe: Bathroom Fixes & Kitchen Cabinet Ideas"). Small content sites, all created-or-indexed in the last few weeks, all getting distribution through an account they appear not to have had to build.

Their pins only exist in search

Here is the part that breaks the obvious next step. Having found nine interesting publishers, the natural move is to walk each profile and see everything they pin. All six preview profiles returned zero pins. The profile feed is Pinterest's _created tab, and for these accounts it is empty — consistent with the pinCount: 0 in the row, and completely inconsistent with the pins I was holding in my hand from the same run.

Their board is no help either. The boardUrl on those nine pins is /preview_<domain>/_inspo/ — one board, named _inspo, and Pinterest's board endpoint does not serve it. Pass that URL in and the scraper reads it as a profile instead, because a path segment starting with _ is a profile tab (/_created/, /_saved/), not a board slug. Across 60 pins from three keywords, 13 pointed at an underscore board: 11 _inspo and 2 _quick_saves.

So the only place these pins are reachable is the search index. For a traffic-sources report that is fine — search is where you were looking anyway. For "show me everything this competitor publishes" it is a dead end, and the dead end is not in my code.

What it changes practically: don't filter a Pinterest sample by follower count. The obvious cleanup — drop pinners under 10 followers as noise — would have deleted three quarters of the link data on this keyword. The accounts holding the links are supposed to have no followers.

The entry point decides the ID format, and the bill

The same run also settled something I had been guessing at. I pulled one account two ways at once — its board, and its profile — 20 pins from each:

	board feed	profile feed
Pins	20	20
Numeric `pinId`	0 of 20	20 of 20
Engagement lookups charged	0	20

Pinterest has been moving logged-out pin pages to opaque tokens — e1_AfqPShxPH41hAo83UO7dgaAaDCq5sJaNsLtv3S…, 55-plus characters — and its own pin endpoint answers Pin not found on them. I measured that on search results last week, with the search-only Actor. What I did not expect is that two feeds belonging to the same account in the same run disagree: the board feed hands out tokens, the _created profile feed still hands out numeric ids like 1063131055810574252. Numeric ids answer the pin endpoint, so the "exact saves and comments" switch fired on exactly the 20 profile pins and silently did nothing for the 20 board pins.

The bill shows it: profile ×1, pin ×40, pin-details ×20 = $0.072. Twenty detail lookups, forty pins. If you want engagement numbers, enter the account by profile, not by board — and if an Actor charges you per detail lookup, check that it is not charging for the ones that came back empty.

The boardUrl did not round-trip, and that one was mine

While measuring the above I fed a boardUrl from a search result straight back in as an input, which is the most obvious thing a user will ever do:

https://www.pinterest.com/cornerrenovation/kitchen-design-tips-guides-%2B-checklists/
→ board not found. Not charged.

Pinterest writes its own board URLs percent-encoded. Its BoardResource wants the decoded slug — kitchen-design-tips-guides-+-checklists. I was splitting the path and passing the segments through raw, so %2B went out as three characters and Pinterest 404'd. Three lines:

// Pinterest's own board URLs come percent-encoded ('…-%2B-checklists'), but its BoardResource
// wants the decoded slug — feeding a boardUrl straight back answered 'board not found'.
const unesc = (x) => { try { return decodeURIComponent(x); } catch { return x; } };
const parts = u.pathname.split('/').filter(Boolean).map(unesc);

Before: 0 pins, board not found, charged $0. After, same URL, same build pipeline: 3 pins, 4 s, each with its outbound link to cornerrenovation.com/blog/…. The try/catch is not decoration — a malformed escape like %E0%A4%A throws, and a bad input should degrade to "slug not found" rather than killing the run. Search and profile inputs re-verified on the same build before the latest tag moved.

And one hypothesis the numbers killed

My first reading of the board feed was "it strips the text fields" — 20 board rows with title, description and link all null. That was wrong, and it was wrong in the most ordinary way: the account I happened to test publishes untitled, unlinked image pins. Its own numeric profile pins, detail lookup and all, came back equally empty. A second board from a different owner, 10 pins, same build: title 10 of 10, description 10 of 10, alt text 10 of 10.

So the board feed is fine. One account's content was the finding. If a field is empty across your whole sample, check a second source before you write it down as an API limitation — I nearly shipped a README note saying the opposite.

Calling it

curl -X POST "https://api.apify.com/v2/acts/lergassy~pinterest-scraper/runs?token=$APIFY_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"searchQueries":["minimalist kitchen"],
       "urls":["https://www.pinterest.com/minimalistliving01/"],
       "maxPinsPerSource":50,
       "onlyWithLink":true}'

onlyWithLink is the switch I use for traffic work: on boho bedroom it would have dropped 14 of 20 rows, because pins uploaded straight to Pinterest point nowhere. Link coverage swings hard by keyword — 19 of 20 on air fryer recipes, 12 of 20 on minimalist kitchen, 6 of 20 on boho bedroom — so your traffic denominator is the linked subset, never the row count.

When not to use this. If you need saves and comments at scale, this is the wrong road: go through a business account and Pinterest's official API, which is what those numbers are for. If you want everything one creator ever pinned, enter by profile and accept that preview accounts will return nothing. And if you are trying to study the sites behind a niche rather than the pins, the links are the whole product — pull them, group them, and then go read the sites.

Three keywords, 60 pins, under thirty cents. The useful output was not a ranking. It was a list of seven live content sites I would never have found by searching the web, plus the knowledge that Pinterest is handing them the distribution.

DEV Community: Matvey

46 Runs, One Route, the Same Cheapest Flight Every Time. The Two Sources Never Once Agreed on Its Price

Both sources agree on the flight. Neither agrees on the price

The seller column explains all of it

And across the rest of the board, the direction reverses

Three joins that look right and are not

The merge mode, and where it helps

Calling it

When this is the wrong tool

Trump Posted 206 Times on 3 November. My Daily Read Saw 28 of Them

The posting rate is not a rate

postedAfter does not define your window — the cap does

Catch a post in the first minute and its engagement is a rounding error

A third of the sample had no text — and then none of it did

What the site will not give you without a login

Calling it

797 of 1,071 TikTok Shop Reviews Came With No Text At All. The 1-Star Ones Talk Twice as Often

Two samples, the same 72%

The complaints talk, the praise does not

What the text is when there is text

Three fields that carried no information, and two that do

The ceiling nobody mentions: 5,000 per filter

How it gets them, in case you are building your own

Calling it

1,084 Mercado Libre Cards Told Me How Many Were Sold. Between Them They Used Ten Numbers

soldQuantity is a bucket wearing a number's clothes

Over three weeks, price moves and the bucket does not

Two layouts of one listing, two different prices

Positions 1–4 and 13–16 are not results

Calling it

168 of 169 Paid 'description' Fields Were Identical to the Free 'summary'. The Real Text Was 17x Longer

The meta tag is not the description

currency is the organizer's currency, not the country's

A series row is one date out of hundreds

Calling it

677 of 785 Telegram Reaction Chips Counted Zero. The Custom Emoji and the Paid Stars Were Fine

Three ways to draw a reaction, one of which broke everything

What reactionsTotal still is not

views is a running counter, and my own history proves it

postId is not a post count — it is a message counter, and albums eat it

Every failure answers HTTP 200

Calling it

One Live Palantir Posting Is Dated December 2009. 535 of My 2,207 Lever Jobs Are Over a Year Old

The field, and what it actually is

The distribution, not the outlier

The cap hides it, and that is how I missed it for weeks

Getting the slug right is most of the work

Three fields I would not trust without looking

Calling it

224 'New' Remote Jobs in Twelve Days. Thirty-One Were Posted in the Last 24 Hours

The runs

The top 25 is not a top 25

Deduplicating by ID does not give you new jobs

"Remote" does not mean "location-free"

Salary: two currencies of time in one column

Age, and what datePosted: "any" really means

Calling it

When this is the wrong tool

One MP4, 27 Runs, 15 Different Transcripts. Lower-Case Them and 26 Are the Same 213 Words

What actually moved

The three coin flips

The one run that was actually different

What I changed in my own pipelines

Calling it

When this is the wrong tool

The Free Press Gave Me 9% of a Paywalled Post. Lenny's Gave 88%. Both Rows Carry the Same Flag

The run

Free posts: the number is honest

Paid posts: the number describes a document you don't have

Two smaller traps in the same dataset

Calling it

When this is the wrong tool

I Averaged All 31 Reviews 2GIS Handed Me and Got 4.36. The Page Says 4.6

The two counters

Then the arithmetic stopped working

The other number that is not about this place

Running it

One thing that genuinely needs a proxy

When not to use it

Nine of Twelve Outbound Links Came From Accounts Pinterest Made Itself. Their Profiles Report 1 Monthly Views

`postedAfter` does not define your window — the cap does

`soldQuantity` is a bucket wearing a number's clothes

`currency` is the organizer's currency, not the country's

What `reactionsTotal` still is not

`views` is a running counter, and my own history proves it

`postId` is not a post count — it is a message counter, and albums eat it

Age, and what `datePosted: "any"` really means