DEV Community

Matvey
Matvey

Posted on

Rows 1–20 Said '4y ago', Rows 21–100 Said '4 years ago'. Neither One Is a Date

I run a visa agency in Bali, and the other half of my week goes into Actors I publish on the Apify Store. YouTube search is the one I use for my own marketing: what ranks for "bali villa tour", who the channels in that niche are, whether my own videos moved. The YouTube Data API charges 100 quota points per search against a 10,000-point daily ceiling — about 100 searches a day — so my YouTube Search Scraper reads the search page the way the website does instead.

Yesterday I ran the same search four times and read the output as a stranger would. The input was frozen, 30 October 2026:

{
  "searchQueries": ["bali villa tour"],
  "resultType": "video",
  "maxResultsPerQuery": 100
}
Enter fullscreen mode Exit fullscreen mode

100 rows in 7.7 seconds. Then "n8n tutorial", then the first search again two minutes later, then one with the upload-date filter on. Six runs, 360 video rows and 50 channel rows — and a column that cannot make up its mind.

The spelling changes at row 21

Row 20 and row 21 of a single run, trimmed to the fields that matter:

{
  "position": 20,
  "title": "Modern Bali Villa Inspired by Nature | A Peek In Paradise S8 EP1 ...",
  "viewsText": "236,649 views",
  "publishedText": "1y ago",
  "duration": "11:15"
}
{
  "position": 21,
  "title": "Bali Villa for Sale - Hidden Luxury Nature Villa in Ubud",
  "viewsText": "38,314 views",
  "publishedText": "6 months ago",
  "duration": "6:26"
}
Enter fullscreen mode Exit fullscreen mode

1y ago against 6 months ago. Not a different unit — a different dialect. The short form across four runs:

run              rows 1-20 abbreviated   rows 21+ abbreviated
bali villa tour        20 / 20                 0 / 80
bali villa tour #2     20 / 20                 0 / 80
n8n tutorial           20 / 20                 0 / 80
bali villa tour, week  20 / 20                 0 / 40
Enter fullscreen mode Exit fullscreen mode

Eighty short-form rows, every one in the first twenty positions, four runs out of four. The boundary has nothing to do with the videos and everything to do with where the bytes came from. Page one of a YouTube search arrives inside the HTML, in the ytInitialData blob served to a logged-out visitor, written in YouTube's compact style: 15h ago, 2w ago, 11mo ago, 1y ago. Everything past position 20 comes from the continuation endpoint the site calls while you scroll, and that one spells the words out: 21 hours ago, 2 weeks ago, 1 year ago.

Any parser has to eat both dialects, and the naive version fails quietly:

# wrong: misses "1y ago" and reads "11mo ago" as 11 months of nothing
if "year" in text: ...

# works on both dialects
import re
UNITS = {"s": 1/86400, "min": 1/1440, "h": 1/24, "d": 1,
         "w": 7, "mo": 30, "y": 365}
def age_days(text):
    m = re.search(r"(\d+)\s*(s|sec|min|h|hour|d|day|w|week|mo|month|y|year)",
                  text or "", re.I)
    if not m:
        return None
    n, unit = int(m.group(1)), m.group(2).lower()
    key = next(k for k in ("mo", "min", "s", "h", "d", "w", "y") if unit.startswith(k))
    return n * UNITS[key]
Enter fullscreen mode Exit fullscreen mode

Note the ordering trap in that next(...): test mo before min, or "month" becomes a minute. The Actor normalises both dialects internally, but it passes publishedText through verbatim — the moment I "clean" it I am inventing precision.

Which is the real problem

Here is the granularity of 200 rows from the two unfiltered searches:

years    105
months    78
weeks      9
days       7
hours      1
Enter fullscreen mode Exit fullscreen mode

183 of 200 rows know their own age only to the nearest month or the nearest year. 3 years ago covers everything from 1,096 to 1,460 days. You cannot sort that column, you cannot filter "published after 1 June" with it, and you cannot tell which of two rows is newer unless they are weeks apart.

Two cheap ways out:

  1. Let YouTube do the filtering. The uploadDate filter is free and it is exact, because it runs server-side before the page is built. I ran "uploadDate": "week" on the same search: 60 rows, and the oldest was 7 days ago on the nose. Zero leakage. (The Actor re-checks the window itself anyway, because YouTube does occasionally mix an older video into a date-filtered page.) If the question is "what is new this week", ask the filter, not the column.
  2. Resolve the handful you care about. Search gives 100 candidates; you rarely need the exact date for all of them. The per-video page carries Jun 14, 2023 as plain text — that is what my YouTube Scraper reads, along with exact likes and comments. Search first, resolve second.

What the search page is exact about

Views. All 200 rows in the two unfiltered runs arrived as comma-separated integers — 274,310 views, 38,314 views — with no 1.2M rounding anywhere, and the four zero-view rows in the filtered run said No views and parsed to 0, not null. For video results, the number is the number.

Channel results are the opposite, and the boundary is sharp. From 50 channel rows across two searches:

388K subscribers     84.4K    30.8K    9.95K    3.74K     <- rounded
737 subscribers      588      115      43       1 subscriber   <- exact
Enter fullscreen mode Exit fullscreen mode

YouTube rounds subscriber counts to three significant figures above 1,000 and prints the exact number below it. That is the site's behaviour, not a scraping artefact — nobody gets the exact subscriber count of a 388K channel from the public page. Two more measured facts about channel rows: videoCount was null on 50 of 50 of them, because the channel card in search carries no video count at all, and 3 of the 50 had subscribersText: null — channels that hide the number. Treat both as missing, not as zero.

position is a photograph, not a rank

This is the one that matters if you came here for rank tracking. I ran the identical search twice, two minutes apart, 100 rows each:

video IDs present in both runs      83 / 100
of those, same position in both      7
largest position shift              47 places
top-20 overlap between the runs     13 / 20
view counts that changed in 2 min   11 / 83
Enter fullscreen mode Exit fullscreen mode

Seven rows out of a hundred held still. A video moved 47 places in two minutes. YouTube search is personalised, A/B-tested and re-ranked continuously, and a logged-out request gets whatever the ranker liked in that second. So position is honest about what it is — where this result sat in this response — and anything built on it has to average. One run a day for thirty days is a trend line; two runs an hour apart is noise. "We rank #4 for X" is defensible only as a median over a week, with the spread shown.

How deep the search actually goes

maxResultsPerQuery allows up to 2,000. YouTube does not go that far on most keywords. I measured the ceiling for free, using the fact that rows removed by minViews are not charged:

{
  "searchQueries": ["bali villa tour"],
  "maxResultsPerQuery": 1000,
  "minViews": 999999999
}
Enter fullscreen mode Exit fullscreen mode

Nothing passes that filter, so the run collected everything YouTube would give, discarded it all and charged nothing: 318 results in 20.8 seconds, 0 rows billed. That is the real depth for this keyword. If you need 3,000 rows about Bali villas, that is twenty keywords at 150 rows each, not one keyword at 3,000 — and any keyword's ceiling is free to check before you commit.

One wrinkle that surprised me the first time I wrote a consumer: the dataset carries one extra row, type: "notice", holding the run's own billing summary. It is free and it is not data. Filter it out.

Calling it from Python:

import requests

TOKEN = "<your Apify token>"
r = requests.post(
    "https://api.apify.com/v2/acts/lergassy~youtube-search-scraper/run-sync-get-dataset-items",
    params={"token": TOKEN},
    json={"searchQueries": ["bali villa tour"], "maxResultsPerQuery": 100,
          "uploadDate": "month", "resultType": "video"},
    timeout=300,
)
rows = [x for x in r.json() if x.get("type") == "video"]
rows.sort(key=lambda x: age_days(x["publishedText"]) or 1e9)   # coarse, and known to be coarse
print(len(rows), rows[0]["title"], rows[0]["publishedText"])
Enter fullscreen mode Exit fullscreen mode

When not to use this

If you need exact upload dates, likes or comment counts for every row, search is the wrong door — it does not carry them, and faking them from publishedText is worse than not having them. If you want one channel's complete back catalogue, a channel scraper walks the Videos tab instead of guessing at search. And if you want guaranteed-complete coverage of a topic, nothing built on YouTube search gives you that: 318 rows was the whole ocean for one of my keywords.

Within those limits it is a boring, fast tool — $0.0005 per result on the free tier and less on paid plans, no fee per run, failed searches and filtered rows free. 100 rows cost me four and a half cents and seven seconds. The publishedText column, though, I treat as a label, not a timestamp.

Top comments (0)