DEV Community: NeverEmpty

The first channelId on a YouTube channel page belongs to a different channel

NeverEmpty — Sat, 29 Aug 2026 14:42:03 +0000

If you resolve a YouTube handle to a channel ID by grabbing the first "channelId" in the page HTML, you are probably getting a different channel. In my sample it was wrong 15 times out of 16 — and the one time it was right is the most dangerous case of all.

I found this the expensive way, after it had silently invalidated a measurement I had already published (since withdrawn).

The check

For each handle I fetched https://www.youtube.com/@name, pulled the first "channelId":"UC..." out of the HTML, and separately pulled the ID from the page's <link rel="canonical">. Then I asked YouTube's public feed what each ID is actually called.

Handle	First `"channelId"` resolves to	`canonical` resolves to
`@veritasium`	Veritasium en Français	Veritasium
`@mkbhd`	The Studio	Marques Brownlee
`@NASA`	Learn With NASA	NASA
`@TED`	TEDx Talks	TED
`@hikakin`	HikakinTV	HIKAKIN
`@3blue1brown`	Mathologer	3Blue1Brown
`@kurzgesagt`	Nightshift – Kurzgesagt After Dark	Kurzgesagt – In a Nutshell
`@Fireship`	Beyond Fireship	Fireship

Eight for eight. I then ran eight more, including the channels that came back as the wrong answers above:

@Computerphile   first → Numberphile                 canonical → Computerphile
@ThePrimeagen    first → The PrimeTime               canonical → ThePrimeagen
@BillieEilish    first → BillieEilishVEVO            canonical → Billie Eilish
@LinusTechTips   first → ShortCircuit                canonical → Linus Tech Tips
@beyondfireship  first → Fireship                    canonical → Beyond Fireship
@Mathologer      first → Mathologer 2                canonical → Mathologer
@nightshift_...  first → Kurzgesagt – In a Nutshell  canonical → Nightshift – …
@LearnWithNASA   first → Learn With NASA             canonical → Learn With NASA   ← same

15 of 16 wrong. And read that last line carefully, because it is the trap inside the trap: on a channel with no featured-channels shelf, the naive method returns the right answer. Test your resolver against one of those and you will conclude your code is fine.

Why this is worse than a loud bug

Look at what it hands you instead. @mkbhd gives you The Studio, whose own description says it carries behind-the-scenes content from the MKBHD studio. @kurzgesagt gives you Nightshift – Kurzgesagt After Dark. @Fireship gives you Beyond Fireship, described as "even more highly-focused blazingly-fast coding tutorials from Fireship". @veritasium gives you Veritasium en Français, which calls itself an official Veritasium channel.

But not always a relative: @3blue1brown gives you Mathologer, an independent channel run by a maths professor at Monash University with no organisational connection to 3Blue1Brown at all. The rule is not "you get a sibling channel". The rule is you get whichever channel the page happens to feature first, and on a channel page that is almost always something on-topic.

So nothing throws. Your rows fill up. Your row count is right. The titles look like the titles you expected. If you are aggregating across channels, you will never see it.

That is exactly what happened to me. I ran a caption survey across twelve channels, verified every number in the write-up against the saved JSON, had the draft adversarially reviewed twice, and published. All of that checked whether the article matched the data. None of it checked whether the data matched the channels I had named. It took an unrelated log line — @veritasium → Veritasium en Français — to notice, hours later.

Where the wrong ID actually lives

I checked the position of that first "channelId" on three pages. On all three it sat inside a gridChannelRenderer in a horizontalListRenderer — a featured-channels shelf — between 37% and 84% of the way through a 1.7–2.5 MB document:

"content":{"horizontalListRenderer":{"items":[{"gridChannelRenderer":{"channelId":"UC…"

Which also explains @LearnWithNASA: no such shelf on the page, so the first "channelId" it finds is the channel's own.

What to use instead

Five markers point at the channel itself, and in this sample all five agreed with each other on every channel I checked:

export function extractChannelId(html) {
  const t = String(html ?? '');
  for (const re of [
    /<link[^>]+rel="canonical"[^>]+href="https:\/\/www\.youtube\.com\/channel\/(UC[\w-]{22})"/i,
    /<meta[^>]+property="og:url"[^>]+content="https:\/\/www\.youtube\.com\/channel\/(UC[\w-]{22})"/i,
    /<meta[^>]+itemprop="identifier"[^>]+content="(UC[\w-]{22})"/i,
    /"externalId"\s*:\s*"(UC[\w-]{22})"/,
    /"rssUrl"\s*:\s*"[^"]*channel_id=(UC[\w-]{22})"/,
  ]) {
    const m = t.match(re);
    if (m) return m[1];
  }
  return null;   // ← not a fallback to "channelId"
}

That last line is the important one. The temptation is to fall back to "channelId" when none of the five match, on the theory that a wrong answer beats no answer. It does not. null is a row you can label "could not resolve this channel". A featured channel's ID is a row that lies to you.

On {22}: it is a validity check, not an over-match guard. Every pattern above is terminated by a closing quote, and " is in neither \w nor -, so UC[\w-]+ would capture exactly the same characters. What {22} buys you is that a token of the wrong length fails the pattern instead of being handed back as an ID. The cost is that if YouTube ever changes the format, all five stop matching and you get null — which, given the rule above, is the failure I want.

Confirming a resolution costs one request

Whatever you extract, the public feed will tell you what it is:

const feed = await (await fetch(`https://www.youtube.com/feeds/videos.xml?channel_id=${id}`)).text();
const name = feed.match(/<title>([\s\S]*?)<\/title>/)?.[1];
console.log(`${handle} → ${name}`);   // this line is the entire fix

The first <title> in that XML is the channel's own name. Print it, compare it to what you asked for. It is one line, and it would have stopped me publishing an article about the wrong channels.

That feed is also where the videos are — up to the 15 most recent, as <yt:videoId> entries — so if resolving the channel was a step towards listing its uploads, you are already there.

About the sample

Sixteen channels, all of them ones I picked by hand, all large, all fetched on 2026-08-29 from one IP with hl=en&gl=US. That is enough to show the naive method is unreliable and that the five markers agree with each other; it is not enough to tell you the exact failure rate on channels unlike these.

The short version

The first "channelId" in a YouTube channel page usually belongs to a different channel — 15 of 16 in my sample.
It sits in the page's featured-channels shelf, so the wrong channel is on-topic and looks right.
Sometimes it is the same creator's other channel; sometimes it is an unrelated channel they feature. Do not assume a relationship.
A channel with no featured shelf makes the naive method look correct. Do not validate your resolver on one.
Use canonical, og:url, itemprop="identifier", "externalId" or "rssUrl". Return null rather than falling back.
Print the resolved channel's own name once. Verifying the numbers is not the same as verifying the subject, and only one of those catches this.

Where this came from

I hit this while building YouTube Channel Transcript Scraper, which takes a channel handle and returns transcripts for its recent videos. Resolution now uses the five markers above, and a channel it cannot resolve produces a row saying which one and why, rather than quietly substituting another.

Re-running the survey that started all this, against the correct channels: 180 videos across 12 channels produced 163 with a caption track, 15 with none at all, and 2 that could not be played — three outcomes, not two. Of the 163, 120 (73.6%) offered only auto-generated captions. Human captions cluster hard: six of the twelve channels had none across their last 15 uploads, while Marques Brownlee (12) and NASA (11) accounted for over half of the 43 that did.

Written with AI assistance. Every handle, ID, channel name, position and ratio above came from live requests executed on 2026-08-29 before publishing.

Your scraper's defaultCountry setting invents phone numbers

NeverEmpty — Fri, 28 Aug 2026 14:09:49 +0000

Most contact scrapers have a setting called something like defaultCountry or region. It looks harmless. It is the fastest way to fill a database with phone numbers that are structurally valid, pass every check you run, and belong to nobody.

Here is the measurement that convinced me to delete mine.

One company, 51 countries

I crawled the contact pages of ibm.com on 2026-08-28 — up to five pages per site, three were reachable, robots.txt honoured. What came back:


Email addresses	37
Phone numbers resolved to E.164	115
Distinct countries in those 115	51
Phone-shaped strings that could not be resolved to E.164	52

The 51: AT AU BE BG CH CN CY CZ DE DK EE EG FI FR GB GP GR HK HR HU IE IL KR LK LT LU LV MO MY NC NG NL NO PF PH PK PL PT RO RS SE SG SI SK TH TR TW UA US VN ZA.

That is one company's own contact pages. Not a directory, not an aggregator. The same crawl over suse.com gave 11 numbers across 8 countries, ionos.com 10 across 7.

So the premise behind a default country — "this is a US site, so bare numbers on it are US numbers" — is wrong before you write a line of parsing code. Corporate contact pages are international by nature. That is what they are for.

What a default country actually does

Take the strings that had no country code of their own and parse them as US:

import { parsePhoneNumberFromString } from 'libphonenumber-js';

let valid = 0;
for (const raw of unresolvedStrings) {          // 52 of them
  const n = parsePhoneNumberFromString(raw, 'US');
  if (n?.isValid()) valid++;
}
// valid === 18

18 of the 52 come back valid. Not "probably valid" — isValid() returns true, .number gives a clean E.164 string, and every downstream check passes.

Most of those 18 really are US numbers, so far so good. The problem is the ones that are not. From the suse.com crawl, a Budapest number printed on the page as a local string:

parsePhoneNumberFromString('361-489-4600', 'US').isValid()   // true
parsePhoneNumberFromString('361-489-4600', 'US').number      // '+13614894600'

+36 1 489 4600 is Budapest. +1 361 489 4600 is a real, dialable US area code. The default country did not fail loudly — it produced a different, perfectly valid number in a different country, and handed it to you as a fact.

A validator cannot save you from this. It validates the number you constructed, not the assumption you made. It is not lying: +13614894600 genuinely is a valid US number. It just is not the number that was printed on the page.

Sometimes the validator does catch it — 0180 0132 00049 from the same site parses as US and comes back invalid, so that one gets thrown away. But you cannot rely on being saved by luck. The failure mode you never see is the one that stays.

The shape that does not lie

Two fields, one rule:

phones        E.164, only for numbers that carried their own country code
phonesLocal   the raw string, essentially as printed, when one could not be resolved

The rule: never synthesise a country code. If the page did not say which country, you do not know which country.

That sounds like losing data. It is not — it is moving it. The 52 unresolved IBM strings are still in the output, still usable by code that has context you do not. A CRM that already knows the account is in Malaysia can resolve 1800-88-8558 correctly; your crawler cannot.

What you lose is the illusion of 52 extra E.164 numbers. That illusion is expensive, because nobody discovers it until someone dials.

While you are there: emails are obfuscated on purpose

Two patterns worth handling, both cheap:

name [at] example [dot] com and its bracketed variants. A plain /\S+@\S+/ extracts nothing usable from those — [at], (at) and {at} forms produce no match at all.
Cloudflare email protection. The address is replaced by <a href="/cdn-cgi/l/email-protection" data-cfemail="a1c4d9c0ccd1cdc4e1...">. It is an XOR encoding: the first byte is the key, each following byte is a character XORed with it.

function decodeCfEmail(hex) {
  const key = parseInt(hex.slice(0, 2), 16);
  let out = '';
  for (let i = 2; i < hex.length; i += 2) {
    out += String.fromCharCode(parseInt(hex.slice(i, i + 2), 16) ^ key);
  }
  return out;
}

This is not a security control — Cloudflare ships email-decode.min.js to every browser to undo it client-side, and their docs note it is enabled automatically on sign-up. It is there to stop naive harvesters, and it works: six lines is the entire difference between finding a site's contact address and reporting that it has none.

And when the fetch fails

The fourth site in my run was chiyodacorp.com. It returned:

{ "domain": "chiyodacorp.com", "ok": false, "reason": "fetch failed",
  "emails": [], "phones": [], "phonesLocal": [] }

Not an absent row. Not an empty row that reads like "this company publishes no contact details". A row that says which site failed and why, so the next run can retry that one and only that one.

A crawl that quietly drops what it could not reach reports a smaller, cleaner, wronger world every time.

The short version

Delete the default-country setting. It is not a convenience, it is a fabricator.
Keep E.164 and un-resolvable strings in separate fields, and never invent the difference.
Handle bracketed obfuscation and Cloudflare's XOR, or accept that you are silently missing contacts.
Emit a row with a reason for every site you failed on.

None of this is hard. All of it is skipped by default, which is why so many contact datasets are confidently wrong.

The crawler these numbers came from

The crawl above ran on website-contacts, which has no default-country setting for the reason described here, and keeps E.164 and un-resolvable strings in separate fields. Worked example: emails and phones from a list of websites.

Written with AI assistance. Every count, country list and parse result above came from a live crawl executed on 2026-08-28 before publishing.

Greenhouse, Lever, Ashby, Workable: 4 job APIs, 1 dangerous bug

NeverEmpty — Fri, 28 Aug 2026 13:35:17 +0000

Most companies running Greenhouse, Lever, Ashby or Workable publish their open roles on a public JSON endpoint. No API key, no OAuth, no browser.

Here they are, with the counts I measured on 2026-08-28. Every number below came back from a live request while writing this — and job counts move daily, so yours will differ.

The four endpoints

# Greenhouse
curl -s "https://boards-api.greenhouse.io/v1/boards/stripe/jobs"
# → 200, jobs[] with 578 entries

curl -s "https://boards-api.greenhouse.io/v1/boards/monzo/jobs"
# → 200, jobs[] with 66 entries

# Lever
curl -s "https://api.lever.co/v0/postings/matchgroup?mode=json"
# → 200, a bare array with 71 entries

# Ashby
curl -s "https://api.ashbyhq.com/posting-api/job-board/linear"
# → 200, jobs[] with 29 entries

# Workable
curl -s "https://apply.workable.com/api/v1/widget/accounts/lyst?details=true"
# → 200, jobs[] with 8 entries

Three of the four are documented: Greenhouse, Lever, Ashby. The Workable one is different — that is the endpoint their embeddable careers widget calls, not the documented Workable API (which needs a Bearer token). It works today and it is public, but treat it as something that can change without a changelog.

Two container shapes, four vocabularies. Greenhouse, Ashby and Workable all wrap the list in jobs; Lever returns a bare array. Inside, nothing lines up — the job title is title in three of them and text in Lever. Greenhouse gives you absolute_url and internal_job_id, Ashby gives employmentType and secondaryLocations, Lever gives categories and hostedUrl.

The part that silently breaks

Here is what a naive integration does:

const res = await fetch(url);
const data = await res.json().catch(() => ({}));
return data.jobs ?? [];   // ← this is the bug

Run that against a company that does not use Greenhouse and you get []. Run it against a company that has zero open roles right now and you also get []. Run it while the endpoint is throwing a 502 and you get [] again. Point it at a Lever board and you get [] every single time, forever, because Lever's response has no jobs key at all.

Four completely different situations, one identical output. Downstream, a week later, someone asks why a client "stopped hiring" — and the answer is that nobody was ever hiring there on that ATS.

I measured this while checking companies for this post:

Request	Result
`boards-api.greenhouse.io/v1/boards/stripe/jobs`	200, 578 jobs
`boards-api.greenhouse.io/v1/boards/ramp/jobs`	404 `{"status":404,"error":"Job not found"}`
`api.lever.co/v0/postings/netlify?mode=json`	404 `{"ok":false,"error":"Document not found"}`
`api.lever.co/v0/postings/plaid?mode=json`	404

Those 404s do not mean those companies are not hiring. They mean the guess about which ATS they use, or the guess about their board slug, was wrong.

So the minimum honest shape is four outcomes, not one:

let res;
try {
  res = await fetch(url, { signal: AbortSignal.timeout(20000) });
} catch (e) {
  return { state: 'fetch-failed', reason: e.message };
}

if (res.status === 404) return { state: 'no-board-on-this-ats' };
if (!res.ok)            return { state: 'fetch-failed', status: res.status };

let list;
try {
  list = pickList(await res.json());        // vendor-specific: jobs[] or bare array
} catch (e) {
  return { state: 'not-json', reason: e.message };   // 200 + an HTML error page
}

return list.length
  ? { state: 'ok', jobs: list }
  : { state: 'board-exists-but-zero-open-roles' };

state costs you one string per row and removes an entire class of "the data looked fine" incidents.

The harder problem: which board belongs to whom

Guessing a slug from a domain name works often enough to be dangerous. Here is a real collision, both measured today:

curl -s "https://api.ashbyhq.com/posting-api/job-board/notion"
# → 200, 134 jobs, first one: "Software Engineer, Developer Platform"

curl -s "https://apply.workable.com/api/v1/widget/accounts/notion?details=true"
# → 200, "name": "Notion", 0 jobs
#   description: "We're a luxury lifestyle agency based in Shoreditch..."

Same slug, two entirely unrelated organisations, both returning HTTP 200. If your resolver tries Workable first and stops at the first 200, you have just labelled a London lifestyle agency as the productivity company.

It gets sillier. apply.workable.com/api/v1/widget/accounts/a — a single letter — returns 200 with "name": "a". A slug existing proves nothing about who owns it. (For contrast, zzqqxxnotarealcompany returns 404 on both Workable and Greenhouse, so the 200s above are real registrations, not a catch-all.)

The only thing that settles ownership is the company's own site. If notion.so links to that board, it is theirs. If it does not, what you have is a lead, not a fact — and it should be labelled that way rather than merged into the same column as verified results.

There is a second trap in the other direction. Investor portfolio pages, accelerator sites and job aggregators link to boards that belong to other companies. Crawl an investor's site, follow every board link, and you will end up attributing their portfolio companies' job boards to the investor. Ask "does this page list other companies' jobs?" before you attribute anything.

Normalising the shapes into one

A usable row needs at minimum:

source        greenhouse | lever | ashby | workable
company       the domain you started from
boardSlug     the slug that actually worked
jobId         vendor id, kept as a string
title         Lever calls this `text`
location      null when the vendor does not give one — not ""
remote        true | false | null   (null when unknown, not false)
department
url           the public apply URL
postedAt      ISO 8601, not the vendor's raw format

The null rule matters more than it looks. If "unknown" and "false" collapse into the same value, every downstream filter for on-site roles quietly includes the ones you simply had no data for.

The rule, if you build your own

Whatever you wrap these endpoints in, keep the three outcomes separate:

the board does not exist on this ATS (404)
the board exists and has zero open roles right now
the request failed (timeout, 5xx, HTML where JSON was expected)

Collapsing them into one empty array is the bug that costs you a week later, not today. The endpoints at the top of this post are the whole integration otherwise — no key, no proxy, no browser.

If you would rather not build it

I keep the wrapper described above as a public Actor. It applies the same three-outcome rule and the field names in this post:

ats-jobs-api — all four boards behind one input
ats-board-finder — give it a company domain, it tells you which ATS the company runs (worked example)

Single-vendor runs, if you only need one: Greenhouse · Lever · Ashby · Workable · remote roles across all four

Written with AI assistance. Every endpoint, count, status code and error body in this post was executed live before publishing, on 2026-08-28.