My site checks whether a YouTube channel is in the Partner Program, using only what a logged-out visitor can see. In September I wrote a post here about the five public signals it reads and how they turn into a verdict. One line from that post:
Likely monetized — enough subscribers, memberships on, ads running.
The same checker feeds a country table that I publish as an open dataset. In the September snapshot, that line decided which countries came last. This is the post-mortem.
The rule
Each check scores five signals as pass, fail or unknown:
- S1: 1,000 or more subscribers
- S2: a Join button, meaning channel memberships are on
- S3: ads on recent long-form videos
- S4: a store on the channel
- S5: 500 or more subscribers
Only Partner Program channels can offer memberships or connect a store, so S2 and S4 are strong evidence when they show up. S3 is weaker, because YouTube can run ads on channels outside the program. In September the top tier needed the first three together:
likely_monetized = S1 and S2 and S3
A Join button, ads or a merch shelf without that full set meant "possibly monetized".
What the September table showed
The September snapshot covers 3,080 trending channels in 50 countries, around 60 per country. Sorted by the share rated likely monetized, the last eight rows were:
| Country | Checked | Likely | Possibly | Likely not | Unable |
|---|---|---|---|---|---|
| Sri Lanka | 60 | 5 | 29 | 1 | 25 |
| Russia | 60 | 5 | 33 | 11 | 11 |
| Iraq | 60 | 2 | 45 | 2 | 11 |
| Pakistan | 60 | 1 | 36 | 2 | 21 |
| Azerbaijan | 60 | 1 | 32 | 5 | 22 |
| Kazakhstan | 60 | 0 | 41 | 1 | 18 |
| Tanzania | 60 | 0 | 48 | 2 | 10 |
| Bangladesh | 60 | 0 | 38 | 2 | 20 |
India and Malaysia came next, with 8 likely monetized channels each.
YouTube Help lists the locations where creators can offer channel memberships. When I transcribed that list on 20 October it had 100 entries (the source link and my copy are in the repo). These eight are exactly the eight countries in the file that aren't on it.
Here's the join, with the verdict mix for each group:
import pandas as pd
def read(path):
return pd.read_csv(path, keep_default_na=False) # "NA" is Namibia, not a missing value
sept = read("data/youtube-monetization-signals-by-country-2026-09.csv")
offered = set(read("data/youtube-channel-memberships-locations-2026-10-20.csv")["iso2"])
sept["memberships"] = sept["country_code"].isin(offered)
short = {"likely_monetized": "likely", "possibly_monetized": "possibly",
"likely_not_monetized": "likely_not", "unable_to_determine": "unable"}
mix = sept.groupby("memberships")[list(short)].sum().rename(columns=short)
print((100 * mix.div(mix.sum(axis=1), axis=0)).round(1))
likely possibly likely_not unable
memberships
False 2.9 62.9 5.4 28.7
True 29.6 46.9 5.1 18.4
The False row is 480 channels in 8 countries, the True row 2,600 in 42. The likely-not share was about the same in both groups: 5.4% in the eight countries and 5.1% in the other 42. The gap is at the top. In the eight countries most channels stopped at "possibly".
Unable to determine is higher too, and the rule explains at least part of that. When the ad check couldn't be read, a channel with a Join button still reached "possibly". One without a Join button ended up as unable unless it had a store, and in these eight countries most channels had no Join button to fall back on.
Across all 50 countries, that one yes/no column accounts for 56.7% of the variance in the likely share.
The floor wasn't quite zero: 14 channels in the eight countries were rated likely, so each of them showed a Join button. I haven't looked into why those 14 had one.
Why nothing flagged it
None of the checks I had complained. In every row the four verdict columns added up to the channel count. Unknown signals never counted as failures. The verdict function had unit tests, and they checked that the code did what the rule said. It did.
What nothing tested was an assumption about the data: that every signal the top tier requires can show up in every country. For memberships that isn't true, and YouTube publishes the list that says so. The bottom of my ranking mostly reflected where YouTube offers one feature.
How it was found, in the wrong order
I changed the rule before I knew about the country problem.
I recheck about a dozen well-known channels on a schedule. In late September most of them, MKBHD and PewDiePie among them, had ads on every sampled video but no Join button, so they sat at "possibly monetized". Many partner channels never turn memberships on. MKBHD was also the example on my home page, and a checker that won't commit on MKBHD makes people doubt it. That was the reason for the change on 29 September.
The country pattern showed up on 20 October, when I joined the September file against YouTube's memberships list while writing up the data. By then the new rule had been live for three weeks.
The new rule
From 29 September, a channel with 1,000+ subscribers and ads on its recent long videos also counts as likely monetized if it has a store or 100,000 or more subscribers:
likely_monetized = S1 and S3 and (S2 or S4 or subscribers >= 100_000)
The 100,000 line assumes that a channel that big, with ads on its recent long videos, is almost always in the program. It's an assumption, and the October numbers below give one reason to doubt it.
On 1 October three more changes shipped:
- The ad signal counts videos instead of taking a share. It passes with ads on two or more sampled long videos, or on every video that could be sampled, and fails only when none of at least three had ads. Anything else stays unknown. In September it passed at two thirds or more.
- Channels set as made for kids no longer get marked down for having no memberships or store, since YouTube switches both off for them.
- The store check looks for the Store tab as well as the old merch shelf. YouTube had moved merch from the channel home page to that tab. When I tested the check on 1 October it was returning "no store" for every channel, including ones that clearly have a store. The page still came back with a 200, so nothing raised an error.
Re-scoring every stored check
Each check stores its signals, and the verdict is computed from them, so the new rule could be applied without going back to YouTube.
The API already re-scored cached results on read, so anyone checking a channel saw the new verdict right away. The country table doesn't take that path. It counts a stored verdict column in the database, and the trending sample from September isn't part of the scheduled refresh. Without a batch job, September verdicts would have stayed until those channels were checked again, and the country table would have mixed results from two rules.
So a one-off script walked every stored check, re-applied the current rule and wrote back only the verdict and the signal list. It makes no requests to YouTube. Each write is conditional, so if a live check refreshed the row in the meantime, the live result wins. Trimmed down, the write looks like this:
await db.from("checks")
.update({ verdict: next.verdict, signals: { ...row.signals, list: next.signals } })
.eq("channel_id", row.channel_id)
.eq("verdict", row.verdict) // still the verdict we read
.eq("checked_at", row.checked_at); // and nobody re-checked the channel since
The script also refuses to run from a checkout that's behind main, because an old copy of the rule would re-score everything the old way. When it finishes, it refreshes the country table straight away instead of waiting for the Monday recompute.
Re-scoring only re-applies a rule to evidence that's already stored. It can't recover a signal the original check misread, so a Store tab missed before 1 October stays missed until that channel gets checked again.
October
The table recomputed on 19 October is the October snapshot in the repo. Totals for the published rows:
| Verdict | September (3,080 channels) | October (4,382 channels) |
|---|---|---|
| Likely monetized | 784 (25.5%) | 2,247 (51.3%) |
| Possibly monetized | 1,521 (49.4%) | 778 (17.8%) |
| Likely not monetized | 158 (5.1%) | 306 (7.0%) |
| Unable to determine | 617 (20.0%) | 1,051 (24.0%) |
The likely share went up in all 50 countries from the September file. The eight without memberships now rank anywhere from 6th to 48th of 51. Iraq went from 2 of 60 to 40 of 60 and is sixth. Tanzania went from 0 of 60 to 35 of 60.
Unable to determine went up as well. The new ad signal leaves more cases unknown on purpose, and the new lookups are a different mix of channels, so I can't split that rise between the two.
Before reading the new order as a fact about countries, two caveats.
The sample changed along with the rule. October includes channels people looked up on the site, and more than half of the growth is in two countries: the US went from 66 channels to 457, India from 60 to 367. Only 28 of the 50 September countries moved by six channels or fewer. For those 28, the rank correlation between the September and October shares is 0.49. Across the full tables, 4 of the September top ten are still in the October top ten.
The new rule has a suspect of its own. Russia wasn't on YouTube's list of countries where the Partner Program is open when I last checked that list on 1 October. It still went from 5 likely monetized channels out of 60 to 31 out of 59. Some of those could be big channels outside the program that YouTube runs ads on anyway, which is the case the 100,000-subscriber route lets through. Public pages can't tell me which.
The September file stays on Zenodo as it was published, and the repo's changelog says what changed and when.
What I'd do differently
- Before ranking groups on a tier, list every signal the tier requires and check where each one can exist. Here that was one YouTube Help page.
- Look at the whole verdict mix per group, not only the top tier. If a group's channels pile up in the middle while its "likely not" share looks normal, check the rule before the group.
- If the product re-scores on read but the statistics count a stored column, ship the batch re-score in the same change.
- Keep the old snapshot as it was, and publish the rule change next to the new one, so anyone comparing the two can see why everything moved.
If you've shipped a scoring rule over grouped data, how do you test for this? The join caught it here, but only because I went looking.
The data, the analysis script and both snapshots are on GitHub. The live table, recomputed every Monday, is on MonetizationKit.
I built and run MonetizationKit. The numbers were checked against the data files and analysis output in the repo before publishing.
Top comments (1)
the part worth stealing is storing the signals instead of the verdict. without that, fixing the rule would have meant rechecking all 3,080 channels from scratch. most scoring pipelines bake the judgement in and lose the evidence, which is why their postmortems stay vague.