On 9 November I published an open dataset: aggregate numbers from about 44,000 hours of 24/7 live streams that ran on our service in September and October. Platforms, resolutions, bitrates, uptime, how long streams stay on air. CC BY, with a CSV.
The day after, one of its headline numbers turned out to describe a single account.
This is how that happened, how we found it, and the check I now run before publishing any product statistic.
The number
The report said paid streams run longer than free-trial ones: a median continuous session of 11.3 hours on paid plans, against 2.2 hours on the trial. It looked right. People who pay for a 24/7 streaming service should keep streams up longer than people trying it for a day. A believable number that matches what you expected is the most dangerous kind.
I had also sent the report to a few people who maintain statistics pages, and several of those letters quoted "about 11 hours".
The smell
The number kept bothering me: paid streams so often ending at about half a day. So I plotted how long paid runs last, in 30-minute bins. A real "how long do people stream" distribution should be smooth and long-tailed. This one had a cliff: the bins from 10.5 to 11.5 hours were about twice as tall as their neighbours, and at 12 hours the count fell by more than half.
A cliff like that usually means a clock somewhere: a timeout, a token expiry, a platform limit. So the first job was to find the clock.
Suspects
Our own cap. The orchestrator has a maximum stream duration. It is 45 days, nothing overrides it, and the logs we checked contain no line saying a stream hit it. An 11-hour run is nowhere near it anyway.
The platform. YouTube does reset ingest connections now and then, and it's plausible it does so on a schedule. To test that without trusting our own servers, we had two small control machines streaming to YouTube around the clock, one in the US and one in Germany, outside our cluster. Between 5 and 10 November YouTube reset them 13 times, after anything from 0.5 to 41 hours. Only one of the 13 fell between 10 and 14 hours. Twitch, for comparison, cut its stream at exactly 48 hours, twice. So YouTube has no 12-hour clock, in either region.
Billing. A balance running out could stop streams, but not at the same hour of the day, day after day.
None of these were it.
The answer was a GROUP BY
The query that solved it was embarrassingly simple: group the runs in that 10 to 14 hour spike by account.
Almost all of them came from one account. That account runs many streams and restarts them on its own schedule, about twice a day. Nothing on our side was cutting anything; it was one customer's normal routine, and it was so large that it was the paid median. That account alone produced more than two-thirds of all paid sessions in the period.
Our paying cohort is small, as it is for most young products. In a small cohort, one heavy user's habit doesn't skew the average a little. It becomes the answer.
Same data, three answers
Here's the paid median session, computed three ways from the same rows:
| Method | Median paid session |
|---|---|
| Every session counts once (what we published) | 11.3 h |
| Same, without the heaviest account | 19.7 h |
| One median per account, then the median of those | 22.2 h |
None of these is a calculation error. They answer different questions. "What does a typical paid session look like?" is dominated by whoever creates the most sessions. "What does a typical paying customer do?" needs each customer to count once. The report claimed to answer the second question with the arithmetic of the first.
The share of long sessions moved even more. Measured per session, 11.6% of paid sessions lasted a day or more. Measured per account, more than 8 in 10 paying accounts kept a stream up for at least a day, and more than half for at least a week.
The free-trial median of 2.2 hours didn't move at all: over a thousand accounts, and none of them has more than 1% of the sessions.
What we changed
- Comparisons between paid plans and the trial are now per account: "more than 8 in 10 paying accounts…", not "11.6% of sessions".
- The paid median is gone. It moves by hours depending on the weighting, so it describes the weighting, not the streams.
- Account shares are rounded to 5% and written as words on the page, so they don't reveal how many paying accounts there are.
- Shares of hours on air stay as they were. They answer "what does the airtime look like?", and there no single account holds more than about 12% of any split.
- The methodology says so in one line, and the page shows the date it was updated.
Two other splits had the same problem in a milder form. "60 fps on paid plans" and "single-video loops on paid plans" were each leaning on a handful of accounts. Both are now per account too, and the direction held: paying accounts use playlists and 60 fps more often than trial accounts.
The check I run now
Before any product statistic leaves the building:
- Plot the distribution, not just the median. Cliffs and spikes are questions. Find the clock, or find the person.
- For every split, compute the top account's share. If one account holds more than about a fifth of the rows in a group, the group's median is that account's habit.
- Decide whether you're describing rows or people, and weight accordingly. "Sessions", "hours" and "accounts" are three different populations.
- Keep a privacy floor. We publish nothing for a group of fewer than five accounts, and round shares so the size of the group can't be backed out.
- When you blame the platform, run a control. Two cheap servers streaming next to production settled the "is it YouTube?" question in a few days, which reading forum threads never would have.
- Check before you pitch. I'd sent the 11-hour figure to people before checking it. The corrected page is the only fix I can offer them.
The corrected report and the CSV are here: 24/7 streaming in practice. The CSV is also archived on Zenodo as 10.5281/zenodo.24103783. If you find another number in it that one account is holding up, I'd like to hear about it.
I run TheLoops, a service that keeps pre-recorded videos streaming 24/7. This post was drafted with help from an AI assistant; the data, the checks and the mistakes are ours.
Top comments (0)