The Chrome UX Report (CrUX) is a dataset of real Chrome users' performance browsing 5+ million websites. This week in its latest release, Time to First Byte (TTFB) was added as a new metric. TTFB is a measure of how long the web page took from the start of navigation (clicking a link) to the first byte of the response arriving to the client. There's a lot that happens in between there, from DNS resolution and SSL negotiation, to server-side rendering and database lookups. Both network and server response times are critical for web performance because they delay the first byte of the response. All of this must happen before the website ships megabytes of images and JavaScript, which slow down the loading experience even more. So if the TTFB isn't fast, no amount of front end optimization has a chance at providing a fast experience to users. This makes it a great metric to measure hosting providers.
I've shared this data with a number of people already, and a lot of the time their first reaction is to roll their eyes as if to say "not another TTFB study". But this is not like other TTFB studies. The Chrome UX Report is the only dataset of its kind (that I know of) to report on real user experiences as opposed to lab-based testing. This makes it uniquely equipped to answer age-old questions like "How fast is a given website?" because its methodology is based on how users actually experience the performance.
Today I'm releasing a new study called Is my host fast yet? a leaderboard of host performance using the CrUX dataset. It also leverages the HTTP Archive dataset to identify websites' hosts by looking for their signatures in response headers. For example, some hosts self-identify in the X-Powered-By header.
The way it works is by querying the datasets together on BigQuery, aggregating the number of websites for each host, and their distributions of TTFB performance:
#standardSQL
SELECT
CASE
WHEN platform = 'seravo' THEN 'Seravo'
WHEN platform = 'automattic.com/jobs' THEN 'Automattic'
WHEN platform = 'x-ah-environment' THEN 'Acquia'
WHEN platform = 'x-pantheon-styx-hostname' THEN 'Pantheon'
WHEN platform = 'wpe-backend' THEN 'WP Engine'
WHEN platform = 'x-kinsta-cache' THEN 'Kinsta'
WHEN platform = 'hubspot' THEN 'HubSpot'
WHEN platform = '192fc2e7e50945beb8231a492d6a8024' THEN 'Siteground'
WHEN platform = 'x-github-request' THEN 'GitHub'
WHEN platform = 'alproxy' THEN 'AlwaysData'
ELSE NULL
END AS platform,
client,
COUNT(DISTINCT origin) AS n,
SUM(IF(ttfb.start < 200, ttfb.density, 0)) / SUM(ttfb.density) AS fast,
SUM(IF(ttfb.start >= 200 AND ttfb.start < 1000, ttfb.density, 0)) / SUM(ttfb.density) AS avg,
SUM(IF(ttfb.start >= 1000, ttfb.density, 0)) / SUM(ttfb.density) AS slow
FROM
`chrome-ux-report.all.201906`,
UNNEST(experimental.time_to_first_byte.histogram.bin) AS ttfb
JOIN
(SELECT _TABLE_SUFFIX AS client, url, REGEXP_EXTRACT(LOWER(CONCAT(respOtherHeaders, resp_x_powered_by, resp_via)),
'(seravo|x-kinsta-cache|automattic.com/jobs|x-ah-environment|x-pantheon-styx-hostname|wpe-backend|hubspot|192fc2e7e50945beb8231a492d6a8024|x-github-request|alproxy)')
AS platform
FROM `httparchive.summary_requests.2019_06_01_*`)
ON
client = IF(form_factor.name = 'desktop', 'desktop', 'mobile') AND
CONCAT(origin, '/') = url
WHERE
platform IS NOT NULL
GROUP BY
platform,
client
ORDER BY
n DESC
The website renders the results in order of fast TTFB performance:
The datasets update monthly, so we'll be able to track how hosts improve (or not) over time. It's also easy to add new hosts, so please contribute if you know how to detect anything we're missing.
Top comments (4)
Perhaps Siteground and Automattic are last because they are hosting companies specialized in WordPress, and WP has a reasonable overhead even for the simplest of requests.
Well... Yes and no.... Yes, WordPress does have reasonable overhead for simple requests, if not optimized.
But that should rather slow down hosters which are NOT specialized in WP than the ones that are specialized.
WPEngine for example is quite far up that list (#5) and they also specialize in WP-Hosting but obviously give you a fair amount of optimization out of the box (Just a wild guess).
Still... Interesting to see.
Indeed, WPEngine ranks well in that list.
I know from personal experience that Siteground provides excellent optimization out of the box for WordPress websites. Automattic, being the creator of WordPress, probably does, too.
Cached WordPress can be pretty fast, but I was guessing Github, which tops the list, would serve some kind of pre-compiled HTML, which is what WordPress would end up doing, too, but with a small overhead - just a wild guess, too
But anyway, WPEngine in 5th is a though fact to sustain my arguments.
And you're right. I think GitHub Pages are ment, which only support static pages. I guess what makes the difference is the use of a cdn.