loading...
Cover image for How to choose a median result when running Google Lighthouse multiple times

How to choose a median result when running Google Lighthouse multiple times

rozenmd profile image Max Rozen Originally published at maxrozen.com on ・2 min read

According to the docs, running Google Lighthouse five times against the same URL will reduce the variability of your results by around half if you pick the median result.

The problem is, how do you pick the right result to use as your median, or representative result?

A naïve approach would be to take the median of the performance score, however since the performance score itself is a weighted average of other scores, and is more prone to outliers, it isn’t as useful as you’d think.

Lighthouse actually exports a function to help you calculate which result to use: computeMedianRun.

How to use computeMedianRun in Node

const spawnSync = require('child_process').spawnSync;
const lighthouseCli = require.resolve('lighthouse/lighthouse-cli');
const {
  computeMedianRun,
} = require('lighthouse/lighthouse-core/lib/median-run.js');

const results = [];
for (let i = 0; i < 5; i++) {
  console.log(`Running Lighthouse attempt #${i + 1}...`);
  const { status = -1, stdout } = spawnSync('node', [
    lighthouseCli,
    'https://example.com',
    '--output=json',
  ]);
  if (status !== 0) {
    console.log('Lighthouse failed, skipping run...');
    continue;
  }
  results.push(JSON.parse(stdout));
}

const median = computeMedianRun(results);
console.log(
  'Median performance score was',
  median.categories.performance.score * 100
);

Under the hood, computeMedianRun finds the run that’s closest to the median First Contentful Paint (FCP), AND the closest to the median Time to Interactive (TTI).

FCP and TTI are used because they’re the earliest and latest moments in a page’s lifecycle.

(This is an article posted to my blog at maxrozen.com. You can read it online by clicking here.)

Posted on by:

rozenmd profile

Max Rozen

@rozenmd

Frontend Dev at Atlassian. Writes about React & building a SaaS while employed. Building PerfBeacon.com.

Discussion

markdown guide