Deep-dive into Cloudflare's Speed Test

This blog post originally appeared on my blog.

There are lot of web apps available to test your home internet connection. Your ISP will have their own tool, I use fast.com from Netflix and Netgear Genie to make sure I am connected to Internet or not. Couple weeks ago Cloudflare has released its home grown solution to test the home internet connection called Speed Test. You can access it from https://speed.cloudflare.com/ This blog post will deep-dive into Cloudflare's Speed Test.

Why another speed test app?

As I mentioned earlier, there are hundreds of web apps and mobile apps available to test the internet connection. But why Cloudflare has to (re)create again. I believe Cloudflare is the new Google. Here is the excerpt from Cloudflare:

Although there are a slew of speed testing tools out there, none of them give you precise insights into how they came to those measurements and how they map to real-world performance. With speed.cloudflare.com, we give you insights into what we’re measuring and how exactly we calculate the scores for your network connection.

Cloudflare's speed test doesn't have a fancy domain name like fast.com. It is hosted under sub-domain of cloudflare.com.

How to get started?

Unlike other web or mobile app, there is no start button in Speed Test (let's call Speed Test). E.g. Speedtest.net has a GO button, if a user want to start the test, they can simply click on that button.

After visiting https://speed.cloudflare.com/, your test will start immediately. After all the tests are completed, you can see the results instantly.

Deep-dive into Cloudflare's Speed Test

Speed Test displays key metrics such as Download speed, Latency, and Jitter in the main dashboard. Detailed measurements will get displayed just beneath the main dashboard. Server Location will get displayed in the right pane as shown above.

Privacy Policy

As mentioned in the Cloudflare Privacy Policy, they do not sell your data. Performance data is collected, anonymized and is governed by the terms of Cloudflare. For more details you can refer this link https://www.cloudflare.com/privacypolicy/

Let's talk performance

After you launch the Speed Test, it will connect you to the nearest data center. You can see the map in the right pane as shown below. My closest data center is Chicago as shown below.

Server Location

If you zoom in to the Cloudflare data center, it will not reveal the exact location of data center, instead it will display the airport location. It uses Anycast network (from Cloudflare) to detect the nearest data center.

Server Location using IATA code

After connecting to the nearest data center, Latency and Jitter test will kick off.

Latency will be measured using 20 iterations and the median will be considered. Lower the number better the performance. Latency is nothing but time to first byte from browser perspective.

Measurements and Chart representation

1. Each dot represents the individual measurement
2. Rectangle represents the measurements from 25th percentile to 75th percentile
3. Solid line inside the rectangle represents the median
4. Dotted line represents average.

Here is the zoomed in image.

Latency Measurement Chart

Here is the definition of Jitter. Lower the number is better.

Jitter

After Latency and Jitter measurement, the app will start downloading small files starting from 10kB and it reaches the threshold based on the performance.

Important Note:

As mentioned above, the app will first connect to the data center and runs latency and jitter tests and then it will kick off the download tests. But this will cause Bufferbloat. This issue has been raised in Cloudflare GitHub repository by Toke Høiland-Jørgensen https://github.com/tohojo. Here is the issue id.

The suggestion from Toke is to continue the latency tests during the download tests. This suggestion will improve the metrics and performance.

As you see my network performance, it was able to reach till 25 MB download test. More the file size, lesser the number of iterations. If your network performance is poor, you may not be able to reach for 10MB download test.

Download Measurements

• 10kB has 10 iterations
• 100kB has 10 iterations
• 1MB has 8 iterations
• 10MB has 5 iterations
• 25MB has 3 iterations

Interesting Facts

• Max file size that you can reach is 100MB :)

• If you open the file which Speed Test is downloading is nothing but string of zeroes and some random string at the end.

If you click on the chart, you can see the detailed metrics.

Raw Data

For the final download internet speed, it will calculate the overall 90th percentile. You can also download the metrics by clicking on Download button as shown below.

Download Results

Here is my raw data:

time	Formatted Timestamp	direction	bytes	latency	bps	Formatted Mbps	ttfb	duration	responseSize
1591643367414	8:09:27 PM	download	0	30.13502098	5492134.678	5.492134678	152.135	0.890000025	1611
1591643368111	8:09:28 PM	download	100000	31.51511296	174902506.4	174.9025064	49.51499996	4.585000046	101241
1591643368748	8:09:29 PM	download	0	27.91007098	5489820.271	5.489820271	51.90999998	0.835000013	1573
1591643370080	8:09:30 PM	download	0	33.08994298	1062008.745	1.062008745	98.08999998	1.144999987	1152
1591643370386	8:09:30 PM	download	0	29.075071	1460869.656	1.460869656	53.075	0.689999957	1126
1591643370590	8:09:31 PM	download	0	32.26987401	1231515.158	1.231515158	50.27000001	0.824999996	1127
1591643370919	8:09:31 PM	download	0	30.24003999	1651612.887	1.651612887	47.23999999	0.620000006	1128
1591643371292	8:09:31 PM	download	0	31.28508002	1296202.594	1.296202594	65.28500002	0.789999962	1128
1591643371527	8:09:32 PM	download	0	29.82011304	5209091.213	5.209091213	47.82000004	0.879999949	1573
1591643371898	8:09:32 PM	download	0	29.37009201	1228571.438	1.228571438	50.37000001	0.839999993	1129
1591643372098	8:09:32 PM	download	0	48.58994699	1385234.85	1.38523485	67.58999999	0.745000027	1129
1591643372255	8:09:32 PM	download	0	26.69980204	1664516.112	1.664516112	43.70000004	0.620000006	1129
1591643372376	8:09:32 PM	download	0	28.50486301	905263.1519	0.905263152	59.50500001	1.140000008	1129
1591643372487	8:09:32 PM	download	0	31.49487398	5942856.816	5.942856816	49.49499998	0.770000042	1572
1591643372584	8:09:33 PM	download	0	28.88513399	1563636.455	1.563636455	43.88499999	0.659999961	1129
1591643372667	8:09:33 PM	download	0	32.67480202	1155555.538	1.155555538	49.67500002	0.720000011	1104
1591643372750	8:09:33 PM	download	0	29.80004001	1011764.739	1.011764739	46.80000001	1.019999967	1129
1591643372870	8:09:33 PM	download	0	29.14511296	1885271.254	1.885271254	47.14499996	0.645000022	1152
1591643373006	8:09:33 PM	download	0	56.45492601	443290.0412	0.443290041	78.45500001	2.310000011	1128
1591643373102	8:09:33 PM	download	0	28.10509204	6366667.085	6.366667085	49.10500004	0.719999953	1573
1591643373174	8:09:33 PM	download	0	27.77011297	1220740.709	1.220740709	45.76999997	0.675000018	1103
1591643373272	8:09:33 PM	download	10000	30.41511297	102460756.9	102.4607569	48.41499997	0.79000002	11118
1591643373682	8:09:34 PM	download	10000	39.750206	66472129.26	66.47212926	55.75	1.220000035	11137
1591643373853	8:09:34 PM	download	10000	41.03020603	105756870.2	105.7568702	57.03000003	0.764999946	11113
1591643373969	8:09:34 PM	download	10000	33.92511102	111856553.1	111.8565531	74.92500002	0.724999991	11137
1591643374053	8:09:34 PM	download	10000	28.50004001	82552194.72	82.55219472	45.50000001	1.025000005	11577
1591643374144	8:09:34 PM	download	10000	28.56506101	116684892.9	116.6848929	42.56500001	0.694999995	11137
1591643374277	8:09:34 PM	download	10000	38.88990298	102713921.4	102.7139214	86.88999998	0.79000002	11143
1591643374407	8:09:34 PM	download	10000	38.11016402	82759187.61	82.75918761	60.11000002	0.979999953	11138
1591643374531	8:09:35 PM	download	10000	32.61020398	111090406.6	111.0904066	71.60999998	0.730000029	11137
1591643374620	8:09:35 PM	download	10000	31.30987403	114208451.8	114.2084518	49.31000003	0.709999993	11136
1591643374731	8:09:35 PM	download	100000	30.39518498	165389732.9	165.3897329	49.39499998	4.870000004	101681
1591643375035	8:09:35 PM	download	100000	51.87004002	138888312.4	138.8883124	68.87000002	5.774999969	101260
1591643375157	8:09:35 PM	download	100000	40.05478101	98209430.42	98.20943042	60.05500001	8.165000007	101235
1591643375320	8:09:35 PM	download	100000	30.74004001	113958208.7	113.9582087	47.74000001	7.035000017	101212
1591643375436	8:09:35 PM	download	100000	29.59521598	101120806.8	101.1208068	55.59499998	7.930000022	101236
1591643375522	8:09:36 PM	download	100000	32.359998	194393210.4	194.3932104	55.36	4.125000036	101234
1591643375644	8:09:36 PM	download	100000	34.385278	68490476.06	68.49047606	51.385	11.76000002	101681
1591643375757	8:09:36 PM	download	100000	60.48020599	149920000.2	149.9200002	76.47999999	5.349999992	101259
1591643375855	8:09:36 PM	download	100000	29.11989504	134430847.5	134.4308475	44.12000004	5.96499996	101235
1591643376033	8:09:36 PM	download	1000000	51.000061	103707593.7	103.7075937	65	77.23499998	1002232
1591643376253	8:09:36 PM	download	1000000	37.56491398	82013085.54	82.01308554	72.56499998	97.66500001	1002226
1591643376434	8:09:36 PM	download	1000000	34.27996802	97937372.39	97.93737239	50.28000002	81.78499999	1002226
1591643376640	8:09:37 PM	download	1000000	33.57990501	69885387.87	69.88538787	58.58000001	114.665	1002676
1591643376846	8:09:37 PM	download	1000000	36.30009199	93192321.14	93.19232114	57.29999999	85.94999998	1002235
1591643377023	8:09:37 PM	download	1000000	29.92496798	102189436.1	102.1894361	45.92499998	78.38000002	1002201
1591643377213	8:09:37 PM	download	1000000	38.399832	114295890.4	114.2958904	62.4	70.08000003	1002232
1591643377378	8:09:37 PM	download	1000000	34.02996798	126017621.1	126.0176211	50.02999998	63.56000004	1002210
1591643378017	8:09:38 PM	download	10000000	36.27003796	155131678.5	155.1316785	76.26999996	516.265	10012132
1591643378707	8:09:39 PM	download	10000000	39.07500797	138248324	138.248324	72.07499997	579.34	10012598
1591643379435	8:09:39 PM	download	10000000	33.78005901	132977885.5	132.9778855	70.78000001	602.275	10012157
1591643380149	8:09:40 PM	download	10000000	32.380059	136674065	136.674065	69.38	585.985	10012119
1591643380826	8:09:41 PM	download	10000000	54.29975799	153141530.7	153.1415307	100.3	522.975	10012149
1591643382446	8:09:42 PM	download	25000000	29.70008799	140018808.9	140.0188089	96.69999999	1429.96	25028662
1591643383906	8:09:44 PM	download	25000000	39.27496201	162604349.7	162.6043497	124.275	1231.34	25028655
1591643385432	8:09:45 PM	download	25000000	29.27995203	160793569.1	160.7935691	104.28	1245.23	25029122

90th percentile formula is =PERCENTILE.EXC(G2:G57,0.9). Column G should have the speed in Mbps.

CSV Data

After your test is done, you can retest it. During the test you can pause the test.

Upload Speed Test

Upload Speed test has been temporarily turned off by Cloudflare due to some incorrect measurements.

Note: we've temporarily disabled upload testing while we investigate reports of incorrect upload speed reports on very fast connections.

From Cloudlfare blog

Network Trace

Here is the captured HTTP requests from developer tools in Edge.

Network Trace

• 170 HTTP requests
• ~30 seconds to finish the test
• ~68 MB resources
• ~61 MB transferred

Tech Stack

• Cloudflare Workers
• Wrangler
• Anycast network

Speed Test is the open source project in GitHub.

Pros

I ran multiple tests in Speed Test. It is one of the best tool to measure the performance.

• Its light weight
• Results download in CSV format
• Intuitive user interface
• Neat graphs and insights about measurements
• Responsive web app
• Open source

Cons

• You cannot change the server location
• Still the measurement needs more accuracy
• No start button

Conclusion

Cloudflare's Speed Test is very light weight web app to measure the performance of your home network. It is fast and the results are intuitive. As of now, no upload tests and the accuracy can be improved while enabling latency measurement during download tests.

What is your favorite app to measure your internet performance? Please let me know in the comments.

Comments

[danim-hub]

I like your application very much.
What are you using for charts? They look very dynamic