loading...

Some Undocumented Facts About Cloudflare's CDN

richardkazuomiller profile image Ricky Miller ・3 min read

I've been a satisfied user of Cloudflare for a while, but I ran into some issues which I couldn't find the answers to online so I though I'd share them in case anyone else ran into them in the future.

I noticed that my site built on Cloudflare Workers was running slower than usual. This was peculiar, because the pages I was loading weren't making any requests to my backend. I've been doing this web development thing for a while, so based on the amount of latency I had a hunch that my requests were going to an edge server somewhere on another continent.

If you've ever wondered where Cloudflare is serving your data from, you can do it easily in your browser. Just open the following URL:

https://{your-domain}/cdn-cgi/trace

You should see something like:

fl=22f173
h={your-domain}
ip={your-ip}
ts=1562687410.788
visit_scheme=https
uag={your-user-agent}
colo=NRT
http=http/2
loc=JP
tls=TLSv1.3
sni=plaintext
warp=off

The key section is the colo= bit. NRT is the Tokyo PoP, which is what I want because I am also in Tokyo.

However, when I was investigating the issue, I was getting SJC, which is San Jose, CA, USA. That's very far from Japan.

Cloudflare says that they serve you locally with any ISP that interconnects with them. Because of this, the conclusion that people often jump to when connections to Cloudflare aren't routed in the shortest way possible is that it's because of the ISP. However, I checked on four different Internet connections – two home landlines, one mobile, and GCP's Tokyo region – all of which shared Internet exchange points (IXPs) with Cloudflare and they were all routed outside of Japan. There might be edge cases where even if two ISPs or CDNs share an IXP aren't directly peered, but since all four were not connecting the way I had hoped, I figured there was something else going on.

The traceroute from GCP was particularly curious.

traceroute to {my-website} (104.27.161.149), 30 hops max, 46 byte packets
1 172.17.0.1 (172.17.0.1) 0.005 ms 0.005 ms 0.002 ms
2 104.27.161.149 (104.27.161.149) 113.895 ms 113.949 ms 114.294 ms

Only one hop, which is pretty cool. This is because GCP is interconnected with Cloudflare. However, the latency was more than 100ms which is a lot. Connections between devices within Japan are usually under 20ms. If they're in the same region, under 10ms. This told me that even though GCP and Cloudflare are directly connected, requests were not being served locally as the documentation says they should be.

After a few hours of back-and-forth with Cloudflare's support team, they told me that they sometimes reroute traffic in times of heavy load and, although I was paying for Cloudflare Workers, traffic to my site was being rerouted because my domain was on a free plan and they prioritize traffic based on how much the owner of the domain pays.

I went ahead and upgraded to the Pro plan and Cloudflare started serving requests from my local PoP almost immediately.

While it would have been nice to have some indication of this in the Cloudflare dashboard, or at least some documentation that this happens periodically, it's not the end of the world and I still like Cloudflare.

So, long story short, if your free Cloudlare domain is slower than you'd like, consider upgrading to Pro or, if cost is an object, adjusting your infrastructure to serve directly from a server closer to your users.

Posted on by:

richardkazuomiller profile

Ricky Miller

@richardkazuomiller

I have a love-hate relationship with the web

Discussion

pic
Editor guide