Hello everyone! Recently, I learned about how the web works, thanks to one of TryHackMe’s modules that focuses on just that. I had always known there were various steps behind that scenes that went into bringing the user to their desired website, even before stepping foot into the cybersecurity world. And I was right; there are so many definitions and functions in place that go into taking users to their desired webpage. And that is what I will be dedicating this post to!
This looks like it was too much to memorize at first when I was learning it. But by slowly breaking down what each part does, trying to understand what each element does, I was able to understand all the stuff that works under the hood when searching for a domain.
Step 1 - Search for the domain
The very first step here is to enter the domain you’re looking for in your browser. I will be using https://youtube.com throughout the post, but you can replace this with any other website. The next step is performed as soon as you click “enter” or “search.”
Your computer will handle everything else from here, so you just kick back and wait as your computer does all of the magic.
Step 2 - Local DNS Cache
Once you type in the domain, your computer will be doing the rest of the magic from here on out.
Your computer stores cache, which is just information about a webpage stored in your computer. Local DNS Cache is cache stored locally inside your computer that holds the recent domains you have visited before. Your computer will search if the desired domain’s IP address is inside of this list.
If YouTube is in the Local DNS Cache, you skip straight to step seven! If not, then you proceed to the next step.
Step 3 - Recursive DNS Resolver
If the domain you’re searching for isn’t in the Local DNS Cache, then your computer will go to the next step of checking the Recursive DNS Resolver. The Recursive DNS Resolver is run by your internet service provider (ISP), but you can have your own as there’s lots to pick from.
Your computer goes to the Recursive DNS Resolver, which has its own cache of visited domains. If the Resolver doesn’t see that YouTube’s IP address is in its cache, then it will go to other DNS Servers to see if they have the domain, too.
If the Recursive DNS Resolver was able to find the domain you’re searching for at any point, then you skip straight to step seven! If not, then you proceed to the next step.
Step 4 - Root DNS Server
If the Recursive DNS Server can’t find the desired domain cached anywhere, then the computer heads to the Root DNS Server. The Root Server can be described as the DNS backbone of the internet. It holds Top-Level Domains (TLDs)—and a top-level domain is the text you see to the right of the period; it can be .com, .edu, .gov, .net, and so on.
What happens here is that the computer goes to the Root DNS Server solely to identify your TLD. We already know that YouTube’s TLD is “.com,” but this begs the question, why are we identifying the TLD? We’re grabbing it because it is important to the next step.
Step 5 - TLD Server
We’re identifying our TLD so we can head to the TLD Server. A TLD Server is just a server that, well, holds onto TLDs. Since the computer knows that YouTube’s TLD is “.com” after traveling to the Root DNS Server, the computer heads to the TLD Server that focuses on .com TLDs.
The computer may now be in the .com TLD, but that doesn’t mean we go through a big list of all other domains that have .com as their TLD until we find YouTube’s domain and IP address (much like how I initially thought when first learning this). Instead, we go to the next step now that we’re in the TLD Server.
Step 6 - Authoritative DNS Server
The computer headed to the TLD Server so it can then go to the Authoritative DNS Server. The Authoritative DNS Server is the final step that goes into finding the IP address of our desired domain, as this is a server that holds the domain’s information.
There are all kinds of information we can pull through a DNS Record Type—which lets us grab information from a domain. Here are a few common types!
- A Record: Tells you the domain’s IPv4 address.
- AAAA Record: Tells you the domain’s IPv6 address.
- CNAME Record: Tells you the domain’s alias (another domain).
Once the computer arrives at the Authoritative DNS Server, it’ll request the A Record of the domain we’re looking for, as that returns the IP address (in IPv4) of YouTube! And now, we finally have the IP address after all of that work!
The computer then stores the domain and IP address to the Local DNS Cache so it doesn’t have to do all of this again. But… we’re not done just yet! We have the IP address, but now we actually have to connect to it.
Step 7 - Web Application Firewall
Whether you were able to grab the IP address of YouTube in the second/third step (Local DNS Cache/Recursive DNS Resolver), or went through the long journey of steps 1-6, you now have to send a request to access YouTube! So, when firing this request to the IP address, you have to go through the Web Application Firewall.
A Firewall works like a security guard that determines what traffic is and isn’t allowed to enter a network based on a set of rules. This is done to ensure that a network is as safe as possible, as the rules placed can help make sure there isn’t any suspicious traffic going on.
A Web Application Firewall does the same thing, it’s a firewall that sits in between the requester and the application itself. It checks for your request to see if it breaks any of the set rules; if it doesn’t, then your request goes onto the next step.
Step 8 - Load Balancer
Once your request is through the web application firewall, it goes through the Load Balancer right afterwards. A Load Balancer is used to make sure a website always has high availability, and that is done through two things:
- Sending traffic to other servers to prevent massive traffic.
- Gives a fail-over (standby) server if one server becomes unresponsive.
Some companies may have multiple web-servers that are there to take you to the same website, existing solely to handle traffic. Load Balancers have a few different algorithms, but a common one is the weighted algorithm that’ll send your request to the server with the least traffic.
In this step, your request will go through the Load Balancer to then take you to the server that the algorithm selects, which may be the least busy one.
Step 9 - Connects to Web Server Port
Once the Load Balancer sends your request to the server selected by its algorithm, you then get connected to the web-server through its port. Since YouTube uses the HTTPS protocol, the request will go through port 443.
If you were to replace the domain in this example with some other domain that uses the HTTP protocol, then your request would be connected to the web-server on port 80.
We’re not quite finished here, however. We still have some more to go over.
Step 10 - GET Method
The request is finally connected to the server that we’ve been trying to access ever since we typed out the domain name to watch ourselves a nice YouTube video. On this step, we need to use what is called an HTTP Method, and a specific one at that, to grab the website’s assets.
An HTTP Method is used to tell the web-server the intended action we’re going to be making. There’s all kinds of HTTP methods, but there are four common ones you will be seeing the most:
- GET: Used to grab information from the web-server.
- POST: Used to submit new data to the web-server.
- PUT: Used to update existing data in the web-server.
- DELETE: Used to remove data from the web-server.
As you could see from the step name, we’re using the GET method, and that is because we’re requesting the websites assets from the web-server. The web-server follows your request by giving you the website’s HTML, CSS, JavaScript, and so on, to then…
Step 11 - Render Everything
…render everything. Your browser, after making that GET request, got all the assets of the website. And now, everything is rendered to your device. The HTML, the CSS, the JavaScript, all of the images that go into the thumbnails, your computer is now rendering YouTube and you’re free to watch whatever video you want after all of that.
Reflection
To think that all of this happens under the hood when I’m just trying to access YouTube to listen to my favorite music playlists to go through the day is insane! It all happens so quickly… to think that all eleven of these steps occur instantly when I was first entering a website like https://dev.to, for instance, is mind-blowing.
Even writing about this was quite the little adventure, so I hope you feel much of the same while you were reading this! It was a fun write, using what I remember and some of my notes to write about what happens when you search for a domain. Writing this down also helped solidify the concepts I learned along the way, like HTTP methods for instance.
I hope you learned something new! If I got any details wrong throughout the post, or if you would like to add anything, please do drop a comment down below! If you didn’t see any problems, a nice comment below would be much appreciated, too. I’d love to discuss! Thanks for reading!!
This has been cross-posted to my LinkedIn account. Any readers from there, drop a "hello!" down below!
Top comments (0)