Have you ever wondered what happens when you type a website address like google.com or youtube.com into your browser’s address bar and hit enter? Sure, if you are connected to the internet, you are presented with the desired web page. But exactly what goes behind the scenes is what we will be discussing in this blog.
Structure
This blog will be divided into two parts, the first part will give a complete high-level overview of the whole process that starts with a client typing in a URL then, then converting the URL to an IP address (using DNS), establishing a connection to share resources and finally rendering the web page.
In the second part, we will learn about DNS and how it actually resolves the client-provided domain name into the corresponding IP address.
Part 1: The Overview
1. URL Parsing
Let’s just first understand what a URL is-
URL stands for Uniform Resource Locator. It is composed of four parts namely- Scheme, Domain, Path and Resource, as shown in the image below.
If you are confused about path and resource, consider a file system and think of path as the directory path and the resource as the actual file (.txt / .cpp file or anything).
The process starts with the client entering a URL into the browser’s address bar. The browser then parses this URL, which basically means it destructures it into components (see the diagram above). Now that the browser has the domain name extracted, the process of DNS resolution can start.
2. DNS Resolution
The machine does not understand English, that’s what we humans understand and thus are provided to type the domain name of whatever site we need to refer to. To fulfill our request, the browser needs to know at exactly what address the web page is stored, and that address must be a bunch of 0’s and 1’s!
In other words, it is required to convert the domain name into an IP address of the server that currently stores the files to render the required web page. After obtaining the IP, a connection can be established between the client’s machine and the server.
As soon as we hit enter, the browser first looks into something known as a DNS Cache. A cache is something where recently used information is stored for retrieval to reduce the load on servers. They are of two types-
- Browser DNS Cache — Domain name and IP address mapping stored in the browser. If you are using Chrome, you can see your browser's DNS cache at chrome://net-internals/#dns
- OS DNS Cache — You can see the stored IP addresses in your computer by running ipconfig/displaydnson your terminal.
If the value is not found in these caches, the request goes on to the DNS resolver. For now, let’s treat it as a black box that receives a domain name and sends the corresponding IP address back to the browser. The actual working of DNS will be discussed in detail in Part 2 below.
3. Establishing a Connection
With the IP address obtained from the DNS resolution process, the browser can establish a TCP (Transmission Control Protocol) connection with the web server through a three-way handshake.
If the website uses HTTPS (secure), an additional step called SSL/TLS handshake occurs to establish a secure connection. You can read more about it here.
4. Sending an HTTP Request
Once the connection is established, the browser sends an HTTP (Hypertext Transfer Protocol) request to the web server. The request includes information about the resource you want to access (e.g., the path in the URL) and any additional data if you’re submitting a form or interacting with the website.
5. Receiving the HTTP Response
After processing the request, the web server sends back an HTTP response to the browser. The response contains an HTTP status code, headers (e.g., Content-Type, Cache-Control), and the requested content itself (e.g., HTML page, images, CSS files).
6. Rendering the Web Page
Upon receiving the HTTP response, the browser starts rendering the web page. It interprets the HTML content, loads any additional resources referenced in the HTML (e.g., images, stylesheets, JavaScript files), and executes any JavaScript code.
Finally you can interact with your favourite website and all this happens in few seconds, pretty neat isn’t it?
Part 2: DNS Resolution
Now let's expand the black box we created in step 2.
DNS stands for Domain Name System, it is essentially a distributed and decentralized collection of servers that works together in order to convert the domain name into a corresponding IP address.
There are mainly four types of servers in DNS -
- DNS Resolver
- The Root Server
- Top Level Domain Server (TLD)
- Authoritative Name Server
We will be going through each of their functionality in detail. But before that, we need to know how our computer gets access to this DNS system. So, to answer that let’s learn about ISPs.
1. ISP
After a hurtful cache miss in the browser and OS cache storage, the browser turns to an ISP (Internet Service Provider). The ISP looks into its own cached records and if it is not able to find the domain name, it passes the IP address of the DNS Resolver.
Some examples of ISP’s are verizon, AT&T, etc.
In India some of the popular ones are Airtel, Reliance Jio, etc.
2. DNS Resolver
A DNS resolver is anything that returns an IP address corresponding to a domain name, previously our browser, computer, and ISP acted as a DNS resolver. Since we met with cache miss in all the above three, we need to contact a higher authority, a place where we will be definitely able to get our IP address.
This place is nothing but the network of servers that we call DNS, and the thing that passes our query to this DNS system is the DNS resolver, it basically acts as an entry point to the DNS.
The address of a DNS resolver is provided by your ISP. But you can also change it by tinkering with the network settings. Changing your DNS can lead to a faster and more secure browsing experience. Maybe now’s a good time to learn what a VPN does (but we will go off-topic, instead learn about it here :)).
Some popular examples are Cloudfare's 1.1.1.1 and Google’s 8.8.8.8
We will be discussing a ‘recursive’ type of DNS where we basically tell the DNS resolver to not come back until it has the IP address.
There is another type which is called — ‘iterative’. In this, the client makes a series of requests itself until it found a match. In case of a miss, the current server directs the client to try on some other server. Learn more about it here.
3. Root Server
The root nameserver maintains an up-to-date index of TLD nameservers, each of which handles a specific TLD, such as .com, .org, or .net. The root nameserver also maintains a cache of records, if it is not able to find the queried domain name, it sends back the the address of the appropriate TLD server. (For example, the address of the .in TLD server is sent if the query was for amazon.in).
The resolver then queries the specified TLD server.
There are a total 13 sets of these root servers (A-M) which are strategically placed around the globe. They are managed by 12 organization. The map below shows the distribution of these root servers
4. Top-Level Domain (TLD) Servers
The TLD servers manage information about domain names under their specific TLDs. When the DNS resolver queries the TLD server for the IP address of the requested domain (e.g., “google.com”), the TLD server either provides the IP address directly(cache system) or refers the resolver to the authoritative name server responsible for that domain.
5. Authoritative Name Servers
These are the final stop in the DNS resolution process. The authoritative name servers store the DNS records for specific domain names. Each domain typically has at least two authoritative name servers for redundancy and fault tolerance. When the DNS resolver contacts the authoritative name server for the requested domain, it receives the IP address of the server hosting the website associated with that domain.
6. IP Address Returned
The client (finally) gets back the IP address from the DNS resolver, which caches the IP address in its local cache for future use.
Step 2 described in Part 1 is now complete, and we can move on with the rest of the steps as described.
Notice how caching mechanism is involved in every step of DNS resolution. And how the DNS system is distributed among various servers. This distributed network and the caching mechanisms are necessary in order to make the URL resolution process seamless and efficient.
If you learned something new, be sure to show your support and check out my other blogs —
- The Backend Development Primer [Part 1/2]
- The Backend Development Primer [Part 2/2]
References
- All the illustrations in this article are taken from ByteByteGo. Check them out :)
- A short video by ByteByteGo (Part 1)
- DNS Resolution explanation by PowerCert Animates Videos (Part 2)
- A short video by ByteByteGo (Part 2)
- https://www.computerhope.com/jargon/d/dns-resolver.htm
- https://www.cloudflare.com/learning/dns/what-is-recursive-dns
Top comments (0)