Behind the click: unraveling the mystery of the Internet

#devops #learning #softwareengineering #networking

Ever wondered what happens behind the scenes when you type "google.com" and instantly see the Google homepage?
This article will unravel the mystery in a way that's easy to understand, even for non-tech enthusiasts. Let's embark on a journey from typing a simple web address to accessing the vast world of the internet.

When we enter a web address in our browser, we're essentially requesting a resource stored on another computer. This computer, technically referred to as a server, is always online and ready to respond to our requests.

When we type "google.com" and hit enter, our browser first asks the question, "Who is google.com?" This process is known as a DNS lookup, where DNS stands for Domain Name System. Because computers identify each other on the internet with IP addresses, our browser needs to find the IP address of google.com. This is similar to what happens when you call 'Dad' on your phone. While the name 'Dad' is easy to remember, your phone can't actually call the name 'Dad'; it has to translate Dad to a phone number. Like phone numbers, IP addresses are not typically memorized.

How does our browser translate google.com to an IP address?

In most cases, our browser first checks its own cache to see if it has a recently retrieved IP address for the domain name. This cache helps speed things up by avoiding redundant DNS lookups for frequently visited websites. If the cache doesn't have the address, the browser then checks the 'hosts file' to see if google.com is mapped to an IP address locally on our pc. If the IP address is not in the hosts file, the browser initiates a DNS lookup process to find the IP address for the domain name 'google.com'.

DNS lookup process

Now that our browser is sure it doesn't have the IP address cached or stored locally on our PC, it begins a hunt for the IP address externally. It first queries our resolver, which is our ISP in most cases. If the resolver doesn't know the IP address, it asks the root servers. These root servers act like a phone book index. They point the resolver in the right direction for the Top-Level Domain (TLD) server it needs to contact next. There are 13 root servers in the world.

Based on the TLD for the domain name, the resolver will query the appropriate TLD nameserver. For example, the ".com" TLD would have its own nameserver.
The TLD nameserver then directs the resolver to the authoritative nameserver for the specific domain name "google.com." This authoritative nameserver holds the actual IP address for the domain name.

Once the IP address is obtained, it is returned to the DNS resolver, which caches the result for future use. The browser also caches the IP address upon receipt. This helps improve performance by reducing the need for repeated DNS lookups for frequently accessed domain names.

Now that our browser knows the IP address that google.com points to, it uses a set of protocols called TCP/IP (Transmission Control Protocol/Internet Protocol) to establish a connection with the server. Once connected, it sends what's called an HTTP request to the Google server. This request is the browser's way of asking the server to provide the content associated with google.com. The request we're making is quite interesting and encompasses enough details to write an entire article about!

The Google server receives the request, processes it, and sends back a response. This response includes the content of the webpage (HTML code, multimedia images, etc.) formatted according to the browser's understanding.

Our browser receives this response, downloads the HTML content, and begins to render it for us to see. This process involves interpreting the HTML code and displaying it as a visual representation—the web page we see every day.

If you're curious...

You can Press Ctrl + Shift + I on Windows, Command + Option + I on Mac, or use the curl utility on Linux. This will let you see the data exchanged between your browser and the web server, including the HTML code that makes up the webpage.
Typing in 216.58.215.174 in your browser bypasses the need for a DNS lookup by your browser and takes you directly to google.com (Note: IP addresses can change, so this might not always be the exact IP address for Google's server).
Here's a nifty trick you can do with '/etc/hosts' on Linux.

Conclusion

Our exploration has unveiled the intricate technology that transforms a simple web address into a fully loaded webpage in milliseconds. This remarkable feat is a testament to the extraordinary work of the many innovators who designed and continue to develop the Internet's infrastructure. These pioneers laid the groundwork for the global network that shapes our lives today. While this article focused on core concepts without delving into excessive technical details, it equips readers with a solid understanding of how the internet functions. This knowledge is valuable not just for software engineers and developers, but for anyone curious about the inner workings of the technology that has become an essential part of our world. Feel free share any additional insights or thoughts in the comments.