HTTP Under the Hood: Before HTTP

I was recently reflecting on a phrase given to me by a university professor a couple of years ago:

Newton’s genius is difficult for us to appreciate because we encounter gravity as an already-established idea. What seems obvious to us had first to become thinkable. His greatness was not simply in seeing things fall, but in questioning what everyone else had learned to regard as ordinary.

Then I found myself thinking about things that I already use or know, but I don't fully understand. And one of those things is HTTP.

I'm pretty sure that HTTP is one of those things that every developer has been involved with but does not understand at all. And please, don't get me wrong; what I'm trying to say is that one thing is to use a library or framework that wraps everything for you, and a different thing is to understand what that library or framework is doing under the hood.

The intention behind this first article is to understand what happens before HTTP in multiple senses:

1. What happened before HTTP was invented.
2. What happens in our computer before HTTP is invoked or used at all in our browser.

I don't promise something fun for this first part of the series, but stay tuned for more interesting content about HTTP and what's behind every major release of the protocol from HTTP/0.9 and the current versions that we have nowadays.

Before HTTP was invented

The Internet has existed way before HTTP, and if you were born after the '80s like me, this concept might be difficult to digest. People used protocols like FTP and Telnet to exchange information between computers. And even I have used these protocols before; it is difficult for me to imagine using the Internet before the Web existed.

So HTTP was not created because computers couldn't communicate through the internet. The problem was something different.

At the end of the '80s, Tim Berners-Lee was working at CERN, a research organization where many scientists from different universities and institutions worked together. As you can imagine, a lot of information was being exchanged across different computers by that time. Finding information was not always easy, especially when every system had its own way of storing and accessing it.

In 1989 Berners-Lee proposed a system where information could be connected using something that they called "hypertext". Instead of knowing which computer had a document, logging into it, navigating its filesystem, and downloading the file, you could simply follow a "link" to another piece of information, even if that information was stored somewhere else.

You can read more about Tim's proposal here.

This idea eventually became what we know today as the World Wide Web, or simply the Web; a system where information could be connected and accessed through links.

But connecting information through hypertext introduced another problem: if a document points to information stored on another computer, how do we actually ask that computer for it?

The Web needed a simple and common way to request information from remote systems. That protocol would eventually become HTTP. But before looking at HTTP itself, we first need to understand what HTTP relies on to communicate between computers, and to explain that we need to know first about the TCP/IP model.

TCP/IP model

You have probably heard about the OSI and TCP/IP models and how they are used to model the behavior of the communication between computers at different levels or "layers". For the purpose of this article and the upcoming ones, we may be using only the TCP/IP model.

Unlike the OSI model, the TCP/IP model is commonly represented with four layers:

• Application: This is where programs define what the communication means. Protocols like FTP, Telnet, SSH, and even HTTP work in this layer.
• Transport: Handles communication between processes/applications running on computers. Two classic protocols that you have heard about before are TCP and UDP.
• Internet Layer: Defines how one computer can reach others across different networks.
• Network Access / Link Layer: This is how your computer actually communicates with another device on its local network. It also defines the channel used for communications, like Wi-Fi, Ethernet, and so on.

What happens when HTTP is used on our computer?

It's essential that you understand the communication flow that happens each time you type in your browser https://google.com or even when you send a request to some REST API.

Do you remember the TCP/IP model we have described before? So, we will describe what each layer does in the process of making an HTTP request.

Before the HTTP communication can start, your computer—on behalf of your browser—needs to resolve google.com into an IP address using DNS.

Notice that DNS is an isolated process itself, so it can also be explained on top of the TCP/IP model on its own.

1. At the "Internet" layer, once your computer resolves the IP address for google.com, it determines how to reach that destination. Because the IP is outside the local network, the computer selects its default gateway (typically the router) as the next hop while keeping the destination IP unchanged.

2. At the "Network Access / Link" layer, the computer encapsulates the IP packet into a frame addressed to the router on the local network. From there, each router repeats the process by forwarding the packet using new link-layer frames, while the destination IP remains google.com until it reaches the final network.

3. Here's where the "Transport" layer chips in. Your computer needs to establish a connection with the google.com HTTP server at some specific port, mostly common at 80 for plain HTTP connections or 443 for HTTPS connections (this will be explained in detail in further articles).

For this example and upcomming ones, we will assume HTTP over TCP. We will see how this changes in later HTTP versions further in the series.

1. Once your computer has established a connection at some port with google.com, then your browser (client) will start sending bytes through a TCP connection and being routed by your gateway to the google.com server. Notice that TCP takes care of transporting those bytes, but it doesn't know what those bytes mean. Defining their structure and meaning is the job of the application protocol, in this case, HTTP.

You may have inferred two obvious terms from the previous explanation:

1. Client: This is who initiates the HTTP request.
2. Server: This is who serves the HTTP response.

In conclusion, the internet existed long before HTTP was invented; the Web changed the way we access and share information over the Internet, and HTTP became one of the key protocols that made it possible.

Comments

[Ivan Firestone]

Genial