ayka.code

Posted on Aug 5

The Complete Developer's Guide to the HTTP Request/Response Lifecycle

#webdev #networking #go #performance

From DNS resolution to TCP handshakes and raw headers—here is exactly what happens when your client sends a GET request.

Introduction

The HTTP Request/Response Lifecycle is the synchronous network protocol execution loop through which a client establishes a connection, negotiates a security context, and exchanges structured headers and payloads with a remote server over TCP/IP.

In today's landscape of abstract, zero-configuration cloud hosts, it's easy to forget that beneath every Next.js route or AI agent API call lies a physical sequence of raw socket writes, DNS recursive traversals, and cryptographic handshakes.

Understanding these transport-layer mechanics is not academic; it is the boundary between an application that breaks under load and a robust, scalable system that performs gracefully at production scale.

In this guide, we'll bypass framework-specific abstractions and trace the exact, step-by-step physical journey of an HTTP request.

We'll dissect raw socket interfaces, analyze packet sequence structures, and write a functional client using low-level Go socket primitives to see the protocol in its purest form.

1. Phase 1: Domain Name System (DNS) Recursive Resolution

Before a single socket can be opened, the client must resolve the human-readable domain name into a routable physical IP address.

The process of translating dev.to into an IP address is a highly optimized, hierarchical lookup sequence designed to minimize latency and protect root network resources.

The Local Resolution Chain

The operating system resolver first attempts to fulfill the query locally to bypass network latency.

Browser Cache

Modern browsers maintain their own DNS caches with strict, short Time-to-Live (TTL) expiration frames.

OS Resolver Cache

The browser executes a system call (typically getaddrinfo() on Unix-like systems).

The OS resolver first checks its local static mapping file (/etc/hosts) before searching its system-level DNS cache.

Local Router / ISP Cache

If the record is missing, the resolver queries the Local DNS Server configured via DHCP (usually your router or ISP resolver).

The Recursive Resolution Loop

If the record remains unresolved, the recursive DNS resolver initiates a multi-step query loop across the global DNS hierarchy.

[ OS Resolver ] --(1. Query: dev.to)--> [ Recursive Resolver ]
                                               |
        +--------------------------------------+--------------------------------------+
        | (2. Query .)                         | (4. Query .to)                       | (6. Query dev.to)
        v                                      v                                      v
  [ Root Name Server ]                  [ TLD Name Server ]                  [ Authoritative Name Server ]
  (Returns .to NS)                      (Returns dev.to NS)                  (Returns A/AAAA IP Record)

Root Nameservers (`.`)

The recursive resolver queries one of the 13 logical root nameservers.

The root server does not know the IP address of dev.to, but it returns the name servers responsible for the requested top-level domain.

TLD Nameservers (`.to`)

The resolver queries the .to TLD nameserver.

It responds with the authoritative name servers responsible for dev.to.

Authoritative Nameservers

Finally, the resolver contacts the authoritative DNS server.

This server returns either:

an A Record (IPv4)

an AAAA Record (IPv6)

The IP address is then cached locally and returned to the browser.

2. Phase 2: Transport Layer Connectivity — TCP & TLS 1.3

Once the destination IP is acquired, the client initiates transport layer connectivity.

Since HTTP relies on TCP for reliable, ordered delivery of data streams, a TCP connection must first be established.

Port 80 → HTTP
Port 443 → HTTPS

The TCP Three-Way Handshake

Client                                                  Server
  |                                                       |
  | -------- SYN (Seq = X) -----------------------------> |
  |                                                       |
  | <------- SYN-ACK (Seq = Y, Ack = X + 1) ------------ |
  |                                                       |
  | -------- ACK (Ack = Y + 1) -------------------------> |
  V                                                       V

1. SYN

The client transmits a TCP segment with the SYN flag set.

A random Initial Sequence Number (ISN) is generated.

2. SYN-ACK

The server allocates TCP buffers, generates its own sequence number, and acknowledges the client's sequence.

3. ACK

The client acknowledges the server.

The TCP connection is now established.

The Cryptographic Dance: TLS 1.3

TLS 1.3 reduces connection setup from two round trips (TLS 1.2) to a single round trip.

Step 1 — Client Hello

The client sends:

Supported TLS versions
Supported cipher suites
ECDHE public key

Step 2 — Server Hello

The server returns:

Selected cipher suite
Server certificate
Server public key
Handshake signature
Finished message

Both parties independently compute the same symmetric session key.

Step 3 — Client Finished

The client verifies the certificate, validates the signature, computes the shared secret, and sends an encrypted Finished message.

From this point onward, every byte written to the TCP socket is encrypted.

3. Phase 3: Inside the Raw HTTP Request

With TLS established, the browser constructs the actual HTTP request.

Instead of using net/http, we'll manually build and transmit an HTTP request over a raw socket.

package main

import (
    "crypto/tls"
    "fmt"
    "io"
    "log"
    "net"
)

func main() {
    host := "dev.to"
    port := "443"
    address := net.JoinHostPort(host, port)

    // Step 1: Establish low-level TCP Connection
    tcpConn, err := net.Dial("tcp", address)
    if err != nil {
        log.Fatalf("Failed to establish TCP connection: %v", err)
    }
    defer tcpConn.Close()

    fmt.Printf("[+] Established TCP Connection to %s\n", address)

    // Step 2: Wrap TCP connection in TLS
    tlsConfig := &tls.Config{
        ServerName: host,
        MinVersion: tls.VersionTLS13,
    }

    tlsConn := tls.Client(tcpConn, tlsConfig)

    err = tlsConn.Handshake()
    if err != nil {
        log.Fatalf("TLS Handshake failed: %v", err)
    }

    fmt.Printf(
        "[+] Established TLS 1.3 Session. Cipher Suite: %s\n",
        tls.CipherSuiteName(
            tlsConn.ConnectionState().CipherSuite,
        ),
    )

    // Step 3: Construct raw HTTP request
    httpRequest :=
        "GET / HTTP/1.1\r\n" +
            "Host: " + host + "\r\n" +
            "User-Agent: RawGoSocketClient/1.0\r\n" +
            "Accept: text/html,application/xhtml+xml\r\n" +
            "Connection: close\r\n" +
            "\r\n"

    // Step 4: Send request
    _, err = io.WriteString(tlsConn, httpRequest)
    if err != nil {
        log.Fatalf("Failed to write HTTP request: %v", err)
    }

    fmt.Println("[+] Sent Raw HTTP GET Request:")
    fmt.Println(httpRequest)

    // Step 5: Read response
    responseBuffer := make([]byte, 4096)

    n, err := tlsConn.Read(responseBuffer)
    if err != nil && err != io.EOF {
        log.Fatalf("Error reading response bytes: %v", err)
    }

    fmt.Println("[+] Received Raw Response Bytes:")
    fmt.Println(string(responseBuffer[:n]))
}

Deconstructing the HTTP Request Structure

A raw HTTP request consists of three logical sections.

Request Line

GET / HTTP/1.1

GET → HTTP Method
/ → Requested resource
HTTP/1.1 → Protocol version

Headers

Headers provide metadata about the request.

Example:

Host: dev.to
User-Agent: RawGoSocketClient/1.0
Accept: text/html

The Host header is mandatory in HTTP/1.1 because servers often host multiple websites on the same IP.

Blank Line

A mandatory \r\n\r\n separates the headers from the body.

If this were a POST request, the JSON payload would begin immediately afterward.

4. Phase 4: Server Processing and the Raw HTTP Response

When encrypted bytes arrive at the server's Network Interface Card (NIC), the operating system passes them to the listening web server.

Examples include:

NGINX
HAProxy
Apache
Go binaries

Server Processing Loop

TLS Decryption
Parse the HTTP stream
Route the request
Authentication / Authorization
Database queries
Construct the response
Send the response back through the encrypted TLS socket

Raw HTTP Response

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 1042
Connection: close
Date: Wed, 05 Aug 2026 11:56:15 GMT
Cache-Control: public, max-age=3600
ETag: "9b4d186f-52fe"

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>DEV Community</title>
</head>
<body>
    <h1>Welcome to DEV!</h1>
</body>
</html>

Response Structure

Status Line

HTTP/1.1 200 OK

Contains:

Protocol
Status code
Status text

Headers

Examples include:

Content-Type
Content-Length
Cache-Control
ETag

Blank Line

Separates headers from the response body.

Body

The actual resource:

HTML
JSON
Images
CSS
JavaScript

5. Phase 5: Performance Optimization & Caching Boundaries

To protect origin servers and reduce latency, modern applications rely heavily on caching.

Browser Cache

Example:

Cache-Control: max-age=31536000, immutable

This instructs the browser to reuse the resource without contacting the server for up to one year.

ETags and Conditional Requests

Suppose the cached asset expires.

Instead of downloading the file again, the browser sends:

If-None-Match: "9b4d186f-52fe"

If the file has not changed, the server responds with:

304 Not Modified

No body is transmitted, saving bandwidth.

CDN Edge Caching

Content Delivery Networks (CDNs) cache static assets at servers geographically close to users.

Instead of reaching the origin server, clients receive content from nearby edge locations.

Benefits include:

Lower latency
Faster page loads
Reduced origin server load
Improved Largest Contentful Paint (LCP)

Conclusion

The HTTP request/response lifecycle is a highly choreographed protocol sequence executed millions of times per second.

Behind the elegant abstractions of modern frameworks lies a transport layer governed by strict rules of synchronization, validation, and encryption.

By understanding the complete journey your data takes—from DNS resolution to TCP connectivity, TLS encryption, HTTP message construction, server processing, and caching—you gain the systems-level knowledge needed to build faster, more resilient, and production-ready applications.

Follow me

Introduction

1. Phase 1: Domain Name System (DNS) Recursive Resolution

The Local Resolution Chain

The Recursive Resolution Loop

Root Nameservers (.)

TLD Nameservers (.to)

Authoritative Nameservers

2. Phase 2: Transport Layer Connectivity — TCP & TLS 1.3

The TCP Three-Way Handshake

1. SYN

2. SYN-ACK

3. ACK

The Cryptographic Dance: TLS 1.3

Step 1 — Client Hello

Step 2 — Server Hello

Step 3 — Client Finished

3. Phase 3: Inside the Raw HTTP Request

Deconstructing the HTTP Request Structure

Request Line

Headers

Blank Line

4. Phase 4: Server Processing and the Raw HTTP Response

Server Processing Loop

Raw HTTP Response

Response Structure

Status Line

Headers

Blank Line

Body

5. Phase 5: Performance Optimization & Caching Boundaries

Browser Cache

ETags and Conditional Requests

CDN Edge Caching

Conclusion

Root Nameservers (`.`)

TLD Nameservers (`.to`)