Building an HTTP Server From Scratch using TCP.

HTTP is something I interact with every single day, whether I am fetching data on the frontend or building backend APIs. But for a long time, if you had asked me how it actually worked under the hood, I wouldn't have had a good answer. I always treated HTTP like this intimidating, hyper-complex black box that would be miserable to understand, let alone rebuild from scratch. Once I actually dug into the internals, though, I realized the core mechanics are surprisingly straightforward.

I am using Go for this project, but the concepts are entirely language-agnostic. You can follow along in C, Rust, Python, Node, or whatever language you like; as long as it can open a network socket (sorry, HTML and CSS won't cut it here).

To understand HTTP, we first have to talk about TCP/IP, because HTTP is just a text-protocol riding on top of TCP/IP stack.

The Internet Protocol (IP) has one basic job: move raw data from Machine A to Machine B across the world. But it doesn't dump a massive file across the wire all at once. If you download a 10 GB file, IP chops that data into tiny pieces called packets, usually around 1,500 bytes each. It does this because network hardware has physical transmission limits, routers need to share bandwidth fairly among thousands of users, and resending one dropped 1.5 KB packet over flaky Wi-Fi is painless compared to re-downloading an entire 10 GB stream.

Every device on the internet gets an IP address so these packets know where to go. When you want to visit google.com, your computer doesn't magically know Google's physical server address right away. It first asks a DNS server to translate google.com into an actual IP address, like 142.250.190.46. Once your machine has that destination IP, it stamps its own IP address into the packet header so Google knows where to send the reply, and fires the packets into the wild.

The catch is that IP is completely "best-effort." It launches packets into the network and immediately stops caring. If a router gets overloaded and drops your packet, or if packets take different routes and arrive completely out of order, IP won't fix it.

That is why we need TCP (Transmission Control Protocol).

TCP sits right on top of IP to turn that chaotic packet delivery into a reliable stream. It tracks every byte with sequence numbers so out-of-order data gets assembled correctly. If a packet goes missing, TCP notices and asks the sender to retransmit it. It also introduces the concept of ports; so when data finally arrives at a computer's IP address, the operating system knows whether to hand those bytes to your web server on port 80 or your database on port 5432.

So now TCP has solved our biggest headache. We have a solid, reliable pipe open between our client and our server. You throw bytes into one end, and they pop out the other end in the exact right order without getting lost.

Problem solved, right? Not even close.

Because while TCP is great at moving bytes from one place to another reliably, it is completely blind to what those bytes actually mean. It treats everything as one never-ending, continuous river of data (a raw byte stream). If your browser sends a request to load a profile picture, and immediately sends another request for a stylesheet, TCP just mashes all those bytes together in a single stream.

Your server is now sitting there staring at a raw chunk of bytes, completely clueless:

• Where does the first request end and the second one begin?
• Is the client trying to download a file, submit a form, or delete a record?
• Is this data plain text, an image, or JSON?
• If something goes wrong on the server, how do we tell the client?

If we didn't have a standard rulebook, every single developer would invent their own chaotic format. You'd write your own custom protocol where maybe you put an exclamation mark at the end of a message, while someone else uses a random binary flag. Your backend wouldn't be able to talk to any standard browser because neither speaks the same language.

This is where HTTP (Hypertext Transfer Protocol) steps in.

HTTP is nothing more than an agreed-upon rulebook. If TCP is a telephone line connecting two people, HTTP is the grammar they agree to speak so they understand each other.

At its core, HTTP turns that blind stream of TCP bytes into predictable, structured messages. In HTTP/1.1, it does this entirely using plain text:

First, it forces the client to state its intent right on the very first line: like GET /index.html HTTP/1.1. Now the server instantly knows the action (GET), the target (/index.html), and the protocol version.

Next, it uses standard key-value headers separated by clean line breaks; specifically \r\n (CRLF: Carriage Return + Line Feed). Why two characters instead of just \n? Because early internet protocols inherited typewriter conventions from telegraph and terminal days, and now we are stuck with it forever.

Then, it solves the boundary problem with an empty line (\r\n\r\n), which screams to the parser: "Hey, the headers are done! Whatever comes next is the actual body payload."

Finally, the server replies with a standardized response that includes a status code like 200 OK if everything went well, or 404 Not Found if you asked for something that doesn't exist.

That's all HTTP really is. It's not magic, and it's not an intimidating engine. It's just a structured text format running over a raw TCP socket. Once you realize it's just plain text over a byte stream, building one yourself becomes a whole lot less scary.

---

What Does Raw HTTP Actually Look Like?

Before we write the code to parse requests and generate responses, let's look at the exact text format traveling across the wire.

1. The HTTP Request Format

When a client wants something from our server, it sends a plain-text payload formatted like this:

POST /users HTTP/1.1
Host: localhost:8080
User-Agent: curl/8.0.0
Accept: */*
Content-Type: application/json
Content-Length: 26

{"name": "Dev", "age": 22}

• Line 1 (Request Line): Action (POST), path (/users), and version (HTTP/1.1), terminated by \r\n.
• Lines 2–6 (Headers): Key-value metadata lines, each terminated by \r\n.
• Line 7 (Empty Line): A single blank \r\n with no characters. This tells our server: "The headers are done."
• Line 8 (Body Payload): Exactly 26 raw bytes of data matching the Content-Length header.

2. The HTTP Response Format

Once our server finishes processing, it writes back an answer formatted like this:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 35

{"message": "user list endpoint"}

• Line 1 (Status Line): Version (HTTP/1.1), status number (200), and status message (OK), followed by \r\n.
• Lines 2–3 (Headers): Key-value details about what we are sending back, terminated by \r\n.
• Line 4 (Empty Line): A single blank \r\n to mark the end of response headers.
• Line 5 (Body Payload): Exactly 35 raw bytes of data sent down the wire.

Now that we know the format for both directions, let's build the server to handle it.

---

Building an HTTP Server from Raw TCP

I like to break the implementation down into four distinct steps:

1. Initialize a TCP socket and bind it to a local port.
2. Accept incoming client connections in a loop.
3. Read raw bytes from the socket and parse the HTTP request.
4. Construct an HTTP response and write those bytes back over the wire.

1. Setting Up the TCP Listener

In any programming language, listening for network traffic comes down to a few basic steps. You ask the operating system to reserve a port (like 8080), and then you wait for someone to connect.

package main

import (
    "fmt"
    "log"
    "net"
)

func main() {
    // Ask the OS to open port 8080 and listen for incoming traffic
    listener, err := net.Listen("tcp", ":8080")
    if err != nil {
        log.Fatal("[SERVER] Failed to bind to port: ", err)
    }
    // Make sure we release the port when the server stops
    defer listener.Close()

    fmt.Println("[SERVER] Listening on port :8080...")

    // Keep the server running in an infinite loop to accept other connections
    for {
        // Our program pauses right here until a client connects
        conn, err := listener.Accept()
        if err != nil {
            fmt.Printf("[SERVER] Failed to accept connection: %v\n", err)
            continue
        }

        // Pass the connection to a background worker so the loop can keep spinning
        go handleConnection(conn)
    }
}

What is happening here?

• Binding: The operating system locks port 8080 for us. From now on, any data sent to this port comes straight to our app.
• Accepting: Our program pauses and waits. When a client finally connects, the operating system wakes us up and hands us a connection object (conn).
• Concurrency: If we try to process this connection right here in the main loop, our server will freeze for everyone else. So, we hand the connection off to run in the background. Go uses goroutines, Python might use threads, and Node uses its event loop. The idea is exactly the same: move the work out of the way so we can instantly wait for the next person.

---

2. The Request Lifecycle (handleConnection)

Before we dive into the details of parsing, let's look at the whole journey of a single connection.

Our handleConnection function does four things in order: it wraps the raw socket to read from it easily, parses the incoming text into a request we can understand, checks the URL path to see what the user wants, and sends back a text response.

func handleConnection(conn net.Conn) {
    // Always close the connection when we are completely done
    defer conn.Close()
    clientAddr := conn.RemoteAddr().String()

    // Wrap the raw connection in our custom stream reader
    reader := stream.NewReader(conn)

    // Try to make sense of the incoming bytes
    req, err := request.Parse(reader)
    if err != nil {
        // If they sent garbage data, reply with a 400 Bad Request
        res := response.New()
        res.SetStatus(400)
        res.SetBody([]byte("400 Bad Request"), "text/plain")
        _ = res.Send(conn)
        return
    }

    // Prepare a blank response to fill out
    res := response.New()

    // Basic routing: look at the path and decide what to send back
    switch req.Path {
    case "/":
        res.SetStatus(200)
        res.SetBody([]byte("Welcome to my scratch HTTP Server!"), "text/plain")
    case "/users":
        res.SetStatus(200)
        res.SetBody([]byte(`{"message": "user list endpoint"}`), "application/json")
    default:
        // If we don't recognize the path, send a 404
        res.SetStatus(404)
        res.SetBody([]byte("404 Page Not Found"), "text/plain")
    }

    // Push the final formatted text back through the socket
    _ = res.Send(conn)
}

We break this workload into three clear pieces:

• stream: Safely reads a continuous flow of bytes from the connection.
• request: Turns those raw bytes into structured HTTP data (methods, paths, and headers).
• response: Formats our answer into valid HTTP text and pushes it back to the user.

---

3. Reading the Raw Byte-Stream (pkg/stream)

The network doesn't understand neat lines of text; it just gives us raw chunks of bytes. If we try to read one byte at a time directly from the network just to find a newline character, it is incredibly slow.

The universal fix for this is buffering. Instead of reading byte by byte, we pull a large chunk of data (like 4KB) into our application's memory all at once. Once it is safely in our memory, we can quickly scan through it to find our lines.

package stream

import (
    "bufio"
    "io"
    "strings"
)

type Reader struct {
    buffered *bufio.Reader
}

func NewReader(r io.Reader) *Reader {
    return &Reader{
        // Pull data in large chunks to save time and system resources
        buffered: bufio.NewReader(r),
    }
}

func (r *Reader) ReadLine() (string, error) {
    // Read bytes until we hit a newline character
    lineBytes, err := r.buffered.ReadBytes('\n')
    // Clean off the trailing \r\n before handing the string back
    return strings.TrimRight(string(lineBytes), "\r\n"), err
}

func (r *Reader) ReadExact(count int) ([]byte, error) {
    // Create an empty chunk of memory of exactly the size we need
    buf := make([]byte, count)
    // Block and fill the memory completely before moving on
    _, err := io.ReadFull(r.buffered, buf)
    return buf, err
}

We need two different ways to read:

1. Line-by-line: HTTP metadata (like headers) is separated by line breaks. We read line-by-line until we hit the empty line that tells us the headers are done.
2. Exact byte counts: The actual body of a request (like an uploaded image) is just raw data. It might have random line breaks in it. So, once the headers are done, we stop reading lines and instead read an exact number of bytes based on the Content-Length header.

---

4. Parsing the Request (pkg/request)

Now we write the parser to read that incoming text stream into a structured Go struct:

package request

import (
    "fmt"
    "strconv"
    "strings"
    "github.com/devxdh/http-from-scratch/pkg/stream"
)

type Request struct {
    Method  string
    Path    string
    Version string
    Headers map[string]string
    Body    []byte
}

func Parse(reader *stream.Reader) (*Request, error) {
    req := &Request{Headers: make(map[string]string)}

    // 1. Grab the very first line (e.g., "GET / HTTP/1.1")
    line, err := reader.ReadLine()
    if err != nil { return nil, err }

    // Break the line into chunks using spaces
    fLineArr := strings.Split(line, " ")
    if len(fLineArr) < 3 { return nil, fmt.Errorf("Malformed request line") }

    // Assign Method, Path, and Version respectively
    req.Method, req.Path, req.Version = fLineArr[0], fLineArr[1], fLineArr[2]

    // 2. Start reading the headers line by line
    for {
        line, err := reader.ReadLine()
        if err != nil { return nil, err }

        // An empty line means the headers section is completely done
        if line == "" { break } 

        // Split at the first colon we see (e.g., "Host: localhost")
        headerLine := strings.SplitN(line, ":", 2)
        if len(headerLine) < 2 { continue }

        // Lowercase the key so we can easily search for it later
        key := strings.ToLower(strings.TrimSpace(headerLine[0]))
        req.Headers[key] = strings.TrimSpace(headerLine[1])
    }

    // 3. If there is a body, figure out how long it is
    if val, ok := req.Headers["content-length"]; ok {
        contentLength, err := strconv.Atoi(val)
        if err != nil {
            return nil, fmt.Errorf("Invalid content-length: %s", val)
        }
        // Read that exact number of bytes directly into our request body
        req.Body, err = reader.ReadExact(contentLength)
        if err != nil {
            return nil, fmt.Errorf("Failed to read body: %v", err)
        }
    }

    return req, nil
}

I keep two rules in mind when parsing:

• Case-Insensitivity: HTTP header names don't care about uppercase or lowercase. We convert them all to lowercase right away so we don't run into bugs later when searching for "Content-Length".
• Splitting Headers: Some headers have colons in their values (like Host: localhost:8080). We only split on the very first colon we see, so we don't accidentally cut the data in half.

---

5. Writing the HTTP Response (pkg/response)

Sending an answer is doing the exact reverse of parsing. We build a block of text containing our Status Line, our Headers, and a mandatory empty line. Then we push that text, along with any body data, back through the connection.

package response

import (
    "fmt"
    "io"
    "strings"
)

var StatusRegister = map[int]string{
    200: "OK",
  400: "Bad Request",
  404: "Not Found",
}

type Response struct {
    StatusCode int
    StatusText string
    Headers    map[string]string
    Body       []byte
}

func New() *Response {
    return &Response{
        StatusCode: 200,
        StatusText: StatusRegister[200],
        Headers:    make(map[string]string),
    }
}

func (res *Response) SetHeader(key, value string) {
    res.Headers[key] = value
}

func (res *Response) SetBody(body []byte, contentType string) {
    res.Body = body
    // Only set a content type if we actually provided one
    if contentType != "" {
        res.SetHeader("Content-Type", contentType)
    }
}

func (res *Response) SetStatus(code int) {
    text, ok := StatusRegister[code]
    if !ok {
        text = "Unknown"
    }
    res.StatusCode = code
    res.StatusText = text
}

func (res *Response) Send(w io.Writer) error {
    // We use a builder to efficiently glue all our text together
    var builder strings.Builder

    // 1. Start with the Status Line
    fmt.Fprintf(&builder, "HTTP/1.1 %d %s\r\n", res.StatusCode, res.StatusText)

    // Automatically count bytes so the client knows when to stop reading
    res.SetHeader("Content-Length", fmt.Sprintf("%d", len(res.Body)))

    // 2. Add all the headers, making sure to use \r\n
    for key, val := range res.Headers {
        fmt.Fprintf(&builder, "%s: %s\r\n", key, val)
    }

    // 3. Add the blank line that tells the client headers are done
    fmt.Fprint(&builder, "\r\n")

    // Push the constructed text block to the socket
    _, err := w.Write([]byte(builder.String()))
    if err != nil { return err }

    // 4. Finally, push the raw body payload (if we have one)
    if len(res.Body) > 0 {
        _, err = w.Write(res.Body)
    }
    return err
}

Why I build it this way:

• Automatic Content-Length: We count the bytes of our body payload right before sending and set the Content-Length header ourselves. If we don't tell the client exactly how much data to expect, it might hang forever waiting for more.
• Send in one go: We build the entire text block of headers in memory first, and then send it all at once. This keeps things fast and simple.

---

6. Testing the Server with curl -v

Now that every component is in place, we can start our server and test it using curl -v (verbose mode).

You can automate this using a simple Makefile:

.PHONY: start test-get test-users test-post test-404

start:
    go run ./main.go

test-get:
    curl -v http://localhost:8080/

test-users:
    curl -v http://localhost:8080/users

test-post:
    curl -v -X POST http://localhost:8080/users \
        -H "Content-Type: text/plain" \
        -d "This is super horsey TCP/IP payload."

test-404:
    curl -v http://localhost:8080/non-existent-path

Run make start in one terminal, and make test-users in another. Here is what the raw TCP exchange looks like:

* Connected to localhost (127.0.0.1) port 8080
> GET /users HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.0.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 35
< 
{"message": "user list endpoint"}

• Lines starting with > are the raw plain-text HTTP request bytes sent by curl over the TCP socket.
• The empty line right after Accept: */* is the \r\n\r\n boundary our parser scanned for.
• Lines starting with < are the raw response bytes constructed by our response.Send function and pushed down the socket.

Because HTTP is fundamentally plain text over TCP, you could even talk to this server by sending raw text through Netcat (nc localhost 8080).

Building an HTTP server from scratch demystified a lot of networking magic. Hope you got some value form this blog :).