Want to be a better full-stack developer? Know the web and HTTP principles

Building an HTTP Server from Scratch with Nodejs and Typescript

These days, you can spin up a fully functional server in minutes with the help of modern tools.

So why would anyone bother building an HTTP server from scratch?

For me, it's an experiment I am running for the next three months: mastering fundamentals and doing deep work.

There is also a simple truth about working in tech: if I am serious about thriving as a full-stack engineer, I surely need to understand the basics, and not just work with abstractions and frameworks. Plus, there’s a kind of clarity that comes from seeing how all the pieces fit together: from TCP to routing requests. Beyond the technical challenge, there’s finally the simple satisfaction of building something fun and tangible.

Why Build My Own HTTP Server?

1. Learning the Fundamentals: I think that understanding how HTTP servers work at a granular level helps solidify my work as a full-stack engineer. It's one thing to deploy a server using pre-built libraries or services; it’s another to know how requests and responses are handled.

2. Building Something Tangible: There’s unparalleled satisfaction in crafting a tool that serves a real purpose, like delivering a web page that serves an html page with images and text, using only the basics. It’s a powerful way to deepen your technical skills while working on something enjoyable.

---

First, the basics. What Is an HTTP Server?

At its core, an HTTP server is a program that listens for incoming HTTP requests from clients (like a browser or API consumer), processes those requests, and sends back responses. It uses the TCP (Transmission Control Protocol) to establish a reliable connection, ensuring that data packets arrive in order and intact.

TCP vs. UDP

I do mountaineering and one analogy came to me to differentiate TCP and UDP protocols.

• TCP: Think of climbers on a glacier roped together. Everyone reaches the summit securely and in order. That’s TCP: ensuring that data (climbers) arrive in order and intact.

• UDP: UDP are like alpinists climbing individually without ropes. It’s usually faster, but there’s no guarantee everyone gets there. Fast, but less reliable.

Key features of my HTTP Server

1. Handling GET Requests: Responds with a simple “Hello, World!” for basic requests.

2. Support for HTTP/1.1 Protocol: Implements the essentials of this widely-used protocol.

3. Serving Static Files: Fetches and delivers image files stored in the server’s public/images/ directory.

4. Routing: Implements basic routing using if-else statements to handle different endpoints like /api or /.

5. Error Handling: Ensures proper responses for malformed requests, like returning 400 Bad Request for parsing errors.

What I Learned

Sockets Sockets Sockets

Sockets are how servers and clients talk to each other. Before this project, I knew about sockets the way most of us know about cars, we press the gas pedal, and the car moves. It's magic! Now, I’ve seen how there is nothing magical about it, especially when I wrote the createDataHandler function.

createDataHandler is a function that handles incoming TCP data chunks for HTTP requests. It maintains a buffer: a string that accumulates data from incoming chunks.

• For each chunk, it converts the binary data into a string and appends it to the buffer.
• It then checks if the buffer contains \r\n\r\n, which marks the end of the HTTP headers.

• If it finds the marker, it processes the request and sends a response.

  This gave me a clear view of how HTTP works at the lowest level: every incoming request starts as raw data, and the server has to parse it into something usable. Sockets are the invisible ropes that pull these chunks back and forth, and the createDataHandler function is the intermediary which makes something sensible of the data.

  Writing this function showed me just how much happens before a server even begins to respond to a request. It also taught me how fragile things can be. If the buffer isn’t handled properly, by failing to detect the end of the headers for example, the server will just breaks. These are things I've never appreciated when using frameworks.

Routing Isn’t That Simple

Manual routing turned out to be more challenging than I expected, especially when it came to handling content types. Frameworks like Express.js make this super easy, but behind the scenes, there’s a lot happening:

• Content-Type Handling: My server only supported JSON for request bodies, and I had to set the correct MIME type manually for responses. Adding support for more content types, like multipart/form-data for file uploads or application/x-www-form-urlencoded for form submissions, would require additional parsing logic.
• Headers Management: In a basic setup, you’re responsible for setting essential headers like Content-Type and Content-Length. Get these values wrong, and the client either won’t understand the response or will hang waiting for more data. It's what happened when my server was not picking up images/jpeg datatype.

• Parsing Body Formats: Each content type requires its own logic. For example, JSON bodies need to be parsed into JavaScript objects, while multipart/form-data requires handling file streams. This complexity grows quickly as you add support for more formats.

  Express.js automates most of this. It detects the Content-Type header, parses the body accordingly, and sets the right MIME type for responses. Doing this manually gave me a deep appreciation for how much work frameworks save.

While my manual routing works for basic cases, it’s clear that scaling it to handle more complex scenarios would require more effort.

Always Try to Break Your Software

Software is rarely perfect the first time around, a great way to test my server was to run:

hey -n 1000 -m GET http://localhost:8080/

BOOM. Serve Crashes after 800 requests.

This command sends 1,000 simultaneous GET requests, simulating real-world stress. It exposed a critical issue: I wasn’t handling socket timeouts properly. My server would hang because it didn’t close idle sockets.

To fix this, I chose to close the socket after each request, which works great for the scope of this project. But if I wanted to support persistent connections, I’d need to avoid calling socket.end() and implement additional logic. For now, closing the connection after each request is the simplest and safest approach. It’s clean, predictable, and prevents resource leaks.

---

What’s Next?

There’s a lot more I could add to this server, which is also why it's so nice to start a project from ZERO. I get to own every bite of it and can add on to it as needs/curiosity fits.

• HTTPS: Right now, it’s plain HTTP. Adding SSL/TLS would make it secure.
• WebSockets: For real-time communication, like chat apps.
• Authentication: Handle user logins and sessions.
• Better Routing: Add support for more content types and dynamically set MIME types based on file extensions or request content.
• Caching: Improve performance by storing frequently requested data.

---

Why It Matters - Meta Learnings

Writing an HTTP server from scratch was a way to deepen my core understanding of the web and servers. To not take things for granted, and to improve my engineering skills as a full stack developer. It was also interesting to see how constraints (not using a framework like Express) force you to think critically and solve problems creatively, which of course deepens my understanding. And there’s a unique joy in building something functional from scratch! 😃

You can check the project on GitHub!