Let's begin with a simple definition of a web server:
A web server is a computer program that listens for HTTP requests and responds with corresponding HTTP responses.
When I first started to learn about web development I learned about HTML, CSS, and our good friend JavaScript. Once I was comfortable with those I began reading about PHP and what I could do with it. It seemed interesting. I installed XAMPP on my Windows machine and started learning. All I really knew was I had Apache HTTP Server and somehow this ran my PHP which produced my HTML.
Fast forward a few years and I am often building web applications using Node.js.
$ npm init -y
$ npm install express --save
A lot of people like to use Express.js because it's easy to understand and get started with. I was building web servers day in and day out. I don't intend to bash Express.js or Node.js but when I look back I think I've wasted quite a bit of time.
History
It's the early 1990s and Sir Tim Berners-Lee debuted the World Wide Web. Genius contraption. You have a web browser on the client, a web server somewhere in the network, and these two speak with each other using HTTP over TCP/IP. Back then websites were really simple in that they were all static - hand written HTML and CSS.
In maybe two years after the WWW was debuted hundreds, if not thousands, of web servers were out in the wild - mostly universities. With increased usage came some new requirements. One such requirement was the ability to display data in existing systems as HTML documents. Many people wrote their own web servers to solve this - a request would come in and the web server would compile the information from their systems to produce an HTML document and return it. These solutions worked but they weren't portable.
Soon enough we had a lot of web servers that did their own thing to produce dynamic output. It became apparent that you needed a standard for this and so CGI was born. CGI is short for Common Gateway Interface. Here's how it works:
Web server receives HTTP request.
If response needs to be generated dynamically then run CGI script as a new process. Forward HTTP request and other information to script as environment variables.
CGI script generates the HTTP response, headers and body, and writes it to standard output.
Web server picks up the response from standard output of the CGI script and forwards it.
CGI is simply a protocol. The CGI script can be anything really - a shell script, a C program, a PHP script, anything. The CGI specification used C as an example. This was a game changer because we now had a way to generate dynamic content and how we did it did not depend on the web server. Soon enough web servers like the Apache HTTP Server were born. This was around the mid '90s.
CGI was and is a great idea. It very cleanly decouples the handling of an HTTP request from generating a dynamic HTTP response, and it is also very simple to use. There is one drawback though - it isn't scalable. Whenever you make a request that runs a CGI script, the web server will spawn a new process to run the CGI script. So if you make a 100 requests you are spawning a 100 processes for those requests. More often than not, the overhead of creating processes outweighs the cost of generating the response. Because the lifetime of a process is the lifetime of the request itself, applications built as CGI scripts can't reuse resources like database connections when new requests come in. This makes applications slow as well.
The solution to this problem is rather obvious: instead of making a new process to handle every request, make one process that handles all requests. But how do you do it in this post-CGI world? FastCGI. Here's how FastCGI works:
Web server receives HTTP request.
If response needs to be generated dynamically then forward HTTP request to FastCGI server via Unix domain sockets/named pipes/TCP connection.
FastCGI server generates the HTTP response, headers and body, and sends it back via Unix domain sockets/named pipes/TCP connection.
Web server forwards the HTTP response.
The FastCGI server is your application. Just like in the case of CGI, the web server forwards the HTTP request and other information to your application but it does this over a Unix domain socket, or a named pipe, or a TCP connection instead. This allows your application to stay running even after a request has been served. And again just like in the case of CGI, it doesn't matter how you implement a FastCGI server because FastCGI is just a protocol. FastCGI allowed people to write more efficient applications and have more efficient web servers in general.
CGI and FastCGI are milestones in the development of web technologies. There are other protocols and solutions besides these two but they hold a special status being one of the first solutions for these problems.
Today
The history of the WWW is just around 30 years as of April 2020. Most of the things I talked about in the previous section is recent memory and the ideas discussed are still very relevant. In fact a lot of applications and businesses are still running just fine with them.
Web servers are a little more than the simple definition I use in the beginning of this article. They parse different media types, compress responses, handle auth, do redirects, etc. Web servers like Apache HTTP Server and Nginx are general purpose. They will almost always meet all your requirements, including your performance requirements. They'll serve your static content and if you need to send back dynamic content, you can hook up these web servers to your application using protocols like CGI and FastCGI. So then, do you really need to build your own web server? Do I?
I cannot answer that question because all this while I've been building my own web server using Node.js for every single web application I've made. But I think the answer is no - no, I do not need to build my own web server for the vast majority of applications that I will build in my lifetime.
What do you think? Leave your thoughts as a comment.
Top comments (0)