I accidentally implemented HTTP/1.0 while claiming HTTP/1.1

I've been building Ariadne, an Express-inspired HTTP framework written from scratch in modern C++ using raw POSIX sockets. Everything was going smoothly, routing, middleware, and request parsing all worked beautifully.

Then I load tested it with k6. That's when the weirdness started.

As soon as traffic ramped up, my terminal flooded with errors saying:

"unsolicited response received on idle HTTP channel"

Connection resets and aborted requests filled my terminal up. My first instinct was that request parsing was failing under load — dropping bytes, maybe confusing response boundaries somewhere. I pored over my recv loops and buffer management. Nothing looked wrong. The server wasn't crashing. It was successfully handling some requests and failing spectacularly on others, only under high concurrency.

Turns out, the bug had nothing to do with how I was parsing data, and everything to do with how I was managing the TCP connection lifecycle.

The Reveal

My server was lying to its clients. It claimed to speak HTTP/1.1, but it behaved like HTTP/1.0.

Under HTTP/1.1, connections are persistent by default. Unless either side explicitly sends a Connection: close header, both client and server assume the TCP connection stays open and gets reused for future requests.

Here's what my worker thread looked like at the time:

pool.enqueue([this, clientFd] {
    // 1. Read and parse the request
    auto req = parseRequest(clientFd);
    if (req) {
        Response res(clientFd);
        // 2. Route the request and send the response
        executeMiddlewares(0, *req, res);
    }
    // 3. Immediately close the socket!
    close(clientFd);
});

See it? Accept a connection, parse one request, send one response, and close the socket.

Since I never sent Connection: close, k6 correctly assumed — because I'd told it I was speaking HTTP/1.1 — that the connection was still alive, and returned it to its pool. Milliseconds later, k6 grabbed that "idle" socket to send another request. On my end, that socket was already dead.

Hence the error: k6 was getting EOFs and resets on what it believed was a healthy, idle channel.

The Fix

The fix required a shift in how worker threads thought about their job. A worker thread couldn't own a request anymore — it needed to own a connection.

That meant wrapping request parsing and response generation in a loop, keeping the socket alive until the client explicitly asked to close it or an error occurred:

pool.enqueue([this, clientFd] {
    while (true) {
        try {
            // Iteratively parse requests from the same connection
            auto req = parseRequest(clientFd);
            if (!req) break;

            Response res(clientFd);
            bool keepAlive = true;

            // Check if the client wants to close the connection
            auto it = req->headers.find("Connection");
            if (it != req->headers.end() && it->second == "close") {
                keepAlive = false;
            }

            // Set the appropriate response header
            if (keepAlive)
                res.set("Connection", "keep-alive");
            else
                res.set("Connection", "close");

            executeMiddlewares(0, *req, res);

            // Break the loop if we're done with this connection
            if (!keepAlive) break;

        } catch (const std::exception &e) {
            Response res(clientFd);
            res.status(400).send("Bad Request");
            break;
        }
    }
    // Finally close the socket when the loop ends
    close(clientFd);
});

This later on introduced a new bug that I also found out under load tests. That is what the next post will be about.

The Takeaway

Once this landed, the protocol violations disappeared completely and as a side effect, my benchmarks against Node and Go finally became fair comparisons, since all three were now speaking the same protocol correctly.

It's easy to take keep-alive for granted when you're working inside a high-level framework. Write the socket code yourself, though, and you realize HTTP/1.1 is really a continuous conversation over one wire. You can't hang up after your first sentence and expect the other side to still be listening when you start talking again.

Have you ever shipped something, load tested it, and discovered you'd been quietly violating a protocol the whole time? Let me know in the comments. I'd love to hear you guys out.

I'll be back with more next week, till then stay consistent!

Comments

[Matthew Faithfull]

Nice. I will definitely be taking a deep dive into your source code. I've partially implemented an HTTP server layered over my general pipeline framework and network server. Also over platform independent sockets. github.com/mfaithfull/linuxQOR/tre...
I can parse Requests and send a fixed response back, basically saying it's not done yet. Now I need to do the actual server part that sends back a different response depending on what's in the request. I took the problem on a layer at a time, separating the protocol from the networking and getting an Echo client and server working on the same platform first to avoid the kind of issue you've seen.
Are you intending to add support for SSL? I might be up for collaborating as that's a challenge I don't fully understand. I have an 'example' SSL implementation but it's very poor code and hard to follow.

[Nikhil Sharma]

Thanks! I'd love to hear your thoughts after you look through the code.

SSL is definitely something I'd like to explore eventually, but I haven't reached that stage yet. I'll probably integrate OpenSSL once the HTTP side is mature enough (using epoll instead of a thread pool like I do in the current implementation). But I would love to collaborate with you on it!