Greetings, in this article, I will tell you how we can increase the test performance of our HTTP servers in the projects we have developed with Go.
Recently, while I was working on a new feature for an application we were developing, I realized that I could commute to get coffee while waiting for unit tests to complete. And it was not possible to run the tests in parallel. If we wanted to run it in parallel in its current form, this test period could have taken much longer.
Now, let's take a brief look at the content and move on to the details of our subject.
Contents
Go HTTP Internals
- http.ListenAndServe
- net.Conn & net.Listener
- http.Client Dial Function
- net.Pipe
Testing in Different Packages
Fiber
- app.Test Internals
- app.ServeConn
fasthttp
- fasthttputil.NewInmemoryListener
Go HTTP
- httptest.NewUnstartedServer
- Mock InMemListener Example
Our Test Performance Improvement
Conclusion
Go HTTP Internals
Let's take a look at what we need to do when we want to run a simple HTTP server with the Go language.
http.ListenAndServe
When we want to run a server that we wrote using the Go standard library, we use the ListenAndServe method.
Let's take a look at the details of the ListenAndServe method:
Now let's look at the details of the server.ListenAndServe() method:
Finally, let's look at the details of the srv.Serve(ln) method. I've removed some code so it's not too long.
The part that we should pay attention to here is the l.Accept method.
The summary of the process done in the codes I have shared so far is actually:
http.ListenAndServe method creates a server in the background and server ListenAndServe method is run
The server creates a net.Listener object that accepts a request from the port specified by the
net.Listen("tcp", addr)method call and runs the Serve method.The Serve method acts on the net.Conn object that represents the new incoming connections with the
Accept()method of the net.Listener object that it receives as a parameter.
So far, we have seen the process of creating an API in its basic form.
Now, I can hear you slowly saying, "Well, sir, what will they do in real life?"
Don't worry, we'll soon see how we can improve our testing performance using this information.
net.Listen & net.Conn
Now let's take a closer look at these two basic types included in the HTTP package.
We see that these two types are actually interfaces.
Well, what does it mean? We can give special structs that we implement these interfaces as parameters to methods that expect these interface types as parameters. This is an important point for us.
http.Client Dial Function
Now let's look at how the http.Client object is used, which allows us to make HTTP requests with Go.
There are two methods of sending requests using the Go HTTP package.
- The first method is to use the http.Get http.Post methods directly
- The second method is to use the http.Client object
resp, _ := http.Get("http://localhost:8080/test")
Actually, the first method, http.Get, also uses the http.Client object in the background.
Let's take a brief look:
In other words, both methods do the same thing in the background.
Now let's look at the fields that we can customize when creating the http.Client object.
http.Client provides all connection establishment processes with RoundTripper interface. And it uses the Transport object as a RoundTripper implementation in itself.
If we want, we can create this Transport object ourselves when creating http.Client.
The Transport object we created has a method called DialContext. This method is the method that enables connection opening in the background, it returns a value of type net.Conn as a variable.
In a moment, we will combine this information and the information that the server.Serve method we just saw is listening for connections with net.Conn using the accept method of the net.Listener variable that takes as a parameter.
Little by little the pieces are falling into place, aren't they?
net.Pipe
Finally, there is another method I would like to mention: net.Pipe(). It returns two net.Conn type variables as return value.
One of these variables can be used as a writing point and the other as a reading point. So what does this mean? How can we use this?
If you remember, the Listen() method of the Listener object returned the variable net.Conn. The Dial() method of the HTTP client Transport object also returns the variable net.Conn.
Starting from here, when we give one end of the connections returned by the net.Pipe() method to the Dial method of our HTTP client and the other end to the Listen method of our Listener object, the request we made with the HTTP client will create the connection expected by the Listener.
Testing in Different Packages
Now, let's create an API using several different web application libraries common in the Go language and write tests for them.
Fiber
First, let's take a look at the Fiber library, which we often use in our projects and to which I contribute.
Writing a test with Fiber is quite simple because it contains a direct Test method.
Now let's look at a sample Fiber application and how it was tested:
Here we need to pay attention to the line app.Test(r, -1).
We're sending our HTTP request here so we can test it.
So how did this HTTP request go without calling app.Listen(":8080") which we normally do to run a Fiber application and make it listen on a port?
Let's take a look at what's behind the app.Test() method (you can access all the code here):
When we examine the code, we see that Fiber actually creates its own connection objects by conn := new(testConn) in order to test the application, as we explained in the net.Pipe() section.
It writes the HTTP request to this conn object and sends this conn object to the server with the app.server.ServeConn(conn) call.
So what happens when we say app.Listen(":8080") in Fiber app normally? Let's take a quick look at this:
Here we see that Fiber does the net.Listen() operation within itself and gives the listener object it created to the server by saying app.server.Serve(ln). The server listens to the connections from this listener object and passes it as a parameter to the serveConn() method.
In other words, it implements an approach that brings together the Listener and net.Conn objects that we explained at the beginning, allowing us to test it without physically listening to the port.
Fiber is an Express inspired web framework built on top of Fasthttp, the fastest HTTP engine for Go. Designed to ease things up for fast development with zero memory allocation and performance in mind.
⚠️ Attention
Fiber v3 is currently in beta and under active development. While it offers exciting new features, please note that it may not be stable for production use. We recommend sticking to the latest stable release (v2.x) for mission-critical applications. If you choose to use v3, be prepared for potential bugs and breaking changes. Always check the official documentation and release notes for updates and proceed with caution. Happy coding! 🚀
⚙️ Installation
Fiber requires Go version 1.22 or higher to run. If you need to install or upgrade Go, visit the official Go download page. To start setting up your project. Create a new directory for your project and navigate into…
We said that Fiber is using the Fasthttp package in the background. If we want, we can develop high-performance API applications directly using the Fasthttp package.
Now let's see how we can test an application we developed with Fasthttp.
In the code above, we created a simple server and defined a handler, and when a GET request was made to the /test endpoint, we returned the "OK" response.
Here, unlike the previous code, we see a definition like ln := fasthttputil.NewInmemoryListener().
Next, we define a http.Client and give the Dial method of the ln variable we created to the DailContext method of the Transport object.
So does this remind you of anything? Yes, we mentioned above that we can give the net.Listener object to our http.Client variable and test it in-memory.
Fasthttp does exactly this approach with a utility package in the background.
Those who are curious about the details of the code can read it here.
valyala
/
fasthttp
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
fasthttp 
Fast HTTP implementation for Go.
fasthttp might not be for you!
fasthttp was designed for some high performance edge cases. Unless your server/client needs to handle thousands of small to medium requests per second and needs a consistent low millisecond response time fasthttp might not be for you. For most cases net/http is much better as it's easier to use and can handle more cases. For most cases you won't even notice the performance difference.
General info and links
Currently fasthttp is successfully used by VertaMedia in a production serving up to 200K rps from more than 1.5M concurrent keep-alive connections per physical server.
TechEmpower Benchmark round 19 results
Switching from net/http to fasthttp
HTTP server performance comparison with net/http
In short, fasthttp server is…
Go HTTP
Now, let's write a test for an API that we developed with Go's own packages without using any external libraries.
Let's say we developed an API like above. So how do we test this?
We can write our tests by using the techniques we learned above here as well. Go language has a package called httptest which contains some utility methods. In this package, a method named NewUnstartedServer() returns us an unstarted test server object.
Let's take a look at the source code:
When we create a Server with the httptest.NewUnstartedServer() method, a Listener is created in the background that listens to a random port with the newLocalListener() internal method.
When we say start, it listens for connections from the listener in a goroutine with the s.goServe() method.
What comes to mind when we see Listener? As in Fasthttp, if we create an InMemoryListener and give it to our server, we can perform our test without listening a physical port.
In the above code, we run our server by actually listening to a physical port in the test named TestHttpServer. I put the time.Sleep to wait for the server to start. Using the srv.Listener.Addr().String() method, we can obtain the address (including the port randomly assigned by the OS) to which we need to request.
In the test called TestHttpServerInMemory, we performed the http server-client communication via an in-memory listener we created without listening to a physical port.
Here I used the fasthttputil.NewInMemoryListener method for convenience. If we want, we can do the same thing by creating a struct and implementing the Listener interface.
Let's take a quick look at it:
As we saw above, we can provide in-memory server-client communication by implementing the Listener interface ourselves and using the net.Pipe() method. The reason why I gave connection objects to our InMemListener struct over a channel is that the server calls the Accept() method in a loop and waits for a new connection. Thus, only if we give a connection to our Listener object, the Server side will receive a new connection.
So why isn't this in-memory server approach provided by default by Go?
In fact, there is a timely issue about it and there have been some people working on it, but the process has not been finalized yet.
net/http/httptest: optional faster test server
#14200
Look into making httptest.Server have a way to use in-memory networking (i.e. net.Pipe) instead of localhost TCP. Might be faster/cheaper and not hit ephemeral port issues under load?
Our Test Performance Improvements
Now let's see how we improved our own test performance using the methods I described above.
We developed our application with fasthttp and this application was acting as a basic reverse proxy. We also use some functionality provided by fasthttp for reverse proxy operation.
For our tests, we need a reverse proxy (our application) and a fake API to represent the server we are redirecting in the background.
We used httptest.NewUnstartedServer for this and we used fasthttptest.NewInmemoryListener.
Previously, we were actually listening on a port to run test servers in our tests, and this slowed down our testing process quite a bit. When I looked from our CI pipeline, our testing process was taking about 10 seconds.
As a result of our development, after using the in-memory listener approach, all our tests were completed in ~0.3 seconds.
In the image below, we can see all our tests and total uptime.
Conclusion
When we want to test an HTTP server, we can perform our test faster without listening to a physical PORT.
Our tests run much faster with this technique
We can simulate in-memory net operations using net.Pipe(), net.Listener,
net.ConnWe can use the
app.Test()method when testing our applications with fiber.We can create in-memory Listener using
fasthttputil.InMemoryListener()
I hope this was a fun and informative article for you. See you in the next post :)
You can support me on Github: Support mstrYoda on Github
Top comments (3)
Hi Emre,
Tried you example (net.http + fasthttputil.NewInmemoryListener), but got this:
httptest.Server blocked in Close after 5 seconds, waiting for connections:*fasthttputil.pipeConn 0xc00043b0d8 pipe in state active
Hi Gregory, can you share your code?
Unfortunately no, but I found this github.com/bufbuild/connect-go/blo... and it "just worked"