In the previous article we've built a tiny HTTP server that was saying "I AM GROOT" in huge bold letters.
But what was actually happening under the hood?
HTTP: HyperText Transfer Protocol
As the name suggests, it is a protocol to transfer text. I'll tell you more: it's a text-based protocol. Take a look at that:
HTTP/1.1 200 OK
Date: Thu, 06 Jan 2022 07:45:04 GMT
Content-Length: 39
Content-Type: text/html; charset=utf-8
<html><body><h1>I AM GROOT!</h1></html>
That is a HTTP response sent by the web server from the previous article. If you remember, we have only wrote the last part of it - the response body; the header was added by the net/http
package:
-
HTTP/1.1
is the name and version of the protocol -
200 OK
is the code for "OK", as you've probably guessed, it could've been404 NOT FOUND
or500 INTERNAL SERVER ERROR
๐ or any other HTTP Response status codes -
Date
,Content-Length
andContent-Type
as the response headers
But don't trust me on this one! Ask curl
instead!
Curl is a small command-line tool that's "transferring data with URLs since 1998". Chances are you already have it installed on your machine, so just give it a try in your terminal and, if it doesn't work, go and grab it from here:
curl --version
The commands that I use the most are curl <url>
, which just prints the response body to a terminal, and curl <url> -I
, which prints only the headers.
To see the raw HTTP response, similar to the one I showed you, start the server from the previous article and type in your terminal:
curl -D - http://localhost:8080
Congratulations, you can now read raw HTTP! You don't need to, you have a browser for it, but you can ๐
But there's the other side to it - if you can read responses, there must be a way to write requests, right?
Right! It looks very similar to the responses, why wouldn't it:
GET / HTTP/1.1
Host: localhost
User-Agent: curl/7.54.0
Accept: */*
This request doesn't have any body, that's why it ends with two empty lines or, to be precise, two CRLF: Carriage Return and Line Feed \r\n
sequences.
The first line starts with a HTTP Method GET
and a path /
and ends with a familiar protocol version.
And, similar to a response, request headers come next.
In this case we are intending to get a response from the root path, and therefore performing a GET /
request with no body, but we could also try to post something:
POST /some/other/path HTTP/1.1
Host: localhost
User-Agent: curl/7.54.0
Accept: */*
Content-Length: 6
Hello!
Want to try it yourself? Save the request to a file, e.g. raw.txt
and type this in the terminal:
nc localhost 8080 < raw.txt
If you are on Windows and this command is not recognised, install NetCat as suggested here.
You've probably noticed, that the web server we've built responds with the exact same thing, no matter what requests we are making. Let's fix that!
RESTful API
We would like to differentiate between request methods (GET
,POST
,PUT
,DELETE
etc) and between the paths (/
,/items
,/item/1
etc). And, a top of that, wouldn't it be nice for paths and verbs to make sense together? It's easy to guess what POST /image
and DELETE /user/1
do and predict other paths. Congratulations, we've re-invented REST API!
Now let's implement it. That's where we left off the last time:
//main.go
package main
import (
"log"
"net/http"
)
func main() {
address := "localhost:8080"
log.Printf("Server is listening on: http://%v", address)
http.HandleFunc("/", HomePage)
err := http.ListenAndServe(address, nil)
if err != nil {
log.Fatal(err)
}
}
func HomePage(w http.ResponseWriter, req *http.Request) {
w.Write(([]byte)(`<html><body><h1>I AM GROOT!</h1></html>`))
}
Our home page screams I AM GROOT
, so how about we add a way to POST /greeting
?
Add a new function after HomePage
:
func GreetingPage(w http.ResponseWriter, req *http.Request) {
name := "USER"
html := fmt.Sprintf(`<html><body><h1>HELLO, %v! I AM GROOT!</h1></html>`, name)
w.Write(([]byte)(html))
}
For now, we've hard-coded the name as USER
and added it to the page content using fmt.Sprintf
. We've done same as before with the log.Printf
, except this time we did not need to print the resulting string in the terminal, just to get the string itself.
VSCode should automatically add packages you are using to the list of imports, but do check that
fmt
was indeed added there.
Let's add one more handler to the main
function:
func main() {
address := "localhost:8080"
log.Printf("Server is listening on: http://%v", address)
http.HandleFunc("/", HomePage)
http.HandleFunc("/greeting", GreetingPage) // <- Add this line
err := http.ListenAndServe(address, nil)
if err != nil {
log.Fatal(err)
}
}
Now, if you navigate to http://localhost:8080/greeting, you'll see:
And, if you come back to http://localhost:8080/, the content will be the same as before.
Not bad! We've got ourselves a multi-page website ๐
Let's make the server say I AM GROOT?
in response to GET /greeting
and answer with HELLO, <name>! I AM GROOT!
(in extended groot language) every time someone sends a POST /greeting
with the name as a request body (in plain text).
If you've dared to try the bonus challenge from the previous article, you are already familiar with test-driven development. It's okay if you haven't, you will be now ๐
Go comes with a bunch of useful tools to check go vet
, format go fmt
and test go test
, to name a few. The latter is the test runner, as you've probably figured.
Create a file called main_test.go
by the side of main.go
:
package main
import (
"fmt"
"io/ioutil"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestGetGreeting(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/greeting", nil)
w := httptest.NewRecorder()
GreetingPage(w, req)
res := w.Result()
defer res.Body.Close()
got, err := ioutil.ReadAll(res.Body)
if err != nil {
t.Fatal(err)
}
expected := "<h1>I AM GROOT?</h1>"
if !strings.Contains(string(got), expected) {
t.Errorf("Expected to find %q in %q", expected, got)
}
}
func TestPostGreeting(t *testing.T) {
name := "YOUR NAME"
req := httptest.NewRequest(http.MethodPost, "/greeting", strings.NewReader(name))
w := httptest.NewRecorder()
GreetingPage(w, req)
res := w.Result()
defer res.Body.Close()
got, err := ioutil.ReadAll(res.Body)
if err != nil {
t.Fatal(err)
}
expected := fmt.Sprintf("HELLO, %v!", name)
if !strings.Contains(string(got), expected) {
t.Errorf("Expected to find %q in %q", expected, got)
}
}
Every function that starts with a Test
in a test file will be executed when we run go test .
:
--- FAIL: TestGetGreeting (0.00s)
main_test.go:24: Expected to find "<h1>I AM GROOT?</h1>" in "<html><body><h1>HELLO, USER! I AM GROOT!</h1></html>"
--- FAIL: TestPostGreeting (0.00s)
main_test.go:41: Expected to find "HELLO, YOUR NAME!" in "<html><body><h1>HELLO, USER! I AM GROOT!</h1></html>"
FAIL
FAIL web-server 0.010s
FAIL
As expected, both of the tests are failing and we'll make them pass in no time, but first, let me walk you though what you just wrote ๐ค
Both functions are very similar, in each one of them we are using net/http/httptest
package to mock a request to a server. The function GreetingPage
expects a certain type of arguments and, if we pass arguments that do not meet the requirements, the program won't compile.
So what are the requirements? In Go it's called an interface. An interface dictates that whatever your entity is it must have certain methods implemented. For example http.ResponseWriter
, comments aside, looks like this:
type ResponseWriter interface {
Header() Header
Write([]byte) (int, error)
WriteHeader(statusCode int)
}
And httptest.ResponseRecorder
is a struct that has these methods wired to store results in memory:
type ResponseRecorder struct {
Code int
HeaderMap http.Header
Body *bytes.Buffer
Flushed bool
result *http.Response
snapHeader http.Header
wroteHeader bool
}
func (rw *ResponseRecorder) Header() http.Header {
m := rw.HeaderMap
if m == nil {
m = make(http.Header)
rw.HeaderMap = m
}
return m
}
Once again I've stripped the comments for brevity, but I do recommend you to cmd+click
(Mac) or ctrl+click
in your code editor on the names of these methods and take a look at the source code for yourself.
So, we are creating couple of structures that has methods required for request and response arguments:
req := httptest.NewRequest(http.MethodGet, "/greeting", nil)
w := httptest.NewRecorder()
In the TestGetGreeting
we, erm, test GET /greeting
without the body, so we pass http.MethodGet
(it's a constant "GET"), path /greeting
and nil
for body.
In the TestPostGreeting
we change the method to http.MethodPost
(constant "POST") and pass a body:
name := "YOUR NAME"
req := httptest.NewRequest(http.MethodPost, "/greeting", strings.NewReader(name))
w := httptest.NewRecorder()
Requests, just like responses, can be quite long (a two hour Netflix movie in 4K would weight around 11.4 GB) and that wouldn't fit into one package. You see, while HTTP is just a text, it is still being carried between you and the server wrapped in small binary packages (up to 64k bytes tops).
On a lower lever, every time you type https://www.google.com/
in your browser here's what's going on:
- Your browser is looking up the IP address for
www.google.com
, you could mimic it with:
nslookup www.google.com
Server: 192.168.1.1
Address: 192.168.1.1#53
Non-authoritative answer:
Name: www.google.com
Address: 142.250.74.4
- Your browser initiates a
TCP
connection with that address - If the requested protocol is
HTTPS
, it would initiate a secure connection and everything that you send will be encrypted so that only the server could make sense of your request - With the open TPC connection, the client (browser) will start packing your
HTTP Request
and sending over to the server - The server will start sending response packages
- The client will unpack & decrypt the response
- And, when done, close the connection
With that in mind, you'll probably understand why you can't just pass a string to the httptest.NewRequest
and instead create a reader from strings
package, that allows reading byte slices instead:
strings.NewReader(name)
And these lines also start to make sense, right? We need to be able to tell when the whole request body is sent and wait till the whole response body is received to be able to do something with it:
GreetingPage(w, req)
res := w.Result()
defer res.Body.Close()
got, err := ioutil.ReadAll(res.Body)
The defer
keyword means "do it when you're done with the rest", or in technical terms, delays the operation till the surrounding function returns. I prefer to read it as "don't forget to", so, in this case, we use it to eventually close the request body:
defer res.Body.Close()
We could write this line anywhere after the res
variable is defined, it'll work just the same. Very handy when you deal with async operations!
Next, we are reading all the parts of the body with ReadAll
from package ioutil
:
got, err := ioutil.ReadAll(res.Body)
And if something went wrong, for example the connection got lost, we'll stop the test with:
if err != nil {
t.Fatal(err)
}
Where the t
is the argument of the test function:
func TestPostGreeting(t *testing.T) {
//...
}
See the *
in front of the type? Yup, it's a pointer again. In most cases Go allows you treat them as common variables, so we wouldn't guess if it's a pointer or not from just looking at the body of the function. We'll be using pointers over and over throughout the series, but if you want to dig deeper right now - take a look at the gobyexample.com/pointers.
And finally, once we are done with all the preparations, we check if the response we received matches the one we expected and fail the test if it doesn't:
expected := fmt.Sprintf("HELLO, %v!", name)
if !strings.Contains(string(got), expected) {
t.Errorf("Expected to find %q in %q", expected, got)
}
We could compare the exact values, but it's a good idea to check only for the part you are interested in, so that when you decide to add a <title>Groot Server</title>
the tests won't start failing.
Phew! You've made it here! Great job! Even if the tests are still failing...
They say "it's hard while learning, easier when fighting", same goes for the tests: writing the actual implementation is often easier than writing a test. But it doesn't mean you shouldn't! Quite the opposite, as your project grows more and more, you'll often change your code, yet the tests will stay the same - guarding your project from making the same mistakes over and over again.
I hope you'll adapt and preach TDD (test driven development) too ๐ Now let's make the tests pass!
Change the GreetingPage
to:
func GreetingPage(w http.ResponseWriter, req *http.Request) {
var html string
switch req.Method {
case http.MethodHead:
{
w.WriteHeader(http.StatusOK)
}
case http.MethodGet:
{
html = `<html><body><h1>I AM GROOT?</h1></html>`
}
case http.MethodPost:
{
name, err := ioutil.ReadAll(req.Body)
if err != nil {
log.Println("Error /greeting", err)
w.WriteHeader(http.StatusInternalServerError)
return
}
html = fmt.Sprintf(`<html><body><h1>HELLO, %s! I AM GROOT!</h1></html>`, name)
}
default:
{
w.Header().Set("Allow", "GET,POST,HEAD")
w.WriteHeader(http.StatusMethodNotAllowed)
}
}
w.Write(([]byte)(html))
}
And, voilร :
go test .
ok web-server 0.009s
We can be certain it works in the browser and curl too, as the packages we used are also well tested, but never hurts to check:
curl -X POST -d "Me" http://localhost:8080/greeting
Will yield:
<html><body><h1>HELLO, Me! I AM GROOT!</h1></html>
And the GET /greeting
, that the browser performs when you load a page will result in:
And if we try a different request, let's say a DELETE /greeting
:
curl -X DELETE http://localhost:8080/greeting -I
We'll get:
HTTP/1.1 405 Method Not Allowed
Allow: GET,POST,HEAD
Date: Thu, 06 Jan 2022 10:45:36 GMT
Content-Length: 0
By now, you've probably figured how it works, but, just in case, let's go over what we've just implemented.
First, we've replaced hard-coded name
with a data from the request:
name, err := ioutil.ReadAll(req.Body)
Then, we handled errors like a pro, by logging the error and responding with an 500 Internal Server Error code:
if err != nil {
log.Println("Error /greeting", err)
w.WriteHeader(http.StatusInternalServerError)
return
}
Then, because our name
variable is no longer a string, we changed the format to explicit string conversion %s
instead of %v
, which otherwise would write something like HELLO, [10, 20,...]
:
html = fmt.Sprintf(`<html><body><h1>HELLO, %s! I AM GROOT!</h1></html>`, name)
And we wrapped it all with a switch
, branching our responses based on the request method:
var html string
switch req.Method {
case http.MethodHead:{/**/}
case http.MethodGet:{/**/}
case http.MethodPost:{/**/}
default:{/**/}
}
w.Write(([]byte)(html))
We've handled HEAD
, GET
and POST
methods and the default
case handled all the other, stating that the method was not allowed and providing available options as a comma-separated list in Allow
header, like this:
HTTP/1.1 405 Method Not Allowed
Allow: GET,POST,HEAD
Date: Thu, 06 Jan 2022 11:02:14 GMT
Content-Length: 0
But we didn't set html
variable to anything, why did it not crash? Well, that's one of the things I like about Go - most variables are defined with empty values. In case of string it's a ""
and for a number it'd be a 0
. This works for struct
too:
package main
import (
"fmt"
)
type Counter struct {
count int
}
func main() {
var c Counter
fmt.Println(c.count)
c.count++
fmt.Println(c.count)
}
// Prints:
// 0
// 1
And you've already met nil
. That's a starting value for pointers, slices, maps, channels, interfaces and function variables. And therefore this won't work:
func main() {
var c *Counter
fmt.Println(c.count)
c.count++
fmt.Println(c.count)
}
// panic: runtime error: invalid memory address or nil pointer dereference
Unless you give the pointer something to point at:
func main() {
var c *Counter = &Counter{}
fmt.Println(c.count)
c.count++
fmt.Println(c.count)
}
As you've guessed, &
is used when you need to get the pointer from the variable: &Counter{}
creates a new counter and returns its memory address.
And we're done for today! Yay you! ๐ฅณ
Bonus Challenge
Ready for more? Nice!
Here's the challenge for today:
- Add one more endpoint
/google
- Make this endpoint redirect users to
https://www.google.com/
- Start it with a unit test
Till next time! And good luck!
Top comments (0)