#WebDevSampler challenge - My answers in Go

As a first time doing the #WebDevSampler challenge, I used Go, which is a popular backend language, as well as the language I have been coding in professionally for seven years. Ahead are my answers to the 11 exercises in the sampler, which are:

• (1) Get an HTTP server up and running, serving an endpoint that gives the HTTP response with a message like "hello world!".
  • Concept demonstrated: starting an HTTP server and processing an HTTP request.
• (2) Give that HTTP response as HTML, with Content-Type text/html
  • Concept demonstrated: editing the HTTP response using response headers
• (3) Adding another endpoint/route on your HTTP server, such as an /about.html page
  • Concept demonstrated: serving more than one HTTP endpoint
• (4) Serving an endpoint with an image or webpage in your file system
  • Concept demonstrated: serving content from a file system
• (5) Route to an endpoints using more complex route like /signup/my-name-is/:name, for example if I send a request to /signup/my-name-is/Andy I would get back "You're all signed up for the big convention Andy!"
  • Concept demonstrated: Parameterized routing
• (6) Write and run an automated test for your HTTP parameterized endpoint
  • Concept demonstrated: Automated testing with an HTTP endpoint in one of your language's testing CLI tools.
• (7) Escape HTML tags in your endpoint. For example, /signup/my-name-is/<i>Andy should be sanitized so you DON'T display your name in italics
  • Concept demonstrated: Basic input sanitization
• (8) Serialize an object/struct/class to some JSON and serve it on an endpoint with a Content-Type: application/json
  • Concept demonstrated: JSON serialization, which is done a lot creating backend APIs
• (9) Add a POST HTTP endpoint whose input is of Content-Type application/json, deserialize it to an object/struct/class, and then use some part of the object to produce some part of the HTTP response.
  • Concept demonstrated: JSON deserialization
• (10) Now have that POST endpoint save the content to some database (MongoDB, Postgres, Cassandra, any database you want)
  • Concept demonstrated: Database input
• (11) Now make a GET endpoint that retrieves a piece of data from the database
  • Concept demonstrated: Database retrieval

The tools I used for doing this challenge were:

• The Go standard library
• Go's built-in testing command for doing automated test coverage
• Gorilla Mux for parameterized HTTP routing
• SQLite and mattn's SQLite package for the database problems

Now, onward to the answers!

(1) Get an HTTP server up and running, serving an endpoint that gives the HTTP response with a message like "hello world!".

Go code

package main

import (
    // Go's standard library package for HTTP clients and servers
    "net/http"
)

func main() {
    // make an http.ServeMux, a Go standard library object that
    // routes HTTP requests to different endpoints
    rt := http.NewServeMux()

    // Make a catch-all endpoint for all requests going into the
    // server. When the endpoint is hit, we run the function passed
    // in to process the request.
    rt.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        // We have the ResponseWriter write the bytes of the string
        // "Hello world!"
        w.Write([]byte("Hello world!"))
    })

    // create a new server and run it with ListenAndServe to take
    // HTTP requests on port 1123
    s := http.Server{Addr: ":1123", Handler: rt}
    s.ListenAndServe()
}

Starting the program

1. Run go run main.go, or use go install and run the installed binary
2. Go to http://localhost:1123 in a browser or in a program like cURL. You should see the text "hello world!"

(2) Give that HTTP response as HTML, with Content-Type text/html

Go code

rt.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    // Add the header "Content-Type: text/html"
    w.Header().Set("Content-Type", "text/html")

    // Add some HTML <h1> tags to the hello world response. The
    // browser, seeing the response is Content-Type: text/html,
    // will display the response as a big header.
    w.Write([]byte("<h1>Hello world!</h1>"))
})

Starting the program

1. Compile and start the server again, and refresh http://localhost:1123
2. The response should now be displayed as a webpage

(3) Add another endpoint/route on your HTTP server, such as an /about.html page

Go code

// naming the route "/about" makes it so when a request is sent to
// http://localhost:1123/about, the about page is served instead of
// the hello world page
rt.HandleFunc("/about", func(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/html")

    w.Write([]byte(`
      <!DOCTYPE html>
      <html>
        <head>
          <title>About us</title>
        </head>
        <body>
          <h1>About us</h1>
          <p>We've got a website!</p>
        </body>
      </html>
    `))
})

Starting the program

1. Compile and start the server again
2. Go to http://localhost:1123/about. You should now see your about page.

(4) Serving an endpoint with an image or webpage in your file system

Preliminary steps

1. Make a directory inside the directory where main.go is, named "images"
2. Save a JPEG image in that images directory named "gopher.jpg"

Go code

// We are using Handle, not HandleFunc, because we're passing
// in an object of type http.Handler, not a function
rt.Handle(
    "/images/",
    // StripPrefix chops the prefix, in this case "/images/",
    // off of the HTTP request's path before passing the
    // request to the http.FileServer
    http.StripPrefix(
        "/images/",
        // create a FileServer handler that serves files in
        // your "images" directory
        http.FileServer(http.Dir("images")),
    ),
)

Starting the program

1. Compile and start the server again
2. Go to http://localhost:1123/images/gopher.jpg. You now should see your gopher.jpg file

(5) Route to an endpoints using more complex route like /signup/my-name-is/:name

Preliminary steps

For this, one, we'll use the Gorilla Mux library, since that is one of the most popular HTTP routing libraries in Go.

1. Set up a go.mod file with go mod init. go mod is Go's built-in package manager, and it is where your project's dependencies are listed, similar to package.json in Node.js.
2. Run go get github.com/gorilla/mux

Go code

First add Gorilla Mux to your imports

import (
    // fmt is a string formatting package in Go
    "fmt"
    "net/http"

    // Now we're importing the Gorilla Mux package in addition to
    // net/http
    "github.com/gorilla/mux"
)

Then at the start of main, replace the ServeMux with a Gorilla Mux Router and add your parameterized endpoint to it

func main() {
    // now instead of a ServeMux, we're using a Gorilla Mux router
    rt := mux.NewRouter()

    // Our new parameterized route
    rt.HandleFunc(
        // make a parameter in the request path that Gorilla
        // recognizes with the string "name"
        "/signup/my-name-is/{name}",
        func(w http.ResponseWriter, r *http.Request) {
            // the route parameters on the request are parsed
            // into a map[string]string
            name := mux.Vars(r)["name"]

            w.Header().Set("Content-Type", "text/html")
            w.Write([]byte(fmt.Sprintf(
                // use the route parameter in the HTTP response
                "<h1>You're all signed up for the big convention %s!</h1>", name,
            )))
        },
    )

    // Our existing endpoints mostly stay the same as before
}

Finally, update the format of the images directory endpoint to use PathPrefix, which is what you use for path prefixes in Gorilla Mux as opposed to Handle("/path/", handler)

rt.PathPrefix("/images/").Handler(
    http.StripPrefix(
        "/images/", http.FileServer(http.Dir("images")),
    ),
)

Starting the program

1. Compile and start the server again
2. Go to http://localhost:1123/signup/my-name-is/YOUR_NAME. You now should get an HTML response saying you're signed up

(6) Write an automated test for your HTTP parameterized endpoint

Preliminary steps

First, take the logic for setting up our Mux router and move it to its own function

func handleSignup(w http.ResponseWriter, r *http.Request) {
    name := mux.Vars(r)["name"]

    w.Header().Set("Content-Type", "text/html")
    w.Write([]byte(fmt.Sprintf(
        "<h1>You're all signed up for the big convention %s!</h1>", name,
    )))
}

// The router function is pretty much the same as the main
// function as of the last exercise, from when rt is declared
// to when we declared the last handler in the router.
func router() http.Handler {
    rt := mux.NewRouter()

    rt.HandleFunc("/signup/my-name-is/{name}", handleSignup)
    // not shown: The other endpoints' handlers

    // mux.Router, the type of rt, implements the http.Handler
    // interface, which is in charge of handling HTTP requests
    // and serving HTTP responses.
    return rt
}

Now replace the body of main with:

func main() {
    // create a new server and run it with ListenAndServe to take
    // HTTP requests on port 1123
    s := http.Server{Addr: ":1123", Handler: router()}
    s.ListenAndServe()
}

Then, make a new file named app_test.go

Go code (in app_test.go)

package main

import (
    // Standard library package for working with I/O
    "io"
    // Standard library testing utilities for Go web apps
    "net/http/httptest"
    // Standard library URL-parsing package
    "net/url"
    // Standard library testing package
    "testing"
)

// Functions whose name start with Test and then a capital
// letter and take in a testing.T object are run by the
// `go test` subcommand
func TestSignup(t *testing.T) {
    // A ResponseRecorder is the httptest implementation of
    // http.ResponseWriter, that lets us see the HTTP response
    // it wrote after running an HTTP handler function
    w := httptest.NewRecorder()

    // convert a string to a *url.URL object
    reqURL, err := url.Parse("http://localhost:1123/signup/my-name-is/Andy")
    if err != nil {
        // if parsing the URL fails, have the test fail
        t.Fatalf("error parsing URL: %v", err)
    }

    // set up our HTTP request, which will be to the endpoint
    r := &http.Request{URL: reqURL}

    // send the request to the HTTP server by passing it and the
    // ResponseWriter to mux.Router.ServeHTTP(). The result of
    // that HTTP call is stored in the ResponseRecorder.
    router().ServeHTTP(w, r)

    res := w.Result()
    // convert the response stored in res.Body to bytes and check
    // that we got back the response we expected.
    body, err := io.ReadAll(res.Body)
    if err != nil {
        t.Fatalf("error retrieving response body: %v", err)
    }
    bodyStr := string(body)
    expected := `<h1>You're all signed up for the big convention Andy!</h1>`
    if bodyStr != expected {
        t.Errorf("expected response %s, got %s", expected, bodyStr)
    }
}

Running the test

1. From the directory the Go files are in, run go test -v
2. Observe the test passing. If you change the text the sign-up endpoint serves, then the test should now fail.

(7) Escape HTML tags in your endpoint.

Go code (main.go)

First import the net/html package

import (
    // standard library package for working with HTML
    "html"
    "net/http"

    "github.com/gorilla/mux"
)

Then update handleSignup to call EscapeString. In a real production app, we'd be doing more advanced sanitization of user data than this and probably rendering our HTML using a templating library that has sanitization built-in in order to catch more edge cases with malicious input, but EscapeString handles sanitizing HTML characters as a very simple demonstration of input sanitization.

func handleSignup(w http.ResponseWriter, r *http.Request) {
    name := mux.Vars(r)["name"]
    // we use EscapeString to escape characters that are used in
    // HTML syntax. For example, the character < becomes &lt; and
    // > becomes &gt;
    name = html.EscapeString(name)

    // rest of the endpoint stays the same
    w.Header().Set("Content-Type", "text/html")
    w.Write([]byte(fmt.Sprintf(
        "<h1>You're all signed up for the big convention %s!</h1>", name,
    )))
}

Go test code (main_test.go)

func TestSignupHTMLEscape(t *testing.T) {
    w := httptest.NewRecorder()

    // convert a string to a *url.URL object, this time with
    // some HTML in it that should be escaped
    urlString := "http://localhost:1123/signup/my-name-is/<i>Andy"
    reqURL, err := url.Parse(urlString)
    if err != nil {
        // if parsing the URL fails, have the test fail
        t.Fatalf("error parsing URL: %v", err)
    }

    // run ServeHTTP just like before
    r := &http.Request{URL: reqURL}
    router().ServeHTTP(w, r)

    res := w.Result()
    body, err := io.ReadAll(res.Body)
    if err != nil {
        t.Fatalf("error retrieving response body: %v", err)
    }

    // Expect that we thwarted the HTML injection attempt
    bodyStr := string(body)
    expected := `<h1>You're all signed up for the big convention &lt;i&gt;Andy!</h1>`
    if bodyStr != expected {
        t.Errorf("expected response %s, got %s", expected, bodyStr)
    }
}

Running the test

1. From the directory the Go files are in, run go test -v
2. Observe the test passing. If you comment out the EscapeString call, then the test should now fail.

Running the server

1. Compile and start the server again
2. Go to http://localhost:1123/signup/my-name-is/<i>YOUR_NAME. You now should NOT get any italicized text since the <i> tag was escaped.

(8) Serialize an object/struct/class to some JSON and serve it on an endpoint with a Content-Type: application/json

Go code (near top of main.go)

First, import encoding/json, Go's standard library package for serializing objects to JSON

import (
    // Go's standard library for JSON serialization
    "encoding/json"
    "net/html"
    "net/http"
)

Then define this struct and HTTP endpoint

// Define a type with JSON serialization specified by JSON
// Go struct tags
type animalFact struct {
    AnimalName string `json:"animal_name"`
    AnimalFact string `json:"animal_fact"`
}

// add this function into the "router" function
func sendAnimalFact(w http.ResponseWriter, r *http.Request) {
    fact := animalFact{
        AnimalName: "Tree kangaroo",
        AnimalFact: "They look like teddy bears but have a long"+
                        " tail to keep their balance in trees!",
    }
    // Set the Content-Type for the response to application/json
    w.Header().Set("Content-Type", "application/json")

    // load the ResponseWriter into a JSON encoder, and then by
    // calling that Encoder's Encode method with a pointer to the
    // animalFact struct, the ResponseWriter will write the struct
    // as JSON.
    if err := json.NewEncoder(w).Encode(&fact); err != nil {
        // if serializing the response fails, then return a
        // 500 internal server error response with the error
        // message "error serializing response to JSON".
        // If the serialization succeeds though, we're all
        // set and the HTTP response is already sent.
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte(`{"error": "couldn't serialize to JSON"}`))
    }
}

Finally, add the animal fact to the router function

rt.HandleFunc("/animal-fact", sendAnimalFact)

Running the server

1. Compile and start the server again
2. Go to http://localhost:1123/animal-fact. You should get your animal fact in JSON.

(9) Add a POST HTTP endpoint whose input is of Content-Type application/json, deserialize it to an object/struct/class, and then use some part of the object to produce some part of the HTTP response.

Go code (near top of main.go)

First, define a signup struct, its JSON serialization using Go struct tags, and an endpoint to handle a JSON payload

import (
    // Go's standard library for JSON serialization
    "encoding/json"
    "net/html"
    "net/http"

    "github.com/gorilla/mux"
)

// Define a type with JSON serialization specified by JSON
// Go struct tags. For example, `json:"days_signed_up_for"`
// indicates that the DaysSignedUpFor field should be
// serialized as days_signed_up_for, not DaysSignedUpFor
type signup struct {
    Name            string `json:"name"`
    DaysSignedUpFor int    `json:"days_signed_up_for"`
}

// add this function into the "router" function
func handleJSONSignup(w http.ResponseWriter, r *http.Request) { 
    // load the request's Body into a JSON Decoder, and then by
    // calling that Decoder's Decode method with a pointer to the
    // signup struct,
    var s signup 
    if err := json.NewDecoder(r.Body).Decode(&s); err != nil {
        // if deserializing the response fails, then return a
        // 400 Bad Request header and the error message
        // "invalid JSON payload"
        w.WriteHeader(http.StatusBadRequest)
        w.Write([]byte("invalid JSON payload"))
        return
    }

    // use the signup in the response body
    name := html.EscapeString(s.Name)
    days := s.DaysSignedUpFor
    msg := fmt.Sprintf(
        "You're all signed up %s! Have a great %d days at the big convention!",
        name, days,
    )

    // NOTE: in a real production endpoint, if we're taking
    // in a JSON payload we'd probably send a JSON response
    // rather than plain text
    w.Write([]byte(msg))
}

Finally, in the router function, add our handleJSONSignup endpoint

rt.Methods(http.MethodPost).Path("/signup").HandlerFunc(handleJSONSignup)

Running the server

1. Compile and start the server again
2. Send a request to the POST signup endpoint with your HTTP client. For example, in cURL, that would look like: curl -XPOST http://localhost:1123/signup --data '{"name": "YOUR_NAME", "days_signed_up_for": 3}'. You should now see a response indicating how many days you're signed up for.

(10) Save the input of your POST request to a database

Preliminary steps to using my implementation

In the interest of simplicity, we will use SQLite as our database. If we were developing a big web app and planning on the site getting really popular with tons of people wanting data from the database at the same time, we might instead opt for a database like Postgres or MongoDB.

1. Install SQLite and add it to your computer's path
2. Open SQLite's command-line tool from the folder you've been coding the Sampler in. The command to do so is sqlite3.
3. Create your SQLite database with the command .open website.db.
4. Create your database table with the command CREATE TABLE signups (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT, days_signed_up_for INTEGER);

Now we have a database table to store sign-ups! Now for Go to be able to talk to SQLite, or any SQL database with the Go standard library's database/sql package, we need a database driver for the database we're using. We can get the one for SQLite using

go get github.com/mattn/go-sqlite3

Now we're ready to use the actual code.

Go code

First in main.go, import database/sql, Go's standard library package for working with SQL databases, and underscore-import go-sqlite3 so that database/sql registers the SQLite Go database driver. With that underscore import, your Go code is now able to talk to SQLite databases.

import (
    // Go's standard library for JSON serialization
    "database/sql"
    "encoding/json"
    "fmt"
    "html"
    "net/http"

    "github.com/gorilla/mux"
    // register the database driver for SQLite
    _ "github.com/mattn/go-sqlite3"
)

Then, define an object for interacting with our signups database table. This is so that the logic for interacting with the database is not intertwined with the logic for serving the endpoint, making the code easier to test and harder to get unexpected bugs with.

// signupsDB centralizes the logic for storing signups in
// SQLite.
type signupsDB struct{ db *sql.DB }

func newSignupsDB(filename string) (*signupsDB, error) {
    // Open a database/sql DB for the file path, using the
    // sqlite3 database driver.
    db, err := sql.Open("sqlite3", filename)
    if err != nil {
        return nil, err
    }
    return &signupsDB{db: db}, nil
}

// SQLite syntax to more or less say "insert the values of the
// two question mark parameters into the name and
// days_signed_up_for fields of a new item in the signups table
const insertSignupQuery = `
INSERT INTO signups (name, days_signed_up_for)
VALUES (?, ?)`

func (db *signupsDB) insert(s signup) error {
    // run DB.Exec to insert an item into the database
    result, err := db.db.Exec(
        insertSignupQuery, s.Name, s.DaysSignedUpFor,
    )
    if err != nil {
        return err
    }

    // check that only one item was inserted into the database
    // table
    rowsAffected, err := result.RowsAffected()
    if err != nil {
        return err
    } else if rowsAffected != 1 {
        return fmt.Errorf(
            "expected 1 row to be affected, but %d rows were.",
            rowsAffected,
        )
    }
    return nil
}

Now add an init function for initializing our database. Note that this isn't how we'd set up a database connectioin in a production app, but it's probably the simplest way to do this setup.

// in a real production Go web app, we would be structuring
// our HTTP handlers to be data structures instead of plain
// functions so we aren't relying on global variables, but
// for the sampler we'll just initialize our database in the
// init function and panic if that fails to keep things simple
var db *signupsDB

func init() {
    var err error
    if db, err = newSignupsDB("./website.db"); err != nil {
        panic(fmt.Sprintf(
            "error opening db: %v; can't start the web app", err,
        ))
    }
}

Then add db: struct tags for serializing your object in a database as well as in JSON.

type signup struct {
    Name            string `json:"name",db:"name"`
    DaysSignedUpFor int    `json:"days_signed_up_for",db:"days_signed_up_for"`
}

Finally, in the handleJSONSignup endpoint, add this if statement right before where you serve the HTTP response:

    if err := db.insert(s); err != nil {
        // in a real production web app, we'd look in more
        // detail at the error's value to decide the
        // appropriate status code and error message
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte("error inserting signup"))
        return
    }

Running the server

1. Compile and start the server again
2. Send a request to the POST signup endpoint with your HTTP client. For example, in cURL, that would look like: curl -XPOST http://localhost:1123/signup --data '{"name": "YOUR_NAME", "days_signed_up_for": 3}'. You should now see a response indicating how many days you're signed up for.
3. To see that you really put a sign-up in the database entity, open sqlite3 in the command line, open the database again with .open website.db, and finally, run SELECT * FROM signups. You should now see a single item in the database.

(11) Make a GET endpoint that retrieves a piece of data from the database

Go code

First, add a new method to the signupsDB type for retrieving signups by name

// SQLite syntax more or less saying "get the "name" and
// "days_signed_up_for" fields of AT MOST one item in the
// signups table whose name matches the question-mark
// parameter"
const getSignupQuery = `SELECT name, days_signed_up_for FROM signups WHERE name=? LIMIT 1`

func (db *signupsDB) getByName(name string) (*signup, error) {
    // retrieve a single item from the "signups" database
    // table. We get back a *sql.Row containing our result, or
    // lack thereof if the item we want is not in the database
    // table.
    row := db.db.QueryRow(getSignupQuery, name)

    // We deserialize the Row to the data type we want using
    // Row.Scan. If no database entity had been retrieved,
    // then we instead get back sql.ErrNoRows.
    var s signup
    if err := row.Scan(&s.Name, &s.DaysSignedUpFor); err != nil {
        return nil, err
    }
    return &s, nil
}

Then, make a new endpoint that uses getByName to query for signups

func handleGetSignupFromDB(w http.ResponseWriter, r *http.Request) {
    name := mux.Vars(r)["name"]

    w.Header().Set("Content-Type", "application/json")
    signup, err := db.getByName(name)
    switch err {
    case nil:
        // if there's no error, we have a signup, so carry on
        // with sending it as our JSON response
    case sql.ErrNoRows:
        // if we got ErrNoRows, then return a 404
        w.WriteHeader(http.StatusNotFound)
        w.Write([]byte(`{"error": "sign-up not found"}`))
        return
    default:
        // for any other kind of error, return a 500
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte(`{"error": "unexpected error"}`))
        return
    }

    if err := json.NewEncoder(w).Encode(signup); err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte(`{"error": "couldn't serialize to JSON"}`))
    }
}

Finally, add handleGetSignupFromDB to router

rt.HandleFunc("/signup/get/{name}", handleGetSignupFromDB)

Running the server

1. Compile and start the server again
2. Send a request to the http://localhost:1123/signup/get/YOUR_NAME. You should now see the JSON of your sign-up you did in the last step.
3. Send another request to the signup endpoint, this time with the name of someone that didn't sign up. You should now see the JSON of a sign-up not found error.

Comments

[Rizèl Scarlett]

This is interesting! What's the web dev sampler challenge?

[Andy Haskell]

Thanks! Link to it is below, it's a series I made of 11 web development exercises to get started learning backend in a new language, and along the way hopefully learn its ecosystem! So this post is my answers for how I'd do these exercises in Go and the link in this comment is the original questions

dev.to/andyhaskell/introducing-the...

[Rizèl Scarlett]

oh oops silly me. I should've clicked the link in the blog post at the top🤦🏾‍♀️

Thank you!

[Andy Haskell]

No problem, your comment actually reminded me to put that there so thanks for that!

[Casey]

great post, I’ll definitely come back to this as a reference! I recently stated using Go at work and definitely have some knowledge gaps to fill 😄