Building Microservices in Go: REST APIs - Custom JSON Types

Disclaimer: This post includes Amazon affiliate links. If you click on one of them and you make a purchase I'll earn a commission. Please notice your final price is not affected at all by using those links.

---

Implementing Custom JSON types

Creating a nice User Experience for the customers of our REST API should be considered when implement our handlers because in the end they are the driving factor of the popularity and usage of what we created. In this post I cover how to build custom JSON types to enhance the way we define different types that make more sense when using JSON payloads in the requests and responses.

Specifically I'm talking about two types currently in our "To Do Microservice", the first one Priority:

// Priority indicates how important a Task is.
type Priority int8

And the second one Dates:

// Dates indicates a point in time where a task starts or completes, dates are not enforced on Tasks.
type Dates struct {
    Start time.Time
    Due   time.Time
}

Before that, let's talk about how we are receiving data from our clients and how we are sending data back to them.

Introducing encoding/json

The code used for this post is available on Github.

We implemented HTTP handlers in the first post of this series and decided to use JSON as the message format for receiving and sending data, we could have used any other format, like XML for example, but for our use case JSON made sense because it is well supported by multiple browsers and programming languages.

The way we generated those messages is by using the package encoding/json which is included in the standard library. This package defines a lot of interesting types for interacting with JSON, for example the following interface types:

• json.Marshaler: a type implementing this interface indicates it knows how to convert itself into a valid JSON, and
• json.Unmarshaler: a type implementing this interface indicates it knows how to create an instance of itself from a valid JSON value.

Besides those, there are another function types used for marshaling and unmarshaling:

• json.Marshal: is used to marshal types into a slice of bytes, if the type implements json.Marshaler then that one is used, and
• json.Unmarshal: is used to unmarshal a valid JSON, in the form of slice of bytes, into an instance of a Go type meant to represent that argument, if the type implements json.Unmarshaler then that method is used.

In both cases if the types don't implement the mentioned interface types then the default logic will be followed.

By putting all that knowledge in practice we could build something like this:

package main

import (
    "encoding/json"
    "fmt"
    "log"
)

type Number int

const (
    Unknown Number = iota
    One
    Two
)

func (n *Number) UnmarshalJSON(b []byte) error {
    var s string
    if err := json.Unmarshal(b, &s); err != nil {
        return err
    }

    switch s {
    case "one":
        *n = 1
    case "two":
        *n = 2
    default:
        *n = 0
    }

    return nil
}

func (n Number) MarshalJSON() ([]byte, error) {
    var s string

    switch n {
    default:
        s = "zero"
    case One:
        s = "one"
    case Two:
        s = "two"
    }

    return json.Marshal(s)
}

func main() {
    var n Number = 1

    b, _ := json.Marshal(n) // XXX: ignoring error
    fmt.Println(string(b)) // prints out: "one"

    //-

    _ = json.Unmarshal([]byte(`"two"`), &n) // XXX: ignoring error
    fmt.Println(n) // prints out "2"
}

The code above defines a type called Number that marshals itself into a valid JSON string, and unmarshals itself from a valid JSON string into the corresponding value of the Number type.

Receiving data from our clients

Our handlers use another type called json.Decoder for decoding (or unmarshaling) the received payload. For example if we use the create handler, its implementation looks like this:

func (t *TaskHandler) create(w http.ResponseWriter, r *http.Request) {
    var req CreateTasksRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        // ... some code ...
    }

    // ... some other code ...
}

What this json.NewDecoder(r.Body).Decode(&req) call is doing is:

1. Reading the body of the request: r.Body, then
2. Transforming the JSON, and finally
3. Assigning the transformed JSON values into the req variable of the type CreateTasksRequest.

Internally json.*Decoder.Decode() uses the logic defined in json.Unmarshal for doing that conversion, and depending if the fields are implementing json.Unmarshaler or not then that method would be used.

Sending data to our clients

Writing values using the received http.ResponseWriter in our handler is what allows our users to get data back. For example if we use the task handler, its implementation looks like this:

func (t *TaskHandler) task(w http.ResponseWriter, r *http.Request) {
    // ... some code meant to get a task from somewhere ...

    renderResponse(w,
        &ReadTasksResponse{
            Task: Task{
                ID:          task.ID,
                Description: task.Description,
                Priority:    NewPriority(task.Priority),
                Dates:       NewDates(task.Dates),
            },
        },
        http.StatusOK)
}

In this case the renderResponse function is the one writing the data:

func renderResponse(w http.ResponseWriter, res interface{}, status int) {
    w.Header().Set("Content-Type", "application/json")

    content, _ := json.Marshal(res)

    // ... some error validation ...

    _, err = w.Write(content) // XXX: explicitly ignoring error
}

Which then again happens to be using the json.Marshal function as well as any implementation of the json.Marshaler in the types defined as fields.

Defining custom JSON type

If we take a similar approach to what was implemented in the first example, we can define new types in the rest package equivalent to the domain types mentioned in the beginning, however those will need to implement both json.Unmarshaler and json.Marshaler to handle any custom logic.

For example for rest.Priority we could define it as:

// Priority indicates how important a Task is.
type Priority string

const (
    priorityNone   Priority = "none"
    priorityLow    Priority = "low"
    priorityMedium Priority = "medium"
    priorityHigh   Priority = "high"
)

// Validate ...
func (p Priority) Validate() error {
    switch p {
    case "none", "low", "medium", "high":
        return nil
    }

    return errors.New("unknown value")
}

// MarshalJSON ...
func (p Priority) MarshalJSON() ([]byte, error) {
    if err := p.Validate(); err != nil {
        return nil, fmt.Errorf("convert: %w", err)
    }

    b, err := json.Marshal(string(p))
    if err != nil {
        return nil, fmt.Errorf("json marshal: %w", err)
    }

    return b, nil
}

// UnmarshalJSON ...
func (p *Priority) UnmarshalJSON(b []byte) error {
    var s string
    if err := json.Unmarshal(b, &s); err != nil {
        return fmt.Errorf("json unmarshal: %w", err)
    }

    if err := Priority(s).Validate(); err != nil {
        return fmt.Errorf("convert: %w", err)
    }

    *p = Priority(s)

    return nil
}

The complete code include more details besides the Marshaling/Unmarshaling logic; similar steps could be taken for Dates as well.

---

Parting words

When building JSON-based APIs we should consider defining custom types to make our API easier to understand, like what we covered in this post, using humanized values like "none", "low", "medium" or "high" instead of constant integers improves the readability and usage of our APIs, it's an investment we can make and it will pay off as well sooner than later when defining documentation using OpenAPI 3 because those options could be use as a concrete list of supported enum values.

In the next post I will cover OpenAPI 3 giving you more details about how we can build upon all the decisions we made already and how those are starting to make more sense for a final REST API.

Recommended Reading

If you're looking to sink your teeth into more REST and Web Programming I recommend the following books:

• The Design of Web APIs (2019)
• Go Web Programming (2016)
• REST in Practice (2010)