Go generics beyond the playground

While Go as of version 1.16 does not support Generics, there is an accepted language proposal for it, and we can probably expect it to arrive in Go 1.18 or later. But as it turns out, we don't need to wait for it to start experimenting with the new feature. We can start now to write small snippets of code on the go2go playground, or even full Go packages that you develop locally.

By far the easiest way to test out Go generics, is to use the playground. And there is nobody saying that the playground isn't awesome. However awesome though, there is clear limits to how much you can reasonably try out in the playground alone. What if you have enough code to start splitting into files? What if you want to write unit-tests? How would a full package look like with generics? In my view, the best way to try out a new feature, is to actually do something useful. And to do this with generics, we need to venture out of the safety and comfort of the playground.

My hope is that this will inspire you to do your own experiments with Go generics beyond the playground and write something potentially useful. Only then, can we truly see if generics itself, is going to be useful in Go.

In this article, I will go through how I re-wrote a test matching library from scratch with generics as part of the tool-box. My hope is that this will inspire you to do your own experiments with Go generics beyond the playground and write something potentially useful. Only then, can we truly see if generics itself, is going to be useful in Go. If you want, you could use the library in this article for testing your experiments; or you can extend the library and do a pull request.

Before we start: a warning. I will assume basic knowledge about the generics proposal in Go and what the most important design concepts are. If you don't have this knowledge, I would recommend that you first acquire it. E.g. by reading Go 2 generics in 5 minutes, by reading the updated design draft, or by doing your own experiments in the go2go playground first. In the meantime, keep this article on your TO-READ list. With that said, because we are trying to do something potentially useful, this article will be more about package design then about generics itself.

A problem to solve (again) with generics

Starting out with Go generics, in order to do something useful, we need a problem to solve. The problem I have picked for this article is one that I have tried to solve before when designing the test matcher/assertion library that we use to test the Clarify back-end at Searis. But first, you probably have a question: with all the great test matcher libraries we have in Go, why on earth would we want to write a new one? To answer that, it's worth having a closer look at at one of the existing matcher libraries. Does it solve it's mission in a useful way, and with the best possible package design?

In the Go world, by far the most popular Go matcher library still appear to be the assert package from the stretchr/testify repo. It's an old, good and stable library. However, because it's old, and because most (old) Go libraries strived to keep full backwards compatibility, it's also an excellent library to demonstrate some problems that can perhaps be solved by a different design. For this article, we will be content with considering the following code, testing an imaginary function mypkg.Sum:

func TestSum(t *testing.T) {
    a := mypkg.Vector{1,0,3}
    b := mypkg.Vector{0,1,-2}
    expect := mypkg.Vector{1,1,1}

    result, err := mypkg.Sum(a, b)

    assert.NoError(t, err, "unexpected error")
    assert.Equal(t, result, expect, "unexpected result")
}

At first glance, all of this might look fine. What's wrong with it, you might think. Before we get into that, there is one minor alteration I want to do to the code, and you would probably know this if you have ever used the assert library. Because the descriptive text in the assertion is optional, you would probably write them like this:

assert.NoError(t, err)
assert.Equal(t, result, expect)

Which may give the following output in the case of a failure:

--- FAIL: TestSum_assert (0.00s)
    /Users/smyrman/Code/blog/2021-03-generics-beyond-the-playground/mypkg/subtest_sum_test.go:35:
            Error Trace:    subtest_sum_test.go:35
            Error:          Not equal:
                            expected: mypkg.Vector{0, 1, -2}
                            actual  : mypkg.Vector{1, 1, 1}

                            Diff:
                            --- Expected
                            +++ Actual
                            @@ -1,5 +1,5 @@
                             (mypkg.Vector) (len=3) {
                            - (float64) 0,
                              (float64) 1,
                            - (float64) -2
                            + (float64) 1,
                            + (float64) 1
                             }
            Test:           TestSum_assert
FAIL
FAIL    github.com/smyrman/blog/2021-03-generics-beyond-the-playground/mypkg    0.130s
FAIL

So, to the problems. Problems in design are often subtle ones. The ones you can't quite put your finger on. Once you know about them though, they are hard not to se. In this short snippet of code and failure output, there is actually as much as six problems that I want to bring to attention:

1. We have to pass in the t parameter. It's a minor inconvenience, but enough for Dave Cheney to write a quite interesting package for solving it. Not recommended for production use, I might add.
2. The "got" and "want" parameter order is hard to get right. If you are observant, you might have noticed that the code above gets it wrong. This again makes the failure output tell a lie. In my own experience, such lies can lead to significant confusion when debugging a problem. As a rule of thumb if you use the assert package a lot, and tend to continue doing so, the "expected" parameter almost always come first. This is of course not a general rule in Go. E.g. the standard library tend to report the "got" parameter first, and the "want" parameter second.
3. From simply starring at the code (which is what code reviewers generally do) it's not obviously clear what's meant by Equal. It's well enough documented, but from the name alone, it's not clear if it's going to use the equal comparison (==), the reflect.DeepEqual method, or if it can handel equal comparison of two time.Time instances with a different time-zone. At some point, subtile details in how equality is implemented, might come back and bite us. Especially so if any of these details don't match the comparison we usually do in our programs.
4. The descriptive text for what went wrong is optional; and thus easily omitted. This can make debugging failing tests hard, as there is little information to exactly what went wrong (other than two values did not compare equal). Especially so if we have multiple assert statements. Is it the result from the function under test that's wrong? Is this just a sanity check before running the main test?
5. For what the output potentially lack of useful information, it makes up for in redundant information. Why do we need a diff for finding the difference between these simple struct instances? Given the diff is printed, why do we also get the "actual" and "expected" one-liners? Why is the filename and line trace information repeated? Why is the test name repeated?
6. Many of the library assertions, assert.Equal included, lack compile-time type safety. Of course, one might say, but if type-safety is beneficial in the rest of our programs, would it not also be beneficial in tests?

In our test-matcher for Clarify (called subtest), we have managed to to solve many of these problems. In order to explain how, let's start by rewriting the test function for mypkg.Sum:

func TestSum(t *testing.T) {
    a := mypkg.Vector{1, 0, 3}
    b := mypkg.Vector{0, 1, -2}
    expect := mypkg.Vector{1, 1, 1}

    result, err := mypkg.Sum(a, b)

    t.Run("Expect no error", subtest.Value(err).NoError())
    t.Run("Expect correct sum", subtest.Value(result).DeepEqual(expect))
}

Which in case of a failing test, may give the output:

--- FAIL: TestSum (0.00s)
    --- FAIL: TestSum/Expect_correct_sum (0.00s)
        /Users/smyrman/Code/blog/2021-03-generics-beyond-the-playground/mypkg/subtest_sum_test.go:46: not deep equal
            got: mypkg.Vector
                [0 1 -2]
            want: mypkg.Vector
                [1 1 1]
FAIL
FAIL    github.com/smyrman/blog/2021-03-generics-beyond-the-playground/mypkg    0.106s
FAIL

So, let's go through which problems we managed to solve by this, and which one we didn't.

First out, you might note that we do not pass in a t parameter. That is because what subtest does, is to return a test function. This test function can then be run as a Go sub-test, omitting the need for the user to pass in a t parameter. Cleaver? I certainly thought so when I first had the idea.

Next up we solved the ordering issue of "got" and "want" by wrapping them in each their method call. We can see that we first call Value on the result. This actually return what we call a ValueFunc or value initializer. It is not important now to explain why it's a function, but we can perhaps get back to that. The important part now is that this type has methods that let's us initialize tests. In this particular instance, we call the method DeepEqual with the expect parameter. Here we have also attempted to solve the clarity issue of what Equal does, by giving the method a more descriptive name.

Next up, as we pass the func(*testing.T) instance returned by DeepEqual to t.Run, this method require us to supply a name. Thus a description for the check is required. In fact, as we are using Go sub-tests, there is now even a way for us to re-run a particular check in the test for better focus during debugging. This can be particular useful for cases when there are many failing sub-tests and many failing checks in a test.

test -name '^TestSum/Expect_correct_sum'

Next up, we kept our test-output brief without repeating any information. We should note that there definitively is some cases where we would not get enough information from this brief output format. While the library do allow some output customization, this is one of those trade-offs that could probably be improved.

As for the final problem though, compile time type-safety, subtest falls short. We can still pass pretty much anything into the subtest.Value and various test initializer methods, and it won't detect any type mismatches for us before the test run. Besides, does it really make sense to have the same test initializer methods available for all Go types? Should not a subtest.Value(time.Time{}) provide methods that are useful for comparing times, such as analogies to the time.Time methods Equal, Before and After?

If we do the count, we gather that subtest appear to solve five out of the six problems we identified with the assert library. At this point though, it's important to note that at the time when the assert package was designed, the sub-test feature in Go did not yet exist. Therefore it would have been impossible for that library to embed it into it's design. This is also true for when Gomega and Ginko where designed. If these test frameworks where created now, then most likely some parts of their design would have been done differently. What I am trying to say is that with even the slightest change in the Go language and standard library, completely new ways of designing programs become possible. Especially for new packages without any legacy use-cases to consider. And this brings us to generics.

With generics added to our tool-box, can we manage to solve all six problems by starting over?

But why do we even need a matcher library?

Before we start of re-designing a tool, it might be worth while asking ourselves the question, what exactly is the tool trying to solve? And why would anyone ever want to use it? In this case, why would anyone need a matcher library in Go?

And to be fair, you often don't. Go famously don't include any assert functionality, because the Go developers believe that it's better to do checks in tests the same way you do checks in your normal programs. E.g. if you do if err != nil in your program, that's the syntax for doing the same check in yout tests. This way reading Go tests, becomes no different then reading any other go code, and you are less likely to do mistakes.

I believe that perhaps a common misconception could be that a matcher library is needed because checking results is hard. But this simply isn't true. At least not for simple checks. We can prove this by rewriting our example test to not use a matcher library at all:

func TestSum(t *testing.T) {
    a := mypkg.Vector{1, 0, 3}
    b := mypkg.Vector{0, 1, -2}
    expect := mypkg.Vector{1, 1, 1}

    result, err := mypkg.Sum(a, b)

    if err != nil {
        t.Errorf("Expected no error, got: %v", err)
    }
    if !reflect.DeepEqual(result, expect) {
        t.Errorf("got: %v, want: %v", result, expect)
    }
}

As we can see, the check part is quite straight-forward. So why then would we need a matcher library?

My own opinion is that the part of writing tests that might make you consider a matcher library, is the disciplinary challenge of letting the output on failures be both useful and consistent. Especially so, if you have a project with multiple developers. How much team and review discipline do you need to consistently order the "got" and "want" parameter, for instance? Again, if you are observant, you might have noticed that the code above have already failed that test. While the test above won't tell lies, consistent wording, ordering and reporting of test failures, can be important tools in streamlining the debugging process. And while some teams, like the Go team, are extremely good at getting this right, other teams might prefer to just use a library.

To finish up, perhaps we can agree that the right mission statement for a matcher library, might be more about providing consistent and useful information on failure than it is about making complicated checks easy; although it might do that too.

Exploring the original design

We will soon move on to designing, or rather re-designing, our test matcher library with generics. But to do that we must understand a little bit more about the core design of the subtest library.

So, remember this code?

t.Run("Expect correct sum", subtest.Value(result).DeepEqual(expect))

We mentioned the code generates a test, and we mentioned something about lack of type-safety, but we didn't give to much detail around exactly what we mean by this. Actually this code is just a short-hand syntax. As it happens, the long syntax for the same operation reveals the underlying design much better.

t.Run("Expect correct sum", subtest.Value(result).Test(subtest.DeepEqual(expect)))

Or to break the code up even further:

// Step 1: create a "value function" a.k.a. a value initializer:
valueFunc := subtest.Value(result)

// Step 2: create a check:
check := subtest.DeepEqual(expect)

// Step 3: Combine the value initializer and the check into a
// test function:
testFunc := valueFunc.Test(check)

// Step 4: Run the test function as a Go sub-test:
t.Run("Expect correct sum", testFunc)

Obviously, the check portion here can be replaced. E.g. instead of subtest.DeepEqual, we could use subtest.NumericEqual, subtest.Before or even a user defined check. This is powered by a combination of Go interfaces and first-class function support.

While the subtest package has several checks and some formatting helpers, the core design can actually be summarized in very few lines of code.

// The interface for a check.
type Check interface{
    Check(ValueFunc) error
}

// An adapter that lets a normal function implement a check.
type CheckFunc func(interface{}) error

func (f checkFunc) Check(vf ValueFunc) error {
    v, err := vf()
    if err != nil {
        return fmt.Errorf("failed to initialize value: %w", err)
    }
    return f(V)
}

// A value initializer function.
type ValueFunc func() (interface{}, error)

// An adapter to convert a plain value to a value initializer
// function.
func Value(v interface{}) ValueFunc {
    return func() (interface{}, error) {
        return v, nil
    }
}

// A method that constructs and returns a Go (sub-)test function
// from the combination of a value initializer and a check.
func (vf) Test(c Check) func(*testing.T) {
    return func(t *testing.T) {
        if err := c.Check(vf); err != nil {
            t.Fatal(err.Error())
        }
    }
}

So comes the problem to solve with generics. Because what's the point of passing in a NumericEqual check (expecting some numeric value) or a Before check (expecting a time.Time) to a value initializer returning a mypkg.Vector type? Can we, by use of generics, make this fail compilation?

The generic re-design

Finally, the re-design. With generics, or type parameterization, which the proposed Go generics is more accurately called, we can enforce type-safety for the check and value initializer combination into tests. To demonstrate how, here is a re-write of the subtest core-design.

// The check function type.
type CheckFunc[T any]func(func() T) error

// An adapter for converting a value to a value initializer
// function.
func Value[T any](v T) func() T {
    return func() T{
        return v
    }
}

// A function for combining a compatible value initializer and a
// check function into a Go (sub-)test.
func Test[T any](vf func() T, cf CheckFunc[T]) func(t *testing.T) {
    return func(t *testing.T) {
        if err := c.Check(vf); err != nil {
            t.Fatal(err.Error())
        }
    }
}

Each individual check implementation, or check initializer implementation to be exact, can be declared with various degrees of type parameterization.

// A type parameterized check initializer that can be initialized
// for any type.
func DeepEqual[T any](w T) CheckFunc[T] {...}

// A type parameterized check initializer that can only be
// initialized with a comparable type.
func CompareEqual[T comparable](v T) CheckFunc[T] {...}

// A type parameterized check without a compare value.
func ReflectNil[T any]() CheckFunc[T]

// A check initializer that is not type parameterized.
func TimeBefore(t time.Time) CheckFunc[time.Time] {...}

The trick here is that the type used to initialize the check and the type returned by the value initializer needs to match. If we try to combine a subx.TimeBefore check with a mypkg.Vector value initializer, compilation would fail. If we try to combine a mypkg.CompareEqual with the same initializer, it will work if mypkg.Vector is implemented as a comparable type. E.g. if mypkg.Vector is implemented as a struct with three fields, that are all comparable, then it will work.

type Vector struct{
    X, Y, Z float64
}

If however it is implemented as a slice, then using CompareEqual would result in a compile time error.

type Vector []float64

For our rewrite, let's assume the slice implementation. In order to explain what's happening in the rewritten code, we will first show the code without any type inference.

func TestSum(t *testing.T) {
    a := Vector{1, 0, 3}
    b := Vector{0, 1, -2}
    expect := Vector{1, 1, 1}

    result, err := Sum(a, b)

    t.Run("Expect no error", subx.Test(
        subx.Value[error](err),
        subx.CompareEqual[error](nil),
    ))
    t.Run("Expect correct sum", subx.Test(
        subx.Value[Vector](result),
        subx.DeepEqual[Vector](expect),
    ))
}

Which might give the following output on failure:

-------- FAIL: TestSum (0.00s)
    --- FAIL: TestSum/Expect_correct_sum (0.00s)
        sum_test.go2:20: comparison failed:
            got: (mypkg.Vector)
                [2 2 2]
            want deep equal to: (mypkg.Vector)
                [1 1 1]
FAIL
exit status 1
FAIL    github.com/smyrman/subx/examples/vector_sum 0.299s

Looking at this code, you might argue that the subx.Test function has reintroduced the ordering issue of the "got" and "want" parameters. However, it has not. this is because ordering this parameters wrong would lead to a compile-time error. We mentioned that using CompareEqual would fail compilation for the slice implementation of mypkg.Vector. Actually, even subx.DeepEqual[*mypkg.Vector] will cause a compilation error. This type-safety can be a useful tool to prevent simple programming mistakes.

If you read the design proposal, you would know all about type inference, and when it can and cannot be used; or perhaps you discovered how through some trial and error in the playground. With the current type-inference implemented in the go2go tool, the [T] syntax can be omitted everywhere in our example except for when comparing an error to nil. This again makes the code more readable.

func TestSum(t *testing.T) {
    a := Vector{1, 0, 3}
    b := Vector{0, 1, -2}
    expect := Vector{1, 1, 1}

    result, err := Sum(a, b)

    t.Run("Expect no error", subx.Test(
        subx.Value(err),
        subx.CompareEqual[error](nil),
    ))
    t.Run("Expect correct sum", subx.Test(
        subx.Value(result),
        subx.DeepEqual(expect),
    ))
}

PS! Note that due to a limitation with the go2go experimental tool, tests had to be written as internal test, unlike the subtest example where we declared them as external tests. This means, instead of declaring the package as mypkg_test and do an import of .../mypkg, which would better demonstrate real usage of the package, we have to declare the package as mypkg and omit the import.

Some cool things we can do with subx

While not part of the core design, we define syntactic sugar that allows different short-hand methods to be placed on different value-initializer functions similar to subtest.

// Long syntax:
result := mypkg.Sum(2, 3)
t.Run("Expect correct sum", subtest.Test(
    subtest.Value(result),
    subtest.CompareEqual(5),
))

// Short-hand syntax:
result := mypkg.Sum(2, 3)
t.Run("Expect correct sum", subtest.Number(result).Equal(5))

If we have a function that isn't reliable, we can repeat a check.

vf := func() int {
    return mypkg.Sum(2, 2, 1)
}

t.Run("Expect stabler results", subx.Test(vf,
    subx.AllOf(subx.Repeat(1000, subx.CompareEqual(5))...),
))

If we want to run a check outside of a test, we can do that as well.

result := mypkg.Sum(2, 3)
cf := subx.CompareEqual(5)
fmt.Println("CHECK sum error:", cf(subx.Value(result)))

Challenge: Go beyond the playground

Now I have told you how I went beyond the playground to re-design a package with Go generics. Now, it's your turn!

I hereby challenge you to find something useful to do with generics. Re-design a package. Write an ORM with generics. What ever you find that you want to do, following the README instructions of subx should be enough to get you started.

Fair warning: As of the time of writing, writing anything with generics is a bit of a time-travel in terms of editor support. We are now quite spoiled by the features of gopls, gofmt, goimports and more. Not to mention, syntax highlighting. At the time of writing, none of this works for Go generics; at least not out of the box. You have to remove your own blank lines, insert your own imports, and align your own struct tags. Now that you are warned; Godspeed!