Locating failing test cases in table-driven tests in Go

#go #testing

In Go, table-driven testing is widely known as a good practice to follow.

Dividing behaviors and data is great, but with current testing in Go, locating a failing test case is not straightforward. Instead, we always get the line number of the failing test assertion, which isn't helpful.

Check out the example below:

package eg

import "testing"

func TestExample(t *testing.T) {
    testcases := []struct {
        name string
        a, b int
        sum  int
    }{
        {"1+1", 1, 1, 99},
        {"2+2", 2, 2, 4},
        {"4+4", 4, 4, 8},
        // [long lines of code...]
        {"1024+1024", 1024, 1024, -1},
  }
    for _, testcase := range testcases {
        t.Run(testcase.name, func(t *testing.T) {
            if got, expected := testcase.a+testcase.b, testcase.sum; got != expected {
                t.Errorf("expected %d, got %d", expected, got)
            }
        })
    }
}

When we run the test, we will get the following output:

--- FAIL: TestExample (0.00s)
    --- FAIL: TestExample/1+1 (0.00s)
        eg_test.go:100: expected 99, got 2
    --- FAIL: TestExample/1024+1024 (0.00s)
        eg_test.go:100: expected -1, got 2048 # <-- this is the line number of the assertion, not the test case

The same line number regardless of which test case failed. What we actually want is to see the line number where the test case itself failed. The test name barely tells us which test case failed, but it's not enough.
So, to solve this problem, I have crafted a small helper library: go-testutil/dataloc.

https://pkg.go.dev/github.com/motemen/go-testutil/dataloc

Usage

Using the above example, let's modify the code a bit:

@@ -1,6 +1,7 @@
 package eg

 import "testing"
+import "github.com/motemen/go-testutil/dataloc"

 func TestExample(t *testing.T) {
        testcases := []struct {
@@ -96,7 +97,7 @@
        for _, testcase := range testcases {
                t.Run(testcase.name, func(t *testing.T) {
                        if got, expected := testcase.a+testcase.b, testcase.sum; got != expected {
-                               t.Errorf("expected %d, got %d", expected, got)
+                               t.Errorf("expected %d, got %d, test case at %s", expected, got, dataloc.L(testcase.name))
                        }
                })
        }

...and we will get the following output, which shows the location of failing test case, not assertion.

--- FAIL: TestExample (0.00s)
    --- FAIL: TestExample/1+1 (0.00s)
        eg_test.go:100: expected 99, got 2, test case at eg_test.go:12
    --- FAIL: TestExample/1024+1024 (0.00s)
        eg_test.go:100: expected -1, got 2048, test case at eg_test.go:95

Under the hood

To achieve this magic, the library uses runtime.Caller to get the caller's file and line number, and then uses go/ast to parse the file and find the location of the corresponding test case.

Find the call of the form dataloc.L(testcase.name)
Find the for ... range testcases that is the origin of testcase
Find the definition of testcases`
Find the definition of the test case whose name is the runtime value of testcase.name ("1+1" etc.)
- So if the test case name is not statically determined, it cannot be identified.

Happy table-driven testing!