Go Package for better integration tests: github.com/ory/dockertest

When writing tests for code interacting with datastores we usually face a dilemma:

• Should we mock the calls to the datastores? or
• Should we write integration tests using a real datastore?

To be clear, when I say interacting with datastores I mean where we are actually implementing the concrete repository talking directly to the datastore, not the application service type using that said repository.

Let's look at our options.

Please refer to the full example code for more details.

Mocking datastore calls

We have different ways to write our tests depending on what datastore we are using, for example if we are testing database calls that happen to be using database/sql then importing a package like github.com/DATA-DOG/go-sqlmock could work.

In cases where there's no de-facto package used for mocking calls, we could define our own interface type that happens to be defining the concrete calls we use in your code, for example if we are planning to mock memcached calls and github.com/bradfitz/gomemcache it's being used, then something like the following could work:

// MemcacheClient defines the methods required by our Memcached implementation.
type MemcacheClient interface {
    Get(string) (*memcache.Item, error)
    Set(*memcache.Item) error
}

// AdapterMemcached uses a mocked client for testing.
type AdapterMemcached struct {
    client MemcacheClient
}

Where AdapterMemcached could receive the real memcache.Client as well as a type mocking the methods we require, allowing us to successfully write tests like:

func TestAdapterMemcached(t *testing.T) {
    // TODO: Test sad/unhappy-paths cases

    mock := mockingtesting.FakeMemcacheClient{}
    mock.GetReturns(&memcache.Item{
        Value: func() []byte {
            var b bytes.Buffer
            _ = gob.NewEncoder(&b).Encode("value")
            return b.Bytes()
        }(),
    }, nil)

    c := mocking.NewAdapterMemcached(&mock)

    if err := c.Set("key", "value"); err != nil {
        t.Fatalf("expected no error, got %s", err)
    }

    value, err := c.Get("key")
    if err != nil {
        t.Fatalf("expected no error, got %s", err)
    }

    if value != "value" {
        t.Fatalf("expected `value`, got %s", value)
    }
}

Writing integration tests

Mocking datastore calls definitely works for testing purposes, it allows us to focus on our business logic more, however the trade-off is the lack of integration testing until the code is deployed to a live environment.

Before Docker, mocking datastores was the ideal solution but nowadays is really simple to setup a local container running concrete datastore versions without conflicting with the local environment, it's cheap to use a real datastore and to run those tests against them.

A simple solution is usually to run the containers in advance before executing the test suite, however thanks to github.com/ory/dockertest it's easy to actually run the containers before each one of the test cases.

ory/dockertest uses Docker to run and manage containers, the way it works is by using its API to interact behind the scenes to properly set up any container we need, in our case we will need memcached:1.6.9-alpine, so implementing a function like the following should work:

func newClient(tb testing.TB) *memcache.Client {
    pool, err := dockertest.NewPool("")
    if err != nil {
        tb.Fatalf("Could not instantiate docker pool: %s", err)
    }

    pool.MaxWait = 2 * time.Second

    // 1. Define configuration options for the container to run.

    resource, err := pool.RunWithOptions(&dockertest.RunOptions{
        Repository: "memcached",
        Tag:        "1.6.6-alpine",
    }, func(config *docker.HostConfig) {
        config.AutoRemove = true
        config.RestartPolicy = docker.RestartPolicy{
            Name: "no",
        }
    })

    if err != nil {
        tb.Fatalf("Could not run container: %s", err)
    }

    addr := fmt.Sprintf("%s:11211", resource.Container.NetworkSettings.IPAddress)
    if runtime.GOOS == "darwin" { // XXX: network layer is different on Mac
        addr = net.JoinHostPort(resource.GetBoundIP("11211/tcp"), resource.GetPort("11211/tcp"))
    }

    // 2. Wait until the container is available and instantiate the actual client
    //    the value set above in `pool.MaxWait` determines how long it should wait.

    if err := pool.Retry(func() error {
        var ss memcache.ServerList
        if err := ss.SetServers(addr); err != nil {
            return err
        }

        return memcache.NewFromSelector(&ss).Ping()
    }); err != nil {
        tb.Fatalf("Could not connect to memcached: %s", err)
    }

    tb.Cleanup(func() {
        // 3. Get rid of the containers previously launched.

        if err := pool.Purge(resource); err != nil {
            tb.Fatalf("Could not purge container: %v", err)
        }
    })

    return memcache.New(addr)
}

This will run a new container per test suite or test case (depending on how we plan to run our tests), with that initialization we will be able to actually run a memcached docker container for using it during our test, covering the type requiring a memcached.Client as well:

func TestConcreteMemcached(t *testing.T) {
    // TODO: Test sad/unhappy-paths cases

    client := newClient(t)

    c := mocking.NewConcreteMemcached(client)

    if err := c.Set("concrete-key", "value"); err != nil {
        t.Fatalf("expected no error, got %s", err)
    }

    value, err := c.Get("concrete-key")
    if err != nil {
        t.Fatalf("expected no error, got %s", err)
    }

    if value != "value" {
        t.Fatalf("expected `value`, got %s", value)
    }
}

Final thoughts

There's no excuse not to use github.com/ory/dockertest when working with datastores in Go. ory/dockertest simplifies integration tests and reduces the dependencies needed when running our tests because only Docker is required in those cases.

However we should be careful and not overuse it, we need to keep in mind that although we have the option to literally create one container per subtest that may not be the best idea in the end. We should measure the duration of our tests tests to determine what's the best number of containers our suite needs, my recommendation to start would be use one for each test case and depending on the isolation we need perhaps increase that value to match the number of subtests.

In the end ory/dockertest is a must, highly recommended.