Testcontainers: Test applications with external dependencies

paul schmidt — Mon, 20 Feb 2023 19:27:43 +0000

Sometimes you want to test the integration in one of your applications (or microservices - lets call it Service A) with one of your other applications (That one we'll call Service B). Assuming that Service B is already packaged into a docker container, you can use the wonderful library testcontainers for that.

Quick testcontainers introduction

testcontainers is a java library, which can run (docker) containers in your tests, for example a MongoDB server. You can then test your code with a real MongoDB server instead of a mocked one like bongo, flapdoodle and so on. Embedded/Mocked test doubles can outdate quickly and not always behave like a real implementation, so having tests running against a real mongodb is one advantage. You always know, that your code behaves on production exactly like in your tests.

Besides the java library there are also other packages for languages like go, Rust, .Node.js, etc.

Problem: external dependencies

As long as the container you're running is just a standalone piece of software everything is fine. But what if that container you're testing against has also other, external dependencies, like a rabbitmq for example?

Start dependencies for external service

If Service B supports it, you could just start a rabbitmq instance during your integrations tests, configure Service B to use that instance through environment variables and do the testing.

But that gets very messy if Service B has multiple external dependencies. You end up starting a lot of dependencies that don't even belong to your actual service. Also it is not and should not be the responsibility of Service A to start dependencies of Service B.

So that's not a good solution to this problem...

Start service with mocks

In Service B we've abstracted away the external dependency by using an interface (because we're good developers). And we also have written some (unit) tests using some kind of fake implementation instead of the actual implementation:

package queue

// This is a very simplified abstraction, just for the purpose of this post
interface Channel {
    func publishMessage(*queue.Message) (err)
}

//...

// a very naive implementation
package mock

struct inMemoryQueueChannel {
    queue queue.Message[]
}

func NewInMemoryQueueChannel() *queue.Channel {
    return &inMemoryQueueChannel{}
}

func (c *inMemoryQueueChannel) publishMessage(*queue.Message) {
    // ...
}

We can then define a switch in the entry point (or whatever place you construct all of your dependencies) of Service B to control which implementation of the mentioned interface is used.
This switch can simply be an environment variable, like OPERATING_MODE=test_mode or a program argument, e.g. --no-external-depencies.

var queueChannel queue.Channel

if (os.getenv('OPERATING_MODE') == "test_mode") {
    queueChannel = mock.NewInMemoryQueueChannel()
} else {
    queueChannel = rabbitmq.New()
}

This now allows us to start the service without any external dependencies.
Perfect for our testcontainers scenario!

Passing switch to container in tests

In Service A you can now write a test using the testcontainers library:

func TestServiceBConnection(t *testing.T) {
    ctx := context.Background()
    req := testcontainers.ContainerRequest{
        Image:        "service-b-image:latest",
        ExposedPorts: []string{"8080/tcp"},
        WaitingFor:   wait.ForLog("API started, ready to accept connections..."),
        // either pass env var 
        ConfigModifier: func(config *container.Config) {
            config.Env = []string{"OPERATING_MODE=test_mode"}
        },
        // or use option/argument
        Cmd: []string{'app', '--no-external-dependencies'}
    }
    serviceB, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
        ContainerRequest: req,
        Started:          true,
    })

    if err != nil {
        t.Error(err)
    }

    defer func() {
        if err := redisC.Terminate(ctx); err != nil {
            t.Fatalf("failed to terminate container: %s", err.Error())
        }
    }()

    // Talk with the container, do some testing
    // ...
}

We can pass the switch as an environment variable or modify the CMD to run our service with the appropriate flag/option.

Now we can test Service A's integration with Service B without any external dependencies. :party:

Pro tip: Log when the application is ready

When your application is done initializing and ready to serve requests log some statement. We can later use this statement to determine the containers' ready state: