DEV Community

loading...

Test-Driven Development: To mock or not to mock

_toul_ profile image Toul ・9 min read

A reader asked me in my previous post Test-Driven Development: You don't need to know it all how to test functions that interact with databases.

In general, the question being asked is the testing of dependencies that arise in writing code, and there are two main approaches.

First, mocking, which is to assume the dependency that is being tested will just work, so you create a mock object in your tests to simulate the responses to be as close to the real thing as possible.

Second, is to actually test the dependency by creating a real instance of whatever the dependency is.

I prefer the second when possible because mocks don't capture reality, only an idealized version, so there can be times when your tests pass, but the code breaks.

And sometimes when the tests pass and the code breaks it happens to be in production, which leads to an incident and that is not fun.

Enough talking let us take a look at some examples to understand mocking and dependency creation testing styles.

When to Mock: Example with AWS SDK for GO

Suppose, you've written some GO code that is utilizing the AWS SDK and you want to test it. Because there is no way to recreate the AWS API locally, there aren't many other options besides mocking or making actual API calls to the AWS services.

Although, it may be economically possible, in the beginning of a project to eat the costs of calling the AWS API directly, it is not a scalable practice.

Because there's a point where it is simply too expensive to have call the API directly both during TDD or via executing the tests in the pipeline.

Hence, mocking is a path forward for testing in this scenario.

Sample Code

Let's say our GO code's responsibility is to interact with S3 by putting a file into a bucket and looks like something like this:


package S3

import (
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
    "github.com/aws/aws-sdk-go/service/s3/s3iface"
    "log"
    "strings"
)

// Every service within the S3 sdk has in interface object with example lambdaIface, dynamodbIface, and so on
type Deps struct {
    s3     s3iface.S3API
    bucket string
    key    string
    body   string
}

// PutReport puts a report into an S3 bucket
func (d *Deps) PutReport(data []byte, bucketName string, appName string) error {
    body := string(data)
    if d.s3 == nil {
    // start a sessions with AWS 
        sess := session.Must(session.NewSessionWithOptions(session.Options{
            SharedConfigState: session.SharedConfigEnable,
            Config: aws.Config{
                Region: aws.String("us-east-1"),
            },
        }))
        d.s3 = s3.New(sess)
        d.bucket = bucketName
        d.key = appName
    }
    // Build the Input struct
    requestInput := &s3.PutObjectInput{
        Bucket: aws.String(d.bucket),
        Key:    aws.String(d.key),
        Body:   strings.NewReader(body),
    }
    // Attempt to PUT the object.
    _, err := d.s3.PutObject(requestInput)
    if err != nil {
        return err
    }
    return nil
}
Enter fullscreen mode Exit fullscreen mode


then to test it the code might look something like the following:


package S3

import (
    "testing"

    "github.com/aws/aws-sdk-go/service/s3"
    "github.com/aws/aws-sdk-go/service/s3/s3iface"
)

// Create the Mock
type mockedPutObject struct {
    s3iface.S3API
    Response s3.PutObjectOutput
}

// Mock the PutObject call
func (d mockedPutObject) PutObject(in *s3.PutObjectInput) (*s3.PutObjectOutput, error) {
    return &d.Response, nil
}

// Create the Test 
func TestPutReport(t *testing.T) {
    t.Parallel()
    t.Run("Happy Path Test PutObject to bucket", func(t *testing.T) {
        // Instantiate and return an empty output object
        m := mockedPutObject{
            Response: s3.PutObjectOutput{},
        }
        // Instantiate dependency
        d := Deps{
            s3: m,
        }
        // Injecting the dependencies with the mock
        err := d.PutReport([]byte(`{"HappyPathTest": true}`), "bucketExists", "appExists")
        // Because it is mocked it should always work unless the aws sdk for go has changed.
        if err != nil {
            t.Fatal("It should work")
        }
    })
}
Enter fullscreen mode Exit fullscreen mode


Luckily, AWS SDK for GO provides interfaces for each of their services so mocking them for testing is easy.

But, as you can see, the only that has been done is too invoke the mocked method, which is not the same as calling the real method.

To be fair more time could be invested in building out a more robust Mock like the Python based Moto, which is a sophisticated mocking framework that creates robust mocks for tests.

However, then there is the additional burden of maintaining the mocks, which isn't advancing the code base any, and the Mocks are bound to have bugs as well as the main codebase.

So, in other words Mocks add more work and provide only a little bit more peace of mind.

When not to Mock: Create Real Instance of Dependency

Now, let's consider the reader's original question of how to test a Database. Although, it could be mocked and some readers might argue that it should.

I am going to suggest creating the database during the execution of the tests via Docker containers.

However, I caution to only consider doing so when (1) the project can afford longer build times (1-x amount of minutes potentially added, dependent on the test(s)) and (2) there is an easy way to do so.

But, the peace of mind of knowing that the code is actually being tested against a real database is well worth it in my opinion for the 1-x amount of minutes added to the pipeline.

Again, my reasoning for using real instances over mocks is that mocks are only a simulation of the instance and therefore can never be as good as the real instance.

Put another way, although driving a digital car with steering and pedals can simulate driving it can never (at this point in time) simulate all variables of actually driving

With that in mind let's create the test for a database.

Sample Code

Suppose, we are building an application that is to take some EC2 instance metadata and store it into a database for later use by the security or compliance team.

Then the code might look like this:

1. Define the Data Structure

We'll only store a small piece of the EC2 metadata available. So, create an Infrastructure type with ImageId, InstanceId, InstanceType, LaunchTime, MonitoringState, and PlacementZone fields.

Next, create a few sample data points (can call the AWS API through the AWS CLI to get the data or view the AWS console)


// Infrastructure Represents a piece of infrastructure
type Infrastructure struct {
    ImageId            string `gorm:"type:varchar" json:"imageid"`
    InstanceId         string `gorm:"type:varchar" json:"instanceid"`
    InstanceType       string `gorm:"type:varchar" json:"instancetype"`
    LaunchTime         string `gorm:"type:varchar" json:"launchtime"`
    MonitoringState    string `gorm:"type:varchar" json:"monitoringstate"`
    PlacementZone      string `gorm:"type:varchar" json:"placementzone"`
}


var pool *dockertest.Pool

var sampleInfra = []Infrastructure{
    {
        ImageId:            "ami-095850e299e2e009d",
        InstanceId:         "i-001d4990ad1f50796",
        LaunchTime:         "2021-01-26T02:30:22+00:00",
        MonitoringState:    "disabled",
        PlacementZone:      "us-west-2a",
    },
    {
        ImageId:            "ami-095850e299e2e009d",
        InstanceId:         "i-001d4990ad1f50796",
        LaunchTime:         "2021-01-26T02:30:22+00:00",
        MonitoringState:    "disabled",
        PlacementZone:      "us-west-2a",
    },
}
Enter fullscreen mode Exit fullscreen mode

2. Set up Test with Docker Containers

Then, initialize the dockertest library to create a new docker container pool for use.

Note: If testing locally on your computer it will require you to be signed into dockerhub as it will be making calls to it to get the image(s) if they aren't all ready on your local computer.

This also means that it will do the same within your pipeline so make sure that you have the login credentials saved as environment variables and have a login to docker step.


var pool *dockertest.Pool

func TestMain(m *testing.M) {
    // Create a new pool for docker containers
    var err error
    pool, err = dockertest.NewPool("")
    if err != nil {
        log.Fatalf("Could not connect to docker: %s", err)
        os.Exit(1)
    }

    log.Println("Initialize test database...")

    // Run the actual test cases (functions that start with Test...)
    code := m.Run()
    os.Exit(code)
}
Enter fullscreen mode Exit fullscreen mode

3. Create a database and return a connection to the database to be used

To make it easier to write multiple tests that use the Database, we'll create a function that creates a database and returns a database connection.

Note, that this implies that for the testing of the functions that use the database that there will be a database created each time within a docker container.

It is important to keep the test databases separate in my opinion to avoid the scenarios of functions executing out of order (t.Parallel()) and the database being in the wrong state for the test.


func db(t *testing.T) *gorm.DB {

    var db *gorm.DB

    // Pull an image, create a container based on it and set all necessary parameters
    opts := dockertest.RunOptions{
        Repository:   "postgres",
        Tag:          "12-alpine",
        Env:          []string{"POSTGRES_PASSWORD=secret", "POSTGRES_DB=testdb"},
        ExposedPorts: []string{"5432"},
        PortBindings: map[docker.Port][]docker.PortBinding{
            "5432": {
                {HostIP: "0.0.0.0", HostPort: "5433"},
            },
        },
    }

    // Run the dockercontainer
    resource, err := pool.RunWithOptions(&opts,
        func(config *docker.HostConfig) {
            config.AutoRemove = true
            config.RestartPolicy = docker.RestartPolicy{
                Name: "no",
            }
        })
    if err != nil {
        log.Fatalf("Could not start resource: %s", err)
    }

    // Exponential retry to connect to database while it is booting
    if err := pool.Retry(func() error {
        databaseConnStr := fmt.Sprintf("host=localhost port=5433 user=postgres dbname=testdb password=secret sslmode=disable")
        db, err = gorm.Open("postgres", databaseConnStr)
        if err != nil {
            log.Println("Database not ready yet (it is booting up, wait for a few tries)...")
            return err
        }

        // Tests if database is reachable
        return db.DB().Ping()
    }); err != nil {
        log.Fatalf("Could not connect to docker: %s", err)
    }

    t.Cleanup(func() {
        err := pool.Purge(resource)
        if err != nil {
            t.Logf("Could not purge resource: %s", err)
        }
    })
    return db
}
Enter fullscreen mode Exit fullscreen mode

4. Test the DB

Now, we can create the test for the database.

Notice, calling the db.Save() method is the same as calling the Gorm function, so we are actually interacting with a real database in this test. At a glance, it may seem like a lot of work and not very useful for such a simple test.

However, it opens the door to more complex business logic tests that involve database interaction. Additionally, it allows for a way to test the database itself.

Imagine, you wanted to test the performance of MySQL vs PostGres, now all that would be required is to swap out the databases in the db function or split it into two functions gormDB and postgresDB (what I'd do).

If mocking were instead used then it would mean the creation of a whole new set of Mocked objects (more work on top of testing / writing business code). And again, the Mocks still wouldn't be able to supply useful information, such as performance.


func TestSeedDatabase(t *testing.T) {
    t.Parallel()
    t.Run("Seeding database", func(t *testing.T) {
        t.Parallel()
        db := db(t)
        db.AutoMigrate(&Infrastructure{})
        db.Save(&sampleInfra[0])
        db.Save(&sampleInfra[1])
    })
}
Enter fullscreen mode Exit fullscreen mode

5. Altogether

I'm providing the full code in case there are any problems with following along.


package DatabaseTestingWithDocker_test
import (
    "fmt"
    "github.com/jinzhu/gorm"
    _ "github.com/lib/pq" // here
    "github.com/ory/dockertest"
    "github.com/ory/dockertest/docker"
    "log"
    "os"
    "testing"
)

// Infrastructure Represents a piece of infrastructure
type Infrastructure struct {
    ImageId            string `gorm:"type:varchar" json:"imageid"`
    InstanceId         string `gorm:"type:varchar" json:"instanceid"`
    InstanceType       string `gorm:"type:varchar" json:"instancetype"`
    LaunchTime         string `gorm:"type:varchar" json:"launchtime"`
    MonitoringState    string `gorm:"type:varchar" json:"monitoringstate"`
    PlacementZone      string `gorm:"type:varchar" json:"placementzone"`
}


var pool *dockertest.Pool

var sampleInfra = []Infrastructure{
    {
        ImageId:            "ami-095850e299e2e009d",
        InstanceId:         "i-001d4990ad1f50796",
        LaunchTime:         "2021-01-26T02:30:22+00:00",
        MonitoringState:    "disabled",
        PlacementZone:      "us-west-2a",
    },
    {
        ImageId:            "ami-095850e299e2e009d",
        InstanceId:         "i-001d4990ad1f50796",
        LaunchTime:         "2021-01-26T02:30:22+00:00",
        MonitoringState:    "disabled",
        PlacementZone:      "us-west-2a",
    },
}

func TestMain(m *testing.M) {
    // Create a new pool for docker containers
    var err error
    pool, err = dockertest.NewPool("")
    if err != nil {
        log.Fatalf("Could not connect to docker: %s", err)
        os.Exit(1)
    }

    log.Println("Initialize test database...")

    // Run the actual test cases (functions that start with Test...)
    code := m.Run()
    os.Exit(code)
}

func TestSeedDatabase(t *testing.T) {
    t.Run("Seeding database", func(t *testing.T) {
        t.Parallel()
        db := db(t)
        db.AutoMigrate(&Infrastructure{})
        db.Save(&sampleInfra[0])
        db.Save(&sampleInfra[1])
    })

}


func db(t *testing.T) *gorm.DB {

    var db *gorm.DB

    // Pull an image, create a container based on it and set all necessary parameters
    opts := dockertest.RunOptions{
        Repository:   "postgres",
        Tag:          "12-alpine",
        Env:          []string{"POSTGRES_PASSWORD=secret", "POSTGRES_DB=testdb"},
        ExposedPorts: []string{"5432"},
        PortBindings: map[docker.Port][]docker.PortBinding{
            "5432": {
                {HostIP: "0.0.0.0", HostPort: "5433"},
            },
        },
    }

    // Run the dockercontainer
    resource, err := pool.RunWithOptions(&opts,
        func(config *docker.HostConfig) {
            config.AutoRemove = true
            config.RestartPolicy = docker.RestartPolicy{
                Name: "no",
            }
        })
    if err != nil {
        log.Fatalf("Could not start resource: %s", err)
    }

    // Exponential retry to connect to database while it is booting
    if err := pool.Retry(func() error {
        databaseConnStr := fmt.Sprintf("host=localhost port=5433 user=postgres dbname=testdb password=secret sslmode=disable")
        db, err = gorm.Open("postgres", databaseConnStr)
        if err != nil {
            log.Println("Database not ready yet (it is booting up, wait for a few tries)...")
            return err
        }

        // Tests if database is reachable
        return db.DB().Ping()
    }); err != nil {
        log.Fatalf("Could not connect to docker: %s", err)
    }

    t.Cleanup(func() {
        err := pool.Purge(resource)
        if err != nil {
            t.Logf("Could not purge resource: %s", err)
        }
    })
    return db
}
Enter fullscreen mode Exit fullscreen mode

Conclusion

It is important to know when to mock and when not to mock. Additionally, to know the pros and cons for mocking and for creating the real instance.

Discussion (2)

Collapse
lifelongthinker profile image
Sebastian

Very nice post, thanks for sharing your insights.

I'd like to add one point: Testing with mock objects vs. testing with real dependencies is like crossing boundaries and stepping from the domain of unit/method/class testing into the terrain of integration testing.

As you rightly say, there are pros and cons of both approaches. Whatever we do, we should always make our intention explicit and communicate clearly whether our tests rely on any third parties or are self-sufficient.

Collapse
arvindpdmn profile image
Arvind Padmanabhan

Detailed and useful discussion on mocks.

Forem Open with the Forem app