Mock DB for testing HTTP API in Go and achieve 100% coverage

If you have trouble isolating unit test data to avoid conflicts, think about mock DB! In this article, we will learn how to use Gomock to generate stubs for the DB interface, which helps us write API unit tests faster, cleaner, and easily achieve 100% coverage.

Here's:

• Link to the full series playlist on Youtube
• And its Github repository

Why mocking DB

In the previous lectures, we have learned how to implement RESTful HTTP APIs in Go. When it comes to testing these APIs, some people might choose to connect to the real database, while some others might prefer to just mocking it. So which approach should we use?

Well, I would say it’s up to you. But for me, mocking is better because of the following reasons:

• First, it helps us to write independent tests more easily because each test will use its own separate mock DB to store data, so there will be no conflicts between them. If you use a real DB, all tests will read and write data to the same place, so it would be harder to avoid conflicts, especially in a big project with a large code base.
• Second, our tests will run much faster since they don’t have to spend time talking to the DB and waiting for the queries to run. All actions will be performed in memory and within the same process.
• The third and very important reason for mocking DB is: It allows us to write tests that achieve 100% coverage. With a mock DB, we can easily set up and test some edge cases, such as an unexpected error, or a connection lost, which would be impossible to achieve if we use a real DB.

OK, that sounds great. But is it good enough to test our API with just a mock DB? Can we be confident that our codes will still perform well when a real DB is plugged in?

Yes, absolutely! Because our code that talks to the real DB is already tested carefully in the previous lecture.

So all we need to do is: make sure that the mock DB implements the same interface as the real DB. Then everything will be working just fine when being put together.

How to mock DB

There are 2 ways to mock DB.

The first one is to implement a fake DB, which stores data in memory. If you followed my gRPC course then you definitely have already known about it.

For example, here we have the Store interface that defines a list of actions we can do with the real DB.

Then we have a fake DB MemStore struct, which implements all actions of the Store interface, but only uses a map to read and write data.

This approach of using fake db is very simple and easy to implement. However, it requires us to write a lot more codes that only be used for testing, which is quite time-consuming for both development and maintenance later.

So today I’m gonna show you a better way to mock DB, which is using stubs instead of fake DB.

The idea is to use gomock package to generate and build stubs that return hard-coded values for each scenario we want to test.

In this example, gomock already generated a MockStore for us. So all we need to do is to call its EXPECT() function to build a stub, which tells gomock that: this GetAccount() function should be called exactly 1 time with this input accountID, and return this account object as output.

And that’s it! After setting up the stub, we can simply use this mock store to test the API.

Don’t worry if you don’t fully understand it right now. Let’s jump into coding to see how it really works!

Install gomock

First, we need to install gomock. Let’s open the browser and search for gomock. Then open its Github page.

Copy this go get command and run it in the terminal to install the package:

❯ go get github.com/golang/mock/mockgen@v1.4.4

After this, a mockgen binary file will be available in the go/bin folder.

❯ ls -l ~/go/bin
total 341344
...
-rwxr-xr-x  1 quangpham  staff  10440388 Oct 17 18:27 gotests
-rwxr-xr-x  1 quangpham  staff   8914560 Oct 17 18:27 guru
-rwxr-xr-x  1 quangpham  staff   5797544 Oct 17 18:27 impl
-rwxr-xr-x  1 quangpham  staff   7477056 Nov  2 09:21 mockgen

We will use this tool to generate the mock db, so it’s important to make sure that it is executable from anywhere. We check that by running:

❯ which mockgen
mockgen not found

Here it says mockgen not found. That’s because the go/bin folder is not in the PATH environment variable at the moment.

To add it to the PATH, I will edit the .zshrc file since I’m using zsh. If you’re using a bash shell then you should edit the .bash_profile or .bashrc file instead.

❯ vi ~/.zshrc

I'm using vim, so let's press i to enter the insert mode. Then add this export command to the top of the file:

export PATH=$PATH:~/go/bin

Press Esc to exit the insert mode, then :wq to save the file and quit vim.

Next we have to run this source command to reload the .zshrc file:

❯ source ~/.zshrc

Now if we run which mockgen again, we can see that it is now available in the go/bin folder.

❯ which mockgen
/Users/quangpham/go/bin/mockgen

Note that the .zshrc file will be automatically loaded when we start a new terminal window. So we don’t need to run the source command every time we open the terminal.

Define Store interface

Alright, now in order to use mockgen to generate a mock DB, we have to update our code a bit.

At the moment, in the api/server.go file, the NewServer() function is accepting a db.Store object:

type Server struct {
    store  *db.Store
    router *gin.Engine
}

func NewServer(store *db.Store) *Server {
    ...
}

This db.Store is defined in the db/sqlc/store.go file. It is a struct which will always connect to the real database:

type Store struct {
    db *sql.DB
    *Queries
}

So in order to use a mock DB in the API server tests, we have to replace that store object with an interface. I’m gonna duplicate this Store struct definition and change its type to interface.

type Store interface {
    // TODO: add functions to this interface
}

type SQLStore struct {
    db *sql.DB
    *Queries
}

Then the old Store struct will be renamed to SQLStore. It will be a real implementation of the Store interface that talks to a SQL database, which is PostgreSQL in this case.

Then this NewStore() function should not return a pointer, but just a Store interface. And inside, it should return the real DB implementation of the interface, which is SQLStore.

func NewStore(db *sql.DB) Store {
    return &SQLStore{
        db:      db,
        Queries: New(db),
    }
}

We also have to change the type of the store receiver of the execTx() function and the TransferTx() function to *SQLStore like this:

func (store *SQLStore) execTx(ctx context.Context, fn func(*Queries) error) error {
    ...
}

func (store *SQLStore) TransferTx(ctx context.Context, arg TransferTxParams) (TransferTxResult, error) {
}

Alright, now we have to define a list of actions that the Store interface can do.

Basically, it should have all functions of the Queries struct, and one more function to execute the transfer money transaction.

So first I’m gonna copy this TransferTx() function signature and paste it inside the Store interface:

type Store interface {
    TransferTx(ctx context.Context, arg TransferTxParams) (TransferTxResult, error)
}

For the functions of the Queries struct, of course, we can do the same, like going through all of them and copy-paste one by one. However, it will be too time-consuming because this struct can contain a lot of functions.

Lucky for us, the sqlc package that we used to generate CRUD codes also has an option to emit an interface that contains all of the function of the Queries struct.

All we have to do is to change this emit_interface setting in the sqlc.yaml file to true:

version: "1"
packages:
  - name: "db"
    path: "./db/sqlc"
    queries: "./db/query/"
    schema: "./db/migration/"
    engine: "postgresql"
    emit_json_tags: true
    emit_prepared_queries: false
    emit_interface: true
    emit_exact_table_names: false
    emit_empty_slices: true

Then run this command in the terminal to regenerate the codes:

❯ make sqlc

After this, in the db/sqlc folder, we can see a new file called querier.go. It contains the generated Querier interface with all functions to insert and query data from the database:

type Querier interface {
    AddAccountBalance(ctx context.Context, arg AddAccountBalanceParams) (Account, error)
    CreateAccount(ctx context.Context, arg CreateAccountParams) (Account, error)
    CreateEntry(ctx context.Context, arg CreateEntryParams) (Entry, error)
    CreateTransfer(ctx context.Context, arg CreateTransferParams) (Transfer, error)
    DeleteAccount(ctx context.Context, id int64) error
    GetAccount(ctx context.Context, id int64) (Account, error)
    GetAccountForUpdate(ctx context.Context, id int64) (Account, error)
    GetEntry(ctx context.Context, id int64) (Entry, error)
    GetTransfer(ctx context.Context, id int64) (Transfer, error)
    ListAccounts(ctx context.Context, arg ListAccountsParams) ([]Account, error)
    ListEntries(ctx context.Context, arg ListEntriesParams) ([]Entry, error)
    ListTransfers(ctx context.Context, arg ListTransfersParams) ([]Transfer, error)
    UpdateAccount(ctx context.Context, arg UpdateAccountParams) (Account, error)
}

var _ Querier = (*Queries)(nil)

And here you can see it declares a blank variable var _ Querier to make sure that the Queries struct must implement all functions of this Querier interface.

Now what we need to do is just embed this Querier inside the Store interface. That would make Store interface to have all of its functions in addition to the TransferTx() function that we’ve added before:

type Store interface {
    Querier
    TransferTx(ctx context.Context, arg TransferTxParams) (TransferTxResult, error)
}

Next, we have to go back to the api/server.go file and remove this * from *db.Store type because it is no longer a struct pointer, but an interface instead:

func NewServer(store db.Store) *Server {
    ...
}

Note that although we have changed the Store type from struct to interface, our code will still work well, and we don’t have to change anything in the main.go file because the db.NewStore() function is now also returning a Store interface with the actual implementation SQLStore that connects to the real SQL DB.

func main() {
    config, err := util.LoadConfig(".")
    if err != nil {
        log.Fatal("cannot load config:", err)
    }

    conn, err := sql.Open(config.DBDriver, config.DBSource)
    if err != nil {
        log.Fatal("cannot connect to db:", err)
    }

    store := db.NewStore(conn)
    server := api.NewServer(store)

    err = server.Start(config.ServerAddress)
    if err != nil {
        log.Fatal("cannot start server:", err)
    }
}

Generate mock DB

Alright, so now as we have the db.Store interface, we can use gomock to generate a mock implementation of it.

First I will create a new mock folder inside the db package. Then let’s open the terminal and run:

❯ mockgen -help
mockgen has two modes of operation: source and reflect.

Source mode generates mock interfaces from a source file.
It is enabled by using the -source flag. Other flags that
maybe useful in this mode are -imports and -aux_files.
Example:
    mockgen -source=foo.go [other options]

Reflect mode generates mock interfaces by building a program
that uses reflection to understand interfaces. It is enabled
by passing two non-flag arguments: an import path, and a
comma-separated list of symbols.
Example:
    mockgen database/sql/driver Conn,Driver

    -aux_files string
        (source mode) Comma-separated pkg=path pairs of auxiliary Go source files.
    -build_flags string
        (reflect mode) Additional flags for go build.
    -copyright_file string
        Copyright file used to add copyright header
    -debug_parser
        Print out parser results only.
    -destination string
        Output file; defaults to stdout.
    -exec_only string
        (reflect mode) If set, execute this reflection program.
    -imports string
        (source mode) Comma-separated name=path pairs of explicit imports to use.
    -mock_names string
        Comma-separated interfaceName=mockName pairs of explicit mock names to use. Mock names default to 'Mock'+ interfaceName suffix.
    -package string
        Package of the generated code; defaults to the package of the input with a 'mock_' prefix.
    -prog_only
        (reflect mode) Only generate the reflection program; write it to stdout and exit.
    -self_package string
        The full package import path for the generated code. The purpose of this flag is to prevent import cycles in the generated code by trying to include its own package. This can happen if the mock's package is set to one of its inputs (usually the main one) and the output is stdio so mockgen cannot detect the final output package. Setting this flag will then tell mockgen which import to exclude.
  -source string
        (source mode) Input Go source file; enables source mode.
  -version
        Print version.
  -write_package_comment
        Writes package documentation comment (godoc) if true. (default true)

Mockgen gives us 2 ways to generate mocks. The source mode will generate mock interfaces from a single source file.

Things would be more complicated if this source file imports packages from other files, which is often the case when we work on a real project.

In this case, it’s better to use the reflect mode, where we only need to provide the name of the package and the interface, and let mockgen use reflection to automatically figure out what to do.

OK so I’m gonna run:

❯ mockgen github.com/techschool/simplebank/db/sqlc Store

The first argument is an import path to the Store interface. It's basically the simple bank module name github.com/techschool/simplebank followed by /db/sqlc because our Store interface is defined inside the db/sqlc folder.

The second argument we need to pass in this command is the name of the interface, which is Store in this case.

We should also specify the destination of the generated output file. Otherwise, mockgen will write the generated codes to stdout by default. So let’s use the -destination option to tell it to write the mock store codes to db/mock/store.go file:

❯ mockgen -destination db/mock/store.go github.com/techschool/simplebank/db/sqlc Store

Then press enter to run this command.

Now get back to visual studio code, we can see that a new file store.go is generated inside the db/mock folder:

// Code generated by MockGen. DO NOT EDIT.
// Source: github.com/techschool/simplebank/db/sqlc (interfaces: Store)

// Package mock_sqlc is a generated GoMock package.
package mock_sqlc

import (
    context "context"
    gomock "github.com/golang/mock/gomock"
    db "github.com/techschool/simplebank/db/sqlc"
    reflect "reflect"
)

// MockStore is a mock of Store interface
type MockStore struct {
    ctrl     *gomock.Controller
    recorder *MockStoreMockRecorder
}

// MockStoreMockRecorder is the mock recorder for MockStore
type MockStoreMockRecorder struct {
    mock *MockStore
}

// NewMockStore creates a new mock instance
func NewMockStore(ctrl *gomock.Controller) *MockStore {
    mock := &MockStore{ctrl: ctrl}
    mock.recorder = &MockStoreMockRecorder{mock}
    return mock
}

// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockStore) EXPECT() *MockStoreMockRecorder {
    return m.recorder
}

...

In this file, there are 2 important struct: MockStore and MockStoreMockRecorder.

MockStore is the struct that implements all required functions of the Store interface. For example, here’s the AddAccountBalance() function of the MockStore, which takes a context and an AddAccountBalanceParams as input and returns an Account or an error:

// AddAccountBalance mocks base method
func (m *MockStore) AddAccountBalance(arg0 context.Context, arg1 db.AddAccountBalanceParams) (db.Account, error) {
    m.ctrl.T.Helper()
    ret := m.ctrl.Call(m, "AddAccountBalance", arg0, arg1)
    ret0, _ := ret[0].(db.Account)
    ret1, _ := ret[1].(error)
    return ret0, ret1
}

The MockStoreMockRecorder also has a function with the same name and the same number of arguments. However, the types of these arguments are different. They’re just general interface type:

// AddAccountBalance indicates an expected call of AddAccountBalance
func (mr *MockStoreMockRecorder) AddAccountBalance(arg0, arg1 interface{}) *gomock.Call {
    mr.mock.ctrl.T.Helper()
    return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "AddAccountBalance", reflect.TypeOf((*MockStore)(nil).AddAccountBalance), arg0, arg1)
}

Later we will see how this function is used to build stubs. The idea is: we can specify how many times the AddAccountBalance() function should be called, and with what values of the arguments.

All other functions of the Store interface are generated in the same manner.

Note that the current package name that gomock generated for us is mock_sqlc, which doesn’t look very idiomatic, so I want to change it to something else, such as mockdb.

We can instruct mockgen to do that using the -package option. All we have to do is to add -package, followed by mockdb to this command:

❯ mockgen -package mockdb -destination db/mock/store.go github.com/techschool/simplebank/db/sqlc Store

Then now in the code, the package name has been changed to mockdb as we wanted:

// Code generated by MockGen. DO NOT EDIT.
// Source: github.com/techschool/simplebank/db/sqlc (interfaces: Store)

// Package mockdb is a generated GoMock package.
package mockdb

import (
    context "context"
    gomock "github.com/golang/mock/gomock"
    db "github.com/techschool/simplebank/db/sqlc"
    reflect "reflect"
)

// MockStore is a mock of Store interface
type MockStore struct {
    ctrl     *gomock.Controller
    recorder *MockStoreMockRecorder
}

// MockStoreMockRecorder is the mock recorder for MockStore
type MockStoreMockRecorder struct {
    mock *MockStore
}

...

Alright, before start writing API tests using the new generated MockStore, I’m gonna add a new mock command to the Makefile so that we can easily regenerate the code whenever we want.

...

mock:
    mockgen -package mockdb -destination db/mock/store.go github.com/techschool/simplebank/db/sqlc Store

.PHONY: postgres createdb dropdb migrateup migratedown sqlc test server mock

Now whenever we want to regenerate the mock store, we can simple run make mock in the terminal.

Write unit test for Get Account API

OK, now with the generated MockStore, we can start writing test for our APIs.

I’m gonna create a new file account_test.go inside the api package.

There are several API to mange bank accounts in our application. But for this lecture, we will only write tests for the most important one: Get Account API. You can easily based on that to write tests for other APIs if you want.

In the api/account_test.go file, I will define a new function TestGetAccountAPI() with the testing.T input parameter.

func TestGetAccountAPI(t *testing.T) {
}

In order to test this API, we need to have an account first. So let’s write a separate function to generate a random account.

It will return a db.Account object, where ID is a random integer between 1 and 1000, Owner is util.RandomOwner(), Balance is util.RandomMoney(), and Currency is util.RandomCurrency().

func randomAccount() db.Account {
    return db.Account{
        ID:       util.RandomInt(1, 1000),
        Owner:    util.RandomOwner(),
        Balance:  util.RandomMoney(),
        Currency: util.RandomCurrency(),
    }
}

Now go back to the test, we call randomAccount() function to create a new account.

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()
}

Next we need to create a new mock store using this mockdb.NewMockStore() generated function. It expects a gomock.Controller object as input, so we have to create this controller by calling gomock.NewController and pass in the testing.T object.

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    ctrl := gomock.NewController(t)
    defer ctrl.Finish()
}

We should defer calling Finish method of this controller. This is very important because it will check to see if all methods that were expected to be called were called.

We will see how it works in a moment. For now, let’s create a new store by calling mockdb.NewMockStore() with this input controller.

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    ctrl := gomock.NewController(t)
    defer ctrl.Finish()

    store := mockdb.NewMockStore(ctrl)
}

The next step is to build the stubs for this mock store. In this case, we only care about the GetAccount() method, since it’s the only method that should be called by the Get Account API handler.

So let’s build stub for this method by calling store.EXPECT().GetAccount(). This function expects 2 input arguments of type general interface.

Why 2 input arguments? That’s because the GetAccount() method of our Store interface requires 2 input parameters: a context and an account ID.

type Querier interface {
    GetAccount(ctx context.Context, id int64) (Account, error)
    ...
}

Thus for this stub definition, we have to specify what values of these 2 parameters we expect this function to be called with.

The first context argument could be any value, so we use gomock.Any() matcher for it. The second argument should equal to the ID of the random account we created above. So we use this matcher: gomock.Eq() and pass the account.ID to it.

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    ctrl := gomock.NewController(t)
    defer ctrl.Finish()

    store := mockdb.NewMockStore(ctrl)
    store.EXPECT().
        GetAccount(gomock.Any(), gomock.Eq(account.ID)).
        Times(1).
        Return(account, nil)
}

Now this stub definition can be translated as: I expect the GetAccount() function of the store to be called with any context and this specific account ID arguments.

We can also specify how many times this function should be called using the Times() function. Here Times(1) means we expect this function to be called exactly 1 time.

More than that, we can use the Return() function to tell gomock to return some specific values whenever the GetAccount() function is called. For example, in this case, we want it to return the account object and a nil error.

Note that the input arguments of this Return() function should match with the return values of the GetAccount function as defined in the Querier interface.

Alright, now the stub for our mock Store is built. We can use it to start the test HTTP server and send GetAccount request. Let’s create a server by calling NewServer() function with the mock store.

func TestGetAccountAPI(t *testing.T) {
    ...

    server := NewServer(store)
    recorder := httptest.NewRecorder()
}

For testing an HTTP API in Go, we don’t have to start a real HTTP server. Instead, we can just use the recording feature of the httptest package to record the response of the API request. So here we call httptest.NewRecorder() to create a new ResponseRecorder.

Next we will declare the url path of the API we want to call, which should be /accounts/{ID of the account we want to get}.

Then we create a new HTTP Request with method GET to that URL. And since it’s a GET request, we can use nil for the request body.

func TestGetAccountAPI(t *testing.T) {
    ...

    server := NewServer(store)
    recorder := httptest.NewRecorder()

    url := fmt.Sprintf("/accounts/%d", tc.accountID)
    request, err := http.NewRequest(http.MethodGet, url, nil)
    require.NoError(t, err)
}

This http.NewRequest function will return a request object or an error. We require no errors to be returned.

Then we call server.router.ServeHTTP() function with the created recorder and request objects.

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    ctrl := gomock.NewController(t)
    defer ctrl.Finish()

    store := mockdb.NewMockStore(ctrl)
    store.EXPECT().
        GetAccount(gomock.Any(), gomock.Eq(account.ID)).
        Times(1).
        Return(account, nil)

    server := NewServer(store)
    recorder := httptest.NewRecorder()

    url := fmt.Sprintf("/accounts/%d", tc.accountID)
    request, err := http.NewRequest(http.MethodGet, url, nil)
    require.NoError(t, err)

    server.router.ServeHTTP(recorder, request)
    require.Equal(t, http.StatusOK, recorder.Code)
}

Basically, this will send our API request through the server router and record its response in the recorder. All we need to do is to check that response.

The simplest thing we can check is the HTTP status code. In the happy case, it should be http.StatusOK. This status code is recorded in the Code field of the recorder.

And that’s it! Let’s run the test.

It passed. Awesome!

Now I’m gonna show you what will happen if in the getAccount() handler function of api/account.go file we don’t call the store.GetAccount function. Let’s comment out this block of code and just set account to be an empty object.

func (server *Server) getAccount(ctx *gin.Context) {
    var req getAccountRequest
    if err := ctx.ShouldBindUri(&req); err != nil {
        ctx.JSON(http.StatusBadRequest, errorResponse(err))
        return
    }

    // account, err := server.store.GetAccount(ctx, req.ID)
    // if err != nil {
    //     if err == sql.ErrNoRows {
    //         ctx.JSON(http.StatusNotFound, errorResponse(err))
    //         return
    //     }

    //     ctx.JSON(http.StatusInternalServerError, errorResponse(err))
    //     return
    // }
    account := db.Account{}

    ctx.JSON(http.StatusOK, account)
}

Save this file and rerun the unit test.

This time, the test failed. And the reason is due to a missing call to the store.GetAccount function. We expect that function to be called exactly once, but in the implementation, it’s not getting called.

So now you know one power of the gomock package. It makes writing unit tests so easy and saves us tons of time implementing the mock interface.

Now, what if we want to check more than just the HTTP status code? To make the test more robust, we should check the response body as well.

The response body is stored in the recorder.Body field, which is in fact just a bytes.Buffer pointer.

We expect it to match the account that we generated at the top of the test. So I’m gonna write a new function: requireBodyMatchAccount() for this purpose.

It will have 3 input arguments: the testing.T, the response body of type byte.Buffer pointer, and the account object to compare.

func requireBodyMatchAccount(t *testing.T, body *bytes.Buffer, account db.Account) {
    data, err := ioutil.ReadAll(body)
    require.NoError(t, err)

    var gotAccount db.Account
    err = json.Unmarshal(data, &gotAccount)
    require.NoError(t, err)
    require.Equal(t, account, gotAccount)
}

First we call ioutil.ReadAll() to read all data from the response body and store it in a data variable. We require no errors to be returned.

Then we declare a new gotAccount variable to store the account object we got from the response body data.

Then we call json.Unmarshal to unmarshal the data to the gotAccount object. Require no errors, then require the gotAccount to be equal to the input account.

And we’re done. Now let’s go back to the unit test and call requireBodyMatchAccount function with the testing.T, the recorder.Body, and the generated account as input arguments.

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    ctrl := gomock.NewController(t)
    defer ctrl.Finish()

    store := mockdb.NewMockStore(ctrl)
    store.EXPECT().
        GetAccount(gomock.Any(), gomock.Eq(account.ID)).
        Times(1).
        Return(account, nil)

    server := NewServer(store)
    recorder := httptest.NewRecorder()

    url := fmt.Sprintf("/accounts/%d", tc.accountID)
    request, err := http.NewRequest(http.MethodGet, url, nil)
    require.NoError(t, err)

    server.router.ServeHTTP(recorder, request)
    require.Equal(t, http.StatusOK, recorder.Code)
    requireBodyMatchAccount(t, recorder.Body, account)
}

Then rerun the test.

It passed. Excellent!

OK, so the GetAccount API unit test is working very well. But it only covers the happy case for now.

Next, I’m gonna show you how to transform this test into a table-driven test set to cover all possible scenarios of the GetAccount API and to get 100% coverage.

Achieve 100% coverage

First, we need to declare a list of test cases. I’m gonna use an anonymous class to store the test data.

Each test case will have a unique name to separate it from others. Then there should be an account ID that we want to get.

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    testCases := []struct {
        name          string
        accountID     int64
        buildStubs    func(store *mockdb.MockStore)
        checkResponse func(t *testing.T, recoder *httptest.ResponseRecorder)
    }{
        // TODO: add test data
    }
    ...
}

Moreover, the GetAccount stub for each scenario will be built differently, so here I have a buildStubs field, which is actually a function that takes a mock store as input. We can use this mock store to build the stub that suits the purpose of each test case.

Similarly, we have a checkResponse function to check the output of the API. It has 2 input arguments: a testing.T, and a httptest.ResponseRecorder object.

Now with this struct definition, let’s add the first scenario for the happy case.

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    testCases := []struct {
        name          string
        accountID     int64
        buildStubs    func(store *mockdb.MockStore)
        checkResponse func(t *testing.T, recoder *httptest.ResponseRecorder)
    }{
        {
            name:      "OK",
            accountID: account.ID,
            buildStubs: func(store *mockdb.MockStore) {
                store.EXPECT().
                    GetAccount(gomock.Any(), gomock.Eq(account.ID)).
                    Times(1).
                    Return(account, nil)
            },
            checkResponse: func(t *testing.T, recorder *httptest.ResponseRecorder) {
                require.Equal(t, http.StatusOK, recorder.Code)
                requireBodyMatchAccount(t, recorder.Body, account)
            },
        },
    }
    ...
}

Its name is "OK". The account ID should be account.ID. Next, for the buildStubs function, I’m gonna copy its signature here. Then move the store.EXPECT command into this function.

Similar for the checkResponse function. Let’s copy its signature. Then move the 2 require commands into it.

We will add more cases to this list later. For now, let’s refactor the code a bit to make it work for multiple scenarios.

We use a simple for loop to iterate through the list of test cases. Then inside the loop, we declare new tc variable to store the data of current test case.

We gonna run each case as a separate sub-test of this unit test, so let’s call t.Run() function, pass in the name of this test case, and a function that takes testing.T object as input. Then I’m gonna move all of these statements into that function.

func TestGetAccountAPI(t *testing.T) {
    ...

    for i := range testCases {
        tc := testCases[i]

        t.Run(tc.name, func(t *testing.T) {
            ctrl := gomock.NewController(t)
            defer ctrl.Finish()

            store := mockdb.NewMockStore(ctrl)
            tc.buildStubs(store)

            server := NewServer(store)
            recorder := httptest.NewRecorder()

            url := fmt.Sprintf("/accounts/%d", tc.accountID)
            request, err := http.NewRequest(http.MethodGet, url, nil)
            require.NoError(t, err)

            server.router.ServeHTTP(recorder, request)
            tc.checkResponse(t, recorder)
        })
    }
}

Note that the url should be created with the tc.accountID so that it will use the account ID defined for each test case.

We call tc.buildStubs() function with the mock store before sending the request, and finally call tc.checkResponse() function at the end to verify the result.

OK, let’s rerun the test to make sure our happy case still works.

Yee, it passed! So now it’s time to add more cases.

I’m gonna duplicate the happy case's test data. The second case we want to test is when the account is not found. So its name should be "NotFound".

We can use the same accountID here because mock store is separated for each test case. But we need to change our buildStubs function a bit.

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    testCases := []struct {
        name          string
        accountID     int64
        buildStubs    func(store *mockdb.MockStore)
        checkResponse func(t *testing.T, recoder *httptest.ResponseRecorder)
    }{
        {
            name:      "OK",
            accountID: account.ID,
            buildStubs: func(store *mockdb.MockStore) {
                store.EXPECT().
                    GetAccount(gomock.Any(), gomock.Eq(account.ID)).
                    Times(1).
                    Return(account, nil)
            },
            checkResponse: func(t *testing.T, recorder *httptest.ResponseRecorder) {
                require.Equal(t, http.StatusOK, recorder.Code)
                requireBodyMatchAccount(t, recorder.Body, account)
            },
        },
        {
            name:      "NotFound",
            accountID: account.ID,
            buildStubs: func(store *mockdb.MockStore) {
                store.EXPECT().
                    GetAccount(gomock.Any(), gomock.Eq(account.ID)).
                    Times(1).
                    Return(db.Account{}, sql.ErrNoRows)
            },
            checkResponse: func(t *testing.T, recorder *httptest.ResponseRecorder) {
                require.Equal(t, http.StatusNotFound, recorder.Code)
            },
        },
    }

    ...
}

Here instead of returning this specific account, we should return an empty Account{} object together with a sql.ErrNoRows error. That’s because in the real implementation of the Store that connects to Postgres, the db/sql package will return this error if no account is found when executing the SELECT query.

We also have to change the checkResponse function, because in this case, we expect the server to return http.StatusNotFound instead. And since the account is not found, we can remove the requireBodyMatchAccount call.

Alright, let’s run the test again.

Cool! Both tests passed.

Let’s run the whole package test to see the code coverge.

In the account.go file, we can see that this getAccount handler is not 100% covered.

Right now only the not found case and the successful case are covered. We still have to test 2 more cases: InternalServerError and BadRequest.

Once again, I’m gonna duplicate the test data, change its name to "InternalError".

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    testCases := []struct {
        name          string
        accountID     int64
        buildStubs    func(store *mockdb.MockStore)
        checkResponse func(t *testing.T, recoder *httptest.ResponseRecorder)
    }{
        ...
        {
            name:      "InternalError",
            accountID: account.ID,
            buildStubs: func(store *mockdb.MockStore) {
                store.EXPECT().
                    GetAccount(gomock.Any(), gomock.Eq(account.ID)).
                    Times(1).
                    Return(db.Account{}, sql.ErrConnDone)
            },
            checkResponse: func(t *testing.T, recorder *httptest.ResponseRecorder) {
                require.Equal(t, http.StatusInternalServerError, recorder.Code)
            },
        },
    }

    ...
}

Now in the buildStubs function, instead of returning sql.ErrNoRows, I return sql.ErrConnDone, which basically is one possible error that the db/sql package can return when a query is run on a connection that has already been returned to the connection pool.

In this case, it should be considered an internal error, so in the checkResponse function, we must require the recorder.Code to be equal to http.StatusInternalServerError.

Let’s rerun the package test.

All passed. And we can now see that the InternalServerError branch in the code is now covered.

The last scenario we should test is BadRequest, which means the client has sent some invalid parameters to this API.

To reproduce this scenario, we will use an invalid account ID that doesn’t satisfy this binding condition.

So I’m gonna go back to the test file, duplicate this test data one more time, change its name to InvalidID, and update this accountID to 0, which is an invalid value because the minimum ID should be 1.

In this case, we should change the second parameter of the GetAccount function call to gomock.Any().

func TestGetAccountAPI(t *testing.T) {
    account := randomAccount()

    testCases := []struct {
        name          string
        accountID     int64
        buildStubs    func(store *mockdb.MockStore)
        checkResponse func(t *testing.T, recoder *httptest.ResponseRecorder)
    }{
        ...
        {
            name:      "InvalidID",
            accountID: 0,
            buildStubs: func(store *mockdb.MockStore) {
                store.EXPECT().
                    GetAccount(gomock.Any(), gomock.Any()).
                    Times(0)
            },
            checkResponse: func(t *testing.T, recorder *httptest.ResponseRecorder) {
                require.Equal(t, http.StatusBadRequest, recorder.Code)
            },
        },
    }

    ...
}

And since the ID is invalid, the GetAccount function should not be called by the handler. Therefore, we must update this to Times(0), and remove the Return function call.

For the checkResponse, we must change the status code to http.StatusBadRequest.

And that’s all. Let’s rerun the whole package tests!

All passed. Great! And looking at the code of the getAccount handler, we can see that it is 100% covered. So our goal is achieved!

However, the test log is currently containing too much information.

There are many duplicate debug logs written by Gin, which make it harder to read the test result.

The reason is that Gin is running in Debug mode by default. So let’s create a new main_test.go file inside the api package and config Gin to use Test mode instead.

The content of this file will be very similar to that of the main_test.go file in the db package, so I’m gonna copy this TestMain function, and paste it to our new file. Then let’s delete all statements of this function, except for the last one.

Now all we need to do is to call gin.SetMode to change it to gin.TestMode

func TestMain(m *testing.M) {
    gin.SetMode(gin.TestMode)
    os.Exit(m.Run())
}

And that’s it! We’re done. Now let’s go back to our test file and run the whole package tests.

All passed. And now the logs look much cleaner and easier to read than before.

Conclusion

OK, so today we have learned how to use gomock to generate mocks for our DB interface, and use it to write unit tests for the Get Account API to achieve 100% coverage. It really helps us write tests faster, easier, cleaner, safer, and much more robust.

You can apply this knowledge to write tests for other HTTP APIs in our simple bank project such as Create Account or Delete Account API.

I will push the code to Github so that you can have a reference in case you want to take a look.

Thanks a lot for reading and see you soon in the next lecture.

---

If you like the article, please subscribe to our Youtube channel and follow us on Twitter or Facebook for more tutorials in the future.

---

If you want to join me on my current amazing team at Voodoo, check out our job openings here. Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.

Comments

[Apoorv Mishra]

Hey while running the mockgen command to create mock file, i am getting below error, do you have any idea why is there this error, it says go mod file not found, but it is present there.
missing go.sum entry for module providing package github.com/golang/mock/mockgen/model; to add:
go mod download github.com/golang/mock
prog.go:12:2: missing go.sum entry for module providing package github.com/golang/mock/mockgen/model; to add:
go mod download github.com/golang/mock
prog.go:14:2: no required module provides package github.com/acetorscode/learningbank/db/sqlc: go.mod file not found in current directory or any parent directory; see 'go help modules'
prog.go:12:2: no required module provides package github.com/golang/mock/mockgen/model: go.mod file not found in current directory or any parent directory; see 'go help modules'
2021/05/31 19:12:53 Loading input failed: exit status 1

[Phi Hoang]

As the readme on their github repo, there are some workaround solutions:

1. Include an empty import import _ "github.com/golang/mock/mockgen/model".
2. Add --build_flags=--mod=mod to your mockgen command.

I had the same error as you and I did the second one successfully.
mockgen -build_flags=--mod=mod -package mockdb -destination db/mock/store.go github.com/phihdn/simplebank/db/sqlc Store

[Tianze Hua]

thank you so so much, I registered just here to say thank you. How are you able to find this solution? I searched for long didn't find a usable solution lol

[Phi Hoang]

Hi, I don't remember. It might be Google or I might walk around their github Issues page. Then I found it in the readme. :) happy to hear it helps.

[Mark]

Hey, after I completed the TestGetAccoutApI(t *testing.T) function and ran the test, the test fails and prompt below information. What's the matter?

--- FAIL: TestGetAccountAPI (0.00s)
/simple_bank/api/account_test.go:35:
Error Trace: simple_bank/api/account_test.go:35
Error: Not equal:
expected: 200
actual : 400
Test: TestGetAccountAPI
/simple_bank/api/controller.go:243: missing call(s) to *mockdb.MockStore.GetAccount(is anything, is equal to 29 (int64)) //simple_bank/api/account_test.go:23
simple_bank/api/controller.go:243: aborting test due to missing call(s)
FAIL
FAIL github.com/simple_bank/api 0.455s
FAIL