Python unit testing with Mock - Part One

What is this article?

Months ago, I was asked to give a talk to the Puget Sound Python Meetup group. This is the article-rendering of that talk. This talk makes some assumptions that you know the basics about how to actually write a test for python code and need to know how to embed mocks to keep your code safe.

When I was asked to give this talk, I was (and still am) responsible for a big script that talks to tons of external services:

• kerberos
• git
• aurora
• package repo
• jira
• email
• shared libraries

This posed the question of how do I test my code without touching these services?

Note this article is going to be exclusive to the mock test library. Structuring overall tests is another topic and deserves its own article, which I have yet to write at this point. When I get there, I'll let you know!

What is testing?

Testing makes sure your code behaves as expected by running your code and observing the results. By actually running chunks code, it's the most assured way to know that it runs, how it runs, and that it does what is expected.

When writing tests, you can categorize into one of three approximate buckets.

• Unit: Just this small part
• Integration: When all the parts talk to each other and included parts
• Acceptance: When the whole app talks to everything else

This article will focus on the best use of mocks, and that's for writing unit tests. Mocks could potentially have a place in integration tests, and the application wouldn't change.

Some code is easy to run to make sure it does what is expected. Here's a quick function and unit test that you might see:

### code ###
def double_up(number):
    return number * 2

### test ###
def test_double_up():
    assertEquals(double_up(2), 4)

The unit test simply runs the code and compares the received output to the expected output. Just like previously stated, testing means running your code

Now for some fun...

How would you test this

How would you test this block of code without actually deleting anything?

def wipe_directory(path):
    p = Popen(['rm', '-rf', path], stdout=PIPE, stderr=PIPE)
    if p.wait():
        raise Exception('We had a fail')

How would you test this block of code without actually deleting anything?

def delete_everything():
    r = requests.post('http://example.com/',
        data={'delete': 'everything', 'autocommit': 'true'})
    if r.status_code == 200:
        print('All things have been deleted')
        return True
    else:
        print('Got an error: {}'.format(r.headers))
        return False

How would you test this block of code without committing any change to the database?

class DBWriter(object):
    counter = 0

    def __init__(self):
        self.db = DBLibrary()

    def commit_to_db(self, sql):
        self.counter += 1
        self.db.commit(sql)

    def save(self, string):
        sql = "INSERT INTO mytable SET mystring = '{}'".format(string)
        self.commit_to_db(sql)

    def drop(self, string):
        sql = "DELETE FROM mytable WHERE mystring = '{}'".format(string)
        self.commit_to_db(sql)

Trying to actually execute these blocks of code will modify external state. These examples either execute a shell command, talk to an API, or connect to a database, all of which are actions that happen outside of the execution space of the written code.

What is mocking?

unittest.mock is a library for testing in Python. It allows you to replace parts of your system under test with mock objects and make assertions about how they have been used.

Source: https://docs.python.org/3/library/unittest.mock.html

As stated earlier, mocks are primarily used for unit testing. There may be some place in integration testing, highly unlikely in acceptance testing.

Why should you mock in your code?

• Unit test safely: stop the state-changing parts so you can actually run tests
• Write better code: a lovely side-effect of writing thorough tests
• Isolation: create walls between your code and "not-your-code" for safer tests

About the mock library

Mock objects intend to replace another part of code so that it pretends to be that code. In this example, let's assume that visible_method is a function inside of MyClass that runs sub_method. Now let's assume that sub_method does something we don't want to run in the scope of this test. We simply replace sub_method with a Mock instance.

from unittest.mock import Mock
from mycode import MyClass

def test_myclass():
   my_object = MyClass()
   my_object.sub_method = Mock()
   my_object.visible_method()
   my_object.sub_method.assert_called_with("arg", this="that")

This isolates one function from talking to another function within the same class. A mock becomes a synthetic code object that simply runs and returns another Mock object successfully every time (with some exceptions). Unless you're expecting something specific, a raw mock object can drop in almost anywhere.

However, most functions expect something specific, and this is the main purpose of a mock. You can shape a mock up to look like anything. Prop it up to behave like a specific function, module, class, object, etc. and you'll avoid having to do dicey things like call a database API. We'll get more on this in some of the next examples.

Using mock.patch

Sometimes, the dropping-in of an object is not as easy as directly replacing a function in a namespace. Sometimes the object to replace is much deeper down. By adding the @mock.patch decorator to a function, you can replace an object directly in the namespace with a mock.

Here's a function that has subprocess.Popen available in its namespace. We can't directly replace it as we did in the prior example.

def count_the_shells():
    p = Popen(['ps', '-a'], stdout=PIPE, stderr=PIPE)
    if p.wait():
        raise Exception('We had a fail')
    count = 0
    for proc in p.stdout.readlines():
        if "-bash" in proc:
            count += 1
    return count

The subprocess.Popen is inside the function, but it's available in the namespace of the function. By @mock.patching, we replace subprocess.Popen with a mock. The PIPEs no longer are used because of the replacement, so we don't need to patch them.

By examining the action in this function, we see its surfaces that talk to the subprocess object that gets returned:

• Popen runs a command line execution and returns a subprocess object. In this case, p
• p.wait() blocks until it gets back the shell's exit code and returns it as an integer.
• p.stdout is a filelike object that captures STDOUT

Our mock then needs to have these surfaces as well. We don't need to completely replace the behavior of a Popen call, we just need to make the return value look like the return value of a Popen call and the methods that we use.

@mock.patch('subprocess.Popen')
def test_count_the_shells(mocked_popen):
    mocked_popen.return_value.stdout = open('testps.out')
    mocked_popen.return_value.wait.return_value = False
    assert count_the_shells() == 4

Let's take a look at what's going on in this test:

• @mock.patch decorator replaces subprocess.Popen with a mock object. That gets passed in as the first argument in the test function. The test function receives it as mocked_popen
• The Popen call returns a subprocess object. We're now amending the return_value of that object by applying behavior to stdout and wait, which get used in the function
• Now when count_the_shells is executed, it calls the mock instead of Popen and gets back expected values.

We've effectively replaced subprocess.Popen with a mock that behaves like that object as far as the function is concerned. Now you know how to replace a method with a mock that actually mocks something!

But wait, there's more!

Spec and Autospec

What if your mock could just be pointed at a module and automatically look like that module? What if it could be prompted to just respond to functions already defined in a module with mocks? Seems like just the python-majick we have come to expect and love, right? Well, read on!!

A mock will have its own built-ins, but any other call received will simply return another mock. This plastic behavior looks like this:

>>> mock = Mock()
>>> mock.this_is_never_assigned('hello')
<Mock name='mock.this_is_never_assigned()' id='4422797328'>

This prevents accidental calls from blowing up your code, but, leaves room for a lot of error. You can obtain safer instantiation by autospeccing - make the mock behave like more like the thing you're mocking. Passing the spec argument tells the mock to closely behave like another. Mocks instantiated with spec=RealObject will pass isinstance(the_mock, RealObject).

>>> from collections import OrderedDict
>>> mymock = Mock(spec=OrderedDict)
>>> isinstance(mymock, OrderedDict)
True
>>> type(mymock)
<class 'mock.Mock'>

Using spec also affords protection, preventing calls to undeclared attributes. You can declare any additional attributes you wish.

>>> mymock = mock(spec=OrderedDict)
>>> a = mymock.this_does_not_exist()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/twitter/lib/python2.7/site-packages/mock.py", line 658, in __getattr__
    raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'this_does_not_exist'

>>> mymock.this_does_not_exist = "this exists now"
>>> print(mymock.this_does_not_exist)
this exists now

Instantiating with spec_set is an even stricter spec. This prevents amending missing attributes. Attempts to define undeclared attributes will fail on AttributeError

>>> mymock = Mock(spec_set=OrderedDict)
>>> mymock.this_does_not_exist = "o no you didn't"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/twitter/lib/python2.7/site-packages/mock.py", line 761, in __setattr__
    raise AttributeError("Mock object has no attribute '%s'" % name)
AttributeError: Mock object has no attribute 'this_does_not_exist'

Using the create_autospec function is even stricter. This will generate mock functions defined to spec that will enforce function signatures, meaning if the original function expects two positional and one keyword argument, then the mocked function will also expect two positional and one keyword.

>>> def myfunc(foo, bar):
...     pass
...
>>> mymock = create_autospec(myfunc)
>>> mymock("one", "two")
<MagicMock name='mock()' id='4493382480'>
>>> mymock("just one")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 2, in myfunc
TypeError: <lambda>() takes exactly 2 arguments (1 given)
>>>

Appropriate use of spec can help you write cleaner code and catch typos. Since a spec_set locks out unimplemented methods, your test code will fail if it has a typo too.

>>> mock = Mock(name='Thing', return_value=None)
>>> mock(1, 2, 3)
>>> mock.assret_called_once_with(4, 5, 6)
# typo of "assert" passes because mock objects are forgiving

>>> from urllib import request
>>> mock = Mock(spec=request.Request)
>>> mock.assret_called_with
Traceback (most recent call last):
...
AttributeError: Mock object has no attribute 'assret_called_with'
# since "assret_called_with" is a typo, it's not declared. Proper exception caught!

Also, in python 3, you can name your mocks, which shows in the repr of the mock. This is useful for debugging! You can see exactly where in your code your mock blew up because you know which of your mocks was misused!

Introspection

Mocks come with a long list of built-in functions that provide a level of introspection into the life of the mock. There are storage attributes that automatically store every function call in the mock. These are built-in attributes and functions for inspecting how the mock was used:

• called - boolean, true if ever called
• call_count - integer, number of times called
• call_args - mock.call() object with args from last call
• call_args_list - list of mock.call() with all args ever used
• method_calls - track calls to methods and attributes, and their descendents
• mock_calls - list of all calls to the mock object

If you have a mock object passing deep through your module, and you want to make sure it's actually referenced and not missed over, the called method is your friend. Similarly, call_count can confirm how many times, and call_args confirms what args were used the last time.

Now since this is for unit tests, assertions are the common way to write tests. To make this better, mock has built-in assertion functions that reference the aforementioned attributes:

• assert_called - if ever called
• assert_called_once - if called exactly once
• assert_called_with - specific args used in the last call
• assert_called_once_with - specific args are used exactly once
• assert_any_call - specific args used in any call ever
• assert_has_calls - like "any_call" but with multiple calls
• assert_not_called - has never been called

So you don't have to wrap up booleans in if statements, you can just use the built-ins.

Summing Up - Part One

In this article, we covered the usage and features of the mock module in python. We discussed how to apply a mock to an existing test and how to adjust its behavior.

In Python unit testing with Mock - Part Two, we'll cover how to review your code to decide what parts of your code require mocks, how to write them with real code samples, as well as some cases where you might want to mock, but shouldn't.

Comments

[Pedro Algarvio]

mock.patch should preferably be used in a with statement as opposed to using it as a test function decorator.

Here's some more insight on why.

[mistermocha]

Interesting ... although the article you post doesn't indicate that the decorator causes problems, but improper use of return values in the decorator is what causes the leaks.

I'm certainly doing this in my example, and I wasn't aware that it caused this problem. I've been writing tests for years and hadn't known about this. Thanks so much for sharing!

[Pedro Algarvio]

Yes, the wrong usage of return values are the issue that cause leakage.

As far as running into trouble, unless you have a massive amount of tests, you won't actually run out of memory, so, like you, probably 99% of mock users won't hit any issues, but its worth keeping that article in your mind if you see some weird behavior on your tests.

[Jeremy Schneider]

Something may have changed in your webpage processing or the way browsers now deal with it. Starting at the section "About the mock library", the code blocks are not being displayed correctly. For example:

thepracticaldev.s3.amazonaws.com/i...