Mocking Return Values in Python

Unit tests are easier to write when given a set of inputs you know you're always going to get an expected response. It becomes a bit difficult when the same set of inputs can lead to varying responses.
When you work on programs where sometimes you have to make a request to an external API or making requests to other services in your microservices, you can't be sure you're going to get 200 status code on each request.
You'll mostly put in measures to handle when your requests fail but how do you write unit tests to ensure your system works as expected.

Writing Program

Let's build a program that a pings a url and returns True if the status code is 200 and False when it is not.

1. Create a file called client.py
2. Write the following code

# client.py

# Make sure you have requests installed.
# If not, you can install with `pip3 install requests` 
# or `pip install requests`.
import requests

def ping(url):
    res = requests.get(url)
    retun res.status_code == 200

print(ping("https://google.com"))

1. Run python3 client.py in your terminal in the directory where client.py resides. The below should be your response;

❯ python3 client.py
True

Writing Tests

Our program works as expected. It's time to write tests to ensure that we return True when our request is 200 and False when the status code is not 200.

1. Create a file called test_client.py in the same directory as client.py
2. Write the following code;

# test_client.py
import unittest
from client import ping

class TestClient(unittest.TestCase):
    def setUp(self):
        self.url = "https://google.com"

    def test_ping_returns_200(self):
        result = ping(self.url)
        self.assertTrue(result)

if __name__ == "__main__":
    unittest.main()

1. Run python3 test_client.py. You should get a response like below;

❯ python3 test_client.py
.
---------------------------------------------------------
Ran 1 test in 1.113s

OK

All our tests are passing but now let's test for the case when the status code is not 200. Let's add another test to our test

# test_client.py
import unittest
from client import ping

class TestClient(unittest.TestCase):
    def setUp(self):
        self.url = "https://google.com"

    def test_ping_returns_200(self):
        result = ping(self.url)
        self.assertTrue(result)

    # New
     def test_ping_returns_500(self):
        result = ping(self.url)
        self.assertFalse(result)


if __name__ == "__main__":
    unittest.main()

When we rerun python3 test_client.py, our new test fails.

❯ python3 test_client.py
.F
==========================================================
FAIL: test_ping_returns_500 (__main__.TestClient.test_ping_returns_500)
----------------------------------------------------------
Traceback (most recent call last):
  File "/Users/mock/test_client.py", line 46, in test_ping_returns_500
    self.assertFalse(result)
AssertionError: True is not false

-----------------------------------------------------------
Ran 2 tests in 2.076s

FAILED (failures=1)

This is because for now our request returns 200, so we are unable to test for when our ping fails.
To be able to do this we are going to mock our requests and responses. This gives us the ability to determine what response we should get.
This way we can manipulate our requests to return 500 response status code instead of the actual status code.

Mocking requests and responses

First we're going to import the patch decorator and MagicMock object from unittest.mock.
The patch decorator allows us to edit our requests, it takes a target which will be the requests we want to change, which in this case is the one in client.py.
The MagicMock object allows us to create the response we want so we can pass it to requests.
We have add an additional parameter which can be named anyway we want. I'll name it mock_requests. This is the object we can attach our response to.
Let's add these changes to our test_client.py;

# test_client.py
import unittest
from unittest.mock import patch, MagicMock
from client import ping

class TestClient(unittest.TestCase):
    def setUp(self):
        self.url = "https://google.com"

    def test_ping_returns_200(self):
        result = ping(self.url)
        self.assertTrue(result)


    @patch("client.requests")   # New
    def test_ping_returns_500(self, mock_requests):    # New

        # New
        mock_response = MagicMock()
        mock_response.status_code = 500
        mock_requests.get.return_value = mock_response

        result = ping(self.url)
        self.assertFalse(result)


if __name__ == "__main__":
    unittest.main()

Let's dive into what we've done so far.

1. By adding the decorator @patch("client.requests"), we're saying we want to modify the requests function that we imported in client.py. It can be any function, we could have targeted ping itself. You can try that on your own after this. The patch decorator modifies our test method and passes in a second argument to our test_ping_returns_500 method. Thus, we add mock_requests as a parameter to our method. This mock_requests is a mock variant of requests.
2. We instantiate our MagicMock object and assign to mock_response. Our MagicMock object allows us to take the form of any class we want and in this case a Response object.
3. We add a status_code attribute to our mock_response and assign it the value of 500.
4. We proceed to assign our mock_response to mock_requests.get.return_value. We have now modified our requests in that, we're saying, when the requests in client.py calls it's get function, we want the value returned to be that of our mock_response. Thus when requests.get() is called, the value returned is going to be the mock_response.

If we run our test again, we see everything passes now.

❯ python3 test_client.py
..
----------------------------------------------------------------------
Ran 2 tests in 1.609s

OK

We can proceed to do the same for test_ping_returns_200 method but make the status_code 200 so we can add a fake url instead of making requests to https://google.com.

# test_client.py
import unittest
from unittest.mock import patch, MagicMock
from client import ping

class TestClient(unittest.TestCase):
    def setUp(self):
        self.url = "https://api.service.com"    # New

    @patch("client.requests")   # New
    def test_ping_returns_200(self, mock_requests):    # New

        # New
        mock_response = MagicMock()
        mock_response.status_code = 200
        mock_requests.get.return_value = mock_response


        result = ping(self.url)
        self.assertTrue(result)


    @patch("client.requests")
    def test_ping_returns_500(self, mock_requests):

        mock_response = MagicMock()
        mock_response.status_code = 500
        mock_requests.get.return_value = mock_response

        result = ping(self.url)
        self.assertFalse(result)


if __name__ == "__main__":
    unittest.main()

We can run our tests again to ensure everything works well.

❯ python3 test_client.py
..
----------------------------------------------------------------------
Ran 2 tests in 1.609s

OK

Mocking response body

We can also add response body to our mock_response. Assuming, the 3rd party API we're pinging returns a json response with an attribute {"status": "ok"}.

1. Let's modify our function in client.py to return the json response too when the status_code is 200 and None when it is not.

# client.py

import requests

def ping(url):
    res = requests.get(url)
    # New
    if res.status_code == 200:
        return (True, res.json())
    return (False, None)

1. We modify our test_client.py to this;

# test_client.py
import unittest
from unittest.mock import patch, MagicMock
from client import ping

class TestClient(unittest.TestCase):
    def setUp(self):
        self.url = "https://api.service.com"

    @patch("client.requests")
    def test_ping_returns_200(self, mock_requests):

        mock_response = MagicMock()
        mock_response.status_code = 200
        mock_response.json.return_value = {"status": "ok"}  # New
        mock_requests.get.return_value = mock_response


        status, body = ping(self.url)  # New
        self.assertTrue(status)
        self.assertEqual(body["status"], "ok")


    @patch("client.requests")
    def test_ping_returns_500(self, mock_requests):

        mock_response = MagicMock()
        mock_response.status_code = 500
        mock_requests.get.return_value = mock_response

        status, body = ping(self.url)  # New
        self.assertFalse(status)


if __name__ == "__main__":
    unittest.main()

Adding mock_response.json.return_value, we're saying when we call the json function on our mock_response we should get {"status": "ok"}.
Thus, {"status": "ok"} is the value we get when we call res.json() in client.py.

Conclusion

Mocking requests make testing easier especially when you have to deal with 3rd party APIs and even other services in your microservices.
Let's say, you have an API that first makes a request to your auth service to confirm if a user has certain permissions before allowing them access to certain resources.
You can mock the response you have to test that only users with the needed permissions are allowed to access the resource.

Code for this tutorial can be found here https://github.com/quamejnr/python-mock-requests