Mock external dependencies in Python

Testing python modules or code that have any external calls such as third-party API calls or database calls, can get difficult. This is where mocks are helpful.

Mocks are typically used when:

• Calling a third-party API is redundant in testing scenarios (but required in actual production version) i.e. the data from external service doesnt often change however the http overhead skews the results.
• Calling a third-party is has only premium subscription(paid) i.e. it doesn't have a test harness environment
• The response from the third-party cannot be easily controlled by just changing the input i.e. simulating corner-case responses received external API.

There are several ways to implement mock, below, is a simple implementation of a http service mock using the unittest.mock library.

Mock http request

Let us assume there is a service that gives temperature back when requested with city or location:

Sample API's "get" output as seen on browser

A typical calling library within the client code would implement something like:

import requests
import json

class ExternalServiceUtility:
    def __init__(self):
        super().__init__()

    # Function to make call to external service
    def get_current_temperature_using_service(self, location):
        _temperature_service_url = "http://127.0.0.1:5001/gettemparature/{0}" #in real life would be from configuration
        formatted_url = _temperature_service_url.format(location)
        current_temperature = None

        try:
            response_obj = requests.get(formatted_url)

            # check if we got a valid response
            if response_obj.status_code != 200:
                #some logging
                return current_temperature

            # parse the response 
            response_json = response_obj.json()
            current_temperature = response_json["Temperature"]

            return current_temperature
        except Exception as ex:
            # some logging and handling
            return current_temperature

And the method to use this library would look like:

from externalserviceutility import ExternalServiceUtility
def get_actual_temperature(location):
    extutil = ExternalServiceUtility()
    temperature = extutil.get_current_temperature_using_service(location)
    print(temperature)

get_actual_temperature("Regina")

Here we may want to test the application out without calling the http endpoint. Hence the mock implementation would look as below.

from externalserviceutility import ExternalServiceUtility
import requests
from unittest.mock import patch
import json

def alternate_mock_method(args):
    data = '{"Temperature": 100, "Location": "Toronto", "Datetime": "2020-07-23", "Units": "Celsius"}'
    r = requests.Response()
    r.status_code = 200
    def json_func():
        return json.loads(data)
    r.json = json_func
    return r  

@patch('requests.get',side_effect=alternate_mock_method)
def get_temperature_with_mock(location, args):
    extutil = ExternalServiceUtility()
    temperature = extutil.get_current_temperature_using_service(location)
    print(temperature)


get_temperature_with_mock("Regina")

The same implementation can be done for other verbs within library say requests.post

Implementation details:

• Include the unittest mock library and import on the top
• Decorate a method using the "patch". All the calls to the method declared in the first parameter of the patch would be redirected to the function name declared as the second parameter i.e. side_effect.
• Now instead of making calls to the external service, the "alternate_mock_method" implementation would be called for each requests.get call with all the child code of the method "get_temperature_with_mock" method i.e. the method decorated with the patch attribute.
• Note that #get_temperature_with_mock# method requires an additional parameter in the end by syntax.

CAUTION: Note that all the calls to requests.get within the code tree structure within the mocked method would be redirected to custom mock implementation.