How to debug your Python mocks or imports

Hands up if you've spent 20 minutes making a feature then 2 hours testing it 🙋. I recently struggled with mocking an object declared globally in a file I was testing. It was making all my tests fail because its __init__ method made a call to AWS which wouldn't have the right credentials everywhere the test was run.

Because of my relatively complicated directory structure, I was struggling to find exactly the right module to mock. You might have read this relatively simple where to mock section in the unittest docs but here is how to figure where to mock out for any scenario, no matter how complicated.

I first commented out the call to the AWS client in MyClient.py. As it was called globally the NoCredentialsError was causing all my tests to fail before any methods or tests were executing 😵‍💫. Then I inspected the sys.modules dict containing all the modules that your test file can see and the path to them.

# test_app.py
import sys
import json

def test_lambda_handler():
    print(sys.modules)

    assert True == False

It'll show your own modules at the bottom.

...
    "boto3.resources.factory": <module "boto3.resources.factory" from "/usr/local/lib/python3.11/site-packages/boto3/resources/factory.py">,
    "boto3.session": <module "boto3.session" from "/usr/local/lib/python3.11/site-packages/boto3/session.py">,
    "boto3": <module "boto3" from "/usr/local/lib/python3.11/site-packages/boto3/__init__.py">,
    "hello_world.util": <module "hello_world.util" from "/Users/sophiewarner/repos/project/hello_world/util.py">,
    "hello_world.service.my_client": <module "hello_world.service.my_client" from "/Users/sophiewarner/repos/project/hello_world/service/slack_client.py">,
    "hello_world.app": <module "hello_world.app" from "/Users/sophiewarner/repos/project/hello_world/app.py">
}

From here you can just replace what sys thinks is at that path with a mock.

# test_app.py
import sys
import json
from unittest.mock import MagicMock

sys.modules["hello_world.service.my_client"] = MagicMock()
from hello_world import app
event = {...}

def test_lambda_handler(event):
    ret = app.lambda_handler(event, "")
    assert ret["statusCode"] == 200

This will update sys.modules for your entire python session. An improvement on this would be patching it just for one test.

# test_app.py
import sys
import json
from unittest.mock import MagicMock, patch

event = {...}

with patch.dict("sys.modules", {"hello_world.service.my_client": MagicMock()}):
def test_lambda_handler(event):
    from hello_world import app
    ret = app.lambda_handler(event, "")
    assert ret["statusCode"] == 200

✅ No more mocking and patching headaches.