How I Set Up Testing for My Python Project

After setting up static analysis tools last week, it's time to configure a testing framework for Continuous Integration (CI). There are several options for Silkie, my work-in-progress static site generator, but I decided to give Pytest a try. In this blog, I'll show you how I set up:

1. pytest - a Python testing framework
2. pytest-watch - a CLI tool for running tests automatically on changes
3. pytest-cov - a Pytest's plugin for producing code coverage reports

Pytest

I find Pytest easy to set up tests without too much boilerplate code and to add more functionalities with many extensions. You can add it to your project with a single installation command:

$ pip install -U pytest

You can check whether it's already installed with:

$ pytest --version

You can write your test as easy as adding a Python function. First, create a file whose name starts with test_ or ends with _test. Pytest automatically discovers those tests if you name them according to their convention. In my case, I created test_get_file_name.py for testing this function:

def get_filename(file_path: str) -> str:
    """Extract the name of the file from a file path and exclude any file extension"""
    return Path(file_path).stem.split(".")[0]

Then, simply add your test cases as Python functions. For instance, I tried to test whether my get_filename() function works on a file path of both Windows and Unix. I also wanted to check if it can exclude multiple file extensions from the file name:

def test_windows_file_path():
    expected_file_name = "test"
    file_extension = "txt"
    file_path = fr"C:\Documents\{expected_file_name}.{file_extension}"
    file_name = get_filename(file_path=file_path)

    assert file_name == expected_file_name


def test_unix_file_path():
    expected_file_name = "test"
    file_extension = "txt"
    file_path = f"/Users/anonymous/Documents/{expected_file_name}.{file_extension}"
    file_name = get_filename(file_path=file_path)

    assert file_name == expected_file_name


def test_file_path_multiple_extensions():
    expected_file_name = "test"
    file_extension = "rc.txt"
    file_path = f"/Users/anonymous/Documents/{expected_file_name}.{file_extension}"
    file_name = get_filename(file_path=file_path)

    assert file_name == expected_file_name

I also configured Pytest to only search for tests in my tests folder by specifying it in pytest.ini file:

[pytest]
minversion = 6.0
testpaths = tests

Then, I can finally run my tests with this command:

$ pytest

Unexpectedly, I caught an error in my code. The test failed when a Windows file path is passed to the function:

====================================================================================================== test session starts =======================================================================================================
platform darwin -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /Users/ptpham/Projects/silkie
collected 3 items                                                                                                                                                                                                                

tests/unit/test_get_file_name.py F..                                                                                                                                                                                       [100%]

============================================================================================================ FAILURES ============================================================================================================
_____________________________________________________________________________________________________ test_windows_file_path _____________________________________________________________________________________________________

    def test_windows_file_path():
        expected_file_name = "test"
        file_extension = "txt"
        file_path = fr"C:\Documents\{expected_file_name}.{file_extension}"
        file_name = get_filename(file_path=file_path)

>       assert file_name == expected_file_name
E       AssertionError: assert 'C:\\Documents\\test' == 'test'
E         - test
E         + C:\Documents\test

tests/unit/test_get_file_name.py:10: AssertionError
==================================================================================================== short test summary info =====================================================================================================
FAILED tests/unit/test_get_file_name.py::test_windows_file_path - AssertionError: assert 'C:\\Documents\\test' == 'test'
================================================================================================== 1 failed, 2 passed in 0.19s ===================================================================================================

I decided to apply a quick fix to the function:

def get_filename(file_path: str) -> str:
    """Extract the name of the file from a file path and exclude any file extension"""
    ...
    # Replace any backslash(es) in Windows file path with forwardslash(es)
    file_path = file_path.replace("\\", "/")
    return Path(file_path).stem.split(".")[0]

It made the test passed 🥳! With pytest working, let's set up pytest-watch so my tests can run automatically whenever I make some changes to my source code.

Pytest-watch

pytest-watch is a zero-config CLI tool that runs pytest, and re-runs it when a file in your project changes

If you don't want to run your tests manually every time you update your code, you can install this tool with this command:

$ pip install pytest-watch

Then, you just need to run the tool in the root directory:

$ ptw

Pytest-cov

pytest-cov is a pytest's plugin that produces coverage reports. If you want to see how much your tests have covered your codebase, you can install this tool by running this command:

$ pip install pytest-cov

Once the package is installed, you can run pytest-cov by adding --cov=<your-root-folder> to your pytest command. In my case, I'd also like to see which lines of code are not covered by running this command:

$ pytest --cov-report term-missing  --cov=silkie

Conclusion

Setting up a testing framework isn't as complicated as I expected, but it's certainly beneficial to my project's quality.