Chapter 1: The Need For Test Automation

• This chapter is part of the series:
  

  Understanding the Python Tox Ecosystem

  Namah Shrestha ・ Oct 21 ・ 2 min read

  #python #tox #testing #pytest
• At first, we need to understand, how we can test python code without a testing library.
• We will then look into how to test with an existing library.
• Then we will look at how tox fits in the ecosystem.

1.1 Testing Without A Library

• Consider the following code structure of a Python application:
  

  - README.md
  - LICENSE
  - .gitignore
  - app.py
  

• We may have various assert statements written inside app.py. However, these tests need to be executed manually by calling the function.

• Consider the following code for app.py.
  

  def simple_calculator_function(evaluation_string: str) -> typing.Any:
    '''
    Reads an evaluation string.
    Evaluates it and returns the result.
    For example, eval("5*(4+5)") = 45.

    params:
        evaluation_string: str: The string that will be evaluated.
    '''
    try:
        return eval(evaluation_string)
    except Exception as e:
        raise Exception(e)

   def test_simple_calculator_function() -> None:
        assert simple_calculator_function("5*(4+5)") == 45 # test addition and multiplication
        assert simple_calculator_function("10 - (100/2)") == -40 # test subtraction and division. 
        assert simple_calculator_function("'a' + 'b'") == 'ab' # test string concatenation

• As you can see, test_simple_calculator_function will not run automatically unless called. We want to automate this.

• The first step is to create a separate test_file which uses libraries to write tests. The new project code would be:
  

  - README.md
  - LICENSE
  - .gitignore
  - app.py
  - test_app.py
  

• We have added a test_app.py which will work on libraries.

1.2 USING AN EXISTING TESTING LIBRARY

• We have libraries such as the unittest library using which we can write test subclasses that are auto-detected and run automatically.

• Example unittest subclass:
  

  import unittest
  from app import simple_calculator_function
  
  class TestSomething(unittest.TestCase):
      def test_simple_calculator_function(self) -> None:
          self.assertEqual(simple_calculator_function("5*(4+5)"), 45)
          self.assertEqual(simple_calculator_function("10-(100/2)"), -40)
          self.assertEqual(simple_calculator_function("'a' + 'b'"), 'ab')
  

• We can run this code with unittest tool in the shell as:
  

  $ python -m unittest discover -s './' -p 'test_*.py'
  

  • Here, unittest discover discovers all the subclasses of unittest.TestCase, i.e. those classes that have inherited unittest.TestCase.
  • -s flag denotes the start directory of where the tests are located. In this case, we can stay in the current working directory.
  • -p flag denotes the pattern of the names of test files. In this case, the test file naming pattern is test_*.py , where * can be replaced with any name.

• We also have the pytest library which is not inbuilt and needs to be installed separately.
  
  

  $ pip install pytest

• For example, pytest test file.
  

  import pytest
  from app import simple_calculator_function
  
  def test_simple_calculator_function() -> None:
      assert simple_calculator_function("5*(4+5)") == 45 # test addition and multiplication
      assert simple_calculator_function("10 - (100/2)") == -40 # test subtraction and division. 
      assert simple_calculator_function("'a' + 'b'") == 'ab' # test string concatenation
  

• pytest can autodetect tests. We can just run pytest in the shell and all tests will run.
  

   $ pytest

• The test detection algorithm also known as the test discovery standard is is used to detect tests by default unless external modifications are made.
• There are other testing frameworks such as Nose, DocTest, etc. which will not be covered in this article.

1.3 TOX USE CASE IN THE ECOSYSTEM

• Tox is just another automation tool for testing.
• What exactly does it automate?
  • Since we are building a package, we need to make sure that our package runs on different versions of Python.
  • The tests that we have in pytest will run on the local version of the virtualenv that we are currently on. So, it will only check one version of python.
  • We need to make sure these tests run passes on all versions of Python.
  • This is what Tox automates. Multi-version testing.
• How does Tox do this?
  • Tox creates a virtual environment for each version of python and/or the operating system that we specify.
  • It runs the tests in all those environments and makes sure that our package works on those versions as well.
• When we run things on github actions :
  • We need to map Tox commands to github actions , essentially because github actions is completely capable of doing what Tox is doing.
  • We are only installing Tox to automate testing locally on our machines while developing.
  • Since we have a working Tox file, why create another github action section for the thing already automated in Tox.
  • We just map Tox environments to github action environments.