Introduction
You started programming with Python, write code and solve problems. On this article we'll discuss how you can write tests for your code, you can run them quickly to make sure everything works instead of running your app over and over to make sure your changes work.
Our program and making sure it works
We will use a very simple example that will help you get the idea of the problem you may face and the solution tests can bring.
The idea is for the example to be super simple (dumb even) so you can focus on the new concept instead of on the example code iteself.
Let's say we have this program where we ask the user for two numbers and we multiply them.
# file: main.py
from mymath import multiply
print("Mutliply two numbers")
a = input("first number: ")
b = input("second number: ")
print("result:", multiply(int(a), int(b)))
Again, this is a simple program, in the real world you could be asking for data relevant to any problem and do complex calculations with them.
And say we have this implementation for multiplication:
# file: mymath.py
def multiply(a, b):
# sum "a" "b" times, 2*3 -> 2+2+2
result = 0
for _ in range(b):
result += a
return result
Let's run the app on the command line:
$ python3 main.py
Mutliply two numbers
first number: 2
second number: 3
result: 6
$ python3 main.py
Mutliply two numbers
first number: 5
second number: 6
result: 30
So far so good, let's try a negative number:
$ python3 main.py
Mutliply two numbers
first number: -4
second number: 4
result: -16
Still works, let's try again:
$ python3 main.py
Mutliply two numbers
first number: 4
second number: -4
result: 0
Alright, we found a bug!
Testing our program with another program
When your program is larger and there are many more options than just one math operation, it will take you much more program runs to actually find some cases that are not working.
And more importantly, you can test all the things in a blink every time you make changes on your app without having to worry if your change broke something that you didn't notice.
Let's write a little program that do this testing for us:
# file: mymath_tests_v1.py
from mymath import multiply
if (multiply(2, 2) != 4):
raise Exception("result error")
if (multiply(2, 4) != 8):
raise Exception("result error")
if (multiply(2, -2) != -4):
raise Exception("result error")
Let's run it:
$ python3 mymath_test_v1.py
Traceback (most recent call last):
File "mymath_test_v1.py", line 10, in <module>
raise Exception("result error")
Exception: result error
Now, we are checking the same thing that we would check running the app, but with code. And as added value, we are documenting the use cases we support on our app, and the ones that are more relevant when we want to make sure that the program works.
This is basically what we can call a "test", or "unit test". Code that checks that make sure that your code works. There are other types of tests, for other purposes, but that's out of the scope of this article.
Improving our tests
The approach we just explored was only a crude implementation of tests using just conditionals, let's take it a step further.
Instead of manually writing conditionals and raising exceptions, we'll make use of the assert keyword.
It basically works like this:
$ python3
>>> assert(2 == 2) # no error
>>> assert(2 == 3) # throws exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError
Here's how we can use it for our tests:
# file: mymath_tests_v2.py
from mymath import multiply
assert(multiply(2, 2) == 4)
assert(multiply(2, 4) == 8)
assert(multiply(5, 6) == 30)
$ python3 mymath_tests_v2.py
Traceback (most recent call last):
File "mymath_tests_v2.py", line 5, in <module>
assert(multiply(2, -2) == -4)
AssertionError
As you may have noticed, not only our code is shorter but when an assertion fails, there's information on the exception about why it failed and instead of just getting the line where the problem appeared we get the expression we run, and that makes figuring out the problem much easier.
Improving our tests, using a framework
Testing can be improved even further, there are frameworks (sometimes called libraries, modules or packages) that provide some extra tools for us to make writing and running tests even easier, as well as providing much nicer results for tests that pass and fail.
Python comes with its own framework to run tests, it's called unittest. See its documentation
Here's how we would write our test using it:
# file: test_mymath_simple.py
import unittest
from mymath import multiply
class MyMathTest(unittest.TestCase):
def test_basics(self):
self.assertEqual(multiply(2, 2), 4)
self.assertEqual(multiply(2, 4), 8)
self.assertEqual(multiply(2, -2), -4)
If you're not familiar with class you can ignore the details and instead of writing MyMathTest you can write whatever you want, and all the functions have to start with test_.
You have many self.assert prefixed helper functions to test different things, we can stick with equality comparison for now.
Let's run these tests:
$ python3 -m unittest -v test_mymath_simple.py
test_basics (test_mymath_simple.MyMathTest) ... FAIL
======================================================================
FAIL: test_basics (test_mymath_simple.MyMathTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/ivan/data/Devel/blog/test_mymath_simple.py", line 9, in test_basics
self.assertEqual(multiply(2, -2), -4)
AssertionError: 0 != -4
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
We are getting more output (partly because of the -v flag) and some of it is valuable: for example 0 != -4, that's exactly the condition that failed. Results are more explicit for us, so it's much easier to figure out the problem.
Remember, here's easy because of our really small example, but on larger programs gets much harder.
Using unittests like in the real world
# file: test_mymath.py
import unittest
from mymath import multiply
class MyMathTest(unittest.TestCase):
def test_small(self):
self.assertEqual(multiply(2, 2), 4)
self.assertEqual(multiply(2, 4), 8)
self.assertEqual(multiply(10, 10), 100)
def test_zero(self):
self.assertEqual(multiply(2, 0), 0)
self.assertEqual(multiply(0, 2), 0)
def test_negative(self):
self.assertEqual(multiply(-2, 2), -4)
self.assertEqual(multiply(-2, -2), 4)
self.assertEqual(multiply(2, -2), -4)
On this example, we have different test "groups", in which we make sure different aspects of our program behave as it should.
Those are run separately and we'll get information about which of them work and which of them fail.
Let's run our new test "suite".
$ python3 -m unittest -v test_mymath.py
test_negative (test_mymath.MyMathTest) ... FAIL
test_small (test_mymath.MyMathTest) ... ok
test_zero (test_mymath.MyMathTest) ... ok
======================================================================
FAIL: test_negative (test_mymath.MyMathTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/ivan/data/Devel/blog/test_mymath.py", line 17, in test_negative
self.assertEqual(multiply(-2, -2), 4)
AssertionError: 0 != 4
----------------------------------------------------------------------
Ran 4 tests in 0.001s
FAILED (failures=1)
You can have many test files, and as long as they start with test_ you can run python3 -m unittest -v and it'll run all your test, wherever they are.
Closing thoughts
Tests allow you to verify your program quickly, frequently and consistently. Using them you can have more confidence on your app and on the changes you make to it.
There's much more to learn about testing, but hopefully this introduction will be enough for you to get started testing your code.
Thre are several third party libraries that provide very nice functionalities on top of what you get from the standard library.
Once you're comfortable writing tests, you can give some of them a try and see how do you like them.
Top comments (3)
What exactly is the point of these tests? It's more code and does not do anything which I couldn't do in PyCharms debugger, right? Everyone is talking about tests, but honestly, I don't get it. Sorry if that's a stupid question - I am currently starting with programming and Python....
Oh, no worries, I appreciate the question.
You're right about pointing out that it's more code, and that's actually something very important to have in mind because it's more code that has to be maintained and made sure it tests what we want to test.
The debugger is a very important tool, but is used to solve a different problem than tests.
You usually would use the debugger to run your application step by step, to help you figure out where a bug might be, check variable values on those steps and so on.
I think that the most important difference is that you can run your app manually or you can use the debugger, but you would check one use case at the time, with tests you can check many many use cases very quickly.
Manually you would check for one scenario, then another, and then another... Using tests you would define those scenarios beforehand, and then check all of them at once, as many times as you want.
So, tests do something you couldn't do manually or the debugger, verify a lot of cases in a fraction of the time :)
Hopefully this explanation helps.
If this still doesn't make much sense let me know.
Thanks, that makes sense!