loading...
Cover image for Writing Good Unit Tests: A Step By Step Tutorial

Writing Good Unit Tests: A Step By Step Tutorial

ice_lenor profile image Elena Updated on ・7 min read

Unit testing (3 Part Series)

1) Unit testing: best practices 2) Writing Good Unit Tests: A Step By Step Tutorial 3) Unit Testing: Mocks

This post was originally published in my blog smartpuffin.com.


Let's imagine we just wrote a method calculating distance between two points on our planet. And let's imagine we want to test it as well as possible. How do we come up with test cases? And what exactly do we need to test?

Bonus: learn a surprising fact about the Fiji islands. 🇫🇯

Prerequisites

I assume you are already familiar with the concepts of unit testing. I'll be using Java and JUnit, but don't worry if you're not familiar with them: the purpose of this tutorial is to learn to write fuller test suites. You'll pick up platform-specific things on the fly.

Here is a link to the repository where I have put the full source code for this post. You can download it and look at it while reading, or look at it afterwards.

And here is my previous article on unit testing topic explaining best practices in unit testing more widely.

Let's go!

Positive cases

Let's start with thinking about our method as of a black box.

We have this API:

// Returns distance in meters using haversine formula
double getDistance(double latitude1, double longitude1, double latitude2, double longitude2);

What can we test?

Putting yourself in a testing mindset

You need to switch from being a developer to being a tester now. This means you have to stop trusting yourself. You can't know your code is working if you didn't test it!

If you tested it manually, you don't know if it's going to work next time someone makes a change - in your code or close to it.

Moreover, you can be sure that whatever code you build on top of this well-tested piece, you have one less problem to worry about.

The feeling of security you have when your code is well-tested is incredible. You can refactor freely and develop new features further - and be sure your old code is not going to break.

Promise yourself a treat if you find a bug. But do be careful: too many cookies are bad for your health.

First test

Okay, we are ready to test! What should we test first?

The very first thing that comes to mind, is:

Calculate the distance between two points and compare with the true real number calculated manually. If they match, all is well!

And this is a great testing strategy. Let's write some tests for that.

I'll pick a couple of points: one in the very north of the Netherlands, one - in the very south. Practically, I want to measure the country from top to bottom.

I clicked at a couple of points on the map and got this:

Point 1: latitude = 53.478612, longitude = 6.250578.
Point 2: latitude = 50.752342, longitude = 5.916981.

I have used an external website to calculate the distance and got 304001.021046 meters. That means the Netherlands are 304km long!

Now, how do we use this?

Thinking about domain area

We need to think about domain area-specific things. Not all of them are obvious from the code: some of them depend on the area, some - on the future plans.

In our example, the returned distance is in meters, and it's a double.

We know that a double is prone to rounding errors. Moreover, we know that the method used in this calculation - the haversine formula - is somewhat imprecise for our oh-so-complex planet. It is enough for our application now, but perhaps, we'll want to replace it with higher-precision calculation.

All this makes us think that we should compare our calculated value with our expected value with some precision.

Let's pick some value that is good for our domain area. To do that, we can remember about our domain area and requirements again. Would 1mm be fine? Probably it would.

And here is our first test!

double Precision = 0.001; // We are comparing calculated distance using 1mm precision
...
@test

void distanceTheNetherlandsNorthToSouth() {
    double distance = GeometryHelpers.getDistance(53.478612, 6.250578, 50.752342, 5.916981);
    assertEquals(304001.0210, distance, Precision);
}

Let's run this test and make sure it passes. Yay!

More tests

We calculated the distance across the Netherlands. But of course, it is a nice idea to double-check and to triple-check.

Your tests should vary - this helps you cast a wider net for these pesky bugs!

Let's find some more nice points. How about we learn how wide Australia is?

@test

void distanceAustraliaWestToEast() {
    double distance = GeometryHelpers.getDistance(-23.939607, 113.585605, -28.293166, 153.718989);
    assertEquals(4018083.0398, distance, Precision);
}
Or how far is it from Capetown to Johannesburg?
@test

void distanceFromCapetownToJohannesburg() {
    double distance = GeometryHelpers.getDistance(-33.926510, 18.364603,-26.208450, 28.040572);
    assertEquals(1265065.6094, distance, Precision);
}
Now that we think about that: are we sure that the distance doesn't depend on the direction in which we are calculating? Let's test that as well!
@test

void distanceIsTheSameIfMeasuredInBothDirections() {
    // testing that distance is the same in whatever direction we measure
    double distanceDirection1 = GeometryHelpers.getDistance(-33.926510, 18.364603,-26.208450, 28.040572);
    double distanceDirection2 = GeometryHelpers.getDistance(-26.208450, 28.040572, -33.926510, 18.364603);
    assertEquals(1265065.6094, distanceDirection1, Precision);
    assertEquals(1265065.6094, distanceDirection2, Precision);
}

Corner cases

Great, we have covered some base cases - the main functionality of the method seems to work.

We can have a cookie now - for going half-way through. 🍪

Now, let's stretch our code to the limit!

Thinking about the domain area helps again.

Our planet is rather a ball than a plane. This means that there is a place where latitude goes from 180 to -180, leaving a "seam" where we should be careful. Math around this area often contains mistakes. Does our code handle this well?

A good idea would be to write a test for it, don't you think?

Screenshot of a map with a two points on it around the Fiji islands

@test

void distanceAround180thMeridianFiji() {
    double distance = GeometryHelpers.getDistance(-17.947826, 177.221232, -16.603513, -179.779055);
    assertEquals(351826.7740, distance, Precision);
}
Upon further thinking, I discovered one more tricky case - distance between two points which have the same latitude; but the longitude is 180 in one point and -180 in the other.
double distance = GeometryHelpers.getDistance(20, -180, 20, 180);
// Question to you, my reader: what is this distance?
Okay, we're done with that weird 180th meridian. We are sure about the 0th meridian though... right? Right? Let's test it too.
@test

void distanceAround0thMeridianLondon() {
    double distance = GeometryHelpers.getDistance(51.512722, -0.288552, 51.516100, 0.068025);
    assertEquals(24677.4562, distance, Precision);
}
What other cases can we come up with? I thought a bit and listed these:
  • How about poles? Let's calculate a couple of distances in the Arctic and Antarctica.
  • How about from one pole to another?
  • How about max distance on the planet?
  • How about a really small distance?
  • If both points are absolutely the same, are we getting 0 m distance?
I'm not adding all these tests here, as it would take too much space. You can find them in the repo. You get the point. Try to think creatively. It's a game of coming up with as many ideas as possible. And don't be afraid with coming up with "too many ideas". The code is never "tested too well".

What's inside

At this point, it can also help to peek inside the code. Do you have the full coverage? Maybe there are some "ifs" and "elses" in the code that may give you a hint? Do google "haversine formula" and look at its limitations. We already know about precision. Is there something else that could break? Is there some combination of arguments which can make the code return an invalid value? Brainstorm about it.

Negative cases

Negative cases are cases when the method is supposed to refuse to do its job. It is a controlled failure! Again, we should remember about the domain area. Latitude and longitude are special values. Latitude is supposed to be defined in exactly [-90, 90] degrees range. Longitude - in [-180, 180] range. This means that in case when we passed an invalid value, our method throws an exception. Let's add some tests for that!
@test

void invalidLatitude1TooMuch() {
    assertThrows(IllegalArgumentException.class, () -> {
        GeometryHelpers.getDistance(666, 0, 0, 0);
    });
}
@test

void invalidLatitude1TooLittle() {
    assertThrows(IllegalArgumentException.class, () -> {
        GeometryHelpers.getDistance(-666, 0, 0, 0);
    });
}

Same tests we'll add for latitude2, and for both longitude parameters.

We're writing Java here, and the double parameters can't be null. But if you're using a language where they can, test for it!

If you're using a dynamically typed language, and you can pass a string instead of the number, test for it!

Side point: dynamically typed languages will require some more extensive testing than statically typed ones. With the latter, compiler takes care of many things. With the former, it's all in your hands.

Wrapping up

You can see and download the source code for this tutorial here. Look at it and try to come up with some more useful testing scenarios.

ExampleUnitTests

This is the source code for the tutorial about writing good unit-test suites.

Tutorial

See the tutorial here:

http://smartpuffin.com/unit-tests-tutorial/

What to do with this

  1. Open the tutorial.
  2. Open GeometryHelpers.java.
  3. Open GeometryHelpersTest1.java. Review the test structure. Run the test.
  4. Open GeometryHelpersTest2.java.
  5. Compare the first test in GeometryHelpersTest1.java with the first test in GeometryHelpersTest2.java. See how even the smallest pieces of domain area knowledge are important?
  6. Think about what else you could test. I'm sure there is something I missed!
  7. Apply the best practices in your own project.
  8. Like the tutorial? Share, like, subscribe, say thanks. I'll be happy to know it!

How to run tests

I run this with Intellij IDEA and Java 1.8+.

  1. Download the source code.
  2. Open the folder with the code in Intellij IDEA.
  3. Open the file GeometryHelpersTest1.java. You'll see green "Run test" buttons next to line numbers. Press the one next…

Here's some followup reading to learn about best practices in unit testing.

For your own projects, select the part that would benefit from testing the most - and try to cover it with tests. Think creatively! Make it a challenge, a competition with yourself!

Unit testing is fun!

Unit testing (3 Part Series)

1) Unit testing: best practices 2) Writing Good Unit Tests: A Step By Step Tutorial 3) Unit Testing: Mocks

Posted on Jan 24 '18 by:

Discussion

markdown guide
 

Nice post, thanks Elena! Another tactic I like to use when creating unit tests is to scan my code for control flow statements and try think of the possible cases that can come from those. A great example of this is languages that allow you to fall through switch statements, where small code changes can lead to big headaches if they're not thought through.

 

Thank you!
I absolutely agree with reading the code and looking for tricky places. I mentioned it in the "What's inside" section.
The switch falling-through is a really good example!

 

If you're doing Test Driven Development I'd expect to also see tests for short and long distances. That would demonstrate that your first test (the short one) was for a distance you measured with a ruler, then the second was added when you ran your code for two points a long distance apart, got the wrong result, and switched to an algorithm that knows we're not living on a plane.

 

I'm not sure I understand the argument about the plane. The distance calculated as if the coordinates are on a plane will never match the distance on the globe.

However, I do agree there should be tests for short and long distances! I mention them in the "Corner cases" section:

How about max distance on the planet?
How about a really small distance?
If both points are absolutely the same, are we getting 0 m distance?

 

The deviation from a plane is about 20cm per km, or one part in 5000. So provided that you're measuring between two points less than 5m apart (a few millionths of a degree apart) the distances do match, to within 1mm.

Incidentally, people may find this link interesting: ianvisits.co.uk/blog/2018/03/06/ho...

No, this is not the case. Here's a simple proof.

Say, we have two points on the equator.
Point 1: latitude 0, longitude 0.
Point 2: latitude 0, longitude 1.

Distance between them, measured as if they were flat coordinates: 1. (Units: degrees.)

Distance between them, measured as if they're on the planet Earth: 111.19 km.

111 (km) != 1 (degree).

Moreover, measuring distance in degrees doesn't make sense because one degree longitude is not equal to one degree latitude; and one degree longitude means different number of kilometres depending on latitude. The closer to the pole you go, the less kilometres "fit" in 1 degree longitude. (Note to self: I need to write an article about that and add pictures. It would be easier to explain with pictures.)


I suspect that you might be talking about specific projections, such as UTM, where for certain subset of coordinates distances between them, measured using the Pythagorean theorem, will roughly match the real on-the-planet-surface distance.

But if you're using such projection, your points' coordinates are also specified in that projection, and not in latitudes and longitudes.

To compare the distances, you'd have to first convert your point latitude and longitude to the coordinate system coordinates and then perform the calculation according to the Pythagorean formula. That's possible, but it complicates tests too much.

 

Good post and notes. Also, I like always to use naming standards: test+methodName+situation

 

Oh nice! Yes, I too believe that the name should be long and descriptive.

I wrote about more "mundane" things, such as naming or placing tests, in here:
dev.to/ice_lenor/unit-testing-best...

 
 

For this type of functionality, people had invented property testing to have unbiased input data.

 

I am struggling a bit to apply property-based testing in this case. Can you give me an example, please?

 

All the random tests with {lat, lon} ∈ {[-90, 90], [-180, 180]} should pass returning a value that is less than 20K km approx.
All the random tests with {lat, lon} not in the range above should pass returning a meaningful error.
Then you run N tests with a proper latlon, manually check values, record a seed and re-run tests after with the same seed.

Thank you for the example. This seems like a great "casting net" for all sorts of general bugs and problems.

Still, I'd add some manually-written tests on top of these. I wouldn't rely on randomised tests more than I would rely on my domain knowledge. For example, a set of randomised tests wouldn't be able to "tell" you about problems you might experience around 180th meridian or around poles. They don't know the reason behind these problems. A human, on the other hand, is able to (re)read the requirements, think critically and figure out all sorts of problems using a creative systematic approach.