Testing in Go: testing floating point numbers

This is cross-posted from my blog.

I've been working on a library that includes some vector manipulations in Go, trying to follow good development practices and starting with writing my tests first. But early on, I ran into a bit of a problem: floating points.

The problem

Now, we know that floating points are an issue in programming, by the very nature of how they're represented in memory. I loved Julia Evans's brief explanation of how floats work in her Linux Comics Zine (scroll down to the sixth panel), and for a more in-depth explanation, see this piece by Carl Burch.

My problem: after writing my implementation of a vector addition function, I couldn't get the tests to pass, because comparing equality of floating point numbers is...hard. I'd be comparing an expected result of {2,3} to an actual result of {1.99999999, 3.00000000002}. Those values don't pass muster for traditional equality.

I looked up best practices of how to unit test functions that returned floating point numbers and overwhelmingly found the recommendation of "don't". (Not helpful, y'all.) One recommended using a tolerance -- that is, if the absolute difference between the expected number and the actual number was under a specified tolerance, you counted them as being equal.

That was pretty easy to implement, and it looked like this:

got := Add(tt.vectors)
if math.Abs(got-tt.want) > tolerance {
    t.Fail("Got %v, expected %v", got, tt.want)
}

Defining a comparer

However, I recently started to reimplement my library, and in the process switched over to testing using the cmp package. The cmp package is a much more robust way of testing for equality, and in exploring it I found that you can define a comparer function that makes this a lot cleaner. This is the comparer function from the documentation, which uses a bit more refined logic than my initial one above:

const tolerance = .00001
opt  := cmp.Comparer(func(x, y float64) bool {
    diff := math.Abs(x - y)
    mean := math.Abs(x + y) / 2.0
    return (diff / mean) < tolerance
})

Then, when you run your test, you use the comparer function when testing for equality:

if !cmp.Equal(got, tt.want, opt) {
    t.Fatalf("got %v, wanted %v", got, tt.want)
}

Your tests are much more clean, and you can easily use a comparer function that works for your use case.

Postscript

However... you have to be careful with your comparer function! The one above is straight from the cmp docs. But straight away I ran into a test failure that I didn't expect: when trying to perform operations on zero vectors, it failed, hard. Which led to this exchange between me and my tests:

Frustrated tweet, Twitter

Yep, when all the values in a vector are zero, I wind up dividing by zero. So my actual comparer function looks like this:

opt  := cmp.Comparer(func(x, y float64) bool {
    diff := math.Abs(x - y)
    mean := math.Abs(x + y) / 2.0
    if math.IsNaN(diff / mean) {
        return true
    }
    return (diff / mean) < tolerance
})

Of course, the same logic (diff / mean) would have had the same issue without using the cmp comparer option, so this amusing story is about the general case.

But, that said, it took 8 lines of code to make my tests actually test what I care about. So if you're doing scientific computation, don't forget about your comparer function!

Go test!

All code in this blog post is released under the Apache 2.0 license and can be found on GitHub.

Comments

[Adrian B.G.]

Nice, in Unity3D (C#) we used a function that is builtin in the engine:

 // Compares two floating point values if they are similar.
        public static bool Approximately(float a, float b)
        {
            // If a or b is zero, compare that the other is less or equal to epsilon.
            // If neither a or b are 0, then find an epsilon that is good for
            // comparing numbers at the maximum magnitude of a and b.
            // Floating points have about 7 significant digits, so
            // 1.000001f can be represented while 1.0000001f is rounded to zero,
            // thus we could use an epsilon of 0.000001f for comparing values close to 1.
            // We multiply this epsilon by the biggest magnitude of a and b.
            return Abs(b - a) < Max(0.000001f * Max(Abs(a), Abs(b)), Epsilon * 8);
        }

"approximately" equal was implemented in all types that had floating precision, including 2D and 3D vectors, you never need the "normal" equality because things were never exactly the same, especially when objects were affected by forces in a physics simulation like a physics engine.

I think epsilon can be calculated in Go using the formula (a very small floating value)

epsilon := math.Nextafter(1.0,2.0)-1.0

[julia ferraioli]

That makes tons of sense to have an approved way to do it in Unity3D, considering how critical floating point precision is to physics simulations! (I'm looking at you, KSP.)

Yeah, there are likely far better comparison functions -- I'm still playing around with one that'll work for me. For this though, I was just demonstrating how to use the comparer option, and simple is always best for blog posts :-)

Thanks for sharing!

[Adrian B.G.]

Definitely, but your post made me curious how that function works, I used it but never saw its implementation.

Now I'm curious what is the most optimal way in Go to do it, to compare the first X decimals in a float, in Go, so your post worked! it made me a better dev.

[julia ferraioli]

🎉