Testing Go CLI Tools

Testing Go CLI Tools (part 1)

This has all started from a very simple tweet from Mat Ryer

I don’t use func main In #golang 😂

But seriously, I always make a new little “run” function and then immediately call that, passing in args and io.Writer for stdout etc.

It’s the easiest way I’ve found to make func main testable. https://t.co/8BQ0EEmU5x

— mat_ryer.go (@matryer) February 12, 2020

What is going on?

Interlude in to my testing journey

So I have a confession to make - I like testing my code. I read about it first in Joel Spolsky’s book “Joel on Software”. Unit testing was a thing that I did a little 20 years ago when I was writing C for mobile phones and then C++ for Windows applications (before there were Apps). There was no common unit testing framework that people always looked to, we just cobbled it together with what we had.

Then I moved to a company that did C# and discovered that there WERE frameworks for you to use. In this case, NUnit. Well, that was a revelation. Everything was tested, every code path and every combination of if statement. It was great to feel like your code worked!

Lets move forward a few years. Unit testing again in C++ with CPPUnit, meeting actual people whose full time job it was to test your software. I even learned some of the theory behind testing, passing a proper QA certification - ISTQB. Only the basic one mind, but it was a start. TDD became a thing that I tried and failed to do. Testing still involved all the code paths.

So now we arrive at the present day. I’ve become less bad. I do TDD before I’ve even created the file that the Production code will go in. I test behaviours and not implementation. Occasionally I even make it so I can refactor my code without needing to change the tests. I’m not so worried about getting all of the code paths checked multiple times but I still care when there are gaps.

Testing CLI tools

It’s actually quite hard to test everything in a command line tool. When you think about it, how is a Unit Testing framework going to help you to parse command line arguments? What happens when some of the output goes to STDOUT? What about STDERR? And how do we get any kind of code coverage statistics?

BATS

My first attempt at solving this uses a great framework called BATS. This is essentially a Bash based Unit Testing framework. You can use this to to make calls to you CLI tool and validate the output. You can see how I’ve done this in a couple of places:

1. goclitem: a simple, opinionated, Go CLI application template
2. abbreviate: a tool for abbreviating long strings (it’s a pre-cursor to goclitem, which is why the structure is a little different)

Here we can see that I’m building the tool and then actually invoking it, testing my expectations against the real fully built tool. It’s great. BUT…

• We can’t check the code coverage
• We can’t use the nice testing tools that Go and 3rd party packages provide

Calling main()

This was the next obvious approach. This didn’t work that well. There was none of the control offered by the regular unit tests but there was difficulty intercepting the output.

Using run

Now we get on to Matt Ryer’s comment. What is it trying to do? Essentially this allows us to have an entry point to the CLI without having be locked in to main. In part 2, I’ll dig in to how this is used.