Avoid the Code Crawl by Reading Tests

#tdd #tests #opensource #documentation

TL;DR

Test driven development is fantastic
Reading tests can supplement/replace documentation
Go read tests.

A story as old as time

As a developer, you likely rely on libraries, packages, modules, etc. When you do, the documentation is critical to understanding and using the code effectively. However, there are often situations where the documentation is incomplete or non-existent.

Your natural instinct is to sigh and start digging through code hoping to find any comments or obvious code snippets to help you accomplish your goal.

Depending on how well the the library is maintained this might be a quick struggle and you'll find the glorious use_me_and_only_me_init(), but more than likely you have found a collection of functions and chunks of data that have been cobbled together that loosely do what you want them to.

There has got to be a better way to do this, and depending on the presence of tests there is!

What are tests?

For those of you who are new to tests or test driven development (TDD), tests are sections code to test code. This is done by breaking functionality down, throwing data at it, then checking the output.

If I have a test for two plus two it should always assert four, if it doesn't then you can't push your code until it's fixed or addressed in the issue queue. This prevents new code from breaking old features/logic.

The great thing about TDD is, if done properly, everything gets tested and that means the developer accidentally creates examples of their code exactly how it is supposed to be used.

In documentation, developers have a tendency to document and give examples for things they think are important, as a result a lot of features can go undocumented(This can be mitigated by using things like Doxygen, but that's a different article). With test cases everything has to be covered.

What about a real life example?

I have done this in almost every language/platform that I've developed in but in this case I just went through the Awesome-Go repo until I found a project that had weak documentation.

I came across an FTP client to be used as a package. First thing you notice is that the documentation is limited to the simple installation instructions and the GoDocs.

Going through the ftp package in ftp.go you can quickly see that there is no obvious list of operations outside of the occasional comment. This is what a developer commonly sees when digging through library code.

// Connect is an alias to Dial, for backward compatibility
func Connect(addr string) (*ServerConn, error) {
  //...
}

// Dial is like DialTimeout with no timeout
func Dial(addr string) (*ServerConn, error) {
  //...
}

// DialTimeout initializes the connection to the specified ftp server address.
//
// It is generally followed by a call to Login() as most FTP commands require
// an authenticated user.
func DialTimeout(addr string, timeout time.Duration) (*ServerConn, error) {
  //...
}

At this point we can go through and start throwing paint on the walls to see if we can get something to work, or we can look at the tests.

Navigating through the *_test.go files you can quickly see that client_test.go is the file we want to be looking at. Now, I know people out there might not write in Go but this idea is universal so bare with me.

Looking through the file you can see function after function named things like TestConnPASV(), TestConnEPSV(), and testConn(). Almost immediately we see which one would most likely interest us, testConn().

// Author's Note: Modified to be simpler.
func testConn(t *testing.T, passive bool) {

    c, err := DialTimeout("localhost:21", 5*time.Second)
    if err != nil {
        t.Fatal(err)
    }

    err = c.Login("anonymous", "anonymous")
    if err != nil {
        t.Fatal(err)
    }

    err = c.ChangeDir("incoming")
    if err != nil {
        t.Error(err)
    }
        //...
}

It becomes pretty clear how to start an FTP connection, run commands, and use helper methods like ChangeDir() and List(). If you pay extra attention you can also see how to implement things like error checking for bad logins as well as IPv6.

// Author's Note: Modified to be simpler.
func TestWrongLogin(t *testing.T) {
    c, err := DialTimeout("localhost:21", 5*time.Second)
    if err != nil {
        t.Fatal(err)
    }
    defer c.Quit()

    err = c.Login("zoo2Shia", "fei5Yix9")
    if err == nil {
        t.Fatal("expected error, got nil")
    }
}

After a whopping 45 seconds we are now ready to take those sections of code and modify them a bit to work for our use case.

Due to weak documentation this could have taken up lots of precious development time with lots of frustration. But thanks to well written tests we learned how to implement this package quickly and accurately.

Takeaways

There is something that has to be said about using tests as documentation. It isn't always going to work, sometimes developers write bad or incomplete tests. But that shouldn't stop you from experimenting with this type of development.

Next time you're up to your neck in undocumented code, look at the tests!