DEV Community: Vincent Fahrenholz

Why TDD might not be that underrated

Vincent Fahrenholz — Thu, 17 Mar 2022 15:46:50 +0000

What triggers this post?

A while ago, the native web GmbH published this video on youtube (german language):

While I genuinely like the video and do not by any means oppose its content, I want to share a different point of view

Quick recap: What is TDD

TDD is the practice consisting in writing the test first and engaging in what is called the TDD-Cycle (also commonly known as Red-Green-Refactor-(Commit): Write a failing microtest, write the according program code, see the test going green, refactor your program code while still keeping the test green. While it can be severely head-damaging at first, with some practice you will find yourself not only producing better - that is, more testable, better designed and better isolated - code, you will also find yourself with a coherent guide to your development as well as an understandable documentation of your intent while developing afterwards.

The essentials of the video

In the video, Golo Roden says, in essence, that TDD is a great programming technique by any means, but falls short when exploring and learning. His core statement is that, in order to do TDD in a not-so-painful way, you need to know pretty well where you are going, or else you might find yourself needing to invest an excrutiating amount of energy refactoring your tests as you go.

A different point of view

First of all: this behaviour is by all means perfectly understandable, and is an often implemented one when coming to TDD. There's absolutely nothing wrong with it in my mind (I myself thought that way for a very long time), and I fully get that dogmatism won't lead you anywhere.

I happen to use TDD for almost everything, excluding maybe small, easy one-shot-scripts. I happen to use it a lot for exploration, and want, in the course of this blog post, to show how I deal with the (very much real) problem of having to adapt existing tests. My goal is to encourage you to try it out if you think that that can't possibly be a happy way of working. So let's dig in.

The most underrated feature of TDD

While it is pretty common knowledge that TDD will give you better structure and test coverage (I will come back to this point later though), there is one feature which I find missed in nearly every discussion I witness about TDD: the guidance which it brings to you while developing. To me, it is one of, if not the most singular, important benefits of the practice.

When I code, I need to get into a flow to be proficient. I manage this flow by managing my cadence: the more a problem I face is complex to solve, the more I slow down my cadence and decompose my work into smaller steps. TDD is the most efficient practice I've come across so far to enable me to do that, because I only ever write the test for the next step, and only ever work on this next step. When not using TDD, I find myself jumping back and forth, having trouble deciding what to focus on and ending up in yack-shaving more often than not. The cycle helps me to concentrate and to ask myself, on every step along the way, if this is really the thing I should solve now. This is the main reason why I don't want to deprive myself of it when exploring something: it channels and streamlines my work.

Changing tests and requirements along the way

I'll not lie to you: you will need to make some changes in your tests when you don't know what you get into. There are, however, several options you could explore in order to mitigate this problem.

Test-Tables

The "normal" way of implementing TDD is by writing microtests. Assuming, to take the example of the video, you want to implement a function isValid(IBAN string) , you will have several Tests which could look like this:

func TestIsValidIBANValid(t *testing.T) {
    given := "DE123456789123456789" //valid IBAN
    when := isValid(given)
    assert.True(t, when)
}

func TestIsTooShortIBANInvalid(t *testing.T) {
    given := "DE123" //invalid IBAN: too short
    when := isValid(given)
    assert.False(t, when)
}

func TestIsIBANWithoutLeadingCharsInvalid(t *testing.T) {
    given := "12123456789123456789" //invalid IBAN: missing chars
    when := isValid(given)
    assert.False(t, when)
}

There are however other possibilities to still leverage one test per test-case while grouping tests. In go, we use test-tables and table-tests for this, in PHP there is the data-provider which gives you the same possibilities. Other languages will have similar tools baked into the test-framework.

The concept is pretty simple: you define a test which is then executed in isolation for several, extensible outputs. The prior example will then look like this:

func TestIsValid(t *testing.T) {
    var table := []struct{
       testCase string
       given string
       then bool
    } {
      {testCase: "valid IBAN" given: "DE123456789123456789", then: true},
      {testCase: "invalid IBAN: too short" given: "DE123", then: false},
      {testCase: "invalid IBAN: missing chars" given: "12123456789123456789", then: false},
    }

   for _, tc := range table {
      t.Run(tc.testCase, func(t *testing.T) {
         when := isValid(tc.given)
         assert.Equal(t, tc.then, given)
      }
   }
}

This might not look like a microtest anymore, but: every test runs in isolation and checks one single condition. The only change is that you now only have one place where you need to make some changes when, for instance, you'll need to change requirements to include a validation on country level: you'll then need to pass a string countryCode to your function, which would result in this:

func TestIsValid(t *testing.T) {
    var table := []struct{
       testCase string
       given string
       withCode string
       then bool
    } {
      {testCase: "valid IBAN" given: "DE123456789123456789", withCode: "DE", then: true},
      {testCase: "invalid IBAN: too short" given: "DE123", withCode: "DE", then: false},
      {testCase: "invalid IBAN: missing chars" given: "12123456789123456789", withCode: "DE", then: false},
    }

   for _, tc := range table {
      t.Run(tc.testCase, func(t *testing.T) {
         when := isValid(tc.given, tc.withCode)
         assert.Equal(t, tc.then, given)
      }
   }
}

You'll have to rewrite your test-cases a bit, but not that extensively.

Test helpers

I don't seek to apply DRY principles to my tests too much. I find it way more useful in the long run to have my tests easy to be read in a linear way, and I like the profound documentation this adds to my code. Nevertheless some test helpers are very useful to me: when I instantiate a class for instance, I try to wrap this in a helper function in order to reduce the amount of changes I have to make if my instantiation has to change - for instance if I find out that I need another parameter passed to my constructor.

IDE tools

Changing a function signature is something which is well covered in IDE-tooling. If you use it, you can mostly identify at once where you need some adaptation.

Also, most IDEs (in most languages) will tell you about failures well before you compile if you bother looking. No need to even bother compiling before these errors are fixed.

Smaller steps

You need to adapt tests along the way, understood. The task gets easier the smaller the isolation layer you work upon is, as the scope of the change get's smaller. If your function is short enough, and does less enough things, You will have a smaller change delta. I typically start with the smallest possible thing and focus on broader API afterwards, when the smallest possible things are done.

Be pragmatic

Even when doing TDD you are allowed to deviate. If you sense that something will be a hassle as you might need to add several things, then don't test it right now. This is typically what I do for constructors as long as I don't know which objects I will have to pass to my class in the end.

So there's nothing overrated in TDD?

I would argue there is, but at a different level: I'm talking about the T of TDD.

A common misconception, in my opinion at least, is to see TDD as a testing technique. In my mind, it really is solely a development technique. The tests you produce during TDD are there to guide your development. They will result in a near-100% test-coverage, which means: every line of code (or most of them at least) will be covered by a test. This doesn't mean that your code implements the correct behaviour.

In order to make sure that this happens, you have to change perspective, and go from the development realm to the test realm. How might your code break? What edge-cases (both technical - off by one errors for instance - and logical - can my customer, whose IBAN I am trying to validate be living in a different country than the one in which their bank account lies? are austrian IBANs (they're shorter) validated correctly?) are there hidden in the problem you are trying to solve? What happens if an upfront service sends bizarre data? What happens if an underlying service fails?

TDD will help you to build the thing right. It will enable you to produce easily testable, well structured and maintainable code. It won't help you building the right thing. And it won't necessarily help you identifying regressions once your business requirements change. That is still on you to test after having completed your development - of course in an automated way.

Collection objects in PHP

Vincent Fahrenholz — Fri, 14 Sep 2018 14:30:38 +0000

The problem with arrays

In PHP arrays are a very flexible data structure, which accepts almost everything and can be combined any way you like. This is actually good: unlike other languages, PHP lets you even combine associative and numeric arrays, thus offering you a great deal of freedom. But there are times when you just need type-safety.

One way to deal with it would surely be to check every item of the array when using it, thus creating some code snippets close to this:

$array = [1,2,3,4,5]

foreach ($array as $item) {
    if (!is_int($item)) {
        throw new \Exception('wrong type');
    }

    //do your stuff here
}

While this obviously works, it's a bit quirky:

in order to factorize all occurrences, you need some sort of globally accessible function
it is, in fact, the wrong place to check it. Ideally, you would want to check the type of a value when creating the array, and don't bother when using it because you are sure that it's ok.

The solution: collections

A collection is a class that abstracts the array for you. A basic collection class would look like this:

class IntCollection {
    private $values = [];

    public function addValue (int $val, $key = null): void
    {
        //the typehint ensures that you won't have any non-integer values
        if ($key === null) {
            $this->values[] = $val;
        } else {
            $this->values[$key] = $val;
        }
    }

    public function deleteValue($key): void
    {
        unset($this->values[$key]);
    }

    public function get($key): int
    {
        return $this->values[$key];
    }
}

That's already pretty cool, but not enough: you can't use it as an array. This means for instance that

If you are refactoring, chances are that you have to change multiple code parts in order to replace an array with this object
you can't count the items in it
you can't use it in standard PHP control structures like foreach

The better solution

Luckily, PHP provides us with the tools to make all of this possible. For this, we need to refactor our collection class in order to use some of the core languages interfaces.

An interface is an API definition for a class. When implementing an interface, you tell the world that your class will work a certain, standardized and predefined way. When you look at it the first time, it may strike you as "just like an abstract class", but it's not quite right:

all methods declared in an interface must be public
abstract classes can contain variables and implemented methods, interfaces can't
a class can only extend one other class, but may implement as many interfaces as you like

In order to build our array-like, type-safe collection, we'll look at three PHP interfaces

Countable => Tells the system your class can be used in functions like 'count()'
Iterator => Tells the system that your object can be iterated through
ArrayAccess => implement this and you'll be able to use your collection like an array (for instance accessing it via $collection[$key];)

Our finished IntCollection would look like this:

class IntCollection implements \Countable, \Iterator, \ArrayAccess
{

    private $values = [];
    private $position = 0;

    /**
     * This constructor is there in order to be able to create a collection with
     * its values already added
     */
    public function __construct(array $values = [])
    {
        foreach ($values as $value) {
            $this->offsetSet('', $value);
        }
    }

    /**
     * Implementation of method declared in \Countable.
     * Provides support for count()
     */
    public function count()
    {
        return count($this->values);
    }

    /**
     * Implementation of method declared in \Iterator
     * Resets the internal cursor to the beginning of the array
     */
    public function rewind()
    {
        $this->position = 0;
    }

    /**
     * Implementation of method declared in \Iterator
     * Used to get the current key (as for instance in a foreach()-structure
     */
    public function key()
    {
        return $this->position;
    }

    /**
     * Implementation of method declared in \Iterator
     * Used to get the value at the current cursor position
     */
    public function current()
    {
        return $this->values[$this->position];
    }

    /**
     * Implementation of method declared in \Iterator
     * Used to move the cursor to the next position
     */
    public function next()
    {
        $this->position++;
    }

    /**
     * Implementation of method declared in \Iterator
     * Checks if the current cursor position is valid
     */
    public function valid()
    {
        return isset($this->values[$this->position]);
    }

    /**
     * Implementation of method declared in \ArrayAccess
     * Used to be able to use functions like isset()
     */
    public function offsetExists($offset)
    {
        return isset($this->values[$offset]);
    }

    /**
     * Implementation of method declared in \ArrayAccess
     * Used for direct access array-like ($collection[$offset]);
     */
    public function offsetGet($offset)
    {
        return $this->values[$offset];
    }

    /**
     * Implementation of method declared in \ArrayAccess
     * Used for direct setting of values
     */
    public function offsetSet($offset, $value)
    {
        if (!is_int($value)) {
            throw new \InvalidArgumentException("Must be an int");
        }

        if (empty($offset)) { //this happens when you do $collection[] = 1;
            $this->values[] = $value;
        } else {
            $this->values[$offset] = $value;
        }
    }

    /**
     * Implementation of method declared in \ArrayAccess
     * Used for unset()
     */
    public function offsetUnset($offset)
    {
        unset($this->values[$offset]);
    }
}

An even better solution: using the SPL

The SPL is a core library from PHP in which you'll find many very useful things if you bother looking. Unfortunately, it is almost as widely unknown as it is useful. For our problem here, it contains a class called ArrayObject, which does everything we did by hand before and already is serializable. By extending the object, our final collection class will be written in literally no time:

class IntCollection extends \ArrayObject
{
    public function offsetSet($index, $newval)
    {
        if (!is_int($newval)) {
            throw new \InvalidArgumentException("Must be int");
        }

        parent::offsetSet($index, $newval);
    }
}