Test Driving Arduino

Originally posted on christopherjmcclellan.wordpress.com.

I gave a workshop for the Columbus Arduino and Raspberry Pi Enthusiasts (CARPE) Tuesday night. We had a great time and I thought it would be a good idea to share the experience and my motivation creating the workshop here.

The Arduino IDE isn't what I'd call good. It's great for someone new to programming or microcontrollers. It gets you from zero to blinking a light in seconds, but lacks a lot of features that seasoned programmers rely on. It has a myriad of problems.

• No file browser.
• No terminal.
• No code navigation (like "Go to declaration").
• Code is hidden away where it's hard find and get into source control.
• No way to create automated unit tests.
• Arduino libs aren't always well behaved (SoftwareSerial and OneWire are notorious for globally disabling interrupts).

A few months ago I set out to see if I could some of these problems. My main goal was to be able to TDD microcontroller code. The secondary goal was to be able to use whichever editor I liked. I figured I would probably need to use a different editor anyway.

Before I got started, I picked up a copy of "Test Driven Development For Embedded C" by James Grenning. I wanted to know if what I wanted to do was even possible before I got started. It turns out that it's largely very testable, even if not many people out there are doing TDD for embedded systems. There are some things that are legitimately hard to test, but I'll get to that later.

James gives an example of doing TDD for a LED Driver in the book. That inspired me to create the LED Driver Kata. The premise is pretty simple. You need to create a LED driver to light up some LEDs memory mapped to an 8 bit register. The problem is, the hardware isn't ready yet. How do we write and, perhaps more importantly, test code for hardware that doesn't exist yet? James gives examples in his book, but the most important for me was the idea of faking the register out by passing a reference to an unsigned integer to the driver. This allows us not only to observe the results of the code, but also means the driver code isn't dependent on the hardware. It looks something like this.

TEST(LedDriverTests, TurnAllLedsOn)
{
volatile uint8_t leds = 0xFF;
LedsOn(&leds);
BYTES_EQUAL(0x00, leds);
}

The other core concept is cross-compiling your code. The code under test gets compiled along with your tests for the host machine. Your tests run on your development machine. Once the tests pass, then you compile for the target device's architecture.

Now that I knew it was indeed feasible to test drive embedded code, I set out to implement the kata.

The secret here is that Arduino is built on top of the pre-existing AVR toolchain. Arduino uses the avr-gcc compiler and AVRDude under the hood to compile and upload to the board. You can see this yourself by going to File -> Preferences and selecting the "Show Verbose Output" options.

Arduino installs the toolchain inside it's application directory, but I found it convenient to just install the tools separately. If you're running Windows, the toolchain is available via WinAVR. If you're running a *nix system, just go check out the AVR docker image I created. It lists out the different apt packages you'll need, including the CppUTest testing framework.

I started out compiling by hand, but it didn't take me long to start fumbling my way through creating a Makefile. In fact, the first time I did the kata, I spent far more time working on crafting a Makefile than I did working on the code. The LED Driver kata turned out to be 2 katas in one. Thankfully, I only needed to work that out once. I turned that Makefile into a reusable template for all my AVR projects. Just clone the project add some code. I've actually been using this setup for the digital thermometer I'm building too. It works really well.

• I can use any text editor and terminal.
• I've TDD'ed most of the implementation.
• No worries about Arduino libs misbehaving, because I'm not using them.

There are a few downsides to this approach though.

• No Arduino libs + no Arduino IDE means no serial monitor. To be honest though, I've rarely missed it because I have a test suite.
• Not everything is unit testable.

For example, consider some external device that requires a very quick pin pulse.

void ShiftRegister::pulse(uint8_t bitmask)
{
    // pull the pin high to clock the data,
    // then back to low so we choose when to send it.
    port.DataOut |= bitmask;
    port.DataOut &= ~bitmask;
}

There's no way to observe the side effect from the test. By time the test has finished executing, the register is back in its original state. I suppose this could be solved with a layer of indirection and some mocking, but the code is obviously correct by sight. I'm okay with leaving this as it is.

Anyway, that's a fairly long winded way of providing some context for what we did at the workshop the other night. I cloned a fresh copy of the template and introduced the group to the kata. Next I gave them the rules of the game. We'd be Mob Programming the kata together. The person at the keyboard would type what the group told them to type. That person isn't allowed to think. If they have an idea, they must give the keyboard to someone else and instruct that person on what to type next. When we were done, we'd upload it to an Arduino hooked up to a bank of 8 LEDs.

I got the group started by writing the first test. Things were a bit bumpy out of the gate. I was live coding in front of an audience after a full day's work and couldn't get the test to pass. After a few minutes someone pointed out that I was setting the value to the exact opposite of the expected value. I thanked him, laughed, and passed off the keyboard. People were very hesitant at first, but pretty soon the group had found their rhythm. One by one tests would go red then green. Once in a while I'd interject with some insight or a refactor to try.

Eventually I wasn't paying much attention to the code at all. I started wandering around talking to the folks who were a bit too reserved to actively participate. I found there was an interesting mix of questions. Some people were asking me about TDD, some about microcontrollers and embedded programming. There were very few people in the room who had done both and, as far as I could tell, none who had done TDD with embedded C++.

Suddenly I heard "Hey Chris! We're done! How do we upload this thing to the board?" They took me by surprise. Before we started, I was worried that we wouldn't have time to finish the kata. I ran back to the keyboard and wrote a quick loop in the main to flash the lights in sequence, then burned the program. It worked. It. Just. Worked. We all kind of marveled at the power of the mob and TDD for a moment as our device driver worked the very first time we flashed it to the board.

Well... it almost worked. There was one light that wasn't lighting up. I instantly suspected the loop I wrote without any tests, but then someone piped up, "The code works. The tests pass and we all saw you write that loop. The code is right. It's a hardware bug." By golly they were right too. We fiddled with the breadboard for just a moment before that stubborn little light lit up.

This was precisely the point of the whole exercise. We knew the code worked because every single line of code was tested and reviewed by an entire room full of programmers. This narrowed the search space for the bug down to the only place it could have been. It was fixed within moments.

I really couldn't be happier with how things went. Even though I screwed up by live coding instead of preparing the first test before hand, the power of collective problem solving and test driven development really shined through. Most importantly though, we proved that not only can embedded code be tested, but that people with little experience can do it if given the right tools and support.