You’re Not Praised For The Bugs You Didn’t Create

The story of a programming prize strategy

The picture of a colorful entangled web of yarn. It's all connected using a plastic circle.

Recently I was reading a technical walkthrough. In that post, the author was making progress mostly by fixing bugs in the code. The author knew the solution to the problem he was solving. He was debugging with the least amount of tests, and the tests were all end to end.

That reminds me of a company where its software had so many bugs they couldn't afford any new feature. The owners of that company started giving prizes to developers as an incentive to reduce the number of bugs: developers who fixed most bugs would get a reasonably good reward at the end of the sprint.

After a few months, people started to realize the number of bugs was increasing, not decreasing. It turned out that developers were creating more and more bugs, only to fix them and get the prize.

The owners of that company gave the best example of Goodhart’s law.

If there's a good prize, everyone is going to game the system to get that.

It's easy for shareholders and non-technical stakeholders to perceive the effort and cost toward fixing bugs. It has a direct correlation with the loss of money. However, fixing them doesn’t yield progress. Bugs slow you down. The best way to make progress is not to create them in the first place.

However, that's not easy.

When a measure becomes a target, it ceases to be a good measure.

— ‘Improving ratings’: audit in the British University system, p. 308

There are many dependencies and moving parts in a real system. Software is hard and complex, sometimes chaotic. The complexity is not only in the code; it’s also in the communication structure of the people behind it. Coming up with simple solutions such as "use TDD" or "create microservices" is naive to the level of being borderline ridiculous.

You can’t remove software complexity altogether, but you can tame it.

That reminds me of an effect I’ve experienced multiple times. When you start writing naive code in a dysfunctional system, you get a sense of progress in the beginning. As complexity increases, you start to see bugs, and it becomes challenging to develop new features.

In the beginning, complexity is low; everything is fast and Agile™. After some time, things start getting slower and slower, until you start fixing bugs from features developed in the last week. Later on, you start fixing bugs from the previous attempt to fix other bugs, that's when you face the nightmare.

The perception from outside the team is that they are getting slower and slower over Time.

A diagram which shows “Time” in the horizontal dimension and “Complexity” in the vertical dimension. As Time flows forward, complexity increases exponentially.

The more complexity increases, the more significant the frustration becomes of everyone involved.

Here’s another side of this spectrum. When most developers have the skills to understand the tradeoffs of every code they write and the value of every discussion they have, things stop getting slower and slower over Time.

However, they get slow at the start!

In the beginning, complexity is considerably higher than usual. The perception is that development is going slower, which may concern non-technical stakeholders. However, as Time goes by that complexity doesn’t increase as much. The speed at which developers build new features remains the same. Estimations start to become more accurate because the team has a standard pace that doesn't change very often.

In summary, the delivery becomes predictable.

A diagram which shows “Time” in the horizontal dimension and “Complexity” in the vertical dimension. As Time flows forward, it maintains the average complexity from the beginning.

In the second situation, sometimes complexity gets lower due to the paying off of Technical Debt, sometimes it gets higher due to the accrual of it. While complexity can increase exponentially, it’s impossible to decrease exponentially, no matter how much Technical Debt you pay.

The perception out of the team is that everyone is slow all the Time. That perception remains the same until one day the consistency wins over the exponential complexity.

A diagram which shows “Time” in the horizontal dimension and “Complexity” in the vertical dimension. This diagram has both examples from before, only now they’re together in the same diagram. As Time flows forward, the complexity of the first example starts low and increases exponentially. The complexity from the second example starts high but rarely changes. At some point, both of them intersect, that’s when the payoff of the second example is evident.

This has an interesting effect.

In the first example, everybody perceives enormous progress in the beginning. Eventually, when complexity is too high to avoid noticing, somebody may pop the question with a frustrating face : "how did we get to this state?".

In the second example, there's a similar reaction, albeit in the opposite direction. Everybody perceives a lack of progress in the beginning, but at some point, everyone realizes that complexity doesn't increase. The team can deliver at the same pace when others can't. That's when somebody pops up the question with a smiley face : "how did we get to this state?".

Later in a project, you’ll hear the rhetoric: "how did we get to this state?". You know the project is a success if that comes with a smile.

Be careful; there's a trap.

Many projects start with the second example in mind but end up with exponential complexity. The bugs keep coming, only now they are harder to fix. The system had too much complexity from day one.

That's much worse than following the path of the first example.

A diagram which shows “Time” in the horizontal dimension and “Complexity” in the vertical dimension. There are two examples. As Time flows forward, the complexity of the first example starts low and increases exponentially. Similarly, the complexity of the second example starts high and grows exponentially, becoming worse in the end.

Although you can only discover these effects in hindsight, there are ways to go in a given direction. You can get to the second example by applying known fundamental software principles, techniques, and practices into software development. Likewise, you can get to the other examples by ignoring them.

That's the reason I write about fundamentals: to prevent others from repeating the same mistakes by becoming aware of their possibilities.

A Dilbert cartoon where there's a manager in the first scene saying "Our goal is to write bug-free software. I'll pay a ten-dollar bonus for every bug you find and fix". In the next scene, the developers scream: "Yahoo!" "we're rich!" "Yes! Yes! Yes!". In the final scene, the manager says: "I hope this drives the right behavior." Later, one of the developers says: "I'm gonna write me a new minivan this afternoon!". Click here for the Source.

The traditional mindset is:

Everyone expects software without bugs. If you didn’t create a bug, you’re just doing your job.

There are places out there who still believe in that quote. The reality is that if everyone expects software without bugs, then there’s no reason to praise someone when they fix it.

Incentives like giving prizes to people who do a good or bad job won’t help to generate software with better quality. It uses the developers Time to fix problems instead of creating value. Prize incentives could make sense in certain situations, but it doesn't make sense for programming.

The right incentive for us, as programmers, is to make sure we love the work we do and the people we work with. Make sure we can have the right technical challenges, work in an environment with high psychological safety, learn different things, work with more skilled developers, and other things.

There are other incentives worth pursuing to create high performing teams. Giving prizes for people who fix bugs is not one of them.

I'm sure nobody's going to praise me for the bugs I didn't create. However, if I strive for bug-free code, I might be able to go home knowing that the system won't collapse overnight.

For me, that's good enough for a prize.

Thanks to Ian Tinsley, Tim Schraepen, Al Ernst, and Dustin Paluch for their insightful inputs to this post.

Thanks for reading. If you have some feedback, reach out to me on Twitter, Facebook, or Github.

Wanna chat in person? You can find me in the Sydney Software Crafters meetup.

---