Technical debt as a metaphor is not serving our profession well. It was meant to help us talk to business people and make better decisions about our software projects. But it has largely been a failure. Part of the problem is that business people aren't afraid of future interest payments.
In business school, I learned about the power of leverage: what it is, how to get it, how to measure it, and how to manage it. Debt is a tool. My fellow business school grads are comfortable with leverage and debt. So when the programmers come to the business people complaining about technical debt, the business people are unconcerned. And why would they be? The metaphor is misleading; technical debt is only superficially similar to financial debt.
What's at stake
I've noticed that business people tend to be better at convincing programmers to damage the long term health of their code base for short term gains, than programmers are at convincing business people about the importance of having healthy software with few messes and low levels of technical debt.
This is really unfortunate for three reasons:
- once you allow your project to turn into a mess, it's really difficult to clean it up (in many cases it might be impossible)
- the decision to take a shortcut or make a mess for short term gain is very often sub-optimal in the long term. If the business people truly understood the risks and costs of these shortcuts, they would likely make better choices
- trying to maintain and extend a software project full of crappy code is frustrating, soul-crushing work. Many programmers quit their jobs to get away from low quality code--it's that serious
No matter how much technical debt your project has, you can always make it worse. You'll always face both internal and external pressures to take a shortcut here, avoid a refactoring there, and release that new feature without automated tests.
Therefore, we need to do a better job at defending the code with the business people. It's their job to advocate for the business objectives and it's our job, as programmers, to advocate for the code. Don't believe me? Read this: Software Engineering Code of Ethics and Professional Practice
How to advocate for the code
You need to educate stakeholders in your company about the software development process. Your job is not done until you are sure they understand the trade-offs and risks involved in taking shortcuts. I'm going to use the remainder of this post share my tactics.
Find a hook to get business people interested in how your company creates and maintains software
You need to get the business people involved in your process. The more they internalize your struggles and challenges, the better.
Appeal to efficiency
You'd be hard pressed to find a business person who didn't want to discuss an opportunity to have the software developers deliver:
- more features, more quickly
- fewer defects
- faster response time to requests for changes
- more predictable release schedules
- lower development costs
- lower programmer turnover
Your general thrust here could be that your company is missing out on high value opportunities. And that you'd like some advice from the business people on how you can capitalize on those opportunities and make things run more smoothly/efficiently/effectively.
Appeal to risk avoidance
Depending on your audience, you might find another approach to be more persuasive. Business people are trained to manage risk and you can use that to your advantage by framing the situation as one of managing increasing risks.
Here are some risks of making messes in your code:
- competitors release superior software
- defects soar
- costs soar
- unable to meet regulatory requirements
- unable to add new features
- progress grinds to a halt
- data loss/corruption/breaches
- lawsuits/fines
- lose customers
- bad publicity
- lose use of the software completely
- good programmers leave the project
- and here's the biggie: risks interact in unpredictable, non-linear ways
One of these two strategies will likely get the attention of your business people.
Get them involved in your meetings
The best thing my team ever did was adopt SCRUM. Our product owner really 'got it' once he attended a few of these meetings. He also became a captive audience for an hour or so per week during these meetings. I took full advantage of the opportunity to increase his understanding whenever I could.
These meetings make abstract ideas about 'going slow' or 'bad code' more concrete. For example, we look at all the stories where we missed our estimates by a wide margin at our retrospective and talk about why that happened. But it's quite another thing to listen to a programmer take 20 minutes to explain all the reasons why a simple change to the system took 40 hours (10 times the estimate). When that happens sprint after sprint, the business people can't help but buy into it. I find it especially helpful if you get them involved in the root cause analysis and help you formulate solutions to the problems.
The meetings also allow the business people to get a sense of the team. How do the programmers handle themselves? Are they competent? Are they complainers? How's the morale? Are they lazy? Do they care about the business? The answers to these questions aren't nearly as important as allowing the business people to reach the inevitable conclusion that 'this programming thing' is harder than it appears from the outside.
Make a distinction between technical debt and crappy code
Technical debt has expanded from its initial definition to include almost any undesirable quality of code. At my workplace, we are resisting this trend. We make a distinction between technical debt and crappy code.
Technical debt:
is a deliberate decision to take a shortcut to achieve a short term goal at the expense of an unknowable long term cost.
Crappy code:
is any other kind of shortcut that is not technical debt.
It's our policy to only allow technical debt into the code base as a result of a considered decision made by the business people and the programmers. Our software is pretty mature so we only allow technical debt into the code base to deal with emergencies. For example, if we discover a critical defect in production, we might 'hack' in a solution to resolve the problem immediately and call that technical debt. And then we'll make a story to fix it properly later.
Whereas we allow a limited amount of technical debt into our code base, we have a zero-tolerance policy for crappy code. We made this policy after a long series of discussions and root cause analyses. We decided that the only reasonable way to move forward with our project was to stop allowing crappy code into the code base. To that end, all code is reviewed and it doesn't get merged with master until it meets our quality objectives (see our code review checklist). If that means a change takes ten times longer than we originally estimated, so be it.
Of course, it's not as black and white as that. Programmers make judgment calls every day. But our talk, actions, and behaviors are all aligned towards the production of zero lines of crappy code. We've made it part of our culture.
Show them the research on the cost of crappy code
Business people generally have no idea how bad crappy code is for the health of their software. So, I've made it my job to educate them.
I lent my copy of "The Software Project Survival Guide" (Steve McConnell) to two senior people in my company. This book is getting dated but the first 5 chapters are absolutely worth their weight in gold.
I show anyone who will pay attention to me page 4 of "Clean Code" (Robert C. Martin). On this page Uncle Bob describes the total cost of owning a mess. And I especially love pointing to the graph at the bottom of the page (reproduced below) and asking them where they think we are on it.
I also show people my copy of "Facts and Fallacies of Software Engineering" (Robert Glass) and use it to stimulate software engineering discussions.
I advocate aggressively for our code, our team, and our sanity. I'm really trying to get three points across:
- developing software is hard, expensive, and prone to outright failure so we should take it seriously
- as a profession, we have some idea about which practices influence the risks, schedule, quality, and cost of software development
- we should adopt the practices associated with the outcomes we want
Show them examples of crappy code in your project
It's pretty easy to convince people that crappy code is bad in theory. But it's another thing to convince them that they own crappy code.
I've gone as far as literally emailing screenshots from my IDE to our senior people. I found one email I wrote last year that had screenshots showing:
- all the static analysis errors for one of our projects. It showed over a thousand serious warning and errors and thousands of weak warnings and notices.
- the static analysis errors for a single 3,189 line file containing 5 languages and over 500 warnings
- all the errors and warnings for a single file flagged down the right side of the editor window. It was 95% covered with warning indicators
I wrote that email to:
- make our code quality problems visible
- show why our code is so expensive to change
- and advocate for not making the situation worse
Quality and speed are not opposites
I love this one. Steve McConnell published an article titled Software Quality as Top Speed. He argues that:
In software, higher quality (in the form of lower defect rates) and reduced development time go hand in hand.
This article is well argued, written by an expert, and it is backed by research, which is clearly referenced in the foot notes. Read it. Bookmark it. And realize that it is a powerful weapon in your arsenal to combat suggestions that your team would go faster if they'd just stop doing so much QA or upstream planning.
Bend over backwards to make the business successful
All your efforts will be for nothing if the business people think--even for a second--that the best interest of your business are not your top priority. You can't just say that you aren't going to accept any technical debt from now on no matter what. You can't say you are going to do things "the right way" from now on and they'll just have to get over it.
You're all on the same team. And your objective is to make as much money for your company as possible. So when the business people say they need a new feature in 5 weeks and you think it will take 20 weeks, you (collectively) have a problem that needs solving.
And you, as the software expert, have plenty to contribute. Why do they need this feature? Do they need the whole feature all at once? Could it be broken into phases and you just deliver the first phase in 5 weeks? Could you change the spec to make the feature easier to implement? Could you build a completely different feature in five weeks that delivers the same sales and profit impact? Could you integrate with an API and cut your development effort? Could you deliver a minimal version of the feature in five weeks and follow up immediately with an enhanced version? Could you outsource some of the work?
There are lots of possibilities besides hacking something together, and kicking it out the door. And if you've been laying a good foundation with the other steps, perhaps you can keep things from coming to that.
Act like a professional at all times
You need to act like a professional if you want to be treated like a professional. Whining and complaining are not the behavior of professionals. But that's not all there is to it. You need a plan to manage your software development process effectively and deliver results. You need to make honest estimates. You need to hold yourself accountable for the outcomes of your team. You need to be truthful and avoid exaggeration.
For example, I avoid blanket statements about our code base. Some code is bad, some isn't so bad, and some is actually pretty good. So I never say that we can't make a change. But I will say that this change would require substantial modification of that really ugly module without tests I was telling you about last week. The original author is long gone and we don't really know how it works. I would say this change would take a many months of effort; would you like me to work up a detailed estimate?
In a case like this, my estimate would include a range that communicates the uncertainty involved in a change of this magnitude. My estimate for this change might be 3-8 person months with 85% confidence. That means I think there it an 85% chance that we could deliver this change in 3-8 person months and that there is a 15% chance the actual effort could exceed that range on either end.
And no, I can't do better than that. That's the estimate. We can create another estimate if we modify the scope of the change but I always give my best estimate the first time. And I don't allow myself to be bullied into lowering it.
The real wins come when you understand where the business people are trying to go and you can make suggestions and plans to help them get there faster or at least avoid the land mines along the way.
When all else fails, you might have to quit
Sometimes you just won't be able to convince the people you work with that anything other than hack on top of hack on top of hack is possible. They don't care about software engineering research. They don't care about best practices. They don't care if you have to deal with crappy code. They don't care if your life is miserable. They don't want to talk about how you develop software. They just want results, and the cheaper, the better.
If you find yourself in this situation, you have some decisions to make. Let me just say two things. First, there are companies that actually care about the quality of their software and the morale of their developers. And second, high quality software developers are in huge demand so you shouldn't have too much difficulty finding another job if you wanted to make a change.
Wrapping up
When programmers add crappy code to a project with a long life expectancy both research and hard-won experience show they are almost certainly taking on more cost and risk than they realize. You'll face all kinds of internal and external pressures to deliver features faster. But always remember it's your job to advocate for the best interest of your code and your business.
Oldest comments (18)
I could not have said it better myself. This is tremendous advice.
Thank you. High praise.
I've tried, with slightly inconsistent results, to encourage "non-technical" folks to contribute to learn how to make pull requests and do some basic coding. I think it helps foster some cross-domain empathy in the same sorts of ways you've outlined. It requires next-level buy-in and sometimes falls off, but it's been a good exercise at least.
But it's all a two-way street. Coders can't have contempt for others' perspectives. Mutual respect goes a long way.
Yeah, we haven't gotten to the "let's get the non-tech people coding" stage but we are working on basic empathy and teamwork. And that seems to be enormously helpful in getting more people rowing in the same direction.
As a developer with a fair amount of business-minded people in my inner circle, I couldn't agree more that empathy is key. I also agree with Ben's method of introducing "non-technical" people to the basic practices of a programmer, as it's able to demystify the work that is often written off as extraneous or too complex.
Consequently, creating a common ground for communication that is fundamental in developing an empathetic relationship.
Thank you for your comments, David. Let me see if I can clarify my thoughts.
What I'm trying to communicate in the first quote is that business people may not understand the consequences of pushing the devs to take a shortcut. They have their own priorities and objectives and if they can get the devs to deliver a new feature in x weeks, then that helps the business people meet their short term goals and they are happy. But if the devs sat the business people down and explained all the consequences of adding this shortcut to the other 2,000 shortcuts already in the system and how each added shortcut makes it that much harder to do anything in the future, and that their two best devs just quit over the low quality of the code base, they might be able to convince the business people to do something less distructive to the health of the software. Most devs don't push back against the business people or at least take the time to explain the damage certain proposed changes are likely to do to the software. When devs fail to advocate for their code, that's not very professional behavior in my books (and according to the Software Engineering Code of Ethics and Professional Practice).
In my second quote, I'm arguing that the devs can't afford to take a hard line either. There may, in fact, be very good reasons to damage the health of the software to achieve an important short term business goal. For example, it might be worth hacking in some new business rule if it meant keeping your biggest client and avoiding bankruptcy.
Both these quotes support the overall thesis of my post, which is that we need to have better cross-functional communication to make our maximum contribution to the success of the businesses for which we work.
I forgot to address one point. A dev adding a 300th method to a class at my work isn't creating technical debt, he's writing crappy software that's not going to pass code review.
So we are probably in slightly different work situations. You're battling crappy code so, yeah, you probably aren't aware of the effect of the business people when you've got stuff like that going on.
Excellent write-up, thank you for sharing!
We read (and applaud!) all the same books. ;-)
Awesome. Thanks.
I agree. If a significant number of your teammates don't really give a crap, it's hard to make things better with improved communication.
I'm so bookmarking this !
High Praise. Thank you.
I think there is a distinction between technical debt and crap code, but I don't find the need for debt to be "intentional". As I meantioned in my article on debt, you'll just accumulate debt over time even if you're not intending on doing it.
Any new requirement has the potential of rendering existing code sub-optimal. Often it's not until several such new requirements arrive can this realization be made.
"once you allow your project to turn into a mess, it's really difficult to clean it up (in many cases it might be impossible)" -- If this is true then I should be charging more for my services. :D
"If the business people truly understood the risks and costs of these shortcuts," -- This is part of the job description of a programmer: Making business understand the risks and costs, of course, that first we require understanding them ourself.
"trying to maintain and extend a software project full of crappy code is frustrating, soul-crushing work." Yes agree, and starting to wonder if specializing in soul-crushing work may be a lucrative field. :)
You are correct on several counts. Debt will accumulate over time even if no intentional debt is accepted and crappy code is not counted as debt.
I've thought about specializing in soul-crushing work but it probably wouldn't be that lucrative most of the time. You'd need to find a client who somehow:
The overlap in the venn diagram of those four factors might exclude the majority of projects.
With that said, I'd love to hear from someone who makes it work. I think Paul Jones (leanpub.com/mlaphp) makes some money doing consulting on this kind of thing but I don't know how much.
Thanks for the read. This was a great article! I think that bringing the business people closer to the development process is a good idea. I myself am a developer, yet I find that I'm regularly surprised and annoyed by how much more difficult something is to actually do compared with describing it conceptually. I imagine it must be even harder for non-technical people to do deal with the frustration of how long features take to implement.
Thanks for reading.
Yes, I think non-technical people are often very frustrated with developers. There might be very good reasons for it but we (software developers) are absolutely terrible about keeping our commitments.
Great article. I am actively cultuvating these conversations with management at my new employer. I’m going to keep this bookmarked.
Awesome. Thanks for reading and commenting.