There are hundreds, if not thousands, of articles about how to pay down your technical debt and most of them miss step one. They don't tell you how to make the time to repay your technical debt. It's unlikely that you'll be able to convince management to let you stop working on features and bug fixes to repay your technical debt for long enough to truly make a difference. And since it's always hard to quantify the impact of repaying technical debt, it rarely gets to the top of anyone's backlog. Well, I'm going show you how to get around this problem.
Stop making messes
In my last post: Technical debt: we need better communication, not better metaphors, I made the distinction between technical debt and crappy code. I also talked about my team's decision to stop letting crappy code into our project. It feels slower at first but that's almost certainly not the case. It's just that we were not seeing all the unplanned work that didn't happen after we didn't push crappy code into production.
So, your first step is to stop making things worse. Use technical debt strategically--and probably sparingly unless you are working for a startup--and stop allowing crappy code into your project because crappy code has no upside.
You shouldn't repay your technical debt in some projects
You may find it unprofitable to repay your technical debt in certain projects or in projects at certain phases of their life cycle. This includes projects that are:
- nearing end of life
- throwaway prototypes
- built for a short life
- will not be changed ever again
- really, really small (any competent developer can probably figure out what's going on in a 100 line script)
If this sounds like your project, keep reading anyway. This post is about getting more efficient so you can repay your technical debt. But you could just as easily use your new efficiency to deliver more features.
How to make time to repay your technical debt
Okay, let me give you an overview of how this all fits together and then I'll spend the rest of this post filling in the details.
Overview
If your company and your team are interested in paying down technical debt--really interested, not just paying lip service to the idea--then making time to repay your technical debt isn't too difficult. Just kidding, it's pretty hard. That's why so few teams are able to do it. But here's the thing: most teams are just drowning in waste and inefficiency. So you almost certainly have plenty of room to improve.
You can use your efficiency gains to repay your technical debt. Just make a deal with management where you agree that any efficiency you pick up gets redirected to repaying your technical debt or gaining more efficiency. If nobody's buying that, you can try pitch it as a two month experiment. Experiments are less threatening than "change" or "new policies."
An example
Let's say you are delivering 30 story points per sprint of features and bug fixes with a "no crappy code" policy in effect. You promise to keep doing that. But if you have extra time because you become more efficient, then you get to use those, say, 5 extra story points to repay your technical debt or clean up messes or invest in more efficiency.
That's the most extreme scenario where management won't let you slow down feature release. But almost every team can convince management to devote some time to cleanups. So let's say you start at 30 story points and management says you can use 5 of those points to repay your technical debt and after a few sprints you get more efficient and pick up another 5 story points. That's probably a realistic starting point.
So apply your 10 story points to your highest priority ideas at the beginning of each sprint. Let's ignore the details for now but just imagine you have a prioritized list of high value ideas at the ready.
If you invest carefully, you should have even more time to deliver your agreed upon features and bug fixes while you continue to invest in cleanups and efficiency. Repeat this every sprint. Measure your progress as you go and correct your course as you learn where to find the easy wins.
What's in it for you?
Make no mistake--repaying your technical debt is a long term project. But the benefits are worth your effort:
- developers who learn how to pay down technical debt and clean up messes without extra resources are very valuable
- crappy code is a major source of low morale and developer dissatisfaction so cleaning up your code base should make your life more pleasant
How to find your waste and inefficiency
Every team is different but I've never heard of a team that couldn't improve. Because this is such a big topic and I don't know what tools and techniques your team is using, I'm just going to enumerate ideas that I think are useful. You can use these ideas as a starting point for your conversations. Pick one or two things that your team thinks will help you the most and work on them until they are automatic. Then repeat. Doing everything all at once is a recipe for chaos and burnout.
How to eliminate your waste and inefficiency
Some of the simplest things have the best payback. Let's look at a few ideas:
- establish a shared sense of urgency. Figure out why you--as a team--want to become more efficient. Talk about it often. Write it on a white board for everyone to see.
- brainstorm initiatives with your team, rank them by likely payback, and work from the top of the list
- do your refactoring/efficiency stories at the beginning of each sprint (if you leave them to the end of your sprint you'll be much less likely to have time for them)
- adopt some kind of agile process with short iterations (SCRUM, KANBAN, etc.)
- manage your code with a VCS like GIT or Subversion
- everyone on your team uses the best IDE money can buy on a fast computer with ample screen real estate
- decide how the code in your project will be formatted, setup your IDE to format your code to the project standard with a pre-commit hook and never format your code by hand again (you can use a language standard instead of arguing about your code formatting - for example PHP has PSR-1 and PSR-2, Python as PEP 8)
- do self and peer code reviews (hour-for-hour code reviews find more defects than testing but they also increase understanding and spread knowledge)
- automate part of your code review with static analyzers, linters, and by turning up your compiler/interpreter warnings as high as you can without overwhelming yourselves. Let your tools find as many errors as possible so you can focus your code reviews on the stuff computers cannot catch
- only work on the highest priority stories and do as few of them at a time as you can (minimize multi-tasking)
- stop wasting time working on low priority stories
- stop talking about the details of things you might work on in a month or two (half of what you talk about won't be done or will change so much that it's a waste of time to talk about it now)
- stop adding stories to the backlog if you have no intention of working on them in the next two sprints (it's a waste of time)
- stop adding "nice to have" features to your stories--stick to the essentials (remember YAGNI: you aren't going to need it)
- stop wasting time in meetings and stand-ups. Be prepared, be on time, get through your agenda, and get back to work as soon as you can
- don't get bogged down in disagreements. If you can't agree on an initiative, dump it and move on.
- stop interrupting your co-workers when they are working (knocking people out of flow is very costly)
- don't work overtime unless it's absolutely necessary (avoid burnout)
- get management to commit to not interrupting your sprint unless it's an emergency
- if there's an easy way to increase the productivity of a team member, consider investing a little time to make that a reality
- do your retrospective meeting, review how things are going, and figure out how to make them go better/faster
- find your optimal pull request size (it matters more than you might think)
- communicate your progress to your stakeholders, get people excited about what you're doing
Measuring productivity is a mistake
Depending on who you believe, measuring individual developer productivity is somewhere between impossible and very, very difficult. If you do go this route, you'll soon find your developers gaming the system. Limit your measurements to team velocity and defects in production. Look for a trend over months. And don't place too much importance on the numbers or people will game them.
Instead of trying to measure productivity, you should look for and eliminate waste. You'll see waste everywhere once you start looking for it. If you somehow get your team to the point where you can't find any more waste, you're amazing! Write a project management book, start your own consultancy, and go on the speaking circuit because we need to know how you did it (I'm not kidding).
Agile is based on bad assumptions but you can improve it
Agile borrowed ideas from manufacturing like Lean and Six Sigma. Lean came out of Toyota and focuses on the elimination of waste. And Six Sigma came out of Motorola and focuses on the minimization of variation. Agile adopted these ideas because they were so successful in manufacturing and we wanted the same success in software development. However, we failed to see that the fundamental assumptions underlying both Lean and Six Sigma don't hold in software development because software development is nothing like manufacturing!
But I have hope. I recently read "The Principles of Product Development Flow: Second Generation Lean Product Development" (Donald Reinertsen). This book blew my mind. Reinertsen's insights are so obvious that I can't believe we haven't figured this out yet. My team is working on implementing his ideas right now.
So if you want efficiency, read Reinertsen's book. Or you can get a taste of his ideas in this video.
Wrapping up
Technical debt and crappy code are everywhere. And while many articles and books can help you figure out how to tackle it, they generally don't help you find the time to repay your technical debt.
Your first step is to stop allowing crappy code into your project. You can make time for code reviews, unit testing, and other QA activities with the time you save not fixing the defects you didn't release into production.
Your next step is to steer some of your team's effort toward increasing efficiency. I've listed over 20 ideas to get you thinking about eliminating waste and inefficiency. And once you're done with the easy stuff you can dive into Reinertsen's book and work on the harder stuff. You can use your increased efficiency to become even more efficient, repay your technical debt, or clean up crappy code. No matter which combination of the above you choose, if you follow this plan you'll still be available to continue releasing new features and improvements. And, maybe, you'll enjoy your job more too.
Have you worked on a project that successfully paid down a pile of technical debt? How'd you find the time to do it? I'd love to hear your about your experiences.
Top comments (7)
LEAN and Six Sigma
I think that software development can learn from things like LEAN and Six Sigma. But the concepts have to be applied/mapped to the right thing.
For example, in software "keeping your area clean" does not mean keeping your desk clean. Your desk isn't the "manufacturing floor" of software. The source code is the "manufacturing floor" of software. So keeping your area clean means keeping the software clean.
James Coplien restated 5S mapped to software development:
5S for Software
Technical Debt
The biggest problem with accrued technical debt when faced with Scrum is the focus on customer value.
Since architecture has little externally visible "customer value", and iteration planning of the backlog is driven by "customer value", architectural activities such as addressing technical debt are often not given sufficient attention. (Props to Philippe Kruchten.)
The only way to avoid accruing technical debt is being willing to say "we're not done" and/or stringently following best practices of Clean Code and strong team discipline.
In the meantime, I'll pick up Donald Reinertsen's book. Thanks for the recommendation!
Great comments and the slides from Philippe Kruchten are excellent. Thanks for sharing.
Let us know what you think of Reinertsen. I'm planning a series of posts on his book but it's such a big area, that there is plenty of room for multiple views.
Nice, I should rename my article too, I actually told the readers how to repay the technical debt, in time without full sprints refactorings. Basically including the time in the next tasks, like you said too.
There are many topics to talk about, including Efficiency vs Productivity, fixing issues is not always the same with writing code.
I would also include at "stop wasting time in meetings and stand-ups." most of the Scrum meetings, but that's a subject for an entire book.
Thanks.
I read your linked article--very nice. I can tell you've been deep in the trenches and survived. I especially agree with your points about continuous small improvements and how long it takes to convert crappy code into less crappy code.
Very nice article!
I have a question (maybe it's an idiom I don't get), what do you mean when you say 'hour-for-hour code reviews'?
Thanks!
Thanks.
Yes. It's an idiom (apologies).
"hour-for-hour code reviews find more defects than testing..."
It means if you spend an hour writing unit tests and an hour doing code reviews, you'll statistically find more defects with the code review.
Here's a link to an article summarizing the research.
Cool! No need to apologize.
It makes a lot of sense.
Thanks again for the great article. I started spreading it through the company.