Accounting for Technical Debt

After this article was posted, Matt Eland wrote a remarkable article with additional information that goes well beyond this one ...

Communicating Technical Debt

Matt Eland ・ Oct 2 '19

#architecture #communication #codequality #management

"Something every software engineering team needs to remember about technical debt - it's almost always hidden. And, hidden debt is so dangerous because it compounds silently." - Unknown

As a development project moves forward and evolves, what are the hidden costs?

Technical debt, while it sounds important, can get lost ... the running joke is that once it gets put into the backlog, it's there; permanently. Unfortunately, this joke more real than we want to admit.

How do we as developers keep Technical Debt from getting lost in the background?

"Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with refactoring."

"The danger occurs when the debt is not repaid. Every minute spent on code that is not quite right for the programming task of the moment counts as interest on that debt."

"Entire engineering organizations can be brought to a stand- still under the debt load of unfactored implementation, object-oriented or otherwise." -- Ward Cunningham

What is Technical Debt?

TECHNICAL DEBT is a term given to the costs associated with defects and other unresolved issues during or at the end of a development project. This concept reflects the implied cost of corrective work that results from selecting a quick and easy solution (often, to meet a time deadline), instead of implementing a more complete solution.

In other words, "Get it done and out the door." On a project timeline this looks good, but in the long term, may require more maintenance costs and a higher risk of defects.

Technical debt may have one or more causes, such as:

1. Time pressure
2. Overly complex technical design
3. Poor alignment to standards
4. Lack of skill
5. Suboptimal code
6. Delayed refactoring
7. Insufficient testing
8. 3rd-Party Code Improvements

Some time is bought by generating Technical Debt, but it is not free. Does this mean technical debt should never be accrued? Yes and no. In an ideal world, The code should always be perfect, and the time should be taken to write all solutions with a mind for the future.

Payment of technical debt can occur immediately or later. A small payment can be made now to meet a deadline, but at what cost later? The costs later can be significant. Consider those potential costs for a moment:

• Rework to analyze and correct the defect later.
• Workarounds that slow productivity.
• Loss of functionality until the issue is corrected.
• Loss of revenue potential.
• Loss of customers due to visible defects.
• Downstream dependencies leading to additional issues and debt.

While creation of technical debt decreases the immediate costs and shortens deadlines, this does not measure the true impact of ongoing debt. Before signing off, be sure there is complete awareness of the ongoing implications. EYES WIDE OPEN is the key mentality to managing technical debt.

Before deciding to take the easy way out:

• Ask if we have to? If time pressure means the non-ideal, expedient solution is the only way out, go on and do it! If there is a little wiggle room, go on to consider ...
• What will the debt cost? How much of a drag will this put on future progress, and is the time being saved now really worth it when looking at the bigger picture?

Planned Debt

From the very start, planned debt can be strategic in nature. This may be from a project developing a prototype or pilot. Part of the implementation is that it is completely understood and agreed on that it is a test run from the very beginning.

This type debt is an expected loss, but a loss that should be controlled and accounted for. Good record keeping is key to tracking all expenses and decisions (including rationale).

Unintentional Debt

This debt us purely due to the human factor, otherwise known as "defects." Unintentional debt often exists because of defects because of human error, poor workmanship, or lack of knowledge. System updates will also cause code to fail, changing downstream functionality or adding security risks. Test automation may be used to identify these risks as fast as the code is written, so that they may be addressed immediately.

Unintentional debt is not limited purely to defects.

• Additional sources of unintentional debt may go back as early as poorly defined or heavily changing user requirements.
• Fairly often, a weak system architecture or poor database design can create dramatic increases in technical debt.

Before agreeing to any changes to a project and its deliverables, the potential downstream effects should be discussed prior to gaining approval.

Unavoidable Debt

This debt is the result of something unplanned, that cannot be avoided. This could be things like changes in dependencies outside of the project (hardware or operating systems or) or additional mandatory requirements being introduced during project development. When these types of things occur, some negotiation of the time, cost, resources, or features originally agreed upon may be in order.

Maintenance Debt

[from SyntaxSeed (Sherri W)]

[Maintenance Debt] is debt that appears over time [and] no amount of careful planning can avoid it. [Also,] it snowballs if ignored.

This is debt that appears as underlying technologies (such as libraries, tools, and infrastructure) go through normal updates. This can be updates, version releases, or deprecation. These types of updates need to be accounted for due to both performance and security updates.

Debt Management

The key to managing technical debt is authentic and clear communication when it comes up. With this transparency, appropriate negotiations and decisions can then be made about how to address the technical debt.

• Challenge the whole team to think beyond the project to identify the ongoing impact.
• Automate testing, especially with repetitive tasks, to be sure of a solid execution every time.
• Monitor the team’s rate of progress. Investigate any slow-down, that could be an indicator of technical debt presenting itself.
• Monitor complexity: With high complexity, developers are more likely to break something when they are trying to fix something else.
• Manage and monitor defects as well as time to investigate and complete.
• Maintain appropriate test coverage. The target range of well written tests is 75 – 85 percent of code covered.
• Ensure the system architecture and database are designed knowing that the system functionality will grow over time.

Conclusion

Technical debt is not bad or good; It's just the nature of the beast when writing code. The key to managing technical debt effectively is understanding its use and addressing the debt in a timely manner. Acquiring technical debt may help to achieve a short-term goal, but one should have a plan in place to address it moving forward. While it is unrealistic to think technical debt can be avoided, it can be managed and minimized.

Comments

[Mick]

I think the real reason technical debt is left unattended is usually to do with the fact that the people running the company and paying the wages don't care about it. That's certainly been my experience in enterprise software development. The dev teams themselves usually have very little say in where their time goes.

"thing" is launched... move onto the next "thing"

Dev Team "Ah but we need to refactor"
Overlords "It's working isn't it... ON TO THE NEXT THING!!"

[Matt Eland]

This is something I think a lot of teams deal with. Why do you think business feels that way and how do you combat it?

[bob.ts]

I can't answer for Mick, but my thoughts are that business has only seen one way work; they assume that the debt can't be too critical since the code works.

They haven't seen the benefits of a strong CI/CD pipeline and the positive impact of proper testing. With these things in place, there is often time to go back and address the tech debt issues properly.

Addressing it in my experience is often the result a slow cultural shift toward code quality ownership, proper testing and pipelines, as well as a focus on writing clean, readable code. Combating the business has to be gradual ... but it can and needs to occur.

[Matt Eland]

I'll probably write an article on my thoughts around actually paying down technical debt over the next few weeks - as well as some thoughts about communicating that debt to business in a way they can understand and make good decisions as a collective group.

I love these discussions, but I fear that the negative feelings in Mick's post are very common and I wonder what we can do as engineers to better expose, communicate, and solve these problems.

[bob.ts]

I'd love to see that article.

I agree about the negative feelings being very common ... in addition, if the developer is not strong (personality, not coding), they can get run-over by management. I've been there ...

Let me know when you get the article done and I'll add a link to this article.

[Matt Eland]

Communicating Technical Debt

Matt Eland ・ Oct 2 ・ 10 min read

#architecture #communication #codequality #management

[bob.ts]

Added a reference to your article!

Excellent work!

[SyntaxSeed (Sherri W)]

I would add a category of 'maintenence debt'.

Debt that appears as underlying technologies (like programming languages, libraries, infrastructure, etc) go through normal updates aka new version releases. Including deprecations.

This is debt that appears over time & no amount of careful planning can avoid it. And it snowballs if ignored.

[bob.ts]

I will look into adding that over the next few days. Great points ... can I quote your last two sentences?

[SyntaxSeed (Sherri W)]

Sure!

[Matt Eland]

Bob, this is great. This squares well with a lot of what I've read and have been thinking about. I sat down to lunch to read Managing Software Debt but read your article instead.

[Boris B.]

Nice one mate !