What is technical debt?
As put simply by Dan Rawsthorne:
Although simplistic, the spirit of this definition is generally at the core of most industry experts’ own definitions. Technical debt is a metaphor that software developers use to refer to “short-term compromises” to a software project’s “code or design quality,” which in turn “make[s] the product more difficult for someone else to continue to develop, test, and maintain in the future.” A key component of the technical debt metaphor is the idea that for any type of debt, some amount of interest needs to be paid. Fundamentally, if there’s no potential for interest to accrue, then you’re not dealing with technical debt—you just have incomplete work and not debt. (Examples of this would include features not delivered, bugs not fixed, etc…)
Technical debt is important for software developers to consider is because code that is hard to work with generally hampers developer’s productivity and results in less stable code.
All too often the term “technical debt” ends up being applied to a wide range of issues, and as such, becomes unmanageable. Let’s talk about what technical debt isn’t: Technical debt is NOT a bug or the lack of a feature. Technical debt is not just a fancy name for “sloppy code”.
Technical debt is generally not visible to a user of the system. It’s visible to developers and those that have to work with the source code in some capacity. That being said, technical debt can contribute to bugs and other user facing/impacting issues.
Is technical debt a code-only thing?
Until this point, I’ve discussed technical debt within the context of code.However, technical debt can be found throughout the technology stack, especially in SAAS (software as a service) organizations like Wikimedia. Outdated operating systems, lagging patch deployments, old libraries and compilers can all contribute to a system’s overall technical debt. Not unlike code-based technical’s impact on developers, infrastructure-based technical debt can make it more difficult and unpleasant for admins and sysops to do their work.
Much of the technical debt blog series will be applicable to technical debt found throughout the stack. However, most of the examples and specifics here will be written from a developer’s/code perspective (because I’m a developer.)
What is the impact of technical debt?
Okay, so we’ve got technical debt. So what? This is really where the financial debt analogy starts to fail us a bit. Unlike financial debt, technical debt isn’t easy to measure either in terms of the cost of repayment or the cost of interest. That’s because some of the costs of technical debt are unknown upfront. Applying this to the financial analogy, it would be like signing up for a range of debt with an undefined interest rate for an undefined amount of time. Does that sound like a good idea?
On the plus side, technical debt doesn’t necessarily incur any interest if the code in question requires no change or maintenance. For example, if you pay off the debt in full via refactoring, before it has slowed you down, it could end up being an interest-free loan. That being said, I’d venture to say that a significant portion of legacy technical debt doesn’t fall into this category. This is because much of the technical debt was identified retroactively due to the impact it already has had.
With technical debt, the interest costs are those associated with working with the code in it’s current and less than ideal state. This could amount to the “extra” time spent in development, or to bugs introduced due to the poor code design or implementation. Another impact that is less obvious, but potentially very impactful, is that of code avoidance. An example of code avoidance might be the development of new extensions versus extending an existing extension or the core code-base itself. The impact of avoidance can be very costly both in-terms of development and operations.
All technical debt is bad—right?
In my experience, most of the discussions related to technical debt, especially the higher-level ones, work from the premise that technical debt is bad and should be avoided at all costs. However, that’s not my experience. Not unlike financial debt, technical debt is a tool that can be used to accomplish more things in the nearer term than would otherwise be possible. Much of our technical debt is more in-line with consumer credit card debt than mortgages or business loans.
Our goal at the Wikimedia Foundation is to transform how we manage our technical debt in order to reduce it and make the debt we do incur be conscious value-driven debt. To help in that, it probably makes sense to take a deeper dive into defining different kinds of technical debt. For that I turn to a piece from Steve McConnell, author of several books including Code Complete, which does a nice job creating a technical-debt taxonomy that is simple to understand but also helps when making decisions about technical debt. He refers to debt in this way:
- I. Debt which is incurred unintentionally
- II. Debt which is incurred intentionally
- II.A. Short-term debt, usually incurred reactively, for tactical reasons
- II.A.1. Individually identifiable shortcuts (like a car loan)
- II.A.2. Numerous tiny shortcuts (like credit card debt)
- II.B. Long-term debt, usually incurred proactively, for strategic reasons
- II.A. Short-term debt, usually incurred reactively, for tactical reasons
In this technical debt taxonomy, the first distinction is between unintentional and intentional debt. This distinction is worth spending some time on.
Unintentional debt is … well, unintentional. It’s not a deliberate decision to incur some debt in order to deliver something faster or easier. Although Steve McConnell labels this as “due to low quality work”, I’d propose that it’s actually more so related to decisions or approaches that were based on the understanding/ability at the time. An example of this is the selection of a design pattern or architecture that seemed reasonable at the time of development, but has since then been identified as complex and difficult to implement and maintain.
Intentional debt on the other hand is known when it is incurred. That doesn’t necessarily mean it’s tracked or thought of as debt, but it requires a conscious decision to cut a corner or pursue a less desirable approach. This is the category that we have the most opportunity to actively avoid through our decisions.
In many circumstances, intentional debt is taken on due to external schedule pressures, however at the Wikimedia Foundation we don’t generally have the same external pressures. That’s not to say that we don’t have self-imposed schedule pressures, but it seems that we are in a better position to resist intentional technical debt that is driven by external pressures. That being said, there are other reasons that could drive the intentional accumulation of technical debt, such as the desire to get timely user/community feedback on new features. This perhaps could be called “prototype” driven technical debt.
Within intentional debt we can further break things down into short-term debt vs long-term debt. In other words, how long (in calendar months) we think it will take to repay the debt. Generally speaking, the longer-term the debt is, the more strategic the decision was in nature. An example of this is the selection a development language like Objective-C, which is inherently more complex to work with, over Swift, which is a more modern and simpler language to work with.
The shorter-term debt tends to be more reactive and in many cases doesn’t provide much value for the debt it creates, such as not following a coding convention that eases readability. This is the kind of debt that is most avoidable. We’ll cover more about how the taxonomy is used in our next two blog posts in this series.
In this post I’ve focused mostly on defining what technical debt is, the impact that it has, and a way to categorize it in a way that supports making decisions. In my next post, I’ll talk more about how we can avoid incurring additional technical debt. I’ll then wrap up this series by discussing how to address currently accrued technical debt.
Interested in joining the discussion? Feel free to join the Technical Debt SIG, attend a regularly scheduled code health office hours session, and/or in IRC #wikimedia-releng using keyword “CH-TechDebt”.
Jean-Rene Branaa, Senior QA Analyst, Release Engineering