What Is Technical Debt and How to Manage It?

Ward Cunningham coined the concept of technical debt in 1992, and several authors have explored it extensively in recent years.

Organizations often need help discussing technical debt because it is associated with poor quality. However, recognizing the existence of technical debt could be a more helpful approach since all applications have technical debt.

When organizations ignore technical debt, it commonly grows without management, which is harmful. We must pay debt, and the longer we take to do so, the more challenging it will be.

On the other hand, a software product should have a downward curve in terms of effort; however, technical debt produces an upward curve. This upward curve is not necessarily bad because it can deliver early benefits to the business but requires adequate management to lower it.

Building on the book Managing Technical Debt: Reducing Friction in Software Development, written mainly by Ipek Ozkaya—a prominent authority on this topic—and our experience at Pragma, this article will provide vital information to understand and manage technical debt adequately.

We'll explore the definition of technical debt and its guiding principles. We'll also explore its costs and causes and give recommendations for its management.

What Is Technical Debt?

Technical debt is the cost of evolving a product. This cost can be incurred to obtain an immediate benefit or for other reasons mentioned later.

We can approach technical debt the same way we understand regular credit. In exchange for future obligations, we obtain immediate benefits. Additionally, we must pay interest, and the interest rate will increase depending on the amount of the loan and the time it takes to repay it.

For example, in a development process, a decision can allow customers to have an early release, which creates technical debt. After some time, they need to implement a new requirement. Without technical debt, that new requirement could take two days, but because of it, it will take five. In this situation, we must decide: either we make a significant effort and repay the technical debt at once, or we can only deal with the interest, which means that we manage to fulfill a short-term requirement, but technical debt continues.

The longer it takes to pay the technical debt, the more expensive it will be to settle it, and the higher the interest will be.
Technical debt is not only incurred intentionally when design decisions are made or implementation is carried out. Technical debt can also be a part of the system's natural evolution, a change in context, or other factors that will be mentioned later.

Moreover, it is crucial to differentiate between defects, vulnerabilities, and technical debt. As previously mentioned, technical debt generates costs in the evolution of the product. Defects result in direct customer dissatisfaction and must be corrected immediately. Likewise, vulnerabilities create security risks and must be corrected in the shortest time possible. However, these three concepts intersect: defects and vulnerabilities can cause technical debt, and technical debt may give rise to defects and vulnerabilities.

Technical Debt Principles

As expressed by Ozkaya, Nord et al. in their book Managing Technical Debt: Reducing Friction in Software Development, the principles associated with the concept of technical debt are listed below:

How Much Technical Debt Costs?

The cost of technical debt distinguishes between technical debt and potential technical debt.Potential technical debt refers to debt that may or may not generate a cost for the evolution of the product. Still, this cost has yet to be generated.

Example: When developing an application, the decision is made to make it for a single language and country because it is the current need. Making it multilanguage and multicountry could mean additional time in terms of development, and the business needs to be in production as soon as possible. As long as the system continues to operate this way, it incurs no cost in evolution. Likely, the potential technical debt will not have to be paid at any time because the application will not ultimately be released in another language or another country. Still, if the decision is to release the application in other languages, the potential technical debt will become technical debt; that is, it will incur a cost in the evolution of the product.
Suppose a long time passes between identifying potential technical debt and paying it. In that case, the system will become more complex, and the cost of paying the technical debt will be very high. This would imply changes in the infrastructure design, the governance of accounts between countries, a multi-tenant architecture, or using a library to manage languages better.

Additionally, technical debt is incurred not only in the initial design or due to incorrect decisions but also by the system's evolution. The latter is crucial because there are architectural decisions that are correct at the time but must evolve with time.

Let us consider a specific situation and time when using Java 8 is a correct decision. At another time, this decision could incur costs in the system's evolution or context, and it is better to use Java 17 or Java 21.

Technical debt management should become a fundamental part of the development life cycle and the system's evolution. Principles 4 and 7 states that technical debt must encompass the entire system, and all code matters.

Ensuring that technical debt remediation is less costly than the benefits achieved is essential. 

Four Stages of Technical Debt

Technical debt usually has the following stages:

1. Occurrence: Technical debt is injected, whether intentionally or not. The benefits of technical debt, if any, begin to be obtained from this moment.
2. Awareness: You become aware of potential technical debt and the importance of repaying the principal as soon as possible so that the interest rate does not increase.
3. Tipping Point: Potential technical debt becomes technical debt, and interest payments must be made. From here, the cost of the technical debt begins to be counted until its remediation. This cost should be lower than the benefits obtained due to technical debt.
4. Remediation: The principal of the technical debt is repaid, and this no longer incurs a cost in the evolution of the product.

What Causes Technical Debt?

As described above, technical debt is not synonymous with poor quality. However, technical debt that is not managed is harmful to the organization.

Technical debt can be intentional, occur without being aware that it is being injected into the system, or even be incurred due to a change in the system’s context or natural evolution. Thus, remember Principle 3: All applications have technical debt.

Technical debt can be due to technical decisions or organizational issues:

Why Does Technical Debt Occur?

Concerning technical decisions, debt can be incurred by:

• The architectural design

The technical debt incurred by architecture is the most expensive in the system, according to Principle 6. Redesigning the architecture implies a structural change and, therefore, could take several days or even months of rework by the development teams. This debt can occur because quality attributes are not aligned with current or future business needs, evolution of the technology that supports the system, uncertainty, architecture smells (architecture designs that do not follow good practices or standards and make system maintainability complex), anti-patterns, structural complexity, or insufficient documentation.

• The code

Debt can be incurred by low code quality, code complexity, incomplete unit or integration tests, unclean code (code smells), unreadable code, or insufficient documentation.

• Components related to production operation

At the production level, elements that allow the system to operate can generate debt; however, due to their age, they are no longer compatible with its new components.
Concerning organizational issues, debt can be incurred by:

• Context
  It is important to identify the context in terms of the causes of technical debt. The company's context can change and incur technical debt, suggesting a crucial role.
  Example: A canonical model can be selected for the messaging management of payment-related APIs; this decision is correct and does not incur technical debt. Yet, when the country's regulation establishes that all payment-related messaging must be based on the global financial reporting standard ISO 20022, the system acquires technical debt.
  
• Processes
  The lack of sufficient documentation can generate debt. The same goes for insufficient automation, poor change management, and misalignment between the processes and systems operating or being incorporated into the system, either because they are new or because a regulation integrates them into the system's context.
• People
  Technical debt can occur due to teams needing more experience, distributed teams, poor communication, and non-dedicated teams.
• Business
  Technical debt can occur due to timelines and pressure of time to market, uncertainty, misalignment between business objectives and the product or feature to be delivered, customers' change of context, and a lack of requirements.
  

It is essential to differentiate between the cause of the technical debt, the consequences of the technical debt, the symptoms to identify the technical debt, and the technical debt itself, as shown in the following table:

Dealing With Technical Debt: How to Decide What to Do First 

When managing technical debt, making informed decisions from both a technical and business perspective is crucial. According to Principle 9, technical debt depends on the system's evolution. Knowing the company's business and technical roadmap is essential to identifying the system's evolution and, thus, understanding what technical debt must be paid as a priority.

An example of this at the business level exists when the roadmap involves releasing the system in other countries and languages. In this case, the organization must prioritize the payment of the technical debt related to this system's feature.

At a technical level, we can see an example when a company decides to stop paying for licensing a software product or service starting the following year. In this situation, the development team needs to remove all the components related to the service before the licensing contract's renewal date, which becomes a technical debt that needs to be prioritized. 

Otherwise, the company must take on another licensing period.
The following chart shows different paths an organization can take to deal with technical debt. 

Registration and Management of Technical Debt

Finally, the registration and management of technical debt are very relevant. Below are several strategies for effective management.

One of the fundamental factors is to record technical debt decisions and make them visible to the organization, plan the debt’s payment with a specific capacity, follow up on it in the development life cycle, and assign an owner.

You can use the following form to record technical debt in your organization: 

It is essential to make Principle 8 clear: Since technical debt does not have an absolute measure and, as mentioned, it depends on many factors, it is important to raise awareness in the organization and align technical debt decisions between the business team and the technical team. Although there is no absolute metric for technical debt, below are some metrics that could be useful and some recommendations:

Metrics:

• MTTR: ​​Mean time to repair a failure
• Lead time: The time it takes for the code to run successfully in production after it has been committed.
• Deploy frequency: How often the application deploys new features to end users
• Failure rate: The percentage of releases delivered in production or testing environments that cause failures in the application
• Downtime when an application deployment is performed
• Average time a person stays on the project
• Compliance with the quality attributes established for the project
• MTTD: Mean time to detect the problem
• MTTI: Mean time to identify the root cause of the problem and the possible solution
• Mean time to build new features according to their size: S, M, L
• Unit test coverage percentage
• System availability percentage
• System performance with quantitative metrics
• Percentage of code detected that does not follow the policies or good practices of the organization

Recommendations:

Review the technical debt under each of the following aspects and mark each item with the following legend:

+ Compliant and/or no problems are causing technical debt.
* Partially compliant and/or can be improved because it may contribute to technical debt.
- Non-compliant and/or contributing to technical debt

Organization:

• Make technical debt review and management part of the development lifecycle
• Form dedicated teams per product
• Change management
• Communication channels
• Collaborative work
• Rework cost
• Record the points in the system that create uncertainty and monitor them periodically
• Establish transparent onboarding processes for new team members
• Validate the alignment among the processes that connect directly or indirectly with the system

It is also important to visibly record the context of the system and validate the impact with respect to the technical debt of the following elements:

• Governance
• Company size
• Solution architecture
• System criticality
• Business model
• System change rate
• System age
• Distribution of teams working on the system
• Country regulations
• Systems interacting directly or indirectly with the system under review
• Users of the system
  

Architecture:

• Automate the review of technical debt. For example, establish a DevSecOps pipeline that includes construction, integration, static code validation, infrastructure validation as code, dependency scanning, secret scanning, container analysis, dynamic security analysis, security testing, performance and automated tests
• Use services that help reduce technical debt
• Establish system observability with clear metrics at the RUM, APM, security, infrastructure, transactional audit, and cost levels
• System infrastructure
• Components or services that help deploy and operate the system
• Host architectural documentation in a single place that is easy to access and searchable by team members
• Schedule periodic monitoring sessions regarding the implementation of the proposed architecture
• Architecturally significant requirements
• Architecture fitness
• Architectural problems
• Impact of technology change
• Short- and long-term architecture goals
• Perform periodic reviews of the system to analyze technical debt; at this point, it is advisable to use ATAM
• Visibly record the context of architectural decisions

Development:

• Set minimum standards of good code practices that should be part of users’ onboarding
• Software development processes
• Development environment infrastructure
• Ensure the quality of the system by appropriately implementing the proposed architecture
• Have adequate tools and services for the development process
• Be clear about DoD and DoR
• Code evolution and maintenance

Business:

• Have clear business objectives
• Be clear about the success of the strategy
• Be clear about the resources required for the success of the project
• Have clear communication channels with the customer
• Be clear about the consequences of business decisions
• Carry out a feedback cycle with the project’s stakeholders

Services and Tools to Manage Technical Debt

Below are some services, frameworks, and tools that may be useful to visualize and reduce technical debt:

• Static code analysis
Sonarqube analyzes static code and verifies test coverage and code quality related to code smells and security.
Amazon CodeGuru Security is a security service for static code analysis that relies on machine learning to detect security policy violations and vulnerabilities. It also provides suggestions to address security risks and generates metrics that allow tracking applications’ security posture.
Amazon CodeGuru Profiler is a service that takes runtime performance data from applications and provides recommendations for improvement.

• Unit and integration tests
JUnit is a widely used framework for Java unit testing.
Mockito is a widely used framework for unit and integration testing for Java.
checkov scans and analyzes the infrastructure as code and detects incorrect or non-optimal configurations.
cdk-nag reviews best practices in infrastructure as code made with AWS CloudFormation and is based on a package of rules that can be viewed in this repository.

• Container analysis and secret scanning
aqua trivy is an open-source security scanner to detect vulnerabilities and incorrect or non-optimal configurations.
trugglehog is a tool that scans secrets, passwords, and sensitive data.

• Dependency analysis
OWASP Dependency Check is a software analysis tool that detects vulnerabilities within a project’s dependencies.

• Dynamic security analysis
OWASP ASST is a free project that helps verify security vulnerabilities in a software project.

• Performance analysis:
Sometimes, architectural technical debt can affect the application’s performance. APM services, such as AWS X-Ray, verify compliance with the established metric (the system must make the transaction and respond to the user in less than two seconds) to ensure the architecture works properly.
JMeter is free software that reviews the functional behavior of load tests and measures performance.

• Migration from Java 8 to Java 17 with Generative AI
Amazon Q Developer: At Pragma, we managed to migrate a microservice from Java 8 to Java 17 (previously described example that can contribute to technical debt) in 20 to 60 minutes using this service when this migration usually took 2 to 3 days.

• Observability
Amazon DevOps Guru is a service that uses machine learning to detect anomalous behavior patterns in the system. With this, it is possible to identify problems before customers are impacted.

• Automated testing
Selenium is a testing environment that creates automated software tests for web applications.
Playwright automates end-to-end software testing in parallel across all browsers.
Appium is a free project designed to automate the testing of mobile, web, and hybrid applications.

• Time invested per requirement
WakaTime is a plugin designed to see the time it takes for each requirement to be coded.

Conclusion

To manage technical debt efficiently, it is vital to consider the nine principles described in the book Managing Technical Debt: Reducing Friction in Software Development.

It is also important to understand the context and cause of technical debt, whether intentional or not, and remember that it is not synonymous with poor quality. However, adequate registration and management are required to guarantee that technical debt costs less than the benefits obtained and to perform a periodic review to validate that technical debt has not been incurred.

Deciding what to solve and establishing the appropriate capabilities to do so is relevant.

The more time passes and the more complex the system, the higher the interest rate on technical debt.

Technical decisions or organizational issues can give rise to technical debt.

Technical topics include architecture, code, and production.

Organizational issues are considered from four perspectives: context, processes, people, and business.

In short, technical debt management is the fundamental factor preventing it from incurring higher costs than the benefits obtained. It can be reviewed in terms of organization, development, architecture, and business vision.

Don't let Technical debt accumulate and impact your development velocity. Let Pragma help you create a tailored strategy to address your specific challenges.