Reversing Technical Debt with Cloud

This blog post covers best practices to manage and reverse technical debt by prudently leveraging and operating cloud services.

Technical debt is a metaphor coined by Ward Cunningham, to deal with the cost of making tradeoffs in software development to meet near-term business needs. In the case of financial debt, you take a loan to acquire something you want right away with a promise to pay it back later. There are consequences if you don’t pay it back. Similarly, if software deficiencies are not addressed in a timely manner (“paid back”), it becomes exponentially more challenging (the “debt interest”) to maintain and add new features to your software product. This applies universally to all companies, irrespective of whether they are operating on-premises or in the cloud. But just as financial debt has to be managed responsibly, there is a need to be diligent about managing technical debt. In fact, a key advantage that the cloud offers is the ability to manage technical debt effectively.

What causes technical debt?

There are multiple factors that contribute to technical debt. To state the obvious, bad technology practices and decisions cause technical debt. Examples include poorly written software, inadequate code reviews and testing, a lack of documentation and standards, inflexible software design, and wrong technology choices. Technical debt also results from a lack of IT skills, capacity, and employee turnover.

Business and technology teams interpret product requirements differently, resulting in technical debt. Business requirements focus on functional end-user experiences. Technology requirements focus on non-functional attributes such as resilience, performance, maintainability, and security. These are often not prioritized by businesses. To meet deadlines, IT teams often cut corners (introducing technical debt) to incorporate these non-functional features. Changes in customer preferences, environment, competition, geopolitics, economy, and regulations make it challenging to predict future business models. Thus, building future-proof software is a challenge. Moreover, technology itself is evolving at a rapid pace, making today’s technological choices obsolete tomorrow. With hackers constantly finding new ways to exploit software vulnerabilities, we are in a perpetual state of technical debt.

Best practices to manage technical debt

Technical debt is inevitable. So how do we deal with it? Left unaddressed, it would get harder to make changes in the software, thereby crippling speed and inflating the cost of delivery. On the other hand, if companies spend too much of time chasing perfection, product delivery will be impacted. Technical debt is not necessarily bad. As long as tech debt is deliberate but prudent, with a well-defined execution path to fix deficiencies (“pay it off”) in the future, it is a win-win for both business and technology teams. Here are some best practices for managing technology debt by leveraging cloud services.

Recognize “real” tech debt and make it visible. Technical debt can be misused to justify investments. It’s typically used to justify improving IT systems’ internals rather than adding new user-facing capabilities. Technical debt investments should be tied to business outcomes. Debt should be quantified by business impact, such as poor customer experience, lost revenue, delayed product releases, longer issue resolution times, and decreased employee productivity. Periodic maintenance activities such as backups, 3rd party upgrades, and security patches do not fall under technical debt. These are simply good operational practices.

Technical debt is not just a technology issue. The decision to incur and repay technical debt should involve both business and technology teams. Working backwards from the customer’s needs and prioritizing the most impactful minimum viable product (MVP) features can help business teams mitigate technical debt. By clearly stating when a software feature is “done and ready” and not overpromising deliverables to stakeholders, business teams can reduce churn in requirements. Similarly, IT teams should be open and realistic about the time and resources needed to build new features or pay off debt.

Leverage cloud services. While “lifting and shifting” your on-premises workloads to the cloud is a great option to move to the cloud quickly, it does not address technical debt. In this case, the on-premises debt simply moves to the cloud. To maximize cloud adoption benefits and manage technical debt, firms must rethink and adapt their on-premises processes to the cloud. Deploying a software application requires what AWS calls “undifferentiated heavy lifting” such as managing hosting, bandwidth, contracts, capacity, operations, and coordinating across large teams. If any of these areas are managed poorly, it can have negative consequences and generate technical debt. Cloud platforms provide the ready-made building blocks needed for building and sustaining software. This allows teams to focus time on business value and innovation rather than IT internals. Infrastructure provisioning, code deployment, testing, threat detection, monitoring and alerting, and disaster recovery can be fully automated on the cloud. Self-healing capabilities automatically shut down underutilized infrastructure or scale up in response to demand. Many cloud services, such as serverless computing and purpose-built databases, are available as fully managed services with built-in capabilities for high availability and resilience. This means that IT teams do not have to allocate resources to manage routine tasks.

Adopt agile practices and cloud-native approach. Cloud computing is well-suited to agile development, and it thrives on the principles of experimentation, flexibility, and incremental delivery. Cloud-native technologies support fast and frequent changes to applications without impacting service delivery. The agile development approach emphasizes speed over perfection and improves business-technology collaboration. Both new features and technical debt are visible in a unified “product backlog.” Teams then prioritize the backlog and deliver incremental value in short sprints.

Mature development practices. Code reviews, establishing clear standards for design and development, proper documentation, and automated testing are techniques to reduce technical debt. Beyond coding practices, modern software applications use loosely coupled technologies and focus on event-driven and serverless components. Modern applications also use automated infrastructure, operational, and security tooling to improve software deployment dependability and consistency. Reusable and time-tested software components are a great way to speed up delivery and mitigate technical debt.

Training/Skills development. Adopting new technologies requires equipping employees with the right skills so that they make informed decisions and put those technologies into practice effectively. Take the case of cloud technology: A recent AWS Global Digital Skills study said that 87% of employers report that investments in digital skills training for their workers have allowed their organizations to achieve digital transformation goals more quickly. From the employee standpoint, more than 80% of respondents reported improvements in job efficiency and higher job satisfaction. These factors help companies realize time-to-value more quickly and reduce technical debt.

Measuring Technical debt. Multiple approaches have been proposed to measure technical debt, such as cyclomatic complexity, code coverage, squale rating, code rule violations, and even counting software bugs. Although these provide some quantitative measures of code quality, they do not measure other dimensions, such as the frequency of changes made to the code. Code that scores low on quality but is rarely changed should rank low when prioritizing resolution. Technical debt is not always obvious until we look under the covers. Y2K is a classic example of when, in software, the “year” was stored with two digits (“75” instead of “1975”) in the 1960s and 1970s. With the turn of the century, it became a major debt for all businesses. Technical debt is meaningful when expressed in terms of business impact, such as poor customer experience, lost revenue, longer release cycles, and employee productivity. This simplifies making investment and prioritization decisions.

Final Thoughts

Technical debt is real and cannot be avoided. It happens due to both controllable and uncontrollable factors. The risk of ignoring it is high as it can result in slower time to market, loss of revenue, and negative impacts on team morale. Technical debt needs to be addressed jointly by the business and technology teams. It should be made visible and prioritized. Cloud technologies offer effective means to turn technical debt into technical wealth.

Additional Reading

Agile Alliance, D. W. (n.d.). Project Management and Technical Debt. Retrieved from https://www.agilealliance.org/project-management-and-technical-debt/

Fowler, M. (2009, October 14). TechnicalDebtQuadrant. Retrieved from https://martinfowler.com/bliki/TechnicalDebtQuadrant.html

Fowler, M. (2019, May 21). TechnicalDebt. Retrieved from https://martinfowler.com/bliki/TechnicalDebt.html

Schwartz, M. (2020, December 16). The CIO-CFO Conversation: Technical Debt—An Apt Term? Retrieved from https://aws.amazon.com/blogs/enterprise-strategy/the-cio-cfo-conversation-technical-debt-an-apt-term/

Whelan, J.-L. L. (n.d.). Introduction to the Technical Debt Concept. Retrieved from https://www.agilealliance.org/wp-content/uploads/2016/05/IntroductiontotheTechnicalDebtConcept-V-02.pdf

About the author: