When I was at Thrive Market, we did a massive technical debt paydown (think: no features for a quarter). Leading into that, I led an all-day workshop for engineering management and tech leads around how we get a handle on the technical debt the company had accreted over time. For context, at the time the company was on a legacy code base in PHP, primarily built around a PHP ecommerce framework that was end-of-life for years.. among many other similar issues.

While the bulk of the workshop had to do with socializing an understanding of technical debt, the meatiest bit I want to keep referencing over time has been the scoring mechanism.

The framework#

We brainstormed a list of all the stuff that should be fixed. What’s preventing us from really delivering quickly? For each of those items, we walked through a simple formula: (blast_radius * frequency_of_pain) + (engineering_velocity * trend). Blast Radius and Frequency amplify each other because they’re indicative of how big of an issue this is. Velocity and Trend join up because they track how bad it’s going to be soon. When things are bad now and are going to be worse soon.. we need to tackle it quickly.

Each of these categories was scored on a 5 point scale.

Blast radius: What’s impacted when this is bad? 1 = a single team, 3 = critical path for a product area, 5 = company-wide outage, data loss or security breach

Frequency of pain: How often does this suck? 1 = quarterly annoyance, 3 = weekly friction, 5 = daily impediment

Engineering Velocity Impact: How much does this slow us down? 1 = mildly annoying, 3 = you have to plan around it, 5 = you can’t ship until it’s fixed

Trend: Is it getting worse? 1 = stable annoyance (flat curve), 3 = slowly degrading (linear), 5 = accelerating problem (superlinear / exponential)

From these, we used their aggregate score to say when things needed to be done.

  • score of 31+.. it’s a critical issue. An immediate priority.
  • score of 21-30.. it’s important. Next quarter.
  • score of 10-20.. moderate. Roadmap it.
  • score < 10? Monitor it.

A few examples#

Just to take a few recent examples I’ve encountered.

  1. We have multiple kubernetes clusters. Most of them have the datadog tooling hooked into argocd. One of them does argocd.. but via terraform. This means that if you want to update the datadog operator version across all the clusters, you need one PR for the k8s yamls and another for the terraform repo. Blast radius=1, frequency=1, velocity=1, trend=1; so 1*1+1*1 = 2. Not a big deal.

  2. We have a team who owns a data store. The architectural boundaries are wonky and it causes no end of consternation for both the team and the users of it. blast_radius=5, frequency=3, velocity=3, trend=3; 5*3+3*3 = 24.. we need to focus on this in the next few months.

  3. Our DORA metrics calculation is flawed due to a monorepo multi-counting deploys and releases. blast_radius=3, frequency=3, velocity=2, trend=2; 3*3+2*2 = 13; we need to put this on the roadmap.

Using this#

This isn’t some magic formula that will solve all tech debt. It is a useful tool when trying to understand which of the items in your technical debt list should be done soonest when it comes time to do quarterly planning.