– Hey Cody! So, for the next sprint, we’re going to focus on adding the favorites list to the shopping cart

– Wait, what? But Piotr, we’ve been postponing the refactoring of the Fibonacci generator for 3 sprints now! And that’s not to talk about the amount of tech debt we’ve accumulated to get the automated mailing lists out of the door.

– Those are valid points, but we’re confident that the favorites list will help with the current issue of customer retention - and this is an immediate issue that needs fixing to keep the company afloat, not just a code-perfect problem

– I hear you, but our unit-test coverage dropped to 60%, and our Fibonacci numbers are falling dangerously low. These aren’t about code perfection. We’re taking a big risk by not fixing these issues.

– Well I’ll tell you: why don’t we put a mix of these into the sprint? We can allocate 80% of the sprint to implementing the favorites list, the rest can go towards the technical issues you’re referring to.

– Piotr, you know damn well how this strategy works out! We’ll spend the whole sprint and some more working on that “favorites” thing, and just accumulate more tech debt that we’ll never come back to.

– I’m confident you’ll get this done, we’ll pull it out. But I must press that we need to work on the favorites, I’m the product manager, this is my mission to determine those priorities

– Hey folks, what’s up?

– Hi Vanessa, we’re just discussing the plan for next sprint with Cody

– Oh, and what’s the plan for next sprint?

– 80% favorites list, 20% fixing tech debt.

– Wait, but when are you going to work on refactoring the website header?

– Oh that’s for later, maybe in sprint 25.

– Sprint 25?! but that’s in 3 sprints! Marketing identified that we need to do those standardization A.S.A.P! Please prioritize that.

Vanessa goes away

– So what do we do Piotr?

– You heard her Cody. She’s the VP, I guess that was an executive decision.

Does that sound familiar?

It does very much so to me at least. We have been through almost two decades of Agile, almost a decade of DevOps and Lean Startup. And yet, it seems that backlog management remains vastly artisanal. In my experience, priorities are often defined through a combination of gut-feel, who spoke last, who spoke the loudest, who has the highest status, or who pulled rank.

There is simply no way to compare things that lack a common unit of measure.

~ Reinertsen, Donald G., The Principles of Product Development Flow

Comes in the concept of economic framework, a fantastic tool to prioritize backlogs (among other things), while shifting the discussion from gut-feeling and arguments over what individual backlog items are the most important (subjective), to a discussion around the intrinsic value of those items (objective, or at least a bit more). It also gives leaders a way to formalize the decision framework, and delegate priorities rather than tasks.

Economic framework

The key idea is that product and project attributes that have economic value should be quantified using a standardized and useful unit of measure: life-cycle profit impact.

~ Reinertsen, Donald G., The Principles of Product Development Flow

The economic framework, as introduced by Reinertsen, proposes to determine a transfer function from various proxy variables (conversion rate, response time, etc.) towards economic impact (money). According to him the five usual “sensitivities” are:

  • Cycle time - decrease time needed to release product
  • Product cost - decrease the cost of manufacturing for a product (seems to be less applicable to software than it is to hardware manufacturing)
  • Product value - increase value that the product brings to its customer
  • Development expense - decrease the cost of the product (e.g. engineering, marketing, etc.)
  • Risk - reduce development and manufacturing risks

Said otherwise: bring back everything to economics, i.e. contribution to profit, so that everything can be compared on the same scale. New products and features, bug fixes, architecture enhancements, infrastructure reworks etc.

New features and projects are usually straight forward to evaluate.

For example:

  • Making the cart of an e-commerce website easier to use might increase the cart-to-checkout conversion rate by 1%1 and so its value is 1% × daily cart value.
  • Having more information on customers and doing targeted marketing campaigns might increase visit rates by 5%, increasing revenue by 5% × conversion rate × daily cart value.

Yes! BUT…

Arguably, this is not always an easy thing to do. How do you evaluate the value of fixing technical debt? What are the dollar-amount benefits of continuous delivery for an e-commerce website? These don’t seem trivial to evaluate, let alone evaluate with a degree of precision that allows to prioritize things more effectively than a gut feeling.

For example, continuous delivery reduces the cycle time and as such, lowers the investment on each feature delivered, these are pretty well recognized facts by now2. But by how much, and how to quantize that in economic terms? The economic framework brings this back to the sensitivities illustrated above.

We can take some data and do a back of the envelope calculation2, deploying manually are costing a developer 15 minutes per deployment. They are doing 40 deployments per 2 weeks sprint, that is 10 hours per sprint on deploying to the various environments. In a 5 developers team, developing 30h per week, that is 3.5% of the productive time spent deploying releases, impacting the development expense and cycle time highlighted above3.

All this is nice and well, but how do we transition from these numbers to value? That is where Reinertsen gives his next trick: use the cost of delay as a reference.

Cost of Delay4

If you are going to quantify one thing, quantify the cost of delay

~ Reinertsen, Donald G., The Principles of Product Development Flow

The cost of delay is the recurring value of the profit lost or the money spent on delaying a feature or project. It is measured in currency / time, for example: US$ per day.

Every feature not implemented yet is removing the corresponding cost of delay from the maximum life-cycle profitability . We can also compute the total cost of the backlog, which is the sum of the cost of delay of all features present in it.

The interesting aspect of the cost of delay is that it helps evaluating optimizations (e.g. reducing debt, fixing bugs, …), additions (e.g. adding features), and improvements to the production capacity (e.g. improve process, tooling, diagnostics, etc.)5.

Going back to our examples:

  • cart improvement: 1% of the value put in cart daily, if this latter metric is $500,000, makes it a $500 / day feature.
  • marketing campaign: 5% increased visits, if conversion rate is 4% and cart value is still $500,000 would be 5% × $20,000 = $1000 / day.
  • continuous delivery: it takes 3.5% of the sprint to deploy, therefore the cost of delaying continuous delivery is a 3.5% dip in productivity. That is, every feature in the pipeline is delayed by 3.5%. If that team has a backlog with a cost of delay of $100k / day, then delay Continuous delivery costs 3.5% of that, that is $3500 / day.

By pro-rating these three features with their corresponding implementation cost, you can determine which feature will yield the most return on investment, therefore prioritizing your backlog.

Using this as a delegation framework

An economic framework identifies the transfer function from proxy variables (lead-metric6) to life-cycle revenue (lag-metric6). As such, it highlights how the organization intents to create revenue, and allows to delegate priorities.

For example, if the following function is determined:

projected daily revenue = (daily visits × average basket × conversion rate) × process improvement factor

The message that is sent is that there are 4 ways of impacting revenue:

  • increase the daily visits,
  • increase the average basket,
  • increase the conversion rate,
  • improve the process continuously

By mandating teams7 to use this as a common function to evaluate to cost of delaying features, the areas in which they can have an impact are effectively highlighted. Priorities are justified by the cold hard truth of revenue, rather than passion and status.

Conclusion

When using economic frameworks with the development teams I’ve worked with, the result has always been interesting. If only to switch the discussion towards a less passionate introspection in the value of our work, and seems to yield results that are less arbitrary.

Ideally this framework is implemented at a company level, so that everyone has the same reference. It goes beyond the product development group, and can be applied to non-product development organizations - a consulting company can use that to determine which projects it should take on ; a sales organization can determine which engagements it should focus its efforts on.

One of the main limitation I have found is its apparent complexity. It requires to convince the product management group and the engineering group of that value you get from this investment (data is your friend). It also requires the authorities to yield to the methodology and its result, and by definition they tend to want to mess with those priorities.

Even imperfect answers improve decision making.

~ Reinertsen, Donald G. The Principles of Product Development Flow

As with everything, an iterative approach seems necessary. Something imperfect is better than nothing. The simple fact of switching from raw “high/medium/low” priorities to a set of ponderated factors is already beneficial to the quality of discussions when it comes to defining work.

There is much more to the economic framework than I can explain. It can be used to exploit variability, by leveraging economics and probability of success to maximize profit in uncertain environments, and analyze fast feedback-loops. It is also the basis to manage queues, batch size and measure work in process.

If this interests you, I would highly recommend reading The Principles of Product Development Flow: Second Generation Lead Product Development by Donald G. Reinertsen.

Notes

  1. Use testing and data to validate hypothesis. A side not that decreasing release-sizes and cycle time will let you test those hypothesis faster and correct any errors made at estimation phase. 

  2. These are some made-up numbers, but Nicole Forsgren and Gene Kim find in Accelerate that high-performers have 46 times more frequent code deployments than low-performers, 5 times lower change rate failure, and 440 times lower cycle time.  2

  3. Disregarding other aspects like the lowered cycle-time, increased confidence in the system, etc. 

  4. The notion of cost of delay is pivotal to most of Reinersten’s book. I’m probably not doing it any justice here, and would injunct anyone with interest in the topic to go and read his book. 

  5. The 7 habits of highly effective people suggest that finding the balance between Production and Production Capacity (what they call P/PC balance) is essential to effectiveness. I would argue that Cost of Delay precisely gives a tool to find this balance by putting those two back onto the same scale of measurement. 

  6. A lead metric is a metric that can be measure in short term, such as a system performance, a daily conversion-rate, the number of items in a cart, or the number of projects in a pipeline.

    A lag metric is a metric that is measured on a longer term, such as quarterly revenue, year over year growth, etc.

    Lead metrics usually indicate how the lag metric might be evolving.  2

  7. That being said, it is always a good idea to accompany such rules with their intent, so as not to create a system that teams will try to game.