Setting the Next Target and Measuring Progress Toward Value
By Matt Ring ,Jeff Gallimore ,Betty Junod ,Andrew Davis ,Dwayne Holmes
This post is excerpted from the DevOps Enterprise Forum paper “Measuring Value: Navigating Uncertainty to Build the Right Thing for Our Customers and Our Business.”
Once we understand the overall direction we are trying to go (as a company, product, or initiative), as well as where we are today (through understanding which problems to solve and identifying signals of value), we can then figure out the next target goal we want to reach. With this, we will focus on a couple of tools to help us: an Outcome Canvas and OKRs (objective and key results).
Focus on Product Outcomes, Not Business Outcomes
Product outcomes provide the leading indicators of the business results (impact) that we are seeking from our initiatives. For example, in an e-commerce product, if we want to grow revenue, we should focus on what user behaviors are likely to lead to revenue growth, such as the number of new and returning users, how long people stay on the site, how many items make their way into the shopping cart, and how many shopping cart items make it through purchase.
A useful tool to help capture desired outcomes and success measures (whether from a product or an initiative) is an Outcome Canvas from the book Sooner Safer Happier. The Outcome Canvas helps guide teams and leaders toward crafting an outcome hypothesis, as expressed in the following statement:
Due to {this insight], we believe that {this capability} will result in {this outcome}. We’ll know we’re on the right track when {quantified and measurable leading and lagging indicators}.
The benefit of using this format and canvas is that it also adapts nicely into an outcome-focused OKR. The capability and outcome help from the Objective while the leading and lagging indicators become our measurable Key Results.
We can also use the product success metric frameworks described earlier to help identify which signals of user behavior will tell us we are progressing toward our desired outcomes. When doing this, make sure to choose multiple success metrics and identify those that create healthy tension with each other (e.g., customer acquisition vs. customer retention rates; support call complete times vs. customer satisfaction).
OKRs vs. Product Success Metrics
At this point, you may be wondering how OKRs differ from product success metrics. You can think of product success metrics as those key performance indicators (KPIs) of your product or service. They are the indicators that the product or service is fulfilling the long-term mission it was set out to do. OKRs, on the other hand, are a goal-setting framework that help provide focus on shorter-term (e.g., quarterly) directional results.
For example, let’s consider a personal health tracker. A health tracker (like a Fitbit, Apple Watch, Oura ring, etc.) tracks a number of different human health insights, including heart rate, daily step count, active minutes, blood oxygen levels, sleep patterns, etc. However, an individual may set a specific health goal of losing weight and feeling more energy. To accomplish that, they choose to focus on increasing daily active minutes and the number of hours slept per night. This doesn’t mean the other insights stop being tracked. But for this current goal, the focus is on directionally changing those specific measures.
Measure Progress along the Way
Measuring value takes time and intentional effort. Experiment and align on which measures will best signal whether you are making progress toward your goal. Agree on what time horizon value needs to be realized and the cadence for checking in. Measuring progress along the way helps validate whether we’re providing a good return on our product or initiative investment. Remember, metrics are only a proxy for value creation. Be sure to connect the data in context.
Read the full paper in the Fall 2023 DevOps Enterprise Journal.