5 Steps to Become a More Outcome-Driven Product Team

“Don’t mistake activity for achievement.” -- John Wooden

The best product teams judge themselves not by quantity or speed of releases (‘output”) but by how much those are adding value (“outcomes”). Here are ways to shift your focus towards what is really helping customers in ways that also improve your business.

As Peter Drucker famously said, “what’s measured improves.” This depends, however, on whether you are measuring and thus focusing on the right things. It is our strong belief that the product should be oriented around improving key customer outcomes. In other words, your goals should be aligned around how customers would quantify the value your product delivers. A key outcome can be broken down further into leading indicators and product KPIs that give a narrower focus for individual product squads. A good outcome KPI pyramid gets alignment on how product success will be measured, keeps attention on progress towards customer centric goals and enables teams to quantify their value delivered. 

Once you have defined this set of KPIs, add them to your product metrics dashboard, expose the dashboard internally and review it regularly to discover and diagnose trends. This will provide transparency about progress you are making and help connect these metrics to ongoing decisions in your team’s work.

2. Incorporate Into Planning

The outcome KPIs should not exist in a vacuum, but should be connected to your strategic planning. If you follow Prodify’s Vision-Led Product Management framework, this would include referencing these as the basis for envisioning 10x outcomes, working backwards to create a strategic plan and creating a roadmap aligned around the goals you have established. This can fit into whichever planning process and planning cycles your team uses, however. Whether your plans are multi-year visions, annual operating plans or quarterly OKRs, they should reflect activities that connect to your outcome KPIs. If not, it could be a signal that you are either not fully aligned on the right KPIs or are working on unrelated priorities that may be a distraction.

The longer the time horizon involved the higher on the pyramid you should be mapping to. Over multiple quarters or years you will want to see things that have potential to improve your key outcome. For short term goals you may be focused on influencing either leading indicators or product KPIs at the base of the pyramid, with the expectation that these will build on top of each other and compound over time.

3. Quantify Feature Targets

As each squad kicks off a new feature, a valuable alignment exercise is to briefly summarize the key elements underlying why this has been prioritized and how you will judge its outcome:

• What problem it is meant to address. Features are built for a reason. Grounding this in a meaningful customer need or opportunity helps be specific about why this is being done. It also keeps focus on resolving this problem, not merely launching a predetermined feature. By “falling in love with the problem” you will open yourself up to more possible solutions in case someone on the team has a better approach -- or your first idea isn’t successful.
• The hypothesis for how that will be done. Rarely do you have 100% certainty that a feature change will be successful. This is a reason confidence factors into many prioritization frameworks such as RICE. In reality, what you build is an expression of a hypothesis based on your understanding of the customer need and product judgement. Framing it as such is a reminder that it might not be successful or sufficient. Iterations and other solutions can be considered if that happens.
• Your definition of measurable success. Here is where you connect it back to a KPI and is ultimately how you decide if the feature’s outcome was achieved. This should relate to the outcome KPI pyramid, or at least have a direct correlation to it. Perfection is not the goal here, but you will improve with practice and accumulation of internal data benchmarks. Comparing actual results vs. your estimates also increases your rate of learning from each change.

4. Build, Measure, Learn

The above planning feeds into how you execute the roadmap. As you build your features, make sure that you are set up to measure the outcomes, such as via an A/B test or other tracking. After the launch, analyze the data, summarize results and look for new insights. Equally importantly: share your findings broadly, whether in a visible Slack channel, email, squad update, demo day or a combination of the above. This takes some time to pull together, but has significant benefits, including:

• Connecting the launch to associated measurable outcomes
• Letting your team understand the impact of their work, which is motivating
• Allowing faster learning and making decisions more data driven

Importantly, your result should inform the next steps for this feature, whether it is ramping up volume, rolling back, iterating for further improvement or applying the new information to inform other similar decisions. This connects to an agile approach where one of the goals of a launch is learning that will inform the evolution of your product. Often a deeper dive can help diagnose why a result was observed to help with this. For example:

• Low trial can indicate a discovery problem or a value proposition that isn’t convincing
• Low task completion can indicate that the flow is too complex
• Low repeat usage can indicate that customers didn’t perceive sufficient value relative to the effort 

These often can be addressed, sometimes for a fraction of the initial effort! 

One additional suggestion is to combine this quantitative analysis with a review of qualitative customer feedback and targeted user research. Metrics are great at describing what is happening, but don’t explain why customers are responding to your product the way they are. Reviewing both will give you a more complete picture to inform your findings and decisions.

5. Be Outcome Focused in Recognition

As you debrief and celebrate what the team accomplished, remember “what gets rewarded gets repeated.” If you focus solely on how fast they got something done, it will reinforce that velocity is what matters most -- which is counter to what we are aiming for here. When you are trying to change a culture towards valuing outcomes, it helps to initially over-index on emphasizing the result to keep that emphasis visible and top of mind.

Many features don’t achieve their goals, particularly on the first iteration. That is normal, particularly if you are not just testing “sure bets.” At Groupon we found that only about 1/4 of A/B tests generated a statistically significant lift. Optimizely reports an even lower rate! Some of the biggest gains can be from higher risk/higher reward tests, which may also have a lower success rate. Given this, you can normalize that not all features will succeed and yet still get learning value from having an outcome orientation.

For example, there may be other outcome-related takeaways worth celebrating. These could include an iterative approach with potential to turn a loss into a win, or an important new insight that informs future product decisions. It’s better to know whether something didn’t work than to proceed on a false assumption that it did and thus compound your losses. Instead, by repeating a process focused on outcomes, you will likely find that your success rate increases with learning and your product will make bigger progress in the long run!

Closing Thoughts