Outcomes Over Outputs – Interview with Barry O’Reilly; sought-after speaker, author and business advisor.

Managing Outcomes Over. Outputs : more than just semantics

One of the most consistent observations we have made here at Just3Things when working with our clients is the exponential difference in organizational culture, productivity and effectiveness when leadership and teams manage to outcomes over outputs.  

We spoke to one of the leading experts in this field, Barry O’Reilly.   Barry is the author of recently released Unlearn: Let Go of Past Success to Achieve Extraordinary Results, and co-author of the international bestseller Lean Enterprise: How High Performance Organizations Innovate at Scale  — included in the Eric Ries series, and a Harvard Business Review must read for CEOs and business leaders. 

He works with many of the world’s leading companies, from disruptive startups to Fortune 500 behemoths, to break the vicious cycles that spiral businesses toward death by enabling culture of experimentation and learning to unlock the insights required for better decision making, higher performance and results.

 

 

J3T: Hi Barry! Let’s start at the beginning – what is the difference between outcomes and outputs?

Barry: It feels deceptively simple but can make a huge impact on how you, your teams and organization perform: Outputs are what you expect to ‘produce’, the products, goods and services which result from an intervention. Outcomes are quantifiable changes in behavior you will have caused due to the output you created.

 

Output  Outcome
Launch newsletter signup Capture contact information of 25% of site visitors 
Certify 50 scrum masters  Reduce user story cycle time by 50% 
Invest $1,000,000 in online marketing campaign  Increase user activation rate per application download from 20% to 50% 
Add feature for ‘$50 Gift For Friend Signups’ to our user product page Increase our user growth rate by 20% in the next three months
Deploy scripts to automate software release to cloud infrastructure Reduce lead times for software releases by 50% 

 

J3T: So why is it better for teams and organisations to define Outcomes?

Barry: An outcome is the level of performance or achievement that occurred because of the activity, product, good or service your organization provided—more specifically the change in customer behavior that occurred. Outcome measures are a better indicator of effectiveness than output measures, for example if you completed the activity on time, on budget and scope but no customers use your product would you consider the effort a success? Outcomes quantify and assess how the product or process you created enable customers achieve the impact they are aiming for. 

Spending the time to define outcomes is always trickier than defining output but gives lots of advantages, including:

 

  • Allowing teams to create a shared understanding of why they are working on an initiative and what success looks like. 
  • Open up options for how teams could achieve the outcome they are aiming for. 
  • Increase the meaningfulness of work because teams can tie their efforts to results and gauge if they moving towards the desired outcome they seek (or not).
  • Creating more of an experimental approach – defining customer outcomes and asking teams to take responsibility at this level opens up the opportunities to experiment with outputs. Did X move the dial? No? In which case let’s try Y.
  • Balancing the need for long term planning with a ‘test and learn’ approach. Most large enterprises are adopting agile or lean ways of working based on the promise of delivering better customer outcomes faster — yet the majority still measure output as success. It can be daunting for stakeholders who are used to annual planning cycles and predictive controls such as time, budget and scope to switch techniques to manage uncertainty. Defining outcomes help engage stakeholders in valuable dialogue on what behaviors they expect to see from customers as indicators of success over  planning every feature that they will release that year with little real insight and limited customer feedback of they are useful, valuable or even needed. Ask stakeholders what they expect customers to be doing differently as a result of the output you create, then measuring and monitoring this can be a great way to help them get started. 
  • Not creating a culture of ‘busy fools’ – in many output driven organisations we see lots of activity and people working really hard, but without stopping to ascertain whether what they are doing (output) is creating any value for the customer or organisation (outcome) until the output is full complete. Our mission should be to learn as quickly and cheaply as possible if our effort is driving the results we’re aiming for.

 


J3T: Sounds great! So how do organisations start?

Barry: I encourage teams to think big but start small, and learn fast as they explore how to deploy an outcome-based approach to their portfolio of work. Don’t change everything all at once — start small, tiny in fact. Identify one product, or new feature or backlog item, start there by trying the following:

 

  • Keep asking ‘why are we doing X’ – for example why are we releasing this feature? What difference will it make and to whom? What change in customer behavior do we expect to see? 
  • Don’t get hung up searching for the one perfect metric or baseline data — start where you are and define what customer behavior you would expect or would be different if the output was available to the world].
  • Try to measure rates and ratios over totals — aiming for a percentage increase is better than saying ‘we can’t define this outcome and so we will see if we can get five customers use it’. Instead, say ‘60% of customers must use the feature’ to test its valuableness, and support an experimental approach’
  • Don’t forget that outputs can also be constraints to foster innovation – we typically think about outputs as a control metric (e.g. deliver this project with $1,000 or release this feature by this date) which force feedback cycles to happen, and reflect on our results. Use output metrics within reason as a forcing function for innovation, for example an outcome a customer success team might be responsible for could be increasing customer retention by 20% in the next month. 

 

J3T: What do you see as the key challenges to organisations defining outcomes?

Barry: One of the biggest challenges is that people are so used to managing to deadlines, budget and scope they don’t know how to measure value in any other meaningful way. 

When I present the idea of looking at outcomes they can see the benefit but don’t know how to start writing outcome based measure of success — it is daunting. 

Another barrier is trust between ‘business’ and technology stakeholders in organizations. Sometimes business partners can feel that you are trying to trick them, plus they to feel comfortable managing to output because it’s the way they have always managed initiatives. 

When people are time poor, the pressure is high and they have been sold new process as the way to succeed, their appetite to experiment is much lower. This is why thinking big but starting small is key, and showing people how the method works and could a better approach. The breakthrough comes by working, learning and unlearning together. Review your existing initiative dashboards, notice if the measure all output or outcome based. Try to identify and introduce one new outcome based measure on your dashboard that you can use as a pilot for a new way of work — then scale from there.

 

What if the metric is lagging indicator?

When people start to embrace outcome-based methods they often use metics such as revenue, profitability and customer satisfaction as a measure of success — but these are all lagging indicators. 

Businesses and teams need to think about how to also create leading indicators as the outcomes they are aiming for, and behavior they know mean the customers is on the path of success.

For example, at Netflix increasing revenue through customer retention of the service is a key outcome for the business. But if you’re working there, waiting each month for customer retention results, and even colerating your work to such a top line metric as customer retention, is challenging. 

That’s why creating leading indicators for the outcomes you aiming for is important and more actionable for the team. Therefore, if you’re working on the Search team at Netflix, and know retention is important you can ask a question such as, “what customer behavior in search would be a leading indicator for retention?” Maybe you’ll say the lead time from searching to starting a movie. You can then model and measure how quickly people can search for, discover and starting movie via your search functionality and see if that impact lagging indicators like retention rate of those customers. 

This takes work and skill but ideally you would look at a mix of leading and lagging indicators— you need to be thoughtful and have checks and balances. But getting started is the first step, so start small and learn fast when works for your context. 

 

 

What other behaviours can stand in the way of teams focusing on outcomes?

Leaders struggling to devolve decision making – or on the flip side, teams being afraid to make decisions. 

The incentives in place in a business make a huge difference too – leaders can say that the team should experiment, try out their ideas to succeed and work together toward outcomes but if leadership behavior shift when bonus are on the line, or paid out individuals on completion of projects on time, budget and scope this is totally mixed messages.

For me, the trick is to find  the cadence for your context; how do we get into the rhythm of making smaller more frequent bets and outcomes with leading indicators rather than creating big upfront plans with the output predicted in terms of time, budget and scope ? Every company has different requirements for different levels of fidelity of plans and some of this will also vary by stakeholder. 

Figure out how best your teams can keep stakeholders updated with the bets that they have made and whether this has moved the dial on the outcome metric you’ve agreed together matter. This is a very different way of working than planning everything out at the beginning of the year but the benefits you can realize in terms of organizational culture, productivity and effectiveness once you learn can be extraordinary.

 

J3T: We hear a lot about psychological safety – how important is this?

The quality of safety in organisations will directly impact the quality of information that you get. 

Ensuring that individuals and teams know how to design safe-to-fail bets is important to enable and make sure they feel they can learn from mistakes. How openly and honestly teams can and are willing to share their learning so that others can jump up the learning curve more quickly is an outcome every organization and leadership team to be aiming for. 

You can think big, start small and learn fast tomorrow by asking your teams on a scale of 1-10, how safe do they feel to share open and honest information? Accept the score they give, and start small to improve. Ask what you can do as a leader to help improve that score by half a point. Pick one suggestion for a week, try it and learn fast what works and doesn’t. Role model your efforts to unlearn outdated behavior and thinking as you seek to create a create of experimentation and learning to help others succeed. 

 

 

To learn more about Barry you can;

 

Read Barry’s blog at: www.barryoreilly.com

See what he has to say on Twitter: @barryoreilly

Subscribe to Barry’s podcast at: www.barryoreilly.com/podcast

Join Barry’s newsletter at: www.barryoreilly.com/newsletter

Leave a Reply

Your email address will not be published. Required fields are marked *