Learn from Hypotheses, Not Failures

Business leaders occasionally offer to share with us stories about failed technology-inspired business initiatives so we can analyze what went wrong. Although we appreciate the offer, we typically turn it down. We’ve found that most failures have multiple interrelated causes. As a result, it is difficult to extract a reliable summary of lessons learned.

Perhaps counterintuitively, we’ve found that it’s easier to learn from successes than failures.[foot]This briefing draws from Jeanne Ross, “Why Hypotheses Beat Goals,” MIT Sloan Management Review, April 1, 2019, https://sloanreview.mit.edu/article/why-hypotheses-beat-goals/.[/foot] Invariably, the story of any success includes tales of missteps that required pivots or redos. At a minimum, successful initiatives generate observations that highlight specific decisions and actions participants feel were critical to success. As we collect data that exposes patterns of success, we can derive valuable lessons. In contrast, failures tend to look like a big mess of bad assumptions, decisions, and actions.

Many companies are trying to create environments that “make it safe to fail” or “applaud failures.” We are not trying to disparage the progress leaders have made in accepting that failures are a natural by-product of doing business. We believe this reduces non-productive finger-pointing after failures. But while we think this new attitude is positive, we sense that leaders often assume that failures will lead to valuable learning. Our data doesn’t support that assumption. For that reason, we encourage leaders to design experiments to ensure learning, rather than to simply hope that failed experiments lead to learning.

Failing fast is not a useful goal—learning fast is what matters. How do you learn fast? MIT CISR research suggests there are three imperatives: (1) rely on empowered, cross-functional teams, (2) articulate and test hypotheses, and (3) ask probing questions.

RELY ON EMPOWERED, CROSS-FUNCTIONAL TEAMS

Hierarchies are not designed for learning. They can efficiently bestow past learning from the top to those below. Given the speed of business change, however, past learning is becoming less relevant. So it’s not surprising that companies are increasingly shifting from hierarchies to empowered teams that can avoid bureaucratic bottlenecks.

Teams rely on rapid cycles of learning rather than the experience and assumed wisdom of people at the top of the organization. When teams are cross-functional, they bring many different viewpoints to the table and can thus consider a wide range of concerns. When teams are empowered, they can act fast, so they learn fast.

In an empowered cross-functional team environment, senior leaders retain responsibility for high-level strategies, but those strategies evolve based on market realities, as experienced by their teams. For example, senior executives at Philips established a high-level vision for helping people live healthy lives. The specifics of that strategy continue to take shape as Philips conducts HealthSuite Labs engagements: workshops in which customers define a critical healthcare problem or need and Philips works with them to design a digitally inspired solution. That solution can then describe new go-to-market opportunities.[foot]M. Mocker and J. W. Ross, “Transforming Royal Philips to Reinvent Healthcare in the Digital Age,” MIT CISR Working Paper No. 425, December 2017, https://cisr.mit.edu/publication/MIT_CISRwp425_PhilipsReinventingHealthcare_MockerRoss.[/foot]

Thus, empowered, cross-functional teams do more than rapidly execute strategy. They help formulate strategy—and expose strategies that are not likely to unfold as hoped. But teams will succeed only if they learn how to state expectations as hypotheses.

ARTICULATE AND TEST HYPOTHESES

Although empowered, cross-functional teams will sometimes conduct experiments that fail, it is possible to limit failures by focusing teams on hypothesizing rather than on proposing solutions. When faced with a business problem or opportunity, enthusiastic business people often rush to propose a solution. Rather than investing in answers to big questions and issues, teams should focus on developing meaningful hypotheses to accelerate learning and limit the risk of expensive failures.

A hypothesis emerges from a set of underlying assumptions. It is an articulation of how those assumptions are expected to play out in a given context. In short, a hypothesis is an intelligent, articulated guess that is the basis for taking action and assessing outcomes.

One company that adopted a culture of learning from hypotheses is 7-Eleven Japan. For more than thirty years, the company was Japan’s most profitable retailer. It achieved that stature by relying on each store’s salesclerks to decide what items to stock on that store’s shelves. Many of the salesclerks were part-time, but they were each responsible for maximizing turnover for one part of the store’s inventory, and they received detailed reports so they could monitor their own performance. The language of hypothesis formulation was part of their process. Each week, 7-Eleven Japan counselors visited the stores and asked salesclerks three questions:

• What did you hypothesize this week? (i.e., what did you order?)
• How did you do? (i.e., did you sell what you ordered?)
• How will you do better next week? (i.e., what did you learn and how will you put that learning into action?)

By repeatedly asking these questions and checking the data for results, counselors helped people throughout the company hypothesize, test, and learn. The result was consistently strong inventory turnover and profitability.

Clearly articulating hypotheses makes the path from hypothesis to expected outcomes clear enough that, should the anticipated outcomes fail to materialize, people will agree that the hypothesis was faulty. Hypothesis testing can lead to more thoughtful actions and a better understanding of outcomes, which helps teams learn from both successes and failures.

ASK PROBING QUESTIONS

Many companies embrace the idea of hypothesis testing but struggle to drive results. For example, when business people propose exactly what system or software is needed to address a problem with customer interactions, they are, in a sense, hypothesizing. A company can (and often does) implement the new system to learn if it is successful in addressing the problem (i.e., if the hypothesis is confirmed). But that is exactly the type of hypothesis and test that runs the risk of expensive failure without learning.

That’s because the hypothesis about what system is needed is actually a whole set of hypotheses and assumptions about what the software does, how employees will use it, how customers will respond to the change, and what new interactions will reveal about customers and how best to serve them. By distinguishing individual hypotheses and their underlying assumptions, a company can develop smaller, focused tests that quickly reveal which hypotheses have merit and which are offbase. That learning significantly reduces the risk of expensive failure when implementing new systems.

At 7-Eleven Japan, effective counselors didn’t just ask a salesperson’s hypothesis about what would sell the next week; the counselor asked why the salesperson was making that assumption. If the assumption was dubious, they could further explore it before placing the order. Again, this approach minimized the risk of expensive failures.

It’s not unusual for people to design tests of their hypotheses in a way that attempts to prove the hypotheses right. This can be dangerous. Pharma companies have long hypothesized that drugs in development might prevent or cure a given disease. It’s easy to fall into a pattern of finding a molecule that passes an early test and then excitedly embark on a years-long series of clinical tests to win regulatory approval of the drug. But that focus on developing proof for the regulators rather than exploring hypotheses essential to developing an effective drug can lead to long, expensive failures.

Scientists are looking at ways to reduce the risks associated with their long drug development cycle by identifying underlying hypotheses and testing to learn whether each hypothesis bears out. Most recognize that stopping a clinical phase early is better than spending years trying to prove a drug is effective only to learn that it’s not. Business cases for technology-based
business initiatives pose the same risk. People can easily focus on proving the legitimacy of the business case rather than determining whether a given hypothesis is accurate or not.

The purpose of hypothesis testing is to limit risk, and where possible, preempt failure. Thus, for the hypothesize-test-learn cycle to work, people must get into the habit of asking probing questions. Empowered teams become much more effective when leaders ask why the teams believe their goals or metrics are achievable—and uncover their assumptions—rather than just quietly assenting to their plans or insisting on an alternative.

BALANCE AUTONOMY WITH ALIGNMENT

Understandably, leaders that have progressed their careersin hierarchical environments understand how to succeed in hierarchies. The rules for success are different in companies that rely on empowered, cross-functional teams. In particular, many leaders are legitimately concerned that individual teams, while focused on solving one issue, could create chaos within the bigger organization.

As we’ve studied successful companies’ empowered teams, we’ve identified eight principles (see figure 1) that help leaders balance the autonomy of teams with the need to align those teams’ efforts.[foot]Our principles are drawn mostly from research at Spotify, BNY Mellon, Northwestern Mutual, and CarMax.[/foot] Note that the essence of this set of principles is that leaders organize around the business, data, and technology components driving business success. Leaders can then assign ownership of those components to individual teams. The teams define and pursue missions that maximize the benefits and minimize the costs of their components. Ideally, each team’s mission is autonomous from other teams, although in practice, this is difficult to achieve.

To fulfill their missions, teams establish goals, and hypotheses on how to achieve them. The teams experiment—continuously—to learn what does and does not work. Because teams invariably have interdependencies, they collaborate as needed. Meanwhile, leaders must then trust the teams’ wisdom. These leaders take on the roles of coach and coordinator, and of the resource provider who can eliminate obstacles to team success.

Empowered, cross-functional teams will surely attempt experiments that fail—and hopefully they learn from those failures. But effective teams are not designed with the idea that they should fail fast. They are most successful if they are designed to hypothesize, question, and learn.

LIMIT COSTS OF LEARNING

The role of digital leaders is not to make it safe to fail. Too many failures simply waste resources in the pursuit of vague ideas about what will lead to business success. Rather than making it safe to fail, leaders should make it safe for people to admit what they think they know (but maybe don’t) and help them formulate questions to guide learning.

To make this happen in your company, form cross-functional teams representing multiple interests and empower them with resources to address important business needs. Question their assumptions and help them articulate hypotheses. Then require that they conduct tests that will confirm or disconfirm their hypotheses. In doing so, you set them up for learning that will minimize costly failures.