INNOVATORS' INSIGHTS ISSUE

In August 2007, star Morgan Stanley analyst Mary Meeker estimated that the “overlay” advertising model introduced by video-sharing site YouTube would immediately add $720 million in net revenue to parent Google’s pockets. Unfortunately, former Merrill Lynch analyst Henry Blodget pointed out in a blog post that Meeker had made a mathematical error: Her estimate actually should have been $720,000.

Meeker acknowledged the error, sharpened her pencil, and came up with new estimates ranging from $76 million to $189 million in revenue next year. What a whirlwind week for overlay ads!

This isn’t to pick on Meeker. Trying to come up with accurate forecasts for new-to-the-world innovations is incredibly difficult. In fact, the only thing we could reasonably predict in this scenario is that all of Meeker’s estimates would have been wrong.

You see, there’s a pesky thing about how the world works. Reliable data only exists about the past. And sometimes only the distant past. For example, market research reports summarize the results of decisions made months, if not years earlier.

Making decisions based on a single forecast with a seemingly precise point-estimate often leads incumbents to pass on investing in disruptive opportunities because they just seem too small. Alternatively, they throw money on a “sure thing” that proves to not pan out. The scars from these experiences lead to organizational risk aversion.

Some of these errors are legendary. In the 1950s, IBM hired Arthur D. Little to estimate the potential market for photocopiers. The consulting company added up everything that constituted the market for copying documents at the time — carbon paper, dittography, and hectography — and concluded the market was too small to warrant investment. Of course, as Xerox and other companies made photocopying simple, cheap, and convenient, new applications emerged and the market exploded.

Companies and investors that make critical decisions based solely on a single internal or external forecast run a high risk of making similarly major investment mistakes. As Blodget wrote in his blog about Meeker’s mistaken projections, “the episode clearly reveals the risk of blindly accepting what appear to be carefully developed assumptions, not to mention the estimates derived from them.”

There are three things that can be done to make better decisions about highly uncertain markets. The first is to let patterns inform decisions. By our count, disruption has affected more than 60 different industries, including high-tech industries, low-tech industries, product industries, and service industries. Remember, a simple pattern connects these seemingly disparate developments:

• There is an important problem that can’t be adequately solved by current solutions
• An innovator develops a different way to solve the problem that focuses on simplicity, accessibility, affordability, or customizability
• The innovator adopts an approach that powerful competitors would find unattractive, uninteresting, or difficult to mimic in the near term
• The solutions is commercialized with a business model that creatively maximizes revenue potential while minimizing fixed costs
• Of course, patterns of success in different industries and different contexts can be more complicated, but when you see forecasts of blockbuster growth for a new offering that runs counter to the pattern — or you see meager forecasts for an innovation that is a spot-on — be wary of making decisions based purely on the forecasted numbers.

Secondly, it is critical to remember the importance of focusing on assumptions, not answers. See if there are analogs or benchmarks that can provide insight into whether there are reasons to believe that major assumptions might bear some kind of semblance to reality. Always be on the lookout for simple ways to turn assumptions into knowledge.

Running multiple scenarios can be a useful way to unearth assumptions and understand the range of potential outcomes. For example, Blodget ran different scenarios showing how YouTube’s new model in five years could add revenues ranging from $200 million to $13 billion.

It’s obviously hard to make serious planning decisions when numbers swing that wildly, but the point of the exercise is less about allocating today’s resources and more about gleaning deeper insight into critical assumptions and highlighting early signals that provide greater certainty into the ultimate outcome.

Finally, companies can turn to emerging tools like prediction markets, where people buy and sell virtual shares based on their belief about the likelihood of certain events. Prediction markets take advantage of the “wisdom of crowds” phenomena noted by James Surowiecki: The collective almost always does a better job at answering questions than any one individual.

Prediction markets such as the University of Iowa’s Iowa Electronic Market have proven to provide accurate forecasts of presidential elections and Oscar nominations. Companies such as Hewlett Packard and Eli Lilly use prediction markets as a tool to help with business planning.

Companies need not be beholden to the tyranny of bad forecasts. By developing a competency around pattern-based analysis, shifting focus from meaningless answers to meaningful assumptions, and utilizing a wide range of approaches, companies can increase the confidence with which they move in new directions.