Rotman School of Management

Main Content

Realizing the promise of the data-driven organization

By Bernardo Blum, Avi Goldfarb, and Mara Lederman

Why is data everywhere but in the perceived results of using data? Why have many businesses been unable to leverage the asset that their data represents?

Businesses are now able to collect and analyze data at a much larger scale than previously possible. The business press has proclaimed a new "big data" era for management. As a consequence, when asked in a MIT Sloan Management Review study if there are opportunities in business analytics, managers respond with a strong "yes". When asked if their organizations are trying to improve their use of analytics, managers again response with a strong "yes". However, when asked, how whether they have all the data they need, or whether their organization does data well, the reaction is more muted. 

So, why is data everywhere but in the perceived results of using data? Why have many businesses been unable to leverage the asset that their data represents?

To understand this, it is important to recognize that there is nothing dramatically new about having data at our finger tips. What is new is that technology means that it is a lot easier to collect the data, to store the data, and to analyze the data. Digital activity means that people leave footprints. People leave a trackable path documenting whatever they do online whenever they interact with a business. Improved sensors mean that offline activity is also easily recorded. Together, this means that managers can observe many things they couldn't see before.

This improved measurement means that the role of managers has changed. Profitable use of data will be difficult for organizations that collect data but don't implement changes in management practices.

Specifically, data changes the nature of decision-making. Historically, decisions were made by the most senior person on the project. The highest paid person in the room decided what to do. Good senior managers listened carefully to the opinions and arguments of others and then declared how to proceed.

The problem with this structure is that it creates a culture of advocacy rather than a culture of evidence. In organizations driven by the opinions of the senior management, people get rewarded for convincing the senior management to implement their ideas. Data analysis is used to support the arguments. The analysis is therefore used selectively and somewhat dishonestly. As a consequence, senior managers understand that they may not be able to trust the data they are shown. Instead, when the data are not trustworthy, "going with their gut" might be the best they can do.

Data-driven organizations behave differently. The role of managers is to decide what to test. The data analysis determines what to do.

 Managers decide what to test. Data analysis determines what to do.

It is easy to say that data should be used as impartial evidence. It is harder to make that a reality. There are a number of steps that can help organizations move to a culture of evidence. In the long run, the objective is to have managers who understand the power of data and analysts who understand the business. The managers and analysts need to trust each other, even when the analyst presents results that suggest the manager's idea is unlikely to work.

To implement this, promotions and rewards need to be given for evidence not advocacy. Data-driven organizations reward people for listening to the data, rather than people who have the right hunch.

How does this work in practice? Analysis done at Ebay provides an excellent example of the value of focusing on the evidence rather than rewarding advocacy. For years, Ebay had spent a substantial sum on search engine advertising at Google and, to a lesser extent, at Bing. As in many companies, information on the large number of customers clicking on the ads was used to support the assertion that the ads worked. Advertisements shown to people who searched for "Ebay" seemed to be especially effective. People who searched "Ebay" almost always clicked on Ebay's ad.

A team of analysts was skeptical of the impact of these ads on actual sales at Ebay. They wondered whether people searching for the term "Ebay" would show up on Ebay's website, even without the ad. They were also skeptical about the usefulness of several other search engine ads. If they were right, they would be documenting evidence that the marketing team had wasted millions of dollars over the years.

To test their hypothesis, the analysts ran an experiment. The experiment showed that, indeed, much of their search engine advertising spending was wasted.

That Ebay management allowed this experiment to proceed is impressive. It had the potential to embarrass senior managers because it could have (and did) document wasteful decision-making. In the end, it was hard to argue with the evidence presented.

The role of management was to allow the analysts to collect the evidence through an unbiased experiment. In doing so, the evidence had the potential to save the company millions.

The more organizations can create a culture of evidence around data rather than a culture of advocacy, the more they will be able to profit from the increasingly important asset that is their data.

:Bernardo Blum, Avi Goldfarb, and Mara Lederman are faculty members at the University of Toronto's Rotman School of Management and teach in several of the school's programs. For more information on the upcoming Data Literacy Program visit:

See more articles like this.

© Rotman School of ManagementAASCB