Analytics 101

Why Analytics Projects Still Fail: Feels Like Groundhog Day

Why is it so hard? Why have so many data analytics projects failed to deliver on expectations? Why do analytics programs and initiatives so often run into seemingly inexplicable barriers and resistance?

Much of my consulting work has focused on helping clients better leverage information technology and data to improve business processes, decision-making, governance and risk management. I witnessed first-hand the trials and tribulations associated with efforts to improve capabilities to turn data into information, and information into actionable insights.

Business Intelligence or “BI” has been around for quite some time as a software category and a technology practice area. There are also other categories of technology solutions for information management, including Corporate Performance Management, Financial Performance Management, and Enterprise Performance Management. Whatever the name, the reality was that much of the focus was on reporting, not really on analytics or developing insights to make better decisions. Performance management systems were historically used just to measure and monitor rather than truly improve performance, or to link insights to impacts on an organization’s ability to execute strategy in a dynamic market.

My interest in data and information management grew significantly in the wake of the 2002 Sarbanes-Oxley Act. I led an enterprise governance practice and had responsibility for consulting programs to help clients comply with Sarbanes-Oxley. Most organizations found problems, but not about accounting, per se. Rather, the problems had to do with people, process and technology. Some companies actually found benefit from their compliance efforts, as they provided the impetus to improve information management processes and systems. I co-authored a paper that explained this experience. The April 2016 article, published in Harvard Business Review, was entitled “The Unexpected Benefits of Sarbanes-Oxley.” [1]

At the time, I asked myself, “if organizations struggled to produce information that was required by regulation, i.e., statutory financial reporting, how good were they in other areas?” In 2005, CFO Research Services, in collaboration with Deloitte Consulting LLP, conducted a survey that revealed the pervasiveness of poor information quality. The initial survey summary, titled “IQ Matters”, reported that a majority of respondents did not have ready access to high-quality, reliable, useful information regarding operating and financial performance at their companies.

A 2007 follow-up to the initial survey, titled “Look Closer, Look Further”, found that the needle had not moved much. Yes, companies had improved in mandated information management activities, such as reporting financial results, but they continued to struggle to produce the timely, accurate and insightful information needed for strategic planning, board oversight, governance, making investment decisions, and identifying, monitoring, managing or mitigating risks. Nearly half of senior finance and IT executives surveyed reported that their companies struggled to produce the desired quality of information needed to make good business decisions.

Since then, there have been countless articles, speeches and presentations about why data and analytics projects fail. I have personally written and spoken about the issue often. Many analysts, practitioners and observers have researched the topic and published thoughtful papers over the years. What’s really interesting is how much of what was written more than a decade ago is still relevant today.

Here’s the bottom line: The most commonly cited reason that data and analytics projects and programs fail to fully deliver on their potential has to do with leadership. In some cases, it is a lack of senior executive leadership. In other cases, there is executive “buy-in” or “support”, but not accountability. The hard truth is that you cannot really delegate leadership accountability when it comes to the type of change required. Becoming a data-driven organization and fully leveraging the power of data analytics, requires persistent and continual attention of the senior executives to overcome inertia, change mindsets and defeat counterproductive behaviors.

Embracing analytics is not about reporting and it is not just about having more, better or faster data about existing processes and performance. Indeed, becoming analytics-driven requires fundamental changes at many levels. An April 2014 Harvard Business Review article entitled “Why Your Analytics Are Failing You” summarized it quite well:

“The real challenge is recognizing that using big data and analytics to better solve problems and/or make decisions obscures the organizational reality that new analytics often requires new behaviors. People may need to share and collaborate more; functions may need to set up different or complementary business processes; managers and executives may need to make sure existing incentives don’t undermine analytic-enabled opportunities for growth and efficiencies.” [2]

There is a lot said in those two sentences. There is a human element that often works against the needs of the overall organization. The insights from new analytics can be threatening to the status quo. I have witnessed it myself on many occasions. The insights from new data analytic capabilities are not always welcomed. Sometimes they are challenged and disputed outright. Sometimes passive resistance erodes early enthusiasm. Check out this excerpt from an August 2016 Harvard Business Review article entitled “The Reason So Many Analytics Efforts Fall Short.”

“Only a little more than one in three of the three-dozen companies that we studied met the objectives of their analytics initiatives over the long term. Clearly, driving major innovations with analytics was harder than many executives expected.

Further study of the less-successful cohort revealed that leadership issues were often at the heart of the problems.

… a feeling of vulnerability settled over the other executive team members when the analysis conducted by the analytics group revealed inefficiencies and missed opportunities in their respective functions.

In all too many cases, the CEO devoted little time to trying to manage this dynamic. As we mentioned in an earlier article, the CEO must play a leading role in establishing the analytics function in his or her organization.” [3]

In October 2019, The Enterprisers Project published “5 Reasons Analytics Projects Fail.” Reason #5 was “The Analytics Project Lacks Executive Buy-in.”

“Even when all of the above [i.e., reasons 1-4] are taken care of, initiatives will fall flat if they lack an executive mandate. Change is not easy as the natural human tendency is to resist it. In addition, organizations often have conflicting priorities. This makes new initiatives highly vulnerable in their early stages. If not nurtured carefully, transformation projects stand very little chance of success.

How can you address it[?]: Innovations must be led from the top to see the true benefits. Executives must present a vision for the future and rally people towards it. You need to push firmly to abandon old habits, at times, with unpopular calls to avoid a relapse. Make sure to onboard leaders at the next levels who can champion the initiative and act as change agents.” [4]

How often have you heard or read that technology limitations are at the root of failed data and analytics initiatives? There was a time when there were, in fact, technology limitations. (Do you remember “middleware”?) But for at least the past decade, the limiting issues have been more about leadership. If you have been in the trenches working to bring about transformation through substantial investments in data analytics, you know that people make or break the initiative. In today’s cloud and app-driven world, and with more efficient development practices, executives can get the information they need faster and at lower cost. And yes, they can transform their organizations. The question is how badly do they really want to transform.


[1] “The Unexpected Benefits of Sarbanes-Oxley” by Stephen Wagner and Lee Dittmar, Harvard Business Review, April 2006

[2] “Why Your Analytics are Failing You” by Michael Schrage, Harvard Business Review, April 2014, https://hbr.org/2014/04/why-your-analytics-are-failing-you

[3] “The Reason So Many Analytics Efforts Fall Short” by Chris McShea, Dan Oakley and Chris Mazzei, Harvard Business Review, August 2016, https://hbr.org/2016/08/the-reason-so-many-analytics-efforts-fall-short 

[4] “5 Reasons Analytics Projects Fail” by Ganes Kesari (Lead Contributor), The Enterprisers Project, October 18, 2019

Related Articles

Back to top button
X