In-Depth

In Search of BI ROI

Hard BI benefits -- such as reductions in costs or in employee headcounts -- remain frustratingly elusive, according to a new survey.

• By Stephen Swoyer
• 09/22/2010

When it comes to return on investment (ROI), BI adopters tend to have mixed feelings.

That's one nugget from this year's installment of The BI Survey, which is now published by the Business Application Research Center (BARC).

In the vast majority of cases, the survey finds, adopters realizing "soft" ROI, chiefly in the form of hard-to-quantify benefits such as "faster" and "more accurate" reporting, or via "improvements" in business decision-making and customer satisfaction rates. "Hard" ROI benefits -- such as reductions in costs or in employee headcounts -- remain frustratingly elusive.

In painful point of fact, write authors Barney Finucune and Melanie Mack, headcount reduction is the BI ROI benefit that's least likely to be realized.

"[W]e consistently find that organizations are more likely to report headcount savings in business rather than IT departments, despite the claims that modern BI solutions will deliver self-service reporting, with a much-reduced need for IT authoring of new reports," the pair writes.

This isn't exactly surprising. To the extent that IT is a bureaucratic entity, it isn't going to pursue strategies that reduce or otherwise diminish its power, influence, or prestige.

Enter the self-service model, which permits IT to offer users a carrot -- i.e., the promise of limited or controlled empowerment -- while simultaneously strengthening its bureaucratic stick.

The reverse of the self-service coin is what some in the industry call "Workgroup BI," a user-focused paradigm that -- as envisioned by its most ambitious adherents -- proposes to take IT completely out of the loop.

Consultant-turned-CEO Scott Davis, founder of LyzaSoft, has a typically provocative take on this issue. Business users demand service, responsiveness, and flexibility, says Davis; none of these qualities has ever been associated with IT. The upshot, he argues, is that IT's highly-centralized, top-down style is no longer workable.

"You have to be realistic," said Davis, in an interview earlier this year. "You're not going to be able to impose a priori that sort of top-down model where you control everything centrally," he continued. "What you have to do is adopt that fast-follower approach that [gives you] … traceability and identity [and] allows [you] to see what is happening, understand what the best practices are, and exploit the best practices to bring them into the IT factory."

Davis's isn't a lone voice. In a report published this spring (Wisdom of Crowds), industry luminary Howard Dresner argued that business stakeholders are enjoying more input than ever into the BI buying process. Dresner cited the advent of self-service and Workgroup BI-like tools that are designed for -- and in many cases marketed directly to -- business users. He likewise depicted an entrenched IT oligarchy that hadn't yet come to terms with the diminution of its power and influence.

Suites Don't Guarantee Benefits

This year's installment of The BI Survey doesn't explicitly address the workgroup BI trend. It does, however, rank BI vendors in terms of their assessed Business Benefit Index (BBI) scores. BBI takes into account both "soft" and "hard" ROI benefits, weighing them according.

The products with the highest BBI scores are also those that (from the perspective of customers) consistently deliver both hard and soft BI benefits.

Interestingly enough, the vendors with the highest BBI scores are (with one exception) those with the least suite-centric approaches.

Board International, a Swiss-based BI and performance management (PM) vendor, grabbed the top spot, followed by Jedox PALO, an open-source reporting and analysis vendor based in Germany. Reporting, analysis, and data integration stalwart Information Builders Inc. (IBI) finished third overall.

Other top performers include workgroup analytic specialist QlikTech Inc.; arcplan, an upstart reporting and analysis player with offices in both the EU and the U.S.; and Cubeware GmbH, an EU-centric reporting and analysis vendor based in Rosenheim, Germany. (Because The BI Survey is published in the UK, it includes a few EU-based vendors -- such as Jedox PALO and Cubeware -- that don't yet have operations in the U.S.)

MicroStrategy Inc. was the only traditional BI suite vendor to score above 4.0 in this year's tally. Microsoft Corp.'s SQL Server-based OLAP offering also scored above a 4.0 (SQL Server Reporting Services didn't), and BARC awarded Microsoft's BI "suite" a 4.07 rating.

None of the big BI suite offerings scored above a 4.0.

More to the point, although users of non-suite BI and PM offerings were able to frequently reduce their non-IT employee headcounts (and -- less frequently -- their IT-related headcounts, too), users of BI suite offerings were less likely to realize headcount reductions of either kind.

Similarly, non-suite users frequently reported reductions in both external IT and non-IT costs; BI suite users, on the other hand, had considerably less success on both accounts.