In-Depth

The BI Year in Review

From tablets to takeovers, it was a busy year for business intelligence professionals.

• By Stephen Swoyer
• 12/13/2011

Every December, we pronounce a verdict on The BI Year That Was. This time around, we have an especially full slate, and not much room in which to recap it. So, in the words of the great jazz pianist Horace Silver, let’s get down to the nitty gritty, because this has been one heck of a year in BI.

Interacting with BI Apps and the Impact of Mobile Computing

At some point in the next eight to 18 months, Microsoft Corp. will ship Windows 8, the successor to its Windows 7 operating system. From a BI perspective, Windows 8 isn’t any great shakes, but from a user experience perspective, it’s an excellent indication of where we’re headed.

Windows 8 will ship with a new user interface (UI) based on Microsoft’s Metro design language, which currently powers Windows Mobile, Microsoft Zune, and other technologies. The point isn’t that Metro is new and/or different -- because it isn’t, necessarily -- it’s that the Metro UI that Microsoft has showcased with its Windows 8 beta builds incorporates non-PC-like concepts and methods. It borrows, to a degree, from the mobile or tablet user experience, in which touch -- and not the gesturing of the mouse -- is the primary means of interacting with an application.

In 2011, the evolving BI usage paradigm officially moved beyond the desktop LCD: smartphones and tablets aren’t simply being supported, they’re being actively cultivated. One upshot of this is that nearly every purveyor of BI client offerings has a mobile or tablet strategy that tends to mix support for mobile devices of all kinds -- chiefly via HTML5-based Web applications -- and device-specific support, usually (but not always) limited to Apple Inc.’s iOS devices. We’ll say more about tablets and mobile devices a bit later in this article, as their success -- which most people saw coming -- was a big part of the year in BI.

For now, let’s just say that the way in which we interact with BI is going to change. Drastically. After all, it already has changed, and what we expect from BI applications has changed, too. Even before 2011, innovation in BI had already outstripped the bread-and-butter contexts in which business intelligence had been served up for decades. These old-school interfaces -- viz., spreadsheets, with their cells, rows, and columns; static charts, diagrams, scorecards, and so on -- haven’t so much been replaced as been supplemented: enriched.

They’re still in use -- for the majority of users, they’re still a primary interface -- but today’s BI tools mix a highly interactive, increasingly visual application environment with an increasingly collaborative user experience. They likewise bundle amenities -- e.g., self-service, sophisticated visualization, and analytic technologies -- that once seemed like the stuff of pipe dreams. Furthermore, they’re increasingly social, and that’s key.

Advanced Social Studies

The socialization of BI has been inexorable. Not fast. Not slow. Inexorable.

Over the last 16 months, for example, big-name vendors (e.g., IBM Corp., MicroStrategy Inc., Oracle Corp., and SAP AG) moved to incorporate social media-like features into their BI suite offerings. They were mostly following the lead of smaller BI vendors, such as Lyza Inc., a start-up that launched a fully-formed, social-ready BI platform almost two years ago.

The trend this year might be called Creeping Socialization. It was a phenomenon in which at least a dozen BI vendors kicked off efforts to retrofit their offerings with social media (or social media-like) capabilities. This didn’t just include prominent names (such as MicroStrategy and SAP), but scrappy players like QlikTech Inc., which radically fleshed out its social media feature set with its QlikView 11 release.

Elsewhere, vendors such as data integration (DI) specialist iWay Software Inc. -- along with its parent company, Information Builders Inc. -- and complex event processing (CEP) giant TibCo touted their own achievements in social BI. Even social BI trailblazer Lyza announced a new -- and (not surprisingly) even more sociable -- version of its flagship BI offering.

However, 2011 wasn’t the year in which BI first became social. Nor was it the year in which BI completed its social makeover. It was, instead, the first full year in which the industry gravitated, inexorably and irrevocably, toward social BI.

It was likewise the year in which social media emerged as one of several forces -- all of which have to do with how we use, interact with, and consume BI -- that will fundamentally transform BI as we know it.

No Room for Squares

BI usage models aren’t the only things changing.

This year brought a reconfiguration of the industry pecking order, with the emergence of several small or upstart competitors and the reinvention -- or reorientation -- of a number of established players, too.

Quiterian, Metric Insights, and JackBe were just a few of the BI vendors to make splashes this year. Both Quiterian and JackBe have been around for some time: the former, was -- until this year -- an EU-only player; the latter had its start (back in 2006) focusing on Ajax and SOA application development. Metric Insights, on the other hand, is a new entry, the brainchild of BI consultant Marius Moscovici. The emergence of these and other vendors illustrates one of the chief ways in which the BI industry contrives to remain competitive, innovative, and, yes, interesting -- even in the midst of significant consolidation.

They focus on the gaps; on the missing pieces; on the areas of user or IT frustration; on all of the ways existing BI just isn’t getting it done.

Barcelona-based Quiterian, for example, positions its DDWeb offering as a complement to the established data warehousing and analytic practices that exist in most organizations today. “If you want to test ideas with incomplete data or with less-than-perfect data to see if it makes sense to do something down the line using traditional BI, this is what to do,” said Alberto Saavedra, vice president of U.S. sales at Quiterian, when we interviewed him in August.

JackBe, for its part, released version 3.1 of its Presto product back in March. Presto serves up analytics as reusable mashups of different service-enabled data sources, which (in the JackBe lexicon) are dubbed “mashables.” Add in a wizard-driven self-service component and you have an approach that “make[s] any kind of data source -- regardless of its location -- equally accessible. We fuse that together with our other theme: as much self service as possible. We try to make it easy for non-technical users to mash together as many of the mashables as they want,” said JackBe CTO John Crupi, in a September interview.

Metric Insights says its flagship product, Instant Insight, is an analytics-for-the-rest-of-us kind of tool. “There [are many] companies that do a very good job at the deep analytics, [but these are] tools analysts would normally use,” said founder Marius Moscovici, told us in August. Moscovici positions Instant Insight as a lightweight analytic offering that eschews the monolithic architectures of heavy-duty BI tools. “Rather than taking ... the approach ... where you have to define a metadata layer first and then build the reports or the analysis, here you build the reports first. The metadata is the reports and the metrics."

These and other vendors say they’re exploiting gaps in the existing BI marketplace: i.e., segments, pain points, or use cases that they say are ill-served by existing offerings -- or by existing usage paradigms. It’s the same kind of approach that a previous generation of upstart or insurgent competitors -- e.g., QlikTech, Lyza, WhereScape Inc., and others -- exploited, too. It’ll likely function as an on-ramp for BI innovation, and BI differentiation, in 2012 and beyond.

Changes in Attitude

Even as Quiterian, JackBe, and Metric Insights trumpeted a new spin on traditional BI, established players repositioned themselves to exploit opportunities in the ever-evolving data integration (DI) marketplace.

ETL specialist SyncSort Inc. is a case in point. For decades, ETL has been SyncSort’s bread and butter. It still is, but with this year’s release of DMExpress 7.0, SyncSort is itself evolving -- namely, by combining its traditional focus on hot-rodded ETL with an ambitious vision -- which it dubs ETL 2.0 -- that emphasizes self-service and collaboration and which itself augurs the development of a more platform-like DI vision. Next year will be an interesting -- indeed, perhaps pivotal -- year in SyncSort’s and ETL 2.0’s development.

Vendors such as Composite Software Inc. and Informatica Corp. have been talking up data virtualization (DV) for a few years. In 2011, they seemed to be pitching to an especially receptive audience. Composite, for example, convened its second-annual “Data Virtualization Day” event in midtown Manhattan, drawing several hundred attendees.

Informatica, for its part, trumpeted its DV bona fides, even as other DI players -- from giants such as IBM, iWay Software, and Oracle to specialists including Denodo Technologies and SnapLogic Inc. -- articulated DV visions, or used DV-like language, of their own. This list of would-be DV-ers includes DataFlux, since time immemorial (or at least 2000) the face of data quality for SAS Institute Inc. Two years ago, DataFlux became the face of SAS DI. As of this year, DataFlux added DV to its list of talking points.

Everybody, it seemed, had DV on the brain. Well, almost everybody. Insurgent DI specialist WhereScape isn’t backing away from its bread-and-butter focus on ETL. This year, however, it unveiled WhereScape 3D, a kind of free test laboratory for DI. (Yes, WhereScape 3D is free. It doesn’t even require special glasses!) Although 3D can be used to generate code for Red (WhereScape’s flagship ETL tool), it can also double as a feasibility testing tool: i.e., is such a project possible, and -- more important -- at what time and at what cost?

Maximum Consumption

Few industries have produced as much acquisition activity as the BI space over the last decade. This year, consolidation in the analytic database arena -- long-anticipated, and, starting in 2010, partially realized -- continued apace, with Hewlett-Packard Co. (HP) gobbling up columnar database powerhouse Vertica Inc. Not to be outdone, Teradata Inc. acquired analytic database highflier Aster Data Inc. Along with the former Greenplum Software Inc., which was acquired last year by EMC Corp., Aster was one of the first two analytic database entrants to bring a native implementation of MapReduce to market. There are still plenty of extant analytic database competitors, however. Veterans Infobright, Kognitio, and ParAccel, along with newer entrants including Algebraix Data Corp. and Actian Corp. (the former Ingres Corp.), seem determined to keep things interesting.

Also this year, Oracle Corp. ponied up $1 billion to acquire Endeca Corp., an established vendor that specialized in faceted search. Although search was indisputably Endeca’s bread and butter -- its technology is used to power at least half of the Top 100 e-commerce sites -- the company believed it could make a big splash in the BI market, too.

“Within the last year and a half, we've had a big BI push, but even before that we started working with some early adopter organizations ... [such as] Raytheon about solving business intelligence problems, taking our search technology, adding analytics to it, adding visualizations to it, and bringing it within the enterprise,” said John Joseph, Endeca’s director of product marketing, in an August interview, two months before Oracle acquired Endeca. (This wasn’t Oracle’s only BI-related acquisition in 2011, by the way: it nabbed data profiling specialist Datanomic in April.)

Eight years ago, for the record, the former Business Objects SA ponied up $820 million for the former Crystal Inc. in an acquisition that -- at the time -- was pronounced one of the biggest in BI history. How times change!

A Torrent of Tablets

You knew this was coming, because although 2011 might not have been the Year of the Tablet, it was the year in which the tablet -- as embodied in the category-defining form-factor of the iPad -- was vouchsafed, seemingly in perpetuity, its role in the enterprise.

Enterprise IT organizations are proceeding apace with tablet computing adoption efforts: many are already using iPads (or, less frequently, competitive tablet offerings) in production, while most are at least mulling tablet strategies. In Apple’s Q4 2011 earnings conference call, for example, CFO Peter Oppenheimer claimed that the iPad enjoys almost complete penetration across the Fortune 500.

“Today, 92 percent of the Fortune 500 are testing or deploying iPad within their enterprises, up from 86 percent last year,” Oppenheimer told investors. Yes, there’s a sense in which this claim stretches credulity. After all, if 86 percent of the Fortune 500 was in fact “testing or deploying” the iPad by the end of its first year on the market, you can bet that many more were testing than deploying, and you can likewise question what it means to “test” in this context.

That said, there’s an expectation among market watchers that shops will increasingly tap tablets to complement -- and in many cases to displace -- desktop or laptop computers. This year, for example, tablet shipments will increase by as much as 300 percent -- from 18 million in 2010 to 62.5 million in 2011, according to projections from IDC. Last year, tablet unit sales came in at about 5 percent of PC unit sales (per IDC’s figures); this year, IDC puts that number at 17.5 percent. That’s a simply staggering growth rate.

Even though Apple controlled almost 70 percent of the tablet market at the end of 2011, few, if any, BI industry execs are willing to call it a one-tablet race. What’s more, BI vendors are putting their development dollars where their non-provocative/partner-pleasing public statements are: they’re developing versions of their software offerings for platforms other than the iPhone and the iPad.

"Last year [2010] was the Year of the iPad as an interactive client device. We would hear from our customers, 'We want a native app just for the iPad. We're only going to be iPad. But that tune has changed a bit over the last four or five months. What we're also seeing is that people are bringing their own devices into work, and while many of these are iPads, some of them aren't," said Jake Freivald, product marketing manager with Information Builders Inc., back in August.

That’s why IBI and other vendors are developing HTML5-based supplements to any of the native or platform-specific applications they offer. (If a BI vendor offers mobile or client support, there’s a good chance they’ll likewise offer a native app for the iPhone, if not the iPad.) In fact, a study from software development researcher Evans Data Corp. found that a majority of mobile developers are focusing on Web apps instead of native apps -- in spite of the dominance of the iPad or (in the small-form-factor mobile arena) the iOS or Android platforms.

Check back next year on your tablet of choice -- by the way, market watcher IDC predicts that Apple’s share of the tablet market will drop to about 50 percent by the end of 2011 -- as we recap the The Year That Will Be: 2012.