In-Depth
The Year in Review, The Year Ahead
In 2007, the large, independent, publicly-traded, best-of-breed BI and PM player all but ceased to exist.
It's a tradition. Every December, we look back at the year just past, take stock of what transpired (and of what -- thankfully -- didn't transpire). It even provides a small comfort of sorts, which is most welcome in a year of such upheaval.
Consider 2007, which might ultimately be remembered as The Year of Living Dangerously -- for business intelligence (BI) and performance management (PM) pure-play vendors, at any rate. Few could have guessed it, of course, but 2007 marked the year in which the large, independent, publicly-traded, best-of-breed BI and PM player all but ceased to exist.
First, in late February, Oracle Corp. snapped up the former Hyperion Solutions Corp.; then, eight months later, SAP AG picked up Business Objects SA. The latter acquisition was as big of an October surprise as the industry has ever seen -- and don't forget that Oracle once sprung an October surprise of its own, two years ago, when it nabbed arch-rival Siebel Systems Inc.
Canadian BI powerhouse Cognos Inc. was the last of the Big Time Pure-Plays to fall, just last month, to IBM Corp. In retrospect, says Wayne Eckerson, director of TDWI Research, BI's Big Three fell like so many dominoes. "I think the BI vendors have seen for a long time that the ERP vendors and the big database vendors were going to come into this space, so they want to bulk up to prepare themselves which they did … through acquisitions," Eckerson remarked just after IBM's acquisition of Cognos. "Ultimately, once one of them went and had a partner with big, deep pockets, they all had to go, so it's kind of a domino effect."
Nor is it simply a matter of the BI Big Three going away. Collectively, Eckerson notes, Oracle, SAP, IBM, Hyperion, Business Objects SA, and Cognos acquired more than a score of other BI vendors over the last several years. SAP, for example, picked up PM specialist OutlookSoft earlier this year -- just months before it nabbed Business Objects -- while Business Objects itself nabbed a PM best-of-breed of its own, the former Cartesis. A lot of players have been wiped off the board -- leaving a lot of customers
The upshot, experts say, is a drastically altered -- and still largely confusing -- BI- and PM landscape. Just how altered and confusing is anyone's guess.
From a customer perspective, buying habits will both change and remain (more-or-less) business-as-usual. Industry veteran Cindi Howson, a principal with BIScorecard.com, breaks customers down into two types: "strategic buyers" (customers who look to buy more software from vendors with whom they have established relationships) and "tactical buyers" (customers who focus on short-term success, features, functions, and ease-of-deployment).
In the case of the former, Howson says, industry consolidation gives them more incentive than ever to ditch independent pure-play solutions and standardize, once and for all, on a preferred vendor. In the case of the latter, of course, there's ample opportunity to shop around, play vendors off against one another, and deploy best-of-breed BI technology while at it.
"All these preferred vendors now have [or will have, once their acquisitions close] viable BI products as part of their total software portfolio," writes Howson on her BIScorecard.com blog. "When a preferred vendor has a solid BI solution, the buying approach changes from 'who has the best product for us' to 'why can't we use the product from our preferred vendor?'" she points out. Of course, this approach isn't quite as simple as you might think -- especially given the prevalence of multiple preferred vendors in most organizations: "The biggest battle for market share here will be when a customer considers more than one of the big four vendors to be a preferred vendor -- so if for example, Microsoft and SAP both have toeholds in the same customer account."
This isn't a totalizing shift, however, Howson concedes. "This is not to say that enterprise customers will only buy from their preferred vendors. It does, however, mean that it will be a tougher sell for other vendors," she argues. "BI pure plays who fail to differentiate and clearly articulate where, why, and when they are a better investment than the incumbent software vendor will lose share."
Now more than ever, she says, BI buyers must do their homework. "BI buyers who understand the unique value proposition of their desired BI pure-play … [should] be prepared for a more rigorous cost/benefit analysis when convincing IT management of 'why not the preferred vendor,'" Howson concludes. "The smartest companies will develop clear guidelines of when to use the standard, preferred vendor and when to supplement capabilities with product from another vendor."
Dashboards and Data Visualization
Dashboards (more precisely, dashboard-driven data visualization technologies) figured large this year. In 2007, it was all about eye candy.
First, at the very beginning of the year, the former Cognos Inc. snapped up the former Celequest Corp. in a multi-billion dollar deal. Celequest gave Cognos best-of-breed dashboard capabilities as well as a full-fledged dashboard appliance. Several months later, enterprise application integration (EAI) giant Tibco Software nabbed best-of-breed data visualization specialist Spotfire Inc., augmenting its EAI line-up with Spotfire's data visualization and event-processing capabilities.
Meanwhile, data visualization best-of-breeds (as well as would-be best-of-breeds) continued to innovate: Tableau Software Inc. announced three major revisions or new product introductions in 2007, including a new "dynamic dashboard" technology that exposes its data visualization technology in an interactive dashboard interface. Tableau also delivered an analytic dashboard add-on for Microsoft Dynamics CRM package. Not to be outdone, competitor Advizor Solutions Inc. also announced a new software-as-a-service version of its data visualization software (SalesAdvizor for Salesforce.com's AppExchange) earlier this year.
Elsewhere, BI pure-play MicroStrategy Inc. introduced a new Active Dashboards offering that officials billed as a Web 2.0-ified dashboard technology, while would-be data visualization powerhouses such as Microsoft Corp. and Business Objects prepped eye-candy-laden data visualization and dashboards offerings of their own (Microsoft's PerformancePoint Server 2007 and Business Objects Xcelsius, which now ships as a standard part of Crystal Reports).
Proponents say dashboards and data visualization will continue to be key. In fact, some industry talking-heads argue, 2008 could be an even bigger year for the technology, and that's saying something.
"It's just a great time to be in the data visualization business. There's just data coming from all angles, and the Internet is exacerbating that," says Kevin Brown, vice-president of marketing with Tableau.
He's got a dog in the race, to be sure -- Tableau is one of the most prominent best-of-breed data visualization players -- but, as Brown notes, "We think the market for BI is actually in the tens of millions, as opposed to the tech-savvy users, the sort of TDWI guys -- the guys who've mostly been using BI [up until now]."
Data Integration Still Front and Center
Even the sleepiest of technology segments can undergo an occasional revival. Consider what's been happening in the sleepy data integration (DI) sector over the last three years: when IBM Corp. plunked down over $1 billion for the former Ascential Software Corp. in March 2005, the gloves came off, the sparring turned serious, and just about every major player let fly.
Informatica Corp., for example, found its legs, gathered its wits, and vaulted to the forefront of the DI market by positioning itself as a Data Integration Switzerland. Other players also augmented their DI chops: Oracle (which released a significant refresh of its Oracle Warehouse Builder in the summer of 2006 and acquired ETL specialist Sunopsis a few months later); Sybase Inc. (which acquired EII specialist Avaki and ETL specialist Solonde AG during that fateful summer of '06); IBM -- which fleshed out its unstructured content management strategy with the acquisition of the former FileNet (also in the summer of 2006), and this year bolstered its ETL capabilities by acquiring changed-data-capture (CDC) specialist DataMirror Inc.; Microsoft; and SAP.
Microsoft, of course, released a substantially retooled version of its former Data Transformation Services (rechristened Integration Services) with SQL Server 2005, while SAP got itself a best-of-breed ETL tool (and matching data quality technology) when it nabbed Business Objects a couple of months ago.
Nor is DI innovation a thing of the past: Business Objects, IBM, Informatica, and Sybase Inc. all announced new revs of their data integration or data quality suites this year. These vendors talked up the real-time credentials of their DI offerings, touting a universe of complementary services -- including EII, event-processing, legacy connectivity, and (in the case of both IBM and Informatica) collaboration -- that they say are crucial DI differentiators.
What's ahead? DI advocates think we'll see a host of drivers, including, in particular, master data management (MDM) and regulatory compliance, which will continue to turn up the heat on enterprise DI efforts.
As a result, another established DI technology category (data quality) will become even more important.
"[D]ata quality is fast becoming a key business asset," says Arvind Parthasarathi, senior director of solutions with Informatica. He cites research from market watcher Gartner Inc., among others, which projects that the DQ tools segment could grow to nearly $700 million by 2011 as more and more businesses tackle lingering data quality issues.
"[T]he importance of data quality will influence the business environment dramatically in 2008. The quality of data can affect our day-to-day lives without us even thinking about it," Parthasarathi points out. "From a business point of view, 2007 saw mergers and acquisitions hit a record high; we can only assume that the number will grow in 2008, thus increasing the need for accurate and reliable data. As a result of this trend, organizations continue to be more accountable for the quality of their data [because they must] adhere to governmental regulations and industry compliance and demonstrate accountability to customers and shareholders."
Data Warehousing Reborn
Surprisingly, the bread-and-butter DW segment itself also seems to be seething. This change -- this ongoing transformation -- became most apparent this year, although rumblings were felt as far back as 2002.
Thanks to a number of market drivers, including both the emergence of DW appliances and the push toward real-time DW, the data warehousing segment is as exciting as ever. There are a host of new players, for one thing: not just DW appliance start-ups like Dataupia Inc. (which launched earlier this year); nor, for that matter, would-be appliance giants such as Oracle's Optimized Warehouse for Sun (which launched just last month), Hewlett-Packard Co.'s ambitious NeoView DW appliance (which relaunched earlier this year), or an independent -- and newly invigorated -- Teradata, which separated from parent company NCR Corp. a few months ago.
No, new players means just that: new players. Companies such as ParAccel and Vertica, both of which market high-performance columnar database technologies. Along with Dataupia and Kognitio (which, although an established player in the UK and EU markets, is relatively new to the U.S.), they comprise a bumper crop of 2007 upstarts: companies that, for one reason or another, burst on the scene, and on customers' radar screens, over the last 12 months.
Ironically, many of these start-ups rely on relatively mature technology models: Netezza Inc. more or less invented the DW appliance five years ago, for example. Columnar database structures aren't anything new, either: Sybase has marketed its seminal IQ columnar DW system for more than a decade now, while Sand Technologies, another columnar database specialist, has more than two decades of experience behind it.
Nevertheless, both technologies are hot: IBM, HP, Sun, and Oracle (along with established players Netezza, DATAllegro, and -- depending on how one defines "appliance" -- Teradata) all market DW appliances, while Dell Computer Corp. markets a DW appliance of its own.
Columnar database technologies are sizzling, experts say, because they're uniquely equipped to deal with the twin demands imposed by real-time data warehousing scenarios: high volumes and high performance.
In addition to boosting performance by several orders of magnitude over their relational kith (both ParAccel and Sybase, for example, tout improvements of 100x or more in specific situations), columnar databases are also highly compressible -- much more so than their relational brethren. The result, experts say, is a measurable uptick in sales of columnar database technologies. That, in all likelihood, is a trend that we'll continue to see in 2008.
"For DW, the columnar approach has a lot of advantages," says Philip Russom of TDWI Research. "[T]oday we're seeing a slight uptick in the popularity of columnar database technologies. There are suddenly all these competitors," he continues, noting that the enterprise "proof of concept" for columnar database technology was established by Sybase IQ a decade ago.
"For some applications, they [columnar databases] deliver much faster query response times, because of the structure of the [columnar] database itself. They're much more compressible, too." For customers whose needs outstrip the capabilities of conventional relational DW systems, Russom says, columnar technologies offer a compelling (and cost-effective) alternative.
Collaboration is Key
Finally, more industry players are recognizing the importance of collaboration. Both IBM and Informatica, for example, touted the collaborative capabilities of the next-generation DI suites they delivered this year. Meanwhile, many purveyors of BI client tools have gotten hip to the power of collaboration, too: Business Objects, Cognos, MicroStrategy, and Tableau -- just to name a few -- all sought to emphasize their collaborative chops this year, and with good reason.
On the DI front, for example, collaboration is crucial, experts say.
"Collaboration has become a pressing requirement for data integration in recent years," explains TDWI's Russom. "As the number of data integration specialists continues to grow -- into double digits in some organizations -- so increases the need for development processes and tool functions that help them communicate and collaborate. Likewise, stewards, business analysts, and other business people are joining the 'extended' data integration and data quality team, which brings a new slew of collaborative requirements."
Tableau's Brown, for his part, links the emergence of collaboration -- as a potential killer app, at any rate -- with the growing popularity of social networking technologies. "I think this Web 2.0 world, especially around social networks, MySpace, Facebook, LinkedIn, is creating this expectation for people … to want to interact with other people and post things -- like pictures, music, or thoughts, in a social kind of setting," he concludes.