In-Depth
IDC Report: Dominant DW Vendors Face Challengers
A new study from IDC shows that which companies continue to dominate the data warehousing market and which are enjoying surging growth
Before we assess what happened to the data warehousing (DW) market segment in 2008, it's worth taking a look back -- just one year.
The results, albeit belatedly, are in: the market watchers at IDC have crunched their numbers, highlighted a few salient trends, and drawn their conclusions. The result is IDC's annual DW market study, a mammoth report that's usually six months or more in the making.
This year's study from IDC (Worldwide Data Warehouse Platform Software 2007 Vendor Shares) paints a picture of untrammeled -- if not runaway -- DW industry growth. DW software revenues grew by nearly 15 percent in 2007, easily outpacing overall software spending. IDC says DW software sales have surged since 2005, consistently notching double-digit growth -- typically in excess of 13 or even 14 percent annually.
IDC breaks the DW space down into two sub-segments: data warehouse generation and data warehouse management tools. The former provide nuts and bolts cleansing, transformation, loading, and administrative capabilities; the latter consists of database management software (DBMS) used to manage the data inside a data warehouse.
Let's look at the second category first. The management tools segment includes revenues attributable to sales of DBMS software and accounts for more than three-quarters of the entire DW market, and tends to tip the scales in favor of the big RDBMS vendors. Not surprisingly, the three biggest purveyors of DW management tools are also the top three RDBMS vendors: Oracle Corp. sits at number one, followed by IBM Corp. and Microsoft Corp. Rounding out the top five are yet another prominent RDBMS vendor -- Teradata Corp. -- and BI independent SAS Institute Inc., a perennial ETL market leader.
Although there are few surprises in that market segment, the lineup is more interesting in the market for data warehouse generation tools. IBM -- steward of many integration assets, including those of the former Ascential Software Corp. and DataMirror Corp. -- clocks in at number one, finally surpassing SAS, which topped IDC's tally for 2005 (by a comparatively wide margin) and 2006 (by a hair's breadth).
Another traditional ETL vendor, Informatica Corp., was third in the list, trailing SAS and IBM by less than a point, followed by RDBMS vendors Microsoft and Oracle. The rest of the Top 10 is rounded out by a quintet of stalwart, new, or (in some cases) unconventional vendors, including direct-mail giant Pitney Bowes (steward of the former Group 1 Software), Accelrys Software Inc., Sybase Inc. (not surprising, given the company's ETL and data management stack), Information Builders (steward of integration specialist iWay Software), and Evolutionary Technology International (ETI), which trumpets a unique spin on data integration (see http://65.115.104.36:8005/News/display.aspx?id=8958).
RDBMS Heavyweights Poised for Dominance
The overall market -- data warehouse generation and data warehouse management tools combined -- is full of familiar RDBMS titans. Oracle is the largest combined powerhouse, controlling nearly one-third (31.7 percent) of the entire market. IBM was (a distant) second, with about one-fifth of the market. Microsoft was third, with a 13.5 percent share; Redmond was also the fastest growing of the four largest DW software players, rising 23 percent from 2005 to 2006 and 16.6 percent growth from 2006 to 2007. Teradata and SAS rounded out the top five.
Year in and year out, these five vendors are jockeying with one another for DI market bragging rights. Furthermore, every vendor in this group seems primed for strong growth. IBM, Oracle, and Microsoft now offer preconfigured DW systems (IBM under its own auspices; Oracle and Microsoft in tandem with hardware OEM partners), while Teradata this year introduced several DW appliance systems. SAS, for its part, announced an ambitious new In-Database strategy, aiming to embed analytic functionality directly inside a data warehouse. Teradata was SAS' In-Database launch partner (see http://www.tdwi.org/News/display.aspx?ID=8813); SAS expects to pursue similar efforts with other data warehousing players.
With so much going on in the DW high-end, IDC analysts Dan Vesset and Brian McDonough singled out Microsoft's prepackaged DW push, which it launched with HP and Dell Computer Corp., citing the built-in bona-fides of SQL Server, which ships with an all-in-one data integration (SQL Server Integration Services), OLAP (SQL Server Analysis Services), and reporting (SQL Server Reporting Services) feature set.
The duo is also high on Microsoft's acquisition of the former DATAllegro Corp. "[T]his acquisition will play a significant role in expanding Microsoft's range of offerings on a continuum of data warehousing requirements," Vesset and McDonough write. "As the sizes and performance requirements of Microsoft's DW clients grow, the company will be able to use the newly acquired technology to address the needs of its clients at the high end of the DW requirements spectrum."
Teradata also had a surprising 2007, during which it officially completed its spin off from parent company NCR Corp. IDC felt that the logistics of the move -- and Teradata's need to ramp up its own independent operations -- might result in a slowdown last year. The opposite was the case; Teradata grew at a pace (13.3 percent) with market leaders Oracle and IBM.
"Teradata accelerated its product growth in 2007 as its products continued to gain traction among end users with high data volume requirements and a shared vision for [an] enterprise data warehouse … architecture," Vesset and McDonough write. More importantly, they point out, Teradata has successfully parried several competitive thrusts from without -- particularly from Netezza Inc., Dataupia Corp., and a passel of DW appliance players.
"Revenue data suggests that as an independent company, Teradata has been able to respond to these challenges successfully," they continue, acknowledging that Teradata's own entry into the appliance space -- with low-cost systems priced at under $100,000 U.S. -- played an important (although far from decisive) role in its strong showing. "[P]rice is only one of the criteria for evaluating business analytics appliances and Teradata remains a preferred choice for many organizations that have very large DW platforms," Vesset and McDonough say. "These clients also find value in Teradata's extensive professional, industry and process data models, and high-availability and disaster recovery features."
HP, Netezza, Sybase Surprise; Business Objects Stays the Course
The data warehousing top five, then, is a relatively static line up, with the same quintet of vendors vying for bragging rights year after year.
There's been a lot of jockeying elsewhere in the ranks, however. For example, a trio of vendors -- DW appliance specialist Netezza Inc., computing giant Hewlett-Packard Co. (HP), and RDBMS stalwart Sybase -- posted the biggest 2007 growth spurts. Netezza notched 58.3 percent growth in 2007; the company went public last year. This year, Netezza acquired a specialty analytics provider -- NuTech -- which could give it a competitive edge. In an increasingly fractious DW appliance segment, and with IBM, Microsoft, and Oracle muscling into the DW segment, Netezza will increasingly need all of the differentiation it can get.
"[NuTech] will provide Netezza with experience in advanced analytics … and eventually some of NuTech's technology will find its way directly into the database or will be sold as an extension to the Netezza Performance Servers," Vesset and McDonough write. "[W]ith the entry of many new competitors, whose technology is being proven in the market, and more attention from large database vendors, Netezza is likely to encounter more competition in the coming years."
HP, for its part, grew its share of the DW segment from a comparatively paltry $5 million in 2005 to $8.4 million last year. That growth coincided with HP's acquisition of DW consultancy Knightsbridge Technology (in late 2006) and the much-hyped relaunch of its Neoview DW appliance (in the spring of 2007).
It was Sybase, however, which posted the most impressive overall performance: after all, both Netezza and HP grew their shares from comparatively humble beginnings. Sybase, on the other hand, jumped from $94.3 million in revenue two years ago to $137.5 million last year, a year-over-year surge of 45.8 percent. The company owns best-of-breed analytic technology in Sybase IQ; best-of-breed data modeling technology (in Sybase PowerDesigner); best-of-breed replication technology (in Sybase Replication); best-of-breed data federation technology (in Sybase Data Federation), and -- of course -- credible ETL technology (in Sybase ETL), which Sybase picked up via its acquisition two years ago of German ETL specialist Solonde. It was primarily on the basis of demand for its IQ analytic data warehouse, IDC notes, that Sybase's resurgence was founded.
"Sybase saw a resurgence of demand for its Sybase IQ analytic database in 2007," Vesset and McDonough point out, noting that IQ alone achieved a staggering 77 percent uptick in sales last year. The analyst duo expects this trend to continue, thanks in part to Sybase's move into the analytic appliance space earlier this year (with the aptly-branded Sybase Analytic Appliance; see http://esj.com/business_intelligence/article.aspx?EditorialsID=8960). Another vendor that had a strong showing in 2007 was Business Objects, which -- for most of the year -- was its own independent entry. (SAP AG purchased Business Objects in October of 2007.) Business Objects markets its own successful data integration stack -- comprising Data Integrator (ETL and other DI heavy-lifting), Data Federator (an enterprise information integration tool), and Data Quality (the former Firstlogic technology).
Last year, Business Objects controlled just 1.1 percent of the overall DW market (and 4.7 percent of the market for data warehouse generation tools), but grew its revenues by nearly 25 percent. IDC expects that SAP's stewardship -- which gives Business Objects an opportunity to cross-sell into SAP accounts -- will help that company (which is now an SAP subsidiary) grow its share over time.