In-Depth

Gartner and "Good Enough" Data Integration

Data Integration leaders are under pressure from "good enough" competitors. This gives non-dominant players a golden opportunity to contest the status quo.

• By Stephen Swoyer
• 02/16/2011

Market watchers first starting writing epitaphs for Informatica six years ago, when IBM snapped up the former Ascential Software Corp. for a cool $1.1 billion.

That didn't happen, of course. According to the latest installment of Gartner Inc.'s "Magic Quadrant for Data Integration Tools," Informatica's leadership position in the data integration (DI) space remains largely unchallenged, though six years on Big Blue has made up some ground.

Although Informatica's leadership position hasn't changed, the dynamics of the market have changed. Just as business intelligence (BI) has seen a recent shift toward "good enough" tools -- e.g., point products or user-oriented tools that tend to be positioned over and against larger suite offerings -- the DI market is in the midst of a similar shift.

This has created an opportunity for non-dominant players to contest the long-prevailing status quo. "[E]conomic conditions continue to push organizations to scrutinize their investments and optimize costs. … [T]his manifests as aggressive behavior on the part of buyers when negotiating price with vendors, as well as a growing number of organizations that are seeking solutions with solid basic capabilities [i.e., "good enough"] offered at attractive price points," write analysts Ted Friedman, Mark Beyer, and Eric Thoo in the Gartner report.

As such, the Gartner trio cites the success of Pervasive Software Inc., Microsoft Corp., and Syncsort Inc., three scarred veterans of the DI wars.

Microsoft first vaulted into the data integration fray with its SQL Server 7.0 release 12 years ago, while both Pervasive and Syncsort have been competing -- in one form or another -- for decades. All three have benefitted from the "good enough" trend in DI shopping behaviors. There's also pressure from free and/or open source software (F/OSS), which appeal to buyers for reasons other than pricing. "[T]he level of activity around open-source solutions continues to grow, since these are viewed by buyers as providing a more attractive cost model. Talend, Pentaho, and other open-source providers in this market see increasing benefit from this trend," "Friedman, Beyer, and Thoo note.

Ranking the DI Powers That Be

Gartner has Informatica sitting atop the DI market heap, outpacing competitors IBM, SAP AG, and Oracle Corp. All four populate the "leaders" quadrant of Gartner's DI market forecast, ahead of a group of "visionaries" that includes perennial DI market powers iWay Software (a subsidiary of Information Builders Inc.), SAS Institute Inc., and Pervasive Software.

What's intriguing is that -- although there may not be much separating market challengers iWay and SAS from market leaders Oracle or SAP -- IBM and Informatica far outpace the rest of the field.

Big Blue still trails Informatica by a huge margin on the Y axis of Gartner's Magic Quadrant (which measures a vendor's "ability to execute") and has a very slight edge on its X ("completeness of vision") axis.

In addition to a "visionaries" quadrant that includes iWay, Pervasive, SAS, and F/OSS specialist Talend, Gartner places Microsoft Corp. into its "Challengers" quadrant, lauding Redmond's "ability to execute" even as it (understandably) deducts points for the Microsoft-centric vision that constrains Redmond's score on the "completeness of vision" axis. Two additional players -- DI veteran SyncSort (which will celebrate its 43rd year of business in 2011) and Pitney Bowes Business Insight -- appear in Gartner's "niche players" quadrant. Both vendors score poorly on the "execution" and "vision" axes.

The Missing Z Axis

One problem with Gartner's Magic Quadrant scheme is that it doesn't effectively assess the impact of some of the players -- i.e., the "good enough" DI vendors -- that Friedman, Beyer, and Thoo single out in their summary. A Magic Quadrant scheme that emphasizes both completeness of vision -- but which equates "completeness" with an overarching, platform-centric architecture -- and ability to execute (but which gives greater weight to the vendors with the most resources) puts at a disadvantage small players that focus on delivering "good enough" DI.

In short, some vendors grouse, Gartner's missing a "Z" axis.

"It's a dirty little secret of the industry. For too long we as an industry have been ignoring the fact that we take too long to build things, and this is because we assume [at the beginning] that we have to get them right -- but we don't ever get them right! That's why you see people still doing satellite data marts, still doing their own Excel spreadsheets, still doing [operational data stores]. It's because IT can't get it right the first time," said Michael Whitehead, CEO of DI upstart WhereScape Inc., at TDWI's Summer World Conference last year.

WhereScape, Whitehead continued, approaches DI from a perspective that -- although not anathema to that of the larger DI players -- nonetheless diverges distinctly. "[Y]ou can't understand everything up front, you can't get it all right in the beginning, so [instead] you do it as a small series of deliverables."

WhereScape isn't ranked in Gartner's Magic Quadrant tally. Nor, for that matter, are several of its competitors -- vendors such as Expressor Software and illuminate Solutions Inc. All three first appeared on the scene three years ago. All three claim to have grown both their revenues and their profits over the same period. (All three are privately-held and don't disclose profitability information.) All three likewise claim to benefit from the "good enough" trend in DI.

Good enough DI naturally targets the most glaring weakness of big-vision, big-execution DI: its complexity. "The tools in the market today are used by ETL developers -- very technical people that have no problem coding up solutions," said Bob Potter, CEO of Expressor, in a November interview. With Expressor, Potter continued, "you can have somebody build a data flow and set up a model and [then] through this nice user interface [to the design environment] … you can have non-ETL developers start building data flows, data integrations."

Potter doesn't just stake Expressor's reputation on its ability to empower non-ETL developers. He points to the latency -- which manifests itself in a protracted time to execution -- which he claims attends any Big ETL project.

Expressor is still growing into an enterprise-grade product. Potter and other officials choose to trumpet its strength as an iterative DI environment: it boasts a canonical model based on types, which Potter says lets developers create mappings and reuse them in subsequent projects. (WhereScape touts a similar capability.)

"The business value is faster time to deploy, and the [time that it takes to deploy the] second project and the third project and the nth project is dramatically less."