In-Depth
Data Integration: Full Steam Ahead Despite Weak Economy
Few predict an outright contraction in a DI segment that's been a hotbed of recent activity
Ever since the long-simmering sub-prime problems erupted into a full-blown economic crisis last September, industry watchers have been predicting lean times for IT budgets, with a special emphasis on scaling back costly or ambitious IT spending efforts, particularly in the area of infrastructure spending.
Data integration (DI) is part of that infrastructure, but it's unclear what the prospect of a worsening recession might mean for the ambitious DI efforts that many organizations are either mulling or have already embarked upon, typically as part of enterprise information management (EIM) or master data management (MDM) initiatives.
Not surprisingly, those who may be the most gung-ho about DI also involved in the DI market. ETL and services vendors, for example, speak of the importance (and likely growth) of enterprise DI efforts. Analysts predict an uptick in DI-related activity, too. Few predict an outright contraction in a DI segment that's been a hotbed of recent activity.
There's far from a consensus, however. Industry veteran Vickie Farrell, who heads up product management for Hewlett-Packard Co.'s (HP) Neoview DW platform, makes a compelling case that enterprise DI efforts will, in fact, outlast an incipient IT spending freeze. She cites HP's own internal survey data -- collected from North America, Europe/Middle East/Africa, and other locales -- which suggests an uptick in, not a retrenching of, DI spending activity. (Farrell says HP plans to publish its survey data, complete with analysis, later on this year.)
"A major trend is really data integration. [Customers are] not backing off. They're recognizing that if they want to compete on analytics … if they want to do fact-based decision-making, it's not just a matter of putting BI tools in more hands, but putting more capacity in the data warehouse. They' have to build a data integration foundation that will help them integrate data without having that horrible manual oversight," she says.
You might think that ETL has been commoditized, Farrell says -- and there's a clear sense in which it has been, with canned ETL components available from Microsoft Corp. and Oracle Corp. (IBM Corp.'s DB2 database also has a built-in -- and mostly basic -- programmatic SQL ETL facility) -- but the ugly truth, she argues, is that DI isn't yet a turnkey proposition in most environments; instead, it's an inescapably manual process that unfolds in fits and stops.
"That's not operational at all. In operational analytics you really have to be able to automate data integration and get it right, and make sure that you're not integrating data in a way that isn't meaningful." HP doesn't offer a branded ETL facility (much less a full-blown data integration platform) -- although Farrell demurred when asked about the possibility of HP's developing or fielding such offerings on its own.
HP is also the proprietor of the former Knightsbridge Solutions, which remains one of the most respected brands in DW and business intelligence (BI). (HP also picked up BI and DW expertise with its acquisition of the former Electronic Data Systems Inc. last May.)
Officials say HP has some internal ETL expertise -- including custom-built data integration adapters for several industry-specific applications -- but stress that to the extent that it focuses on DI, it's from a services perspective. "In the supply-chain world, we actually have built a little ETL tool that we don't sell as a product but we're using it with a small set of clients," says John Santaferraro, director of business intelligence with HP. "There's a lot there that certainly could be harvested, and a lot ... that is really about accelerating our services capabilities."
Farrell points to data quality (DQ), master data management (MDM), complex event processing (CEP) and other perceived business-critical initiatives as key DI drivers. Businesses are committed to such projects, she says, because they believe the ROI (which is also a function of getting better, cleaner, more timely data out of their operational systems) is there. As a result, she argues, they aren't reining them in -- in spite of belt-tightening budget cuts. "There's continuous pressure on all of us. [Customers] have to have some kind of technology that enables them to do that operational work without creating synchronization problems and other issues for themselves," Farrell argues.
DI and Industry-Focused BI
Ken Hausman, data integration product marketing manager with SAS Institute Inc., likewise thinks DI (and in a more general sense, BI) projects will weather the storm. He differentiates, however, between DI as a general technology prescription -- namely, as a component of enterprise information management (EIM), DQ, MDM, CDI, or CEP efforts -- and DI as an embedded component of industry-specific products.
To the extent that organizations pursue DI efforts, Hausman says, a growing portion of their spending will be directed toward packaged offerings that solve specific business problems, in addition to over-arching efforts like EIM or MDM. "Business pressure drives what is bought. I think you're going to find that the products that get bought are going to be focused on solving business problems that will be business solutions -- those solutions that best address customer problems will be bought even if they already have ETL, they already have reporting," he says.
Hausman and SAS also have a product in the DI stakes. SAS develops a market-leading DI tool and fields several packaged, industry-focused offerings, such as SAS Solutions for Financial Services (which includes risk management, credit scoring, and anti-money laundering), SAS Solutions for Insurance (claims prediction, cross-sell/upsell, and customer retention), and other dedicated offerings.
SAS DI is embedded in all of these offerings, Hausman concedes. At the same time, he argues, this is hardly a self-serving vision. It is, he suggests, a frank assessment of where the industry is heading. Hausman doesn't think discrete DI tooling is going to disappear; he thinks that more vendors will strive to deliver industry-focused offerings, complete with data integration, reporting, and other canned plumbing. It's in this context, again, that many businesses will opt to pursue and consume DI -- although Hausman stresses that discrete DI tooling (like ETL or data quality) still has a place.
"You have to think about it in terms of having the right tool for the job. Even though I have a bandsaw, a table saw, … a Sawzall, I still have a handsaw, because it's the right tool for the right job. If you're looking to migrate off of one system to another, you can probably use the same tools [e.g., ETL, data profiling, data quality] that are the foundation for this industry solution, just like you can use a handsaw from anybody," he comments. "When there are opportunities to use data integration tools for commodity-type initiatives, I don't think it's going to make that much difference which tool you're using."
Integration Must Go On
There's some agreement among analysts, too. In a recent research blast, Mark McDonald, group vice president and head of research for Gartner Executive Programs, talked up a potential surge in DI-related spending, suggesting that customers will invest "in business intelligence applications and information consolidation in order to raise enterprise visibility and transparency, particularly around sales and operational performance."
With the likelihood of new and more rigorous regulation, such investments will serve a dual purpose, McDonald predicted, inasmuch as they enable enterprises to respond more adaptively to "new regulatory and financial reporting requirements."
Philip Russom, senior manager with TDWI Research, is sanguine -- even though he flags a requisite sloughing-off in the area of general IT project spending. "As firms are reluctant to spend money on anything -- and are unable to get loans for spending -- they are also increasingly reluctant to increase budget for IT," he argues, noting that "corporations and consumers alike are hunkering down and hoarding cash, until the current recession is better understood."
The picture is considerably brighter in the DI and BI segments, Russom says. "Few firms are committing to large, new implementations. For example, I'm not seeing many new ERP or CRM deployments, although ERP and CRM consolidation projects are still occurring," he continues. "Firms are, however, making carefully selected incremental improvements to pre-existing infrastructure. In my BI-oriented world, that means that I've seen companies commit to new tools and new solutions in data integration, data quality, and related stuff like master data management," he concludes.
"One of the interesting wrinkles here is that these are now being considered an infrastructure, whose maintenance needs occasional augmentation. This new mindset -- by both technical and business people -- which considers DI/DQ/MDM to be infrastructure [i.e., akin to networks and broadband] is helping these technologies get -- or keep -- resources, despite a lock-down on general IT spending."