In-Depth

DataFlux and NEON Partner for Mainframe Data Quality

NEON’s Shadow technology gives DataFlux customers native access to information residing in mainframe VSAM, DB2 and IMS data stores

• By Stephen Swoyer
• 02/18/2004

Last week, DataFlux Corp. notched a deal with mainframe data integration specialist NEON Systems Inc.

Under the terms of the partnership, DataFlux will distribute NEON's Shadow mainframe integration software with its own data management solutions. The upshot, officials say, is that NEON’s Shadow technology gives DataFlux customers native access to information residing in mainframe VSAM, DB2, and IMS data stores. Shadow also facilitates access to third-party mainframe databases such as Software AG’s Adabase.

Estimates vary as to how much mission-critical data still resides on mainframe systems—some analyst firms put the figure as how as 70 percent—but it’s a safe bet that many business intelligence (BI) efforts must deal, at one time or another, with data residing on mainframe systems.

Most prominent BI vendors license ETL technology to facilitate access to mainframe data, which usually involves dumping VSAM, DB2, or IMS data into flat files, or loading it into an operational data store. Not all BI vendors take this approach, however: ETL powerhouse Informatica, for its part, last year purchased Striva, a company that marketed mainframe data integration technology similar to NEON’s Shadow.

According to Mike Schiff, a senior analyst with consultancy Current Analysis Inc., NEON’s Shadow integration software—like Informatica’s Striva technology assets—provides a better way to get at mainframe data. “While mainframe data can always be extracted into a flat file and then downloaded to another platform, navigating IMS/DB segments … and Adabas repeating groups and multi-valued fields are not necessarily trivial,” he writes. “NEON’s Shadow technology greatly reduces the complexity associated with accessing these data structures.”

Schiff says that the agreement should be a boon to DataFlux customers, as mainframe data is typically primed for data profiling and associated data quality technologies.

“Mainframe data structures are probably the largest source of data that disagrees with file documentation and an area where data profiling can provide very significant investment returns,” he writes, noting that DataFlux last year incorporated data profiling into version 6 of its DataFlux Data Management Suite.

At the same time, Schiff notes, DataFlux's parent company (SAS Institute Inc.) has already developed powerful data integration technology in the form of its Base SAS and ETL technology. Schiff speculates that SAS’ integration technologies were “overkill” for DataFlux deployments in non-SAS environments.