Data Warehouse Alternative Porting to NT
Jefferies & Co. Inc. officials wanted to hang onto their legacy systems.
The Jersey City, N.J., institutional brokerage firm and investment bank therefore needed a data warehousing solution that could access and integrate data from VSAM, Sybase and other data stores.
What the company decided on was Virtual DB, an unusual solution to the data warehousing problem from Enterworks (www.enterworks.com). Enterworks is porting the server portion of Virtual DB to Windows NT.
Virtual DB essentially takes the field names from several databases throughout an enterprise and mixes them together into a virtual database with pointers to the data, which never move from the operational systems. The modeled database is stored on the Virtual DB server along with any data transformation rules necessary to make data from the various sources usable in the virtual database.
From there, IT staff can develop an application in a business intelligence tool that connects to the Virtual DB via ODBC, as if Virtual DB contained a real database. In the background, Virtual DB grabs the data from the various systems, transforms it and serves it to the business intelligence tool.
"There’s always this knee-jerk reaction to try to retire legacy systems," says Russ Lewis, CIO at Jefferies, which uses an OLAP tool from Gentia Software (www.gentia.com) to access Virtual DB. "What the Virtual DB product allows is to access a lot of the data [in those systems] that’s rich historically."
Most companies buy the product with a small group of users in mind who will run a query across the operational systems, says Lenley Hensarling, vice president of marketing for Enterworks. For example, an executive team could track daily revenue from worldwide offices that use different operational systems. The initial setup takes about three months, but subsequent tweaking of the application to match refined user demands or meet changing business conditions can take a few days, Hensarling says.
Running across production systems with what is essentially a metadata product provides great flexibility but poses some obvious limitations.
Once the user group gets beyond 100 users, the performance of the underlying operational systems begins to lag.
Many customers address this problem by using Virtual DB during slow times in the day to run a batch load into a data mart, much the way a regular extraction, transformation and loading tool works. Then users can be segmented into a select group that needs real-time data and a larger population that can hit the day-old data in the data mart. Jefferies & Co. is one customer using this approach.
"If you have 15 to 50 people in finance who need very broad access in disparate data sources, it’s not clear you ever need to migrate that data. If you have a user community of several thousand, you need to migrate it," Hensarling says.
The other major limitation is query speed. "It’s clearly going to be slower than a data warehouse because you’re pulling data from across those different sources and doing the joins to create those virtual tables," Hensarling acknowledges. "Typically, it’s still in the under-a-minute response time." Hensarling explains that despite the speed limitations, it’s still worth it for many customers because they "couldn’t even hope to have gotten this information before."
Enterworks was expected to release the port to Windows NT last month. The process was slowed when test customers complained the initial port didn’t behave like a native Windows NT Server application. The company is now making application appear in the NT registry and manageable by NT tools, Hensarling says. The completed version is expected sometime this quarter.