Complex Joins and Lookups Can Now Run Outside a DBMS
CoSort V9 introduces multi-file joins and lookup transforms, reducing DBMS overhead and improving accuracy
Innovative Routines International Inc. today announced two new IT productivity features in Version 9 of CoSort: multi-file joins and multi-dimensional lookups.
By defining intersections in database extracts and legacy files, CoSort users can find , transform, and report on related data. When applied to flat files, the join and lookup functions allow CoSort users to reduce query overhead and incorporate other data (such as spreadsheets) into the process.
Users have had no efficient way to compare large files and identify field changes (inserts, updates, deletes). “In addition to offloading DBMSs, multi-file joins offload data integration tools by merging data before it hits the tool,” said Philip Russom, senior manager at TDWI, in a IRI statement. “At the high end, this is useful with the distributed architectures that many users apply to scaling up their data integration solutions. At the other extreme, multi-file joins may eliminate the need for a data integration tool.”
Data cleansing, multi-table joins, and complex computations producing discrete solutions are resource-intensive. If a simple lookup can replace a runtime computation (e.g. mathematic expression or pseudonymization), the result is a significant performance improvement, because retrieving a value in memory is faster than computing that value.
To achieve these fast retrievals, CoSort users specify lookups against set files. Because they reference multi-column files, users receive faster answers to discrete questions, according to the company. According to Russom, “When multi-column files are sources for a data warehouse, multi-dimensional file lookups can generate cubes and other multi-dimensional structures for the warehouse and analysis tools.”
More information is available at http://www.cosort.com.