Attention Data Mart Shoppers: Deals May Be at the Warehouse
What was once a war between data warehouse and data mart advocates is evolving from a fiery debate about which approach is better to a calmer focus on how best to blend the technologies.
"Some people say you've got to develop the warehouse first, in more of a top-down fashion. Some people say you've got to develop the data mart first, from a bottom-up, more grassroots fashion. Most people who've been successful will say you need both," says Wayne Eckerson, vice president of technology services at TDWI.
Several consultants and analysts agree the industry is beginning to move toward a blended approach, with a data warehouse serving as an umbrella or backbone for several data marts. Current debate diverges on how much effort should be put into the data warehouse side.
Eckerson offers an approach that reflects a recognition that the same force that has driven the popularity of data marts, a desire for rapid and inexpensive delivery of functionality, is still important. "You want to move quickly. The best way to do that is map out a blueprint for where you want to go -- at a high level -- and don't spend a lot of time on it. Then, pick a project that implements a portion of that model, and build it quickly, within six to nine months, and get users up and running. Show value quickly. Hopefully, you'll get the attention of senior management to continue funding that project, and others, and enable you to flesh out your architecture iteratively as you go along."
On the other end of the spectrum, Kevin Strange, an analyst at GartnerGroup (www.gartner.com), predicts many companies will turn back to the hard, expensive, labor-intensive and necessary work at the beginning of a data warehousing project. Data marts fit into that strategy as a subset of a data warehouse, he says. "If the right planning is done to incorporate the data mart into the overall data warehouse architecture -- a dependent data mart -- then data marts can be valuable," Strange says.
In any case, says consultant Richard Winter, CEO of Winter Corp. (www.wintercorp.com), a mixed environment employing data marts also helps with query, analysis and reporting needs, which can relieve the load on the enterprise data warehouse. But, Winter cautions, "the collective growth and changing needs of a half dozen data marts can result in dramatic management challenges every few months."
Many vendors and industry specialists often utter the terms "warehouse" and "mart" in the same breath, as if the two approaches are interchangeable. Strange cautions against confusing the two: "Data marts are not small data warehouses."
Warehouses and marts have vastly different missions and architectures. Data warehouses support cross-platform databases tied into different applications all over an organization. Data marts focus on specific business functions, such as sales or accounting, and often are linked to a single application on a single platform.
The difference in scope illustrates why the decision of which data framework to choose can be so important when it comes to future considerations, such as effects on employee time, overall cost and the ultimate value of the projects. The largest data warehouses support more than 6 TB of data. Most data marts store less than 50 GB of data and sustain fewer than 50 users, the incremental approach to building a data warehouse with data marts installed a department at a time involves administering several data marts in an enterprise.
A Meta Group (www.meta.com) study finds data warehouse implementations to be multiyear projects that cost an average of $3 million. A data mart tends to range between $100,000 and $300,000, with an implementation time of three to six months. At the extremes, Eckerson says he’s seen tiny prototype data marts built for $60,000 and giant data warehouse projects balloon to $40 million.
The conventional argument for the data warehouse has been that the discipline and effort of its creation results in a clean, robust, homogeneous version of the data across the enterprise that is useful for any number of business purposes and can change with business needs. Proponents of data marts argue that data warehouse projects tend to be top-heavy, hampered by long installation and delivery times, and too expensive.
The data mart approach has surged in popularity recently, and many vendors offer these solutions. A recent study by Meta Group of 2,200 data warehouse-data mart projects found that data marts had grown to 61 percent of all project architectures, up from 51 percent the last time the analyst firm conducted the survey.
Incidentally, data mart growth has helped drive the Windows NT market, which is widely viewed as appropriately priced and suited for data mart projects. Meta attributes up to 25 percent of all NT sales to data mart projects. Although some vendors say the release of Microsoft’s SQL Server 7.0 database makes Windows NT a serviceable data warehouse platform, many IT buyers still consider the scalability demands of data warehouses too much for Windows NT. Gartner's Strange predicts Microsoft-based technology will not effectively support large warehouse environments until 2003.
With data marts popping up all over the business world, industry observers are getting enough real-world reports to identify some major shortcomings.
"Data marts bring back many of the past mistakes found during the information center concept of the 1980s," Strange says. Those so-called "stovepipe" information centers only supported tactical decisions for particular user areas, he explains. Aside from the predictable data isolation, an unintended problem is emerging. Frustrated with the slow development of data warehouse technology, and pressured by vendor hype and impatient users, IS departments are implementing tactical data marts to avoid the cost and effort of building data warehouse architectures, Strange says. "This opens up a whole new set of administration and management issues, since a collection of data marts is not necessarily easier or less expensive to implement than a data warehouse," he says.
Effective solutions have been few and far between, according to Nagrav Alur, principal with DataBase Associates (www.dbaint.com). "Data mart vendors are not doing very well. The cycles are long -- three to four months -- and they're still trying to sell to the IT people. Everybody buys into the argument that we need some quick and dirty, inexpensive solutions, but the technology has not stepped up to that."