In-Depth

Improving DW Performance: Survey Finds Companies Are Flying Blind

Your data warehouse may very well be a speed demon—but to what extent does its high performance correlate with exceptional business value?

• By Stephen Swoyer
• 03/16/2005

So you’ve built your data warehouse on the latest and greatest server processing power from Hewlett-Packard Co., IBM Corp., or Sun Microsystems Inc. You have the latest and fastest chips from Intel Corp., along with IBM’s and Sun’s own RISC chip designs. And—you’ve left nothing to chance—your storage is 2 Gbps fibre channel from EMC Corp. or Hitachi Data Systems.

Performance-wise, your data warehouse is screaming—but to what degree does its high performance correlate with exceptional business value?

That, says Eric Rogge, a vice-president and research director with consultancy Ventana Research, is the $64,000 question.

“Essential to resolving misunderstandings [between BI professionals and business leaders] is establishing the value of a given level of performance,” Rogge writes. “Performance value can only be done by relating query performance to business performance via translational frameworks.”

Ventana Research recently surveyed 225 IT and line-of-business professionals on this very question.

The result, Rogge says, is a mixed bag of sorts. First, the good news—a majority (55 percent) of respondents reported being satisfied with the ad hoc query performance of their data warehousing solutions. In this respect, there seemed to be a definite correlation between ad hoc query performance and business value: 84 percent of respondents said their users would benefit significantly from a ten-fold improvement in query response times.

But a majority of these—60 percent—weren’t able to establish a monetary value for a performance increase of this kind: In fact, 20 percent said it was worth $20,000 or less. The upshot? “While better ad hoc query performance improvement was clearly beneficial and desirable, IT practitioners had no understanding of the value of improved performance,” writes Rogge.

Now the bad news: Companies are flying blind when it comes to making purchasing decisions to improve the performance of their data warehouses and data marts. For example, 31 percent of respondents said they expected to buy more hardware over the next 12 to 18 months specifically to improve the performance of their data warehouses and data marts. “IT realizes performance is an issue and is willing to spend money on improving performance. Yet, no actual dollar value has been established,” Rogge says.

Companies also seem to be shooting from the hip when it comes to solving performance problems. Ventana found that organizations were more likely to pony up money for hardware (31 percent of respondents) than for software (18 percent)—and more likely to earmark funds for both hardware and software than spend it on training personnel: Only 17 percent indicated a willingness to pay for performance-tuning training, 12 percent for tuning consultants, and eight percent for hiring DBAs with tuning skills. The upshot, Rogge says, is that “IT would rather solve the problem with infrastructure than with personnel.”

There’s some basis for this preference, Ventana found. Although budgets for data warehousing personnel varied significantly across the sample of respondents, they’re nevertheless a significant cost for organizations.

For example, 16 percent of respondents said their data warehousing personnel costs were between $0 and $250,000; eight percent between $500,000 and $1,000,000; and 10 percent between $1,000,000 and $2,000,000. (Thirty-four percent simply did not know.)

“Costs for personnel should be factored into strategies for upgrading the performance of data warehouses,” Rogge suggests. “Trade-offs between additional infrastructure and personnel should be made after both costs are assessed.”

More than 70 percent of respondents said their users would benefit from having more OLAP dimensions or relational database query filters. Of course, when asked to specify a monetary value for this capability, Ventana found that 52 percent respondents could not ballpark an estimate, while 30 percent said this capability was worth $100,000 or less.

In at least one case, organizations persist in flying blind even when they have demonstrable evidence that what they’ve been doing isn’t working. Forty-five percent of respondents to the Ventana survey stated their batch reporting windows would be exceeded within the next 12 months, with 40 percent citing batch windows of two to four hours.

“As with ad hoc query performance, resolution to the problem was most often solved with additional hardware purchases,” writes Rogge, noting that 32 percent of respondents have employed this approach—even though their batch problems are and will probably remain chronic.