In-Depth

OLAP Wars: Hyperion Strikes Back

With a quartet of Itanium 2-based HP Integrity servers and the 64-bit edition of its Essbase OLAP, Hyperion beats Oracle’s market-leading score in a key OLAP benchmark by 39 percent.

• By Stephen Swoyer
• 01/07/2004

All Hyperion Solutions Corp. wanted was a fair fight. That’s what it got, more or less, when it vaulted over Oracle to take the performance crown in an aging industry benchmark that had been all but mothballed until Oracle brought it back into service last year.

In November, Hyperion tapped a quartet of Itanium 2-based Integrity servers from Hewlett-Packard Co.—each was populated with four processors—and the 64-bit edition of its Essbase OLAP server to beat Oracle’s market-leading score in the OLAP Council’s APB-1 Release II benchmark by 39 percent.

For the record, Essbase achieved a result of 119,085 analytical queries per minute (AQM) while Oracle 9i notched 85,719 AQM. Oracle’s result was also achieved using a four-node cluster on HP hardware.

The benchmark results are the side story here, however, as what’s most interesting are the signs of life in a market that some industry watchers had given up for dead. Oracle Corp. kicked things off last April when it unearthed the moribund APB-1 benchmark to demonstrate the capabilities of its Oracle 9i Database Release 2—running on its Real Application Clusters—as an online analytical processing (OLAP) server par excellence.

Since the introduction of Oracle 9i Release 1, the database giant—which traditionally marketed a separate OLAP tool, called Oracle Express Server—has integrated OLAP functionality into the database itself. Oracle says this tight integration between the 9i database and its integrated features—which include OLAP, data mining, and ETL features, among others—give it a powerful competitive advantage. The company sought to demonstrate as much when it announced an APB-1 score that shattered—by more than 7500 percent—the previous performance record, held by Hyperion’s Essbase OLAP server.

There was just one catch: The Essbase system against which Oracle matched its 9i-based OALP server was powered by a four-way RS/6000 system and ran a now-ancient version 5.0.2 release of Hyperion’s Essbase OLAP server. In fact, Hyperion recorded its then-industry-leading OLAP benchmark in April of 1999.

In the interim, the OLAP Council itself has apparently dissolved—dial its last known telephone number, says Mike Schiff, a senior analyst with consultancy Current Analysis, and you’ll ring a retirement community in Washington—and, more importantly, Hyperion has been through several subsequent generations of Essbase. Add to that the fact that the Real Application Clusters and OLAP options are both priced at 50 percent of the cost of the 9i database itself, and you’ve got a less-than-compelling benchmark triumph.

Writing about Oracle’s benchmark triumph in May, Schiff described it as another case of the database giant’s “classic benchmark chutzpah.” Give Schiff credit for prescience: He suggested at the time that Oracle’s move could presage a new round of benchmark wars in an otherwise somnolent OLAP marketplace.

Just how applicable is the APB-1 benchmark, and what can the results achieved by both Oracle and Hyperion tell us about the performance of their products in real-world settings? According to Schiff, the APB-1 benchmark was designed to help quantify some key performance metrics for OLAP applications. There’s a particular emphasis on analytical processing, for example, which—for the purposes of the benchmark—is understood as the amount of time required to complete an incremental update to warehouse data and complete a given set of queries, divided by the number of queries in the data set. The result is the APB-1 metric, which is measured in analytical queries per minute (AQM). As for its applicability, Schiff believes that the APB-1 can still be a good basis for comparisons between OLAP servers, provided that they’re based on reasonably similar hardware and running reasonably current OLAP engines.

The OLAP marketplace probably hasn’t seen such a good controversy since both vendors last wrangled with one another, back in 1998. At the time, Oracle finessed a benchmark result to demonstrate the purported performance advantage of its Express OLAP server relative to Hyperion’s Essbase. In this regard, a little controversy might be a good prescription for an OLAP marketplace that has seemingly been on autopilot for some time now. Schiff suggests that other vendors—such as Applix Inc. and MicroStrategy Inc.—should consider leaping into the benchmark fray, as well.

“OLAP vendors now have a new target to beat, and any that think they can should certainly attempt to do so and get their names on the board,” writes Schiff. “Applix should certainly consider participating in the benchmark in order to bring need publicity to TM1 in general and its 64-bit implementation in particular.”