IBM, Microsoft Push Envelope on TPC-C Benchmark

• 07/19/2000

The two companies and Intel Corp. this month submitted a benchmark to the Transaction Processing Performance Council (www.tpc.org) on a cluster of machines running Windows 2000 that more than triples the best results published so far by Unix/RISC vendors.

"We wanted to validate that the industry-standard server solution platform could really scale," says Brian Sanders, IBM’s (www.ibm.com) worldwide brand director for Netfinity servers, the hardware used in the benchmark. "We’ve broken down a barrier that has prevented people from going over to this industry-standard platform."

The results came just in time for Microsoft (www.microsoft.com), which saw its prized results from the TPC’s OLTP benchmark, the TPC-C, withdrawn a week earlier. Microsoft officials say the withdrawal occurred over a technicality. They vow to have the results recertified shortly.

The combined IBM-Microsoft effort achieved a record 440,879 transactions per minute, more than three times the raw performance of the best Unix/RISC benchmark result and nearly twice the performance of the withdrawn Microsoft result from February. The cost per transaction checked out considerably better than the Unix systems, but higher than other Windows-based benchmarks (see chart).

For the demonstration, IBM connected 32 Netfinity servers running Windows 2000 Advanced Server in a performance cluster using the multinode capabilities of IBM’s DB2 relational database management system. IBM says the multiplatform DB2 software theoretically can scale to 1,000 nodes, although the largest customer installations using DB2 on Windows have no more than 12 nodes.

IBM throttled the system by using eight-processor capable servers, but populated them with only four processors apiece. "We’re trying to set up a configuration that can grow, and that a customer would consider as very viable for a high-traffic site," Sanders says. The 128 processors were the new 700-MHz Pentium III Xeons.

Microsoft is pushing against its traditional scalability limitations in two directions at once: scale out and scale up. Scale out refers to using dozens or hundreds of machines to accomplish one task. Windows 2000 Advanced Server lies near the heart of Microsoft’s scale-out designs. The up-front costs in hardware can be proportionately lower in a scale-out approach, although management requirements for the complex configurations make the long-term costs of such systems less clear.

Scale up refers to the capabilities of a single machine to handle a larger workload, the approach often taken in Unix/RISC environments. Microsoft is progressing in that area with Windows 2000 Datacenter Server, scheduled for release this summer. Unisys Corp. recently demonstrated an Intel-based machine running Datacenter Server across 32 processors. A recent Fujitsu-Siemens TPC-C benchmark of Datacenter on an eight-processor machine, also demonstrated impressive performance.

Michel Gambier, Microsoft’s group product manager of enterprise server marketing, says Redmond is not favoring scale out over scale up. "People will need essentially larger boxes and combine those larger boxes in a scale-out approach," he says.

Gambier calls the benchmark a "tremendous proof point for scale out." He compared the 1.4 billion transactions per day the system could handle with the 25 billion transactions handled in an entire year by Visa.

As for the withdrawal of the TPC-C benchmark results Microsoft had been so proud of, Microsoft and IBM officials say it boils down to an unexpected interpretation of a rule by TPC officials. The interpretation requires a database to allow for updates to the primary key.

IBM officials say the interpretation sent their engineers back to work midway through work on the new benchmark and means the tested configuration won't be technically available until December, although a customer could install and run a similar configuration today.

"That feature has been added to SQL Server. The numbers have been withdrawn and they will be resubmitted when the benchmark is rerun. By adding the feature, the number will not change," Gambier says.

[Infobox] Back on Top

With IBM’s help, Microsoft is back atop the TPC-C rankings despite having its prized results from earlier this year withdrawn. The following are selected TPC-C results: