TPC Changes Web Site Over Microsoft Numbers
Microsoft’sscale-out numbers on the closely watched Transaction Performance ProcessingCouncil’s OLTP benchmark are ruffling feathers again.
Recentlythe TPC reconfigured its Web site to separate out the results of clustered andnon-clustered systems on the TPC-C scalability benchmark.
Clusteredresults have been a part of the TPC-C benchmark results for several years, butMicrosoft and Compaq have recently filled up much of the top 10 performancelist with a number of similarly configured clustered systems.
The TPCsite at www.tpc.org now offers visitorsinterested in the viewing the top 10 results on the TPC-C the option of seeingall results, clustered results, or non-clustered results (see screenshotbelow).
Theofficial explanation for the change was not immediately clear. TPC officialsdid not return calls requesting comment.
Microsoftbriefly got its wrist slapped over the summer when scale-out benchmarksperformed on Windows 2000 Advanced Server with Compaq in February had to bewithdrawn because at least one other TPC member complained the test violatedbenchmark rules.
The systemsused a new feature of Microsoft SQL Server 2000 called Distributed PartitionViews that permits different nodes of a cluster to control different segmentsof a database. The Compaq-Microsoft systems were identical except one clusterhad eight nodes and the other ran 12 nodes.
Thebenchmarks highlight what Microsoft calls its scale-out strategy: buildinglarge system capacities by joining servers through software instead of scalingup with a single large server.
Thechallenge to the February results was that the primary key, the relationaldatabase field that uniquely identifies a record, could not be updated in thedistributed configuration. As a result of the challenge, several non-Microsoftclustered configurations dating back to 1996 also had to be withdrawn.
Publishedreports put Microsoft archrival Oracle behind the rules complaint. Oracle isone of 35 industry members of the TPC, which includes most major hardware,database, and operating system vendors.
Microsoftand Compaq responded by rerunning the original two benchmarks, adhering to thenew rule, and then also running the tests using updated systems with fasterprocessors. In the interim, IBM posted a Windows 2000 Advanced Server clusterof its own using 32 machines and its DB2 database software.
By the timeMicrosoft and Compaq responded with a 24-node system to reclaim the top spot,clustered systems based on Windows 2000 owned six of the top 10 slots in thebenchmark list, but there is little difference among the systems.
If it wasOracle’s doing to get the rule reconsidered, the maneuver backfired. Systemsrunning Oracle databases were pushed way down the TPC-C performance list -- apoint Microsoft has made frequently.
Splittingout clustered and non-clustered systems partially fixes the perception problemfor Oracle, Sun, and IBM divisions that run large-scale, single-system tests.It also provides a way for the TPC to acknowledge criticisms that comparisonsof clustered systems to single server systems are flawed because of thecomplexity involved in configuring and managing clusters.
The recentprofusion of clustered systems is definitely a spike. Prior to the six Windows2000 clusters posted in the last eight months by Microsoft, Compaq, and IBM,only seven clustered systems appear among the online database of valid TPC-Cresults dating back to 1997.
All sevenof the older results used Oracle databases and clustering technologies. Oraclewas last associated with a clustering benchmark -- on Windows -- in February,and hasn’t been involved in a Unix clustering benchmark since September 1999.
Index:
Microsoft Corp., Redmond, Wash., www.microsoft.com
Transaction Processing Performance Council, San Jose, Calif., www.tpc.org
Oracle Corp., www.oracle.com
IBM Corp., Armonk, N.Y., www.ibm.com