Java Benchmark Makes Run at TPC
Citing a need for a benchmark to measure Java application server performance, Hewlett-Packard Co. and Vision Software Tools Inc. (www.vision-soft.com) teamed up with consulting company Netroscope Inc. (www.netroscope.com) to launch a new benchmark called JETMark. The new benchmark will be administered by Netroscope and is being positioned as a possible de facto industry standard, which proponents hope other vendors will endorse.
The benchmark is designed to measure e-business performance, specifically server-side performance, on servers running applications built using Java. JETMark will specify performance in e-transactions processed per hour.
One of the key drivers for developing the benchmark was Vision Software. As a Java development tool supplier, the company has a vested interest in such a benchmark test. Vision Software's Vision Jade produces server side Java code generated from a business rules engine. The company is anxious for industrywide acceptance of a benchmark that will demonstrate performance of transactional e-business Java applications. Most benchmarks measure system performance, but don't specifically highlight Java application performance.
"At the moment, the e-transactions that are considered for the benchmark [include] order entry and order processing and shipping," says Natalie Shaheen, president of Netroscope. "The benchmark is measuring both of these."
The launch of the benchmark will be followed by the first test results, which were generated using a six-way Hewlett-Packard K-class Unix system running an application created by Vision Software. The results are expected to be between 40,000 and 45,000 JETMarks. Until other vendors weigh in with competitive results, the Vision/HP results have no basis for comparison.
The benchmark will be administered by Shaheen, who previously was a member of the Sun Java development team. "Java continues to play an increasing role in electronic commerce," she adds.
JETMark is not alone in its attempt to test e-business applications. In January, the Transaction Processing Performance Council (TPC, www.tpc.org) announced TPC-W, a Web commerce benchmark. TPC-W, however, is still in development. The TPC benchmark is in its third draft form. TPC-W will measure performance in terms of Web interactions per second, and will include a price-performance metric similar to other TPC benchmarks.
Shaheen says the differences between JETMark and the proposed TPC-W benchmark are highlighted by the specifics of the server-side application. JETMark calls for a Java application, while TPC-W doesn't limit testing to Java applications. "TPC results can say a lot about the database, but not necessarily about the system in front of it," Shaheen adds.
JETMark Testing
Measures performance in e-transactions per hour- Tests Java e-commerce applications