SPC to Standardize Performance Measurement

A group of big-name companies have announced the formation of the Storage Performance Council (SPC, Redwood City, Calif., www.gradientsystems.com/spc), an organization focused on investigating and standardizing performance benchmarks for computer storage products. Membership is open to companies that manufacture, integrate or distribute storage products and supporting system components.

The SPC began to form in 1997, when representatives of vendors in the storage industry began holding teleconferences about storage performance measurement methodologies, explains John M. Stephens Jr., SPC administrator. In October 1997, the participants began meeting bimonthly in person. In June, the founding members officially announced the SPC.

The organization's mission is "to define, standardize, and promote storage subsystem benchmarks as well as to disseminate objective, verifiable performance data to the computer industry and its customers." The vendor-neutral standards body plans to address issues from component-level evaluation to the measurement of complete distributed storage systems.

Members involved in the first teleconferences had experience with benchmarks such as those from the TPC (San Jose, Calif., www.tpc.org), but felt that TPC-C and TPC-D were not focused on the details of storage performance. "TPC does an excellent job of calibrating global system performance," says Stephens, "but storage is not defined well. It's overconfigured in those benchmarks so that it is not the bottleneck." Stephens says that the SPC hopes to work with the TPC so that TPC can emphasize CPU performance, while SPC can deal solely with storage performance.

The SPC's first measurement goal is focused on "server-class systems in sizable market segments," explains Stephens. The council will then evaluate measurements at the component level. The top-down approach serves two purposes, says Stephens: First, the SPC measurements will offer enough breadth to get the whole picture of a storage configuration, defining terms broadly enough so as not to exclude any areas of storage. Second, individual storage vendors can get the most detail they need. That is, an storage management vendor can choose to look at the performance area that it needs to evaluate, while a spindle manufacturer can evaluate the details it needs.

"We work very, very hard on performance," says Roger Reich, planning manager for Compaq StorageWorks and one of the early organizers and founding members of the SPC. "We believe it is a key purchasing criteria for enterprise customers. We have everything to gain by having performance benchmarks."

In addition, Reich points out, "there are [currently] no fundamental points of comparison" for users exploring storage technologies. "There's nothing for enterprise solutions. [The SPC] can fix that problem. SPC is uniquely comprised of storage experts whose goal is singular in purpose: storage performance."

In late July, the SPC planned to hold its first editing session of a draft specification for benchmarking. It plans to finalize the specification through the rest of this year and provide a draft version for the public at the beginning of next year.