Columns

How Do You Prove Storage Value?

The storage industry must learn to base purchase decisions on facts, not faith.

• By Jon William Toigo
• 03/01/2002

• "How do I compare different vendor products and identify the best one for my application?"
• "Which storage technology will best reduce my total cost of ownership (TCO)?"
• "How do I develop a financial justification that will persuade senior management to fund my $X million storage acquisition?"

Addressing these important questions requires that you understand how to derive—and prove—the value of a storage investment. Doing that requires a clear understanding of competitive characteristics, administrative costs, return on investment (ROI) over time and the impact of downtime.

Unfortunately, there are as many speeds-and-feeds comparisons, TCO reduction claims, and return-on-investment justifications as there are storage products in the market. It's confusing enough that many IT managers set aside hard data in favor of buying a safe, recognizable brand name. When the time comes to justify choices, they'll reiterate propaganda from vendors' brochures. Unfortunately, the safe choice may not always be the best choice.

Performance Measurement
Objective third-party test data still lags miserably behind the flood of new storage platforms entering the market each year.

Spec.org's SFS testing regime, probably the most useful, sets forth a robust, "real world" working set of operations and simulated data to test the capability of storage connected to servers via the Network File System (NFS). Testing must follow strict SFS methodology and the results must be certified by Spec.org before a vendor can boast about them.

Other vendor-based tools, such as Intel's Iometer, offer more questionable performance measurements. Iometer is a good tool, but like most generic performance measurement tools, its use can be "bent" to support the tester's objectives. I'll avoid the nuisance of hate mail and lawsuits by not naming names, but one network-attached storage (NAS) vendor got into trouble with Iometer misuse last year. The company self-tested with Iometer, publicizing results that were several orders of magnitude over the performance of storage products from established industry leaders.

But the tests were planned to favor the vendor's own architecture—they measured how quickly a file could be retrieved from the memory cache of the vendor's own product. Since the vendor's NAS platform utilized a larger memory cache than rival platforms, the tests were bound to favor that vendor. If the vendor in question had used a real-world set of operations (create directories, find files, retrieve files from and store files to back-end disk drives, and so forth), the results would have been quite different.

Without a real-world set of working operations and data, a methodology for testing across all platforms to be tested, and a policeman to ensure that all tests are conducted in the Yankee Doodle-est way, test results are meaningless.

If you want to know where all the pieces will go when two trains collide, then build a track, run two trains into each other, and take a picture. Mike Linett, President of Zerowait in Newark, Delaware does exactly that. His team takes the application requirements directly from the customer to build a simulated working set. He then tests the performance of each prospective storage platform against the working set to measure performance.

A trusted partner like Zerowait can help IT managers establish a standard test bed and obtain meaningful performance results. I emphasize the word "trusted." Most integrator/resellers have financial arrangements with those vendors whose products they include in their "solutions."

Comparing TCOs
Over time, storage platforms may have hidden costs in the form of administrative staffing requirements, maintenance-related downtime, training and more. To reduce total cost of ownership (TCO) for storage technology, these ownership costs must be attacked.

Some analysts assert that different storage products offer different levels of TCO cost reduction. To see the differences, compare status quo costs (costs to operate existing storage) with costs of operating a new platform. If you repeat this process for each storage platform you're considering, you can readily compare the TCO advantages of several acquisition options.

Others say TCO provides a foundation for a "payback" calculus. The amount of time required for the aggregated TCO cost savings of a storage product to equal the acquisition price is the payback time frame. Comparing the payback time frames for multiple platforms, they claim, provides a practical measure of product value.

But such an approach considers only the costs of a current platform, and none of its benefits. TCO shouldn't be used to justify acquisition decisions directly. But there's hardly a vendor today that doesn't offer TCO payback numbers.

Storage costs vary—sometimes greatly—from IT shop to IT shop. Beware of any vendor claims using "typical" IT shop generic costs. Vendors rarely have cost data specific to the customer's current environment, so using generic costs is rather like a car salesman telling you how much money you'll save with a certain car, without knowing how far or frequently you drive.

In addition, TCO advantages generally hide more than they reveal. In most vendor TCO payback models, it's hard to tell if the savings required the purchase of hardware platforms in their entirety, or if implementing effective storage management software across existing platforms would have yielded the same savings.

Bottom line: TCO analysis can be useful in identifying cost components of the existing storage infrastructure, if applied using direct data from the IT shop. But using TCO-based comparisons to demonstrate payback time frames is foolhardy. Moreover, savvy business or financial managers could view a TCO payback-based business case for a new storage acquisition with suspicion.

Or, as one Fortune 500 executive once told me, "If I took all of the TCO analyses given to me by IT folks to justify equipment acquisitions and totaled the cost-savings, it would quickly become obvious that we could show a profit without ever opening our doors for business."

ROI Back in Vogue
Return on investment (ROI) has come back into vogue recently as a vehicle for justifying the acquisition of storage platforms. Techniques vary, but basically, ROI strives to show investment value in terms of an internal rate of return. The theory is that the difference in operating cost before and after storage investment, expressed as a percentage, can represent the return on that investment.

ROI justifications may use the findings of a TCO analysis. But unlike TCO payback, ROI justifications are framed in investment terminology: "This investment will yield X percent to the organization over the next three years."

A solid ROI case is more difficult to develop than a TCO payback. The mix of financial and technological skills and knowledge required for the successful development of a realistic ROI model is often cited by IT managers as the chief impediment to undertaking such efforts.

Lately, many storage resellers and integrators have been hanging out "ROI services" shingles to bolster sales and create new revenue streams. Some providers, such as Avnet Enterprise Solutions, offer a solid methodology that has been provided to key customers for several years. Others are simply harnessing the ROI "templates" provided by various storage platform vendors that may or may not offer valid assessment models.

There's a lot of snake oil out there. As with TCO "calculators," many vendor tools make some "generic" assumptions and are stilted to favor the vendor's platforms. IT managers seeking to outsource ROI development need to pay heed to the dictum: caveat emptor.

There's no easy answer to the question of how to be sure that you're getting the best value for your storage investment. Clearly, you as the IT manager need good data on the existing storage platform—covering I/O performance, utilization trend analysis and operating costs. This is the real heavy lifting of strategic storage planning.