In-Depth

Access Density: The Cause of Our Storage Woes?

While top administrators are spending money for things they don’t understand, IT professionals fail to see the value in overpriced goods.

• By Jon William Toigo
• 10/25/2005

At a recent Computer Associates roundtable in Detroit, storage growth and its related expense was the pain point most commonly cited by attendees. Only one attendee said his storage was cheap: he was paying about $1 per GB, which he attributed to shrewd negotiation with IBM. (He has promised to share his secrets with me for how he obtains such an unbelievable price, especially with “Tivoli Storage Management bundled in,” which I will share with you when he does.)

Despite this anomaly, the norm was that storage costs in the Detroit-area companies represented at the event were large and growing larger. Despite the improvements in areal densities in disk (how much data you can squeeze on a platter of media), which had grown ten fold over the past ten years, and the cost per GB for disk media, which had dropped 500 percent over the same period (50 percent per year), the cost of arrays themselves (especially when fielded in expensive FC fabrics) had climbed a whopping 128 percent per year.

The obvious explanation proffered by many attendees was that vendors were gouging consumers. They were putting so much functionality into disk arrays that they were propping up their prices on the products despite cost reductions in the underlying component hardware. In a recent column, we cited the example of EMC’s DMX 3, which includes a dual-ported memory cache that EMC spokespersons say is not technically necessary at all, but for which they exact a considerable amount from consumers. Certainly, similar examples can be found in just about all of the products from enterprise-class storage vendors.

Then there are the software value-adds: software that is joined at the hip to the array--controller hardware and sold with the array whether the consumer needs or wants it. As in other cities, Detroit IT mavens complained that they were using less than half of the software bundle, but they were having to pay for all of it.

These are all obvious explanations for the high cost of storage, but a recent conversation with Randy Chalfant, StorageTek (now Sun) Chief Technology Architect, exposed another dimension of the equation. Chalfant offered a partial explanation for the phenomenon of array cost acceleration that was based on access density. The more I listened to him, the more what he was saying made tremendous sense.

The theory of access density is simple. The more storage you put in an array, the more you centralize or route access to that array from users of stored data, be they applications or end users. Since there hasn’t been a significant increase in the capacity of the communications links that connect users to storage arrays, you start to see a performance hit as more and more users crowd their requests for storage services into the same amount of bandwidth.

To resolve the performance hit, storage administrators often seek to balance the workload across more and more storage, leaving a lot of storage in each array under-allocated. The larger the array’s capacity, the less you can effectively use.

Building on Randy’s insight, it seems to me that much of the extra technology being built onto high end arrays is intended to address access density so you can use more of the capacity you buy. Memory caches are trending larger so that more requests can be stacked and acknowledged (before they are processed) in order to free up bandwidth.

In Network Appliance Filers, for example, there is actually a lot of spoofing going on in the controller just to make their RAID 4, Write Anywhere File Layout (WAFL), Network File System (or CIFS/SMB) design work in a balanced way. NetApp uses nonvolatile RAM caches in a manner reminiscent of many channel extenders in the mainframe world, storing write requests and acknowledging that writes have been made before the work has actually been accomplished. That way, as we used to say in the days of mainframe bus and tag cabling, “the channel is not held high.”

Of course, explaining all of this engineering-ese to consumers is not easy, so vendor marketing departments try to sugarcoat explanations for their pricing using generalized assertions of “better, bigger, faster.” NetApp is doing this right now, going around telling everyone that each generation of their gear offers a substantial price/performance advantage over previous generations of gear, despite the fact that performance (which has been tested and articulated at http://www.spec.org) has not improved significantly, and has even declined with each new product they have introduced, while comparisons of street-price data shows that costs have actually increased on a generational basis. It may be wholly legitimate to charge more for a box that features new engineering gizmos to solve the intractable problems of access density, but you have to say that. Like most hardware vendors, NetApp doesn’t.

Instead, dumbed-down explanations are preferred over engineering details to explain why things cost so much. This is essential for marketeers, especially in enterprise-class hardware companies. Why? Because the sales model of the top-tier vendors has changed. They are no longer making a pitch to the IT guy who might comprehend their engineering approach, but to the front office: to Chief Executive Officers, Chief Financial Officers, Chief Operations Officers, and Boards of Directors. These folks don’t know storage from shinola and are singularly unimpressed by new caching algorithms or NVRAM spoofing. In addition, they don’t necessarily trust the guys in their own organizations, who are tasked to understand such things, to tell them which product is a better fit or merits its price tag. But as we all know, they control the purse strings.

One fellow in Detroit was ready to turn in his engineering degree and to go seek a job in the exciting world of fast food. The vendors, he noted, are schmoozing management so well, with golf outings and expensive dinners, that he has a boatload of gear today that belongs at the bottom of Lake Michigan and not in his data center. Sound familiar? In his words, “Michigan is already the dumping ground for all of the solid waste coming out of Canada. Why does my data center have to be the repository for all the waste coming out of Hopkinton and Silicon Valley?”

Why indeed?

There is no easy answer to the issue of why storage costs so much, but it seems clear that at least one explanation is the abdication of common sense around right-sizing storage to applications and balancing capacity with bandwidth and, lest we forget, processor performance on servers.

More on this subject next time. Until then, your feedback is welcomed at jtoigo@toigopartners.com.