Readers Respond

Perfection is indeed a worthy goal….and we try.

A recent column about the EMC DMX 3 prompted mixed responses from readers. Predictably, a few viewed it as a biased attack on their preferred hardware solution. This is the expected outcome whenever a particular vendor is singled out for review.

Wrote one fellow, “I don't always agree with what the marketing at EMC and IBM say, and so I enjoy reading articles that provide a good, competitive analysis. Unfortunately, the article "Bigger can be Baffling" is not one of them. I was hoping to get an overview of the new EMC gear, and a comparison with (at least) IBM and HDS gear. I do a lot of work on the ESS Sharks, and customers are always asking questions comparing IBM to EMC. What I got out of the article seemed like a personal attack. It was full of opinions, and not fact. Most of my customers run very large shops, and they are healthcare, banking, government, and retail. They are the ones who can benefit from a thoughtful analysis, but I can't refer them to this article, as it is just full of what seems like biased statements. It seems like Mr. Toigo has stock in HDS or IBM.” (For the record, I have no stock in any of the aforementioned companies, and the technical points of the piece were correct, as confirmed in a follow-up call to EMC.)

On the other hand, I received several that complimented the article for its insight and acumen. The chief technologist for an EMC competitor wrote a personal note that stated, “Lots of good lessons in this. Adding capacity requires more than just bigger disks and longer loops. As the drives get fatter it takes a longer time to format and rebuild. Hot swapping an average of seven LC-FC drives per month will cost days or weeks in rebuild time if additional functions like RAID 6 are not added. And how many months would it take to migrate a petabyte of storage to the next generation of DMX after the warranty expires and the box is fully depreciated? It would help if the majority of that petabyte was externally attached so that you could simply switch over by dialing the path connection over to the next generation controller. I believe there is a need for big iron, but it has to be balanced with safe multi-tenancy and QoS, and scales with connectivity and functionality. While I enjoyed the article since it was written about our competitor, I am mindful that some of this applies to us as well. Thanks for your insights that help keep us grounded.”

This columnist certainly has no axe to grind with EMC. In past columns, we have pointed out the foibles of products from the company’s competitors, as well—all in an effort to arm consumers with the information that they need to properly evaluate the product offerings and to weigh the public statements of vendors and their paid shills in the analyst community. To the extent that this column helps consumers to think independently and to question the value proposition of the technology products being proffered, it provides a service.

That said, it is also true that storage is pretty dry stuff. Fundamentally, we are talking about a technology that hasn’t changed a great deal in more than 30 years. There have been improvements in capacity based on advances in certain basic component technologies. The introduction of Gigantic Magneto-Resistive (GMR) read/write heads and Partial Response, Maximum Likelihood (PRML) algorithms, combined with Ruthenium-laced sputter coating processes for media, have contributed to astounding areal density improvements. Coming on the heels of these innovations are technologies such as perpendicular recording, which should be appearing on drives from Hitachi within the next year and a half, and which enable huge increases in the amount of data that can be stored on a single platter.

Beyond hardware, there has been a proliferation of standards, dutifully reported here, for connecting boxes together and for enhancing the speeds, feeds, and resiliency of storage itself. Many of these have fallen prey to proprietary implementations by vendors, leading to an endless litany of interoperability problems between products bearing different brand names.

Instead of standards forming the basis for solution-building, they seem to have stimulated an on-going business in plug fests and interoperability tests that produce, in the end, Yellow Pages-sized documents for each of the 11,000 new storage products introduced annually that provide, at best, a clumsy mechanism for identifying with which products each device will or will not operate. The problem with standards is that there are so many of them.

On top of all this hardware and these standards-that-aren’t-really-standards is software. Some of the software for the management, movement or protection of data is developed (or acquired) by hardware vendors, who tend to shape it to serve only their own gear (as part of an effort to freeze out the competition from “their” customer shops). The rest is developed by independent software vendors: smaller ones who have little chance of competing against brand names unless they align themselves with each other or with bigger vendors, and larger ones whose suites of many software products generally include a mixture of gems and junk.

We cover both the innovative smaller guys, who can’t get an inch of ink in other publications, and also the larger suite providers, because they share a common problem: the need to beg, borrow or reverse engineer the interfaces that give them access to proprietary hardware platforms in order to achieve real cross-platform management capability. The battle over storage cost containment and business value is ultimately fought at the software level, not the hardware level. EMC’s former CEO said as much a couple of years ago, when he noted at a Santa Barbara conference that, going forward, there was little to differentiate the hardware of leading vendors, since everyone is selling a box of disk drives. In the future, he said, hardware vendors would need to wrap themselves in “value-add” software for differentiation. Truer words were never spoken. So, we try to keep tabs on what the storage software guys are doing and the vision that shapes their development efforts, while at the same time pointing up the pain of the consumer with the generally poor condition of management in their heterogeneous storage infrastructure.

There is no way that every product can be touched by this column, but we strive to keep abreast of the technologies so that readers are provided with enough information to help them ask the right questions before making a purchase. To ensure that the information sticks, this columnist sometimes resorts to “entertainment”—phraseology intended to anchor key points in the reader’s mind. Such humor, whether ironic or sarcastic, is a time-honored tool of opinion columns, but it is also a two-edged sword, because some readers do not appreciate it. To paraphrase Barnum, you can’t entertain all of the people all of the time.

If this column errs, it is probably in the bias it takes against vendor proprietariness and double-speak, which makes effective product decision-making by consumers a difficult task. However, in our efforts to champion the consumer, we sometimes fail to note that there are a lot of consumers who invite their storage problems upon themselves.

Some outsource their thinking process to the vendor altogether, which is a potentially acceptable strategy if you know the vendor to be offering products of the highest caliber and to have a service and support organization to match. Vendors, however, tend to be uneven in both categories. Even a top drawer vendor can produce a product best described as a “dog” from time to time, and service quality varies widely, based on personal experience and input from readers, often as a function of changes in vendor bureaucracy over time and the knowledge and skills of the support team assigned to an account.

In a recent consultation with a large Fortune 500 manufacturer, I was told of plans to roll out Fibre Channel fabrics in 20 manufacturing plants. I asked whether the team had analyzed application requirements before choosing this strategy and was told, “No, we figured we would put in the infrastructure, then fit the application to it.” Upon further questioning, it was revealed that the application in each plant, a SQL Server database, generated about 360 GB of data in a fairly light transaction load that was deleted every 24 to 48 hours in its entirety: hardly the kind of workload that necessitated a fabric. Moreover, the organization did not have centralized management or a cadre of skilled technicians in every plant to troubleshoot and fix the fabric whenever it breaks down. They were deploying the storage topology because their vendor had described it as “strategic.”

The bottom line: there is no shortage of storage issues that are the result of bad choices by consumers. We do not endorse a Nader-esque view of the universe: big vendor beating up on defenseless consumer. Often, the reason the consumer’s storage infrastructure is in such bad shape is that the consumer let it get that way.

Comparing product offerings, as suggested by the critical commentary at the outset of this piece, is a dicey proposition. We have done so in the case of some products, but there are limitations to what we can do at the level of DMX 3, TagmaStore and Shark. Some of these vendors are delighted to share with us their “competitive matrices”—tools they have created to compare features and functions of their gear with those of their competitor’s gear. We prefer not to run these, as they are usually self-serving for the vendor who provided them. The alternative approach is to test each product ourselves, which we have been able to do in some cases. As a rule, independent testing is hampered 1) by the high cost of large enterprise arrays and 2) by explicit provisions used by EMC, Network Appliance, and others in their sales agreements and warranties that make it illegal for a customer to talk publicly about the performance received from their purchased product.

About the only way to circumvent these hurdles is for consumers to provide us—often anonymously—their own reports on product performance. In evaluating these reports, which we receive fairly frequently, we must carefully weigh the veracity of the report against the possibility that results are skewed by the specific implementation made by the consumer of the product, or the possibility that the report has been “planted” by a vendor. Yes, sometimes vendors do plant reports about their products, either to make their product appear to be better than it actually is or to condemn a competing product by pretending to be a “disgruntled user.”

In a column a few years back, we actually caught one vendor red handed pretending to be a customer of its competitor and stating everything wrong with the product that he never owned or operated. On the flip side, we have documented many, many instances of marketing claims made by vendors about their own gear that just didn’t stand up to the facts.

Truth is hard to come by in the storage industry. We are doing the best we can to provide useful commentary and welcome the continued feedback of readers—positive or negative—as to how we can do better. Please send your comments to jtoigo@toigopartners.com