In-Depth
Time For a SAN Report Card
The old song "Who's Sorry Now" is playing loud and clear in many IT shops.
Remember when you bought your first SAN—well, that channel fabric the industry calls a SAN, anyway? The vendor—most likely a reseller/integrator with a local presence and a lot of face time to dedicate just to you—painted a rosy picture of your new storage world with wide eyes and big phrases such as "limitless scalability," "unparalleled reliability," "simplification through virtualization," and "economies of scale."
The SAN would alleviate downtime, you were promised in a way that seemed to make intuitive sense. After all, with direct-attached storage, any failure in the server made its connected storage unavailable. SANs would fix that problem by severing the umbilical cord that made storage dependent on servers.
Moreover, the SAN would enable more storage to be managed by fewer technicians. Just centralizing the storage physically in one location would mean that you no longer needed to deploy support service personnel in every department or branch office of your company.
With good SAN management tools, a single administrator could manage a lot more storage capacity individually, so staffing costs would decline—which is a nice way of saying that you could start laying off people.
Moreover, the SAN would simplify capacity management, especially with SAN-based virtualization. With infinitely scalable storage pools, there would be no more "disk full" errors, thus further reducing downtime and lowering labor costs to solve capacity allocation problems.
The SAN would bring simplicity to the storage infrastructure and its management as well as to tasks such as disaster recovery backup. You could share that old tape silo you no longer needed for your mainframe among your distributed servers and their storage (now centralized in a SAN). Repurposing your library would lengthen your return on investment and at the same time enable non-disruptive backup processes that were oblivious to the shrinking backup window in the operations schedule.
In short, despite the high price tag, all the unfamiliar technology on which your personnel would need training, the relative immaturity of the underlying protocols, the high price tag, the interoperability issues between equipment providers, the lack of a common management standard, the high price tag, and a boatload of other caveats, SANs were nevertheless "strategic," the "next big thing," the "Big Kahuna," and a technology that "separates the enterprise class from lower orders of storage life."
Analyst after analyst reinforced these arguments. Working desperately to bury their mistaken oracle that network computers would replace WinTel PCs by 2000, Gartner jumped on the SAN-wagon. They said that SANs would be substantially cheaper to own than direct-attached storage topologies for all the reasons already explained to you by your reseller. But when Gartner said it, it sounded eloquent.
Not to be upstaged, and perhaps to direct attention away from their erred prediction that application service providers would replace all shrink-wrapped software by 2000, IDC also began towing the line. One fellow there wrote that SANs would enjoy a 68 percent increase in growth over the subsequent five years, compared with a paltry 9 percent increase in the growth of DAS installations. Big numbers, those—though from an epidemiological standpoint, 68 percent growth in the number of people with Ebola virus might translate to 10 new cases of the disease every year, while a 9 percent growth in the number of folks with the common cold would mean billions of new customers for cold remedy vendors.
So, a lot of folks bought SANs—or, rather, Fibre Channel fabrics. Everything was so new; even the cables and connecters looked different—thinner, more trendy. More like Apple iPod gear than a Big Iron topology.
Think back: When you were first unpacking your gear, didn't the diminutive switches and dental floss cables make you wonder what exactly you were paying $3,000 to $9,000 a port for? But you quickly dismissed the thought and hid the smallish hardware boxes where the bean counters wouldn't look, in order to avoid answering any embarrassing questions.
The Report Card
You dismissed or disregarded that sinking feeling in the pit of your stomach at the time, but chances are it has been nagging at you ever since. Maybe it's time to do a report card on FC SANs, or at least a sanity check, to see whether you have realized the value that was promised by your vendor. I’ve started to take inventory of FC SAN investments by companies all over the world. Here is some of what I am finding in my conversations with medium and large companies that have deployed the technology.
The so-called success of Fibre Channel fabrics is not the result of fulfilling marketing promises. Certainly it isn't attributable to performance. Those who've been around the block a few times and who are now on their third or fourth SAN keep hoping against hope that the new one will deliver something approximating the performance promised in the brochure.
Nor is SAN success about uptime. A recent survey in The Disaster Recovery Guide revealed that SAN failures were the third most common cause of application outages, just behind natural disasters such as Asian tsunamis, California earthquakes, Florida hurricanes, and outages in networks (real networks, that is, not SANs).
SAN management is still sorely lacking. ANSI had to go to the Internet Engineering Task Force to write SAN management MIBs that actually worked. Despite the flood of proprietary fabric-management tools, management remains elusive. SAN management products continue to have more holes in their functionality than Swiss cheese, mainly, it seems, because vendors don't want to appear to be selling commodity wares—which is the impression left by any common management scheme.
In the absence of effective SAN management, most IT mavens find they're being challenged to grow their staff because of the SAN, not to find ways to repurpose all the techs who have been rendered obsolete by the efficiency of the storage topology. SANs have not reduced labor costs in storage and, in these cash-conscious times, hiring approval is hard to come by.
Making matters worse, in some cases, is that the same vendor that sold you the SAN is now going behind your back to management to have the whole infrastructure outsourced to its services organization. The vendor’s rationale? "Networked storage is obviously too difficult for the IT department, with its limited skills, knowledge, and resources."
The so-called success of SANs is not predicated on the ability of the technology to deliver "virtualized storage pools that simplify the provisioning of infrastructure to applications." Truth be told, a dirty little secret with most virtualization schemes—all of which are in-band, regardless of what Gartner and the rest have been saying—is that they offer no intelligent way to differentiate among the capabilities of discrete storage devices in order to create stable pools.
Just ask Hu Yoshida at HDS, or Zia Aral at DataCore, or many other knowledgeable people in the technology side of the industry (as opposed to the marketing side) about this some time. Without granular virtualization and a lot of other, yet-to-be-delivered functionality, there is no way for SANs to deliver what the visionaries promised: not just a drink of storage, but the right drink of storage to applications using this infrastructure.
The SAN Success Factor
To what, therefore, can we attribute the so-called success of SANs? Is it just good marketing and salesmanship on the part of vendors? A demonstration of their ability to sell ice cubes to penguins, that sort of thing?
To some degree, the answer is yes. Some SANs were acquired because a huge industry marketing machine made it seem like the Next Big Thing. But if this were the complete answer, wouldn't we be seeing a huge uptake of Fibre Channel fabrics by small-to-medium-sized businesses? This certainly isn't happening, despite the swarm of analysts who are being paid big bucks to peddle the notion that it is.
In my humble opinion, the answer is rooted less in the marketing machinations of the three-letter acronyms (IBM, EMC, HDS, HP…uh…Q) than in a different set of three letters: CYA.
Having made the choice to acquire a Fibre Channel fabric for whatever reason, perhaps catalyzed by the marketecture of the vendors and the ruminations of their analyst minions, many of the IT folk I talk with believe that they are stuck with the results, good or bad. They have invested so much of their political capital, staked so much of their technical reputation, and committed so much of their scarce budget to deploy the kludge of Fibre Channel cables, switches, HBAs, and controllers that, simply put, there is no exit strategy available to them.
Working constantly to "put lipstick on the pig" is, for most, preferable to the alternative, which is taking the Walk of Shame from the cubicle to the back door of the building and out into the cold, miserable world of humiliating job interviews and potentially long-term unemployment.
I suspect that CYA is what has created the so-called success of SANs. It is what makes me angry every time a vendor smiles smugly when you criticize him for selling half-baked solutions that aren't really ready for business prime time. For him, it's all about the purchase order. For you, it's your career, your kid's college, mom's hip surgery, and all the rest of the realities of your life.
Going forward, my best advice is to back out of the mess by increments, just as you probably got yourself into it. Take a hard look at the kind of storage your applications require, then strategize to find ways to give it to them, whether the solution leverages the Fibre Channel fabric or not. To paraphrase Einstein, getting out from under your SAN disaster will require a different way of thinking than what got you into it. This time around, think application requirements first and storage second.
Your opinions are welcome, whether your SAN report card matches mine or not. jtoigo@toigopartners.com