In-Depth
Rise of the (Non-)Standard Block-Storage Protocols
Several changes today are opening the door once again to more novel and less standards-based storage protocols.
For several years, consumers have been presented a relatively straightforward set of choices for cobbling together connections between external storage and servers. Parallel SCSI was (and remains) a mainstay of external array connections, with Fibre Channel and, more recently, iSCSI, providing a new set of plumbing alternatives aimed squarely at the enterprise setting— the two percent of companies that accounted for up to 60 percent of storage spending.
Along the way, there were numerous protocols that tried—and failed—to garner widespread industry adoption. One vendor in the late 1990s offered a parallel SCSI switch, termed the Jigsaw, that enabled the connection of multiple parallel SCSI arrays to one or more servers. It might have been the right solution, but at the wrong time. The industry was already looking at serializations of SCSI, such as Fibre Channel and Serial Storage Architecture (both developed by IBM, which has also taken a leadership role in the development of iSCSI). The failure of Jigsaw is testimony to the truism that in the storage business, timing is everything.
Another ill-timed storage protocol was Adaptec’s EtherStor, introduced in 2001, then unceremoniously yanked from the market. The technology used Ethernet and UDP, the stepsister of TCP, as a transport for SCSI commands and data. Stories continue to be told about this abortive attempt to do block storage over a network, with the most interesting (though possibly apocryphal –-- we could find no one who knew the whole story) holding that Cisco Systems was behind EtherStor’s demise.
EtherStor came to market while the development of iSCSI standards at the Internet Engineering Task Force had ground to a virtual halt. Adaptec was all but ready to ship its EtherStor adapters when Cisco encouraged them to cease and desist. According to our sources, Cisco, at the time a champion of iSCSI, was afraid that the appearance of alternative network storage protocols at a critical juncture in iSCSI development would balkanize the emerging standard, giving Fibre Channel a permanent edge in the market. (Cisco wasn’t a FC switch vendor then, as it is today.) In a closed-door session at Storage Networking World, only a few months after the announcement of Adaptec’s protocol and product, Cisco allegedly convinced the “Grey Old Lady of Storage” (Adaptec’s name was once synonymous with parallel SCSI) to remove its contender in exchange for most-favored-vendor status in every Cisco implementation of iSCSI. Adaptec acquiesced and EtherStor was never shipped.
Following a different path to networked storage, the founders of LeftHand Networks in Boulder, CO, created another UDP-based block-storage protocol, which was implemented on their early products in 2001. Advanced Ethernet Block Storage (AEBS), the brainchild of LeftHand CTO John Spiers, was arguably the first implementation of IP storage that shipped to market. However, on the advice of many consultants (myself among them) and CEO Bill Chamber’s read of the marketplace, the company embraced iSCSI instead of AEBS in its products once the standard was ratified at IETF.
At the time, it was the safe move and the smart choice—as suggested by the vendor’s profitable status in the hotly competitive storage market today. As usual, start-up storage vendors such as LeftHand needed to compensate for their lack of market tenure and console consumers about the risk of choosing their products by wrapping themselves in the flag of standards. The idea was, even if LeftHand bit the dust, their standards-based products would continue to deliver value. The customer wouldn’t be left holding the proverbial bag.
That was then, this is now. Several changes are occurring today that are opening the door once again to more novel and less standards-based storage protocols.
The market has changed: the preponderance of storage-spending growth is no longer being seen in the Fortune 500, but in the small to medium enterprise (SME). The big companies have almost unanimously latched on to Fibre Channel as their storage plumbing. It is fast and conveniently packaged by vendors that own the top tier of the market: EMC, HDS, and IBM. One tape vendor decries that, while Gigabit Ethernet with Jumbo Frames is a more efficient protocol for doing backups, “you can’t sell a port of anything that isn’t Fibre Channel in the Fortune 500.”
The SMEs, however, have very different values and very different budgets. HP, which has done some interesting bundling of Microsoft’s FC fabric setup wizard and QLogic switches with their own storage—creating a “SAN in a Box” aimed at SMEs—says that the product is the hottest-selling storage option in its channels. To the extent that it is true (and considering the meaning of this assertion relative to the sales of other storage products from HP, which are rather poor the last time we checked), Fibre Channel does not seem to have the traction in the SMEs that it enjoyed in the Fortune 500. Simple reason: it costs too much and is a hassle to own over time.
Does this create an inroad for iSCSI? It might—but only if iSCSI gets back to its roots.
The original argument for the technology was that it was simpler to deploy and easier to manage than FC. A company could buy a $15 network interface card, an $80-per-port GigE switch, and an iSCSI target, and deploy a full-fledged storage area network (sort of). However, in the years since the standard was first introduced, vendors have been “adding value” to iSCSI in an effort to make it fully competitive with Fibre Channel (that is, to sell it into Fortune 500 accounts). In the process, we have seen the evolution of pricey iSCSI host bus adapters ($400 to $750, not a $15 NIC) and specialty iSCSI switches whose prices are on par with FC gear. Bottom line: in going for the big money, the vendors have overplayed themselves in terms of their “SAN for the everyman” with iSCSI.
That brings us full circle— back to block protocols that don’t use iSCSI or TCP at all. At one end of this fast-growing spectrum is Coraid, of Capistrano, CA. The company has created an ATA-over-Ethernet (AoE) protocol that extends the storage protocol used in just about every server in existence to attach storage over a simple Ethernet cable. Their pitch is simple: AoE is open-standards based (what is more open than ATA and Ethernet?) and leverages the cost advantages of Ethernet to provide networked storage (using their own storage target “blades”) that is simple to understand and simple to manage.
Representatives say that they aren’t interested in UDP, TCP or IP— just Ethernet. They smile when they remind us that Ethernet was first invented to attach storage to servers, so they are going back to basics. They smile even more broadly when noting a kind of coup they recently pulled off: they gave their driver software stack to the open-source world and it is now embedded in all Linux kernels, so their gear is automatically recognized by Linux servers. It works with Microsoft as well, they claim, because the OS sees an ATA-attached drive.
Another server backplane extension strategy comes from JMR Electronics in Chatsworth, CA. This month, the company unveiled its PSAN, touted as a high-performance alternative to Fibre Channel SAN solutions that allows IT professionals to easily deploy highly scalable storage solutions at lower cost than comparable Fibre Channel solutions.
This switched-storage solution is also described as standards-based because it leverages the PCI bus architecture of most server mainboards, simply extending the bus to enable the external connection of shared storage. According to the vendor, we can expect to see massive capacity, high availability, and the lowest price per terabyte available in the market today.
Previously cited in this column, IP-based storage is not dead by a long shot. Zetera in Irvine, CA has delivered what is potentially the most disruptive play in networked storage since the invention of two cans and a piece of string. Earlier this year, they partnered with NetGear and launched a “MicroSAN” called Storage Central. This toaster-sized appliance, a proof of concept aimed at the consumer market, plugs into any Ethernet network, obtains an IP address automatically, and uses a proprietary protocol to be shared with any server or workstation on the net.
The NetGear play was brilliant strategy, enabling Zetera to enter the market under the radar of most storage vendors. Successful is an understatement when describing the $129 appliance (you just add the drives of your choice). As indicated by the near impossibility of sourcing one from Best Buy or any other commercial outlet because stock sells out as rapidly as it appears, Zetera will soon become a household name (think TiVo)—and that, company representatives hope, will translate into greater adoption of its next generation of products aimed squarely at business customers.
Zetera’s model is the essence of simplicity: take disk drives, attach them directly to an IP network, assign each a unique IP address, then stripe across the drives using IP multi-casting (a function of any IP switch) to create arrays. So straightforward is the idea, it boggles the mind that no one has done it yet, founders say. And they own the patent.
You can almost begin to hear the vaults opening up at Adaptec and LeftHand Networks: it may be time to blow the dust off their UDP-based storage implementations that have been shelved for several years. In the SME market, the rules are different than in the Fortune 500. Price rules, and “good enough” is more important than brand name or marketing glitz.
SMEs are showing spending growth on storage in excess of 47% per year. Do the math.
Your comments are welcome at [email protected].