In-Depth
Getting Smarter About Storage (Part 3 of 3)
Smart storage isn't about applications running on an array controller; it's about manageability of the infrastructure.
In my previous column, we hinted at this final column in a series covering "smart storage." This time, we want to look at innovative approaches to improving the manageability of storage.
The solutions for making storage easier to manage by fewer people have traditionally come down to three options. First, embed a lot of smart technology on the storage equipment itself, usually as "value-add" software functionality on the array controller. Although these boxes have immediate ROI, usually in the form of short-term labor cost savings, I questioned their longer-term value because of vendor lock-in, barriers to management in a heterogeneous environment, higher acquisition and long-term warranty and maintenance contract costs, and a propensity for downtime given controller complexity.
One alternative is to virtualize the physical array controller in a software abstraction layer. This could be a big improvement over option 1. One benefit of this approach is that value-add software (thin provisioning, de-duplication, etc.) is separated from physical controllers of disk arrays, thereby enabling their value-add functionality to be conceived as "services" that can be applied judiciously to data using all of your virtualized spinning rust. The downside is that most virtualization solutions only virtualize the volumes as presented by the equipment to a server or switch: configuring storage arrays themselves usually requires hands-on work using the element management tools supplied by the vendor of each array deployed.
Although physical arrays, uncluttered by value-add software, might break less often using this approach, storage administrators must still understand the differences between the data pathing schemes, RAID schemes, and other hardware engineering nuances introduced by each vendor on its gear. Unless you use rudimentary equipment, it really isn't as easy as simply aggregating capacity to present volumes to applications and to end users; mixing and matching disk from different vendors might create unstable volumes.
The third approach to smarter storage by driving down management cost is to deploy storage resource management (SRM) tools. SRM vendors, however, will be the first to admit that (1) keeping their software up to date with the thousands of storage products that are introduced into the market each year is a daunting task, and (2) the role of SRM is to provide monitoring and reporting intended to help managers identify problems and to plan their configuration changes. In short, SRM may let you look into your environment, but it doesn't automate most of the tasks involved in configuration or break fix.
Need to Blend Infrastructure Management with Data Management
In a perfect world, data would be routed through storage infrastructure on a policy-directed basis and exposed along the way to those services that are required for its proper storage over its useful life. Doing so requires real-time information about the status of services, which includes both software processes (value-add functions) and hardware resources (disk arrays of different speeds and capacities). For example, data may need to be written first to fast, expensive disk, then just as quickly migrated to slower, more capacious disk. If it is mission critical, it may need to be replicated between a local and a remote repository, and archived for a period of time. The more you can automate of these functions, the better your data is managed and the more efficiently you utilize your storage infrastructure.
A prerequisite to getting to this kind of storage service utilization efficiency, however, is developing a way to collect and report service status in real time. In essence, you need a way of collecting real-time information about your storage processes and resources. You need to know everything from waiting times for the gateway providing de-duplication services to how full volume X is to what the queue depths are on each drive on each device where data might be written. This data is used by a service broker (a piece of software) in conjunction with the policy engine that is routing the data through infrastructure.
Innovation is Happening
Sounds pretty far fetched? Actually, thanks to innovators in the market today, it is a lot closer to reality than you might think.
Throughout the summer, we have been inundated by vendors -- usually the smaller companies -- asking us to write an article on their latest wares. Stand outs are few, but some that have caught our attention because of their potential to deliver the pieces of a service-based metaphor for data and storage management, and in so doing, smart storage.
One was Sepaton, a provider of software- and hardware-based de-duplication solutions that seems to have come up with a different product with enormous potential value. The company was keen to brag over improvements to its flagship wares, including a disk pooling technology reminiscent of the mainframe world, but what really caught my attention was their ContentAware Analytics software. They realized that, in the process of delivering de-duplication solutions, they were also collecting some potentially valuable data on data traffic and storage consumption that had potential use beyond the idea of provisioning storage for their de-duplication engine. Their thought is that such analytics might be useful to firms offering cloud storage services either internally or as a third-party service provider.
The reports they generate are as clean and exacting as what we have seen from Fujifilm's and partner Crossroads Systems' analytics engine for tape environments. Fujifilm currently offers a Tape Environment Analysis Service that leverages Crossroads' Read/Verify Appliance (RVA) to collect and report detailed information about tape operations, including equipment status, error rates, mount frequencies, busy times, etc. that can be leveraged, again, by both tape operations managers within companies and external tape services providers to get more bang for the buck from their tape investments.
Moving out of the domain of hardware management and into the realm of cost-efficiency, Aprigo's Ninja software is looking at service provisioning from the standpoint of process and resource cost and the value of data itself, essentially to support those who want to provision services in the most cost effective way for the data that they are producing. In a really cool innovation, they bring to bear an interesting approach to comparative analysis that we are beginning to see in products such as CA Technologies' ARCserve line of data protection wares: users can see community benchmarks, data aggregated from other users of Ninja, that tell them how their cost-efficiency ratios compare to a community of other users of the product. With the latest ARCserve products, CA Technologies has integrated mechanisms for user communities to suggest changes and to discuss challenges and successes directly within the product itself. Such communal thinking bodes well for both CA and Aprigo because it helps make their agile development models self-validating and user driven.
However, the innovation that goes directly to the heart of realizing smart storage comes from the people at Xiotech Corporation. A short while ago, the company re-introduced its unique Web services-based management model, previously known as ICON but now called ISE Manager, to consolidate branding around the company's Intelligent Storage Element (ISE) technology. ISE Manager uses REST protocols from the W3C standards kit to collect and present information about ISE storage blades.
ISE Manager simplifies the previous SOAP-based approach used in ICON, which required an external software component (ICON Manager) to query the array controller on a Xiotech rig, collect information, then present its findings to a management console. Now, Xiotech places a REST protocol-based stack, called CorteX, directly on the controller. Their arrays send their information like an RSS feed directly to a wide range of devices -- from smartphones and iPads that can run the customized applets such as ISE View or on any laptop or desktop connected to the Windows Microsoft Management Console, with which ISE Manager is readily integrated. (We shot some video of a demonstration conducted during our interview and posted a clip here, in case you missed it).
Xiotech's innovation is important on many grounds. First, it is an open standards-based approach that Xiotech has made readily available to all storage vendors at CortexDeveloper.com. The company has provided the specs on its REST API as well as guidance to users who would like to be able to collect different information about their ISE arrays or to integrate other Web services-enabled products into a broader application (such as infrastructure management). That a company would give away this bit of secret sauce in such a competitive market is remarkable in and of itself; but it is also a harbinger of what Xiotech is ultimately seeking to bring about: a smart storage infrastructure combining their products with anyone else's provided the vendors will just embrace Web services. We will watch their partner ecosystem closely, as CorteX promises to provide a glue layer to join third-party products with the Xiotech ISE blade storage products in a commonly manageable way.
Conclusion
In the final analysis, smart storage isn't about applications running on an array controller, it is about manageability of infrastructure. Given the lean staff sizes in most business IT shops today, fewer people need to be able to manage more IT resources efficiently. That requires unified infrastructure management.
Many vendors understand this, of course. However, most are seeking to leverage the requirement to lock customers into proprietary hardware encrusted with expensive proprietary value add software. A service-based model makes more sense, but it will require adherence by all parties to open standards such as Web services.
Your comments are welcome. [email protected].