In-Depth

Leveraging the Virtual Storage Hierarchy

• 11/01/1999

In a recently published white paper by International Data Corporation (IDC) titled "Virtual Storage Architecture – The Basis for Storage Utilities," storage systems analysts John McArthur and Vernon Turner wrote: "As corporations embrace content-focused computing, storage and information management become more central and critical to businesses. Traditional approaches to storage architecture and information management may inhibit access to content and limit the ability of IS to exploit this content wave to its fullest. In contrast, virtual storage architecture (VSA) meets the content-driven needs of today’s business and will continue to do so into the future."

One of the major reasons for the success of virtual storage solutions is their ability to leverage the entire storage hierarchy, ranging from performance disk to performance tape to archive tape. This inherent ability in the virtual storage architecture to exploit and manage the entire storage hierarchy allows virtual storage solutions to provide the performance and fast data access of disk-based solutions while exploiting the cost advantages of tape-based solutions – characteristics in great demand in today’s companies. By combining the benefits of both tape and disk storage technologies into a single integrated solution, virtual storage solutions are able to provide greater benefit to the user.

Despite the dramatic advances in storage technology during the past several years, most storage solutions today maintain a "physical" view of storage. This traditional storage architecture is based on a one-to-one relationship where every server must maintain control over its own dedicated physical storage and every megabyte of storage must be available to the server at all times.

In traditional storage architectures, the layers of the storage hierarchy remain physically and logically segmented. Host-based applications are required to interact with each layer of the storage hierarchy through separate device-specific interfaces and to directly manage the movement of data to a particular layer, say to capacity tape for permanent archiving of files. Using a traditional storage architecture, once the data is placed within the storage hierarchy, the burden of managing the migration of data from one layer to another falls to the host-based applications or host-driven data migration utilities. Any movement of data between layers within the storage hierarchy typically requires the use of the application host to manage the migration of the data and provide the conduit through which to move the data.

Administration of storage solutions based on traditional storage architectures is also typically segmented. Because of the lack of logical integration of storage components, users must manage the components independently. This is typically done with the use of "home-grown" or third-party storage administration tools, one tool for the disk-based storage products and one tool for the tape-based storage products. There are no globally established policies for managing data and no centralized administrative interface from which to apply management and administrative policies and procedures.

From a user perspective, the traditional storage architecture (and its inherent physical view of storage) offers very little synergy amongst the storage components and has a number of drawbacks. Specifically, it relies heavily on the host to be directly involved in storage management and administration. It generally offers no effective means for centralized data or resource management. The burden of managing the migration of the data within the storage hierarchy is host-dependent and often requires manual initiation. And because of its inefficiencies in using resources, disk and tape capacity utilization tends to be low.

The result: An increase in the total cost, and decrease in the potential value of storage resources. Even with the predictable declines in the cost per megabyte of tape and disk storage and increases in performance, traditional storage architectures are no longer sufficient to support the explosion of data fueled by applications, such as the Internet and data warehousing.

To address these shortcomings, a new generation of storage solution – based on a virtual storage architecture (VSA) – has emerged, and offers dramatic advantages over traditional storage architectures. Virtual storage solutions are designed to emulate a physical device where the characteristics of the emulated device – say a tape drive – are "mapped" over another physical device – say a disk subsystem. The result is the creation of a virtual device that separates logical and physical characteristics from one another. This virtual device is then accessed by the application as if it were the real device.

Because of its logical nature, the virtual device can leverage many of the physical characteristics of the underlying physical device, such as greater reliability, improved throughput performance and reduced data access times. And because of the logical and physical separation, physical devices can then be upgraded or replaced without requiring host or application changes.

One of the most important features of a virtual storage architecture is its ability to create a more synergistic storage hierarchy using an integrated combination of tape and disk to create a "logical" view of storage and data access. Using a logical view of storage, every server operates as though it has access to its own dedicated physical storage. However, it does not require that storage be dedicated to the server or that every megabyte of storage be available at the time the storage transaction occurs.

As a result, virtual storage architectures go well beyond traditional storage architectures by tightly integrating the storage hierarchy and seamlessly managing the migration of data between the layers of the storage hierarchy. Virtual storage architectures address many of the complexities of storage management and are, therefore, much better equipped to handle the explosion of data.

With virtual storage solutions, host-based applications are not required to interact with each layer of the storage hierarchy separately. Rather, applications interact with a storage hierarchy through a common "virtual storage interface" that presents a common device image to the application. An example is a virtual storage solution that presents itself as 64 virtual 3490E tape drives, even though it really consists of a high-speed disk cache, an automated tape library and four high-performance tape drives.

Host-based applications continue to operate as they would with a 3490E tape drive and leave the task of managing the data and generating massive storage efficiencies to the virtual storage solution.

Another feature that differentiates virtual storage solutions from traditional storage solutions is how data is managed and migrated within the storage hierarchy between the storage layers. With virtual storage solutions, the intelligence for managing data is resident in the storage device rather than on the host. In addition to off-loading the storage management workload from the host, this approach provides a centralized approach to managing data and its migration between the layers of the storage hierarchy.

Using a common set of storage management policies, users create a single storage infrastructure that automates the migration of data between storage layers, balances the storage of data within the storage hierarchy and optimizes the placement of data to be within the most cost-effective layer within the hierarchy.

Because of the tighter logical integration of the disk and tape storage devices in a virtual storage solution, storage management is greatly simplified, resources are better utilized, storage administration is more easily accomplished and the total cost of ownership is reduced.

Storage solutions based on virtual storage architectures provide users with the best of all worlds: They combine the benefits of disk storage (e.g., throughput performance and fast access) with the benefits of tape (e.g., cost and security). Virtual storage solutions also help address many of the limitations of disk-only or tape-only solutions by providing benefits, such as:

• Reduced complexity through a single, integrated storage infrastructure

• Reduction of application disruptions through increased availability and reliability

• Reduction of storage management costs through automated data management

• Minimization of unused storage capacity through increased utilization

• Reduction in storage costs through device and resource sharing

• Extension in the life of existing storage assets

• Increased application performance through improved data access times

Virtual storage solutions also offer a number of very real business benefits. By way of example, virtual storage solutions help reduce the amount of storage management through increased storage automation and thereby reduce overall storage management expenses.

With the improved system reliability offered by virtual storage architectures, vis-à-vis traditional storage architectures, virtual storage solutions greatly reduce storage system downtime and thereby improve revenue, reduce expenses and reduce risk. And with the increased throughput performance and reduced device access time, batch and backup windows can be dramatically reduced, providing more application uptime and an increase in revenue.

Today, virtual storage products exploit virtual technology to provide dramatic improvements in performance, availability and total cost of ownership over traditional storage solutions. In fact, virtual disk subsystems have been so successful that they currently account for 30 percent of all enterprise disk subsystems sold today. Users of these products are getting more done with less cost than ever before as a direct result of virtual technology.

The major advances in virtual storage over the next few years will likely come in the area of increased storage management "intelligence," new device images and open systems connectivity. As storage policy and data management intelligence increases, virtual storage solutions will further increase the savings associated with storage management. The introduction of a variety of "outboard" data movers that are managed as part of the virtual storage solution will eliminate the need for hosts in data migration and enable the implementation of true, large-scale storage area networks. The introduction of additional device images, such as NFS, HTTP or CIFS and open systems connectivity, such as fibre channel, will allow virtual storage solutions to integrate the open systems world and provide value without requiring the rewrite or heavy modification of existing applications.

The next generation of virtual storage solutions will also offer further advancements by increasing throughput, access time, and capacity and position these solutions to address issues such as data sharing. Clearly, virtual storage solutions offer significant advantages over traditional storage solutions and help companies remain competitive in an information-intensive world.

About the Author: Don Langeberg is the Manager of the Enterprise Nearline Business Group Marketing Strategy at StorageTek.