Storage Becomes Core Part of W2K

• 09/08/1999

Storage is becoming the backbone of corporate networks. The Internet, e-commerce, file sharing, messaging, and heftier desktop productivity tools are creating a huge demand for storage.

International Data Corp. (www.idc.com) expects storage capacity to grow at a compound rate of 89 percent a year until 2003. The analyst group predicts Windows NT and Windows 2000 will remain the fastest growing segment of the storage systems market with a 30 percent compound annual growth rate forecast through 2002. At that rate of growth, Windows 2000 will become the top storage segment in 2002, with a 38.6 percent share of revenue. Unix, growing at a more modest compound growth rate of 8.8 percent, would drop to second place with 35 percent of revenue.

The rocketing growth rate of the storage market has created a need for methods to manage storage. Using numerous small servers to handle storage isn’t a practical or easily managed solution.

Nikhil Joshi, a program manager for Windows 2000 at Microsoft, explains that the company’s approach to storage management in W2K is to provide management services within the operating system, core sets of base tools, and a platform -- or foundation -- on which developers and ISVs can build third-party value-added management solutions.

While building that foundation, Microsoft enhanced Windows 2000’s storage management capabilities to keep pace with the data explosion. The most significant storage management features built into the operating systems are hierarchical storage management (HSM), removable storage management (RSM), volume management and quota management.

"We can finally say with Windows 2000 that storage is a feature, a principal part of the operating system. In NT 4.0 that wasn’t the case," says Mark Hassall, a program manager in charge of storage for Windows 2000.

Hierarchical storage management is designed to provide a more efficient way to manage data on cost-effective media. To do so, HSM automatically moves files between disks, tapes and optical media. This migration process keeps frequently accessed files in fast local disk storage and moves less-used data to slower, less expensive devices.

HSM relies on remote storage services (RSS), a technology Veritas Software Corp. (www.veritas.com) gained through its acquisition of Seagate Software Inc. (www.seagatesoftware.com). Microsoft integrated the technology into Windows 2000 to transparently shuffle data between devices. RSS monitors the amount of space available on the local hard disk, and when the free space drops below a preset threshold, RSS moves data from the disk to the secondary storage. The directory and property information, however, is kept active. Likewise, when more space becomes free, files can be shifted back to the disk.

The architecture also streamlines the backup process. Rather than repeatedly backing up unused files, the system concentrates backup on the files that are stored on disks and used frequently.

"A huge percentage of the files housed by file servers haven’t been accessed within 30 days of their creation," says Jerry Held, vice president of the storage division at OTG Software (www.otgsoftware.com), a developer of automated data storage solutions. "HSM helps cut down on the great big waste of time created by backing up the same untouched files over and over again."

Microsoft’s Joshi cautions that the effectiveness of HSM relies on ISVs as well as on Microsoft. At this point, ISVs are waiting to see how well it works and are trying to figure out what they’re going to do with it.

"My fear is that companies will roll out HSM and think it is a piece of junk because third-party apps don’t work well with it," Joshi says. "We need a benevolent conspiracy to make HSM reach its potential. We’ve built it into Windows 2000, now we need ISVs to build apps that are HSM-aware."

Removable storage management (RSM), developed by HighGround Systems Inc. (www.highground.com), provides a single set of APIs that allows applications to catalog all removable media -- except floppy disks and similar small-capacity media -- whether housed offline, on shelves or online in robotic libraries.

As a result, RSM provides a common interface to robotic changers and media libraries, enables multiple applications to share local libraries and tape or disk drives, and controls removable media within a single-server system. With RSM, administrators can perform online tasks without shutting down the system or interrupting users.

One of the most significant issues that RSM addresses is the use of expensive tape libraries for a single application.

"It’s ludicrous to have a $100,000 backup library sitting in a network when only one application can access it at a time," Microsoft’s Joshi says.

To avoid this problem, RSM arbitrates connections to a device, such as a tape library, so that multiple applications can simultaneously access the device. RSM, however, is only a media management layer. Once the connection has been arbitrated, it stays clear of the data path.

"RSM makes tape an integral part of the OS, pulls out media management features and allows multiple apps to share storage," says Brandon Hoff, senior manager of technology and business development at storage vendor Exabyte Corp. (www.exabyte.com). "It is very significant because it puts the management of storage in a centralized location, which is key for SANs."

Hoff points out that centralizing management also has clearly quantifiable financial benefits. According to Exabyte surveys, on average 55 percent of a company’s IT budget goes to storage management. Centralizing storage management can bring that percentage down by 15 percent or more in some cases. Some of Exabyte’s large customers have saved between $500,000 and $1 million per year by centralizing storage management.

The volume management in Windows, also developed by Veritas, is responsible for the creation, deletion, alteration and maintenance of storage volumes in a system. Microsoft’s goal for Windows 2000’s architecture is to improve the manageability and recoverability of volumes in a Windows environment.

"We supported software RAID with NT, and with Windows 2000 we continue to support mirroring, striping and RAID, but we used a new driver to make the layers transparent," Joshi says.

The new driver, which sits below the file system, is more dynamic and requires fewer reboots. All configuration information is stored on each disk, along with a unique identifier. So if a volume fails, administrators can recreate it and fix it offline, without rebooting.

The enhanced volume management also enables administrators to grow Windows NT file system (NTFS) volumes, create and recover NTFS-formatted stripes and mirrors and mount volumes to directories, all without taking the volume offline.

"The catch here is that all of this functionality is only available on dynamic disk," Joshi adds.

Dynamic disks, new to Windows 2000, allow administrators to add space to a server without taking it down and to expand the size of the volume without interrupting end users. Dynamic disks can be expanded while a server is online because they have no partitions or logical drives, and disks and volumes can be moved from one system to another. If a server suffers hardware failure, the disks can be moved to another machine and the data brought online.

Microsoft built disk quotas into Windows 2000 to give administrators a tool to manage storage growth across distributed environments. With the aim of preventing common storage problems, such as running out of disk space, the disk quota capability enables administrators to establish limits that control users’ disk space.

The built-in quota manager calculates limits and thresholds by file ownership on a per-user, per-volume basis. This quota can be implemented globally through a group policy or on an individual basis. When the quota limit is reached, administrators can either elect to make an entry into the event log, or to stop write operations for users violating the quota. Because quota information is in the volume, it is flexible for SANs and Microsoft Cluster Service.

According to Mark Leff, director of business development at NTP Software (www.ntpsoftware.com), a third-party Windows NT/2000 quota management software vendor, the ability to control quotas in a consistent manner is an important aspect of Microsoft’s storage management strategy.

"Disk quotas save administrators the headache of finding the easiest, most efficient manner of managing these resources," he says.

The built-in quota management capabilities in Windows 2000, however, are not up to par with third-party tools. Naj Husien, president of W. Quinn Associates Inc. (www.wquinn.com), a maker of quota management software, says Windows 2000 has no directory quotas, no full-fledged reporting and no SNMP support.

"The [disk] quotas are enough for some tasks, but not for enterprise tasks," he explains.

While the operating system’s storage enhancements won’t revolutionize the way companies manage storage resources, they will influence the way companies planning to implement Windows 2000 purchase storage devices, particularly tape libraries and disk storage.

James Gruener, managing director of Windows 2000 platforms at the consulting agency Aberdeen Group Inc. (www.aberdeen.com), says that as companies plan to take advantage of the storage resources in Windows 2000, the first thing to do is plan for the Microsoft Management Console (MMC) to be active in their architecture.

"Microsoft is adding many enhancements by using a snap-in approach, so the MMC is key because it gives users great flexibility as to how they want to approach storage, among other things," he says.

An equally important technology to plan for is SANs.

"Windows 2000 provides the connectability to use large-scale, multinode, multiple switched-fabric SANs," says Peter Loeb, a product marketing manager at Sequent Computer Systems Inc. (www.sequent.com).

Storage vendors, however, are not planning radical changes to their equipment in preparation for Windows 2000.

"Vendors have been working with Microsoft to enable features that already existed in the hardware to work with Windows 2000 software," says Vicki Vollmar, director of marketing for the storage unit at Hitachi Data Systems (www.hds.com).

For instance, Hitachi’s and some of its competitors’ hardware have had the ability to copy -- or mirror -- data to another storage system for data protection purposes. Microsoft added Intellimirror to the software to take advantage of this capability.

Another example is the ability to share storage systems, which allows customers to move to bigger, mission-critical equipment that previously would not have been practical for NT. As a result, mainframe-class tape libraries could be found in Windows 2000 data centers.

Dan Kusnetzky, operating systems analyst at IDC, says sharing storage systems, particularly in a SAN environment, has another advantage.

"Sharing storage can be used to provide higher data availability, and thus to provide higher application availability," he says. "This is something that mainframes and high-end midrange machines have been doing for years, but that NT has previously lacked."

The sum of W2K’s features and the advantages customers can harvest from them, however, are not complete without the use of some third-party tools.

"We’re not saying that we have all the tools necessary for storage management already built into the operating system," Microsoft’s Joshi says. "We are saying that we have laid the foundation for third-party tools to add that functionality."

Brandon Hoff, senior manager of technology and business development at Exabyte Corp., says to take advantage of the storage enhancements in Windows 2000 users must purchase storage equipment that meet three criteria:

• Manageable via a network

• Certified for use in a SAN

• Fibre Channel-capable

Whether a company is currently planning for a SAN or not, purchasing SAN-capable equipment gives companies the option to use the same hardware if they decide to move from an existing infrastructure to a SAN.