An Alternative Route
As weembark on a new year, many of my colleagues will dust off the old crystal balland take a look at what the past can tell us about the year to come.Unfortunately I don’t have the tools for wizard-like prediction of the future.Fortunately, in storage you don’t need to be a wizard to predict the future. Ashort trip to any consumer electronics store should be enough.
Disks thesize of Nebraska have appeared at costs so attractive that only the leastmotivated will have less than 50 GB of available storage on their desktops.It’s going to be needed. Last week I installed my first piece of software wherethe setup requirement included disk space greater than a gigabyte! Today, diskstorage is plentiful and cheap.
That’sgreat for consumers and perhaps for your desktops as well, but what about thoseof us involved with enterprise storage and data warehousing? Once again, ashort trip to a consumer electronics store may hold the answer!
With disksgetting larger, faster, and cheaper, the easy answer is that the future oflarge-scale storage management rests squarely on the traditional disk platform.I’m starting to believe that’s wrong. A look at the consumer marketplace showsthat alternative means of supporting storage is becoming more important. Ithink that will soon be important for enterprise storage architects, as well.
The datawarehouses I’ve had the privilege to see recently are growing at exponentialrates. Warehouse architects are being asked to store enterprise data,e-commerce data, data for historical analysis, data for future projections,summary data, detailed data, and data for which there is no known need.
In thecoming year alternative storage -- whose main characteristics are that it ischeaper, less often cached, and accessed sequentially -- is going to play anever-increasing role in enterprise and warehouse storage solutions. One of themain reasons is that, while the cost of high-performance disk storage continuesto decrease, the cost of alternative storage is coming down even faster.
Another keyreason is that alternative storage fits a large number of applicationssurprisingly well.
Datawarehouse applications, as an example, use storage is a very stable way. Datain a warehouse is often stored as a snapshot at a particular time. Changes tothe data often are not reflected as updates to the original snapshot, butentire new snapshots. The previous copy remains undisturbed and potentiallystill useful. This type of application results in very stable storagerequirements.
Very stabledata is perfect for near-online storage environments, such as siloed tapestorage or CD libraries. The enormous capacity -- and cost per megabyte -- ofnear-online storage makes it perfect for data warehousing. But there’s oneother surprising and compelling reason for the increasing importance ofalternative storage: performance.
“Performance?”I can hear you saying, “Is McFadden a few megabytes short in the common sensearena?” In fact, I think that alternative storage is going to enable higherperformance solutions for large-scale system disk solutions.
After all,alternative storage is typically used for data that is used infrequently. Whenenterprise system architects first looked into data warehousing they found thattraditional disks provided all the storage they needed. As warehouses and otherdisk intensive applications grow, they find that the amount of storage used forinformation that is seldom or never used increases. With the volume of dataincreasing, acres of disk space get used for data that is effectively dormant.Dormant data then looks the same to the operating system as the storage usedfor active queries and reporting.
Moving thatseldom used data to alternative storage means that it no longer competes withthe active information on a physical disk.
Making thiswork requires monitoring of the storage environment. Understanding the activitythat is taking place inside the storage system means, in part, determining whatdata belongs in actively used storage and which belongs in alternative storage.In addition to monitoring the data store, system architects are going to needan operating system that is an effective cross-media storage manager. Windows2000 attempts to provide this service, but in practice enterprise storagemanagers turn to third-party solutions to move data between alternative andtraditional media.
Is there anobstacle to the emergence of alternative storage strategies? Clearly, goodsoftware solutions that integrate data utilization monitoring with cross-mediamanagement are needed. It would be great if they were part of upcoming storageimprovements to Windows 2000. In the past, the alternative media choices werealso scarce and inflexible. This is changing, and changing fast. And you don’tneed a wizard’s crystal ball to see what’s coming. --Mark McFadden is a consultant and is communications director for theCommercial Internet eXchange (Washington). Contact him at mcfadden@cix.org.