In-Depth
Seven Ways to Tackle Unexpected Server I/O Costs
Why the solution to I/O sprawl is I/O virtualization.
By Jon Toor
Why is server I/O shaping up to be the next major revolution in the data center? Two primary reasons: unexpected costs and the promise of a solution. The total spending for server I/O can often total well over $10,000 per server -- sometimes more than the cost of the server itself. At the same time, new technologies, such as virtual I/O, have the promise to slash expenses -- both capital and operational -- and to deliver economic benefit similar to what we’ve achieved with server virtualization.
I/O costs are often difficult to assess because they encompass multiple expense categories and may cross departmental boundaries. When you look at the I/O spend across servers, networking and storage, and factor in all of the elements (such as switch ports, cards, cables, installation costs, and floor space), the total is large and growing. As more servers are virtualized, they require more connectivity to networks and storage: 7 to 16 connections per server is typical. This increases the cost of I/O and may also increase the cost of the server itself. Sometimes, larger and more costly servers, such as 4U devices, are required just to accommodate the ports.
The solution to this I/O sprawl is I/O virtualization. Just as we cured server sprawl with virtualization technology that allowed us to increase server utilization and management efficiency, I/O virtualization lets us consolidate and better manage those resources.
Let’s look at how we got where we are and how emerging technologies such as virtual I/O will help.
Virtualization’s New I/O Challenges
Server I/O has not changed significantly for the past 20 years. Even as servers became vastly more powerful and migrated from departments to data centers, connectivity remained much the same.
Server virtualization, however, changed the game. By significantly altering the server workload, it changed server connectivity demands:
- Increased demand for bandwidth: As virtualization boosts server utilization, it increases bandwidth demands, increasing I/O requirements.
- Pervasive external storage: To achieve virtual machine mobility, external storage became a necessity.
- Increased quantity of physical network connections: Many data centers employ physically separate networks for certain functions. For flexible deployment, servers must be physically connected to more of them – or even all of them.
- New networks deployed: New capabilities, such as virtual machine migration, require new communication paths between devices, creating new networks that must be accommodated.
These new requirements expand costs significantly. One data center manager estimated that connectivity for 60 servers would cost about $600,000 in his shop -- $10,000 per server. That cost includes four 1Gb Ethernet network connections (two physical networks with redundant connections) and two 4Gb Fibre Channel connections per server. This cost includes only server ports and top-of-rack switches, not the core switching.
An often overlooked cost is the server itself. If you’re buying a 4U high server, rather than a 2U or 1U high device, just to get the required number of I/O slots, you’re spending an additional $8,000 (or more) per server. Those large servers consume more rack space, adding facilities costs.
Electricity costs play a role as well. For an I/O intensive server, I/O power and cooling can exceed 300 watts per server, which adds about $800 per server in operating costs over three years.
Big Costs = Big Savings Opportunities
Where there are costs there are opportunities, and here you’ll find both. The total cost of ownership for I/O can quickly exceed $20,000 per server, when you combine the capital and operating expenses. The opportunity lies in the I/O utilization: most I/O ports are used at only a small fraction of their capacity. The trick is to consolidate the physical resources (as we have done with servers), while still providing granular management to accommodate individual requirements. This could both cut the number of assets needed and simplify operations.
Virtual I/O does just this by consolidating the I/O infrastructure, eliminating about 70 percent of the parts and about 50 percent of the capital expense, leading to savings of about $5,000 per server. When you consider that an estimated one million servers will be deployed worldwide with virtualization software, those savings by themselves could total $5 billion per year.
Why Virtual I/O Works in Real-world Data Centers
A solution that consolidates the infrastructure can deliver meaningful savings only if it is usable in today’s data center, not some mythical super-data center of the future. A solution must be non-disruptive from a management standpoint.
Server virtualization succeeded because it let you run on virtual machines the exact same applications and operating systems that you ran on physical machines. The same is true of virtual I/O.
With virtual I/O you still have the familiar storage and network adapters (NICs and HBAs) you have now, but they become software elements rather than hardware. You still have multiple physical ports for connectivity to separate LANs and SANs, but now the adapters (the NICs and HBAs) can be assigned to those ports in software instead of with physical cabling.
Seven Ways to Get Started
As with any transition, the move from physical connectivity to virtual connectivity will happen in stages, so we present seven things to consider now to help you achieve big savings in the near future:
1. Total your own costs: It may take a little research to learn your I/O costs, but you could be a hero for identifying an overlooked opportunity. Look at the costs for servers, switches, cards, and cabling. Don’t forget power, rack space, and the time spent managing the infrastructure. You will know the size of the opportunity when you know the size of the problem.
2. Evaluate your need for flexibility: Are your requirements periodically changing? Are you contemplating a managed services or cloud computing model for your internal IT operations? If so, enhanced flexibility can deliver huge returns. When servers can be assigned new connectivity at a moment’s notice, you are likely to get more from those assets. Plan for unexpected contingencies when assessing connectivity requirements for new servers. Assume any server must access any resource at a moment’s notice. Then you can react more quickly to unexpected changes and better meet new demands.
3.Seek out the bottlenecks: Are your servers I/O constrained? That can lead to reduced asset utilization and unnecessary spending. If you have monitoring tools, such as the ones included with VMware ESX, look at I/O utilization. You may find that I/O is a limiting factor, in which case you have the potential to increase your consolidation ratios.
4. Keep it simple: Since I/O infrastructure can cost as much as the servers themselves, strive for simplicity in your next deployment. A simpler I/O infrastructure costs less to buy and install, and reduces the risk of downtime due to human errors. Next-generation virtual I/O options can help by delivering scalable connectivity with 70 percent less hardware.
5. Look for follow-on savings: One cost-saving decision may lead to another. A simpler I/O infrastructure could help you move to smaller, less-expensive servers. That alone can save you thousands of dollars per server.
6. Innovate, innovate, innovate: Doing things the same way in the future will get you roughly the same cost structure as in the past. Look for the high-cost areas in your infrastructure that might be improved by innovation. Game-changing technologies such as server virtualization, iSCSI storage, and virtual I/O offer more cost-cutting potential than belt-tightening alone. The whole point of virtualization, after all, is to do more with less.
7. Put your ROI in action: Selling your organization on new techniques can take time, but cost-saving ideas only produce benefits when they’re put to work. To help accelerate the process, consider implementing new ideas in a portion of your environment. This lets you refine practices before rolling them out on a larger scale. When you get to that stage, you’ll have real-life data to justify the larger deployment, and you will gain experience that will ensure success.
Summary
Server virtualization redefined the server. I/O virtualization will help do the same for the data center as a whole. The implications are enormous. Start the process by considering your organization goals and your IT environment. Think about your connectivity demands: which areas have high costs and which would benefit from increased agility and performance? Once you have targets, you can consider virtual I/O options in that context to determine the potential benefits relevant to you.
Jon Toor is the vice president of marketing at Xsigo Systems. You can reach the author at jtoor@xsigo.com