In-Depth

Reducing IT Costs in 2000

• 01/01/2000

The CIO is in a meeting with the vice president of operations and the data center manager. Also in the meeting is the capacity planning manager, who has just finished explaining to the executives why the time has come for a mainframe upgrade. The five-month delay in the company’s last upgrade was difficult enough to absorb, and the system clearly doesn’t have the necessary capacity to handle both normal growth and Y2K logic.

The CIO trusts the team. The members have worked well together and responded creatively to find solutions to problems. Nonetheless, the prospect of justifying more than $1 million in spending on hardware and at least twice that on software is disheartening, especially given that the company just missed its earnings estimates for the quarter and Wall Street has pounded its stock price into the ground. Even worse, competitors have begun raiding the IT staff, offering lucrative options packages to top candidates.

The CIO asks the professionals in attendance for alternatives. It’s clear they believe they have none. After all, they have the best systems tools available and a very talented, well-paid IT staff. The systems are tuned to their maximum potential, and if they don’t upgrade the mainframe, the company will be hurt in the marketplace.

The CIO decides to let upper management adjudicate. The options are to upgrade the mainframe or operate at dramatically reduced service levels; they must spend more than $3 million on hardware and software or lose market share. The company’s management supports the upgrade, and life goes on.

As increases in MIPS and storage levels continue to grow at nearly 50 percent per year, hardware upgrades have become commonplace. The good news is that CPU and DASD price/performance improvements are running at about 40 to 45 percent per annum.

The downside is that hardware upgrades continue to be disruptive; they dilute staff time and seriously increase software costs that are tied to installed MIPS levels. Moreover, while hardware costs continue to decline rapidly, at roughly $4,000/MIPS, typical upgrade costs can easily exceed $500,000 in hardware alone.

IT professionals interviewed for this story say that increased use of application performance management (PM) tools cuts IT costs and saves money by reducing pressure on CPU capacity and forestalling hardware upgrades. Automating application PM is also essential to cutting IT costs and making the process manageable. In addition, proactive application PM improves application service levels and system uptime. Such proactive practices lead to better service levels, reduced customer complaints, and increased customer satisfaction.

Attacking performance problems leads to immediate cost reductions by eliminating or deferring the need to upgrade mainframe hardware. Automating the application PM process helps users of products like Programart’s STROBE and BMC’s InTune realize significant extra value by proactively and automatically identifying application job steps that are contributing to performance problems.

We believe that companies using STROBE and InTune can better leverage these products and maximize their value by applying automated management tools to the application PM process.

Mainframe upgrades are a fact of life. Competitive realities, Y2K modifications, falling hardware prices, and advances in operating system technologies have made upgrades even more commonplace.

Unfortunately, products like Programart’s STROBE and BMC’s InTune are widely installed, yet often underutilized. They help put out "performance fires," but they are difficult to deploy proactively. Automating the use of such products can increase CPU headroom inexpensively. Using modern, statistically based tools can move the process from "fire fighting" to "fire prevention."

Automating application PM can provide substantial benefits to IT and can be addressed as a background operational task. The justification for applying increased resources to application PM is related to IT’s contribution to the overall business. Numerous barriers to improved application PM exist, but if they are addressed, substantial cost reduction and improved IT performance are possible.

Mainframe computing remains fundamental to many business processes, such as order entry, credit card verification, stock transactions, and reservation systems. Moreover, an ever-increasing multitude of Web-based and e-commerce applications (that require access to mainframe information) means the performance of these systems can create or destroy customer satisfaction.

At the same time, activities involving regular processing for applications such as accounting, balance reconciliation, inventory replenishment, billing and shipping, place increased pressure on batch windows and have a direct impact on system availability.

With increased customer demand and burdened IT staff, few spare resources are available to accommodate new initiatives. Attention to issues, such as Y2K resolution and e-commerce, take priority and further dilute IT resources.

Each company devotes different levels of effort to application PM. Often, companies manage mainframe headroom issues by using capacity planning techniques to anticipate upgrade needs and tune the I/O subsystem in the best possible way.

They focus tuning efforts on high-resource, consuming applications as time permits. Some companies operate in reduced service modes for extended periods until they can justify a mainframe upgrade.

Ironically, the distraction and costs associated with mainframe upgrades are significant and concomitantly increase software costs that are tied to MIPS levels. Typically, large users experience a 2-to-1 spending ratio between software and hardware. Minimizing installed MIPS levels can often help minimize software costs.

It is common for companies to have products like STROBE or InTune installed; close to 1,500 sites worldwide make use of such offerings. STROBE, an application measurement tool for the MVS and OS/390 environments, repeatedly samples an active program’s status and creates a performance profile identifying the time an application spends in various states (e.g., using a subroutine). Such products are fundamental to PM applications, and most large corporations indicate that these tools generate tremendous amounts of valuable data that help them address performance problems and save money.

Although companies could not identify application performance problems without this information, many companies do not have the time or resources to exploit these products fully, and they are often used under emergency performance management conditions (putting out performance fires).

Most companies indicate the efforts to fully utilize products like STROBE and InTune can be substantial, often requiring meaningful human intervention to sift through reports to identify tuning opportunities. However, by applying automated tools and statistical software to existing application performance measurement software, companies have found they can analyze jobs in the background with limited staff effort and easily identify hard-to-find tuning opportunities without a dedicated, full-time staff. If the performance problems are addressed, a mainframe upgrade can be pushed back, and if the tools are used in a proactive manner, overall IT quality increases.

In addition to the clear cost-cutting advantages, a proactive application PM effort can help avoid degraded service levels until an upgrade is completed, increased customer complaints that require staff attention, and frustrated IT staff who are distracted from more productive tasks.

A logical strategy for IT management involves auto-mating the existing application PM process using modern statistical software tools.

In many cases, the catalyst for a more rigorous application PM effort is customer complaints. Many IT organizations formally track "incidents" and categorize them on a sliding scale from severe —incidents that are so high they trigger a "fire drill" — to minor. The problem is corrected, and it’s back to business as usual until incidents increase again.

An in-depth assessment will almost always uncover performance improvement opportunities, even in companies with more advanced application PM initiatives in place. The key is to automate the process and make it more manageable and predictable.

Notably, the companies that are most advanced with regard to application PM are those that stress the importance of automation through application PM software. They highlight the increased value derived from products like STROBE and InTune and cite additional benefits, such as nondisruptive, automatic identification of problems, dramatic improvement in the number of online transactions and job steps analyzed, and reduction of analysis time for problematic jobs and online transactions. They note that a common framework and terminology improve internal communications needed to resolve application performance problems.

The American economy is well into its sixth year of expansion. From a global perspective, especially when viewed in the context of economic turbulence in Asia, the position of American companies hasn’t looked as strong since the 1960s.

The relationship of IT competitiveness to application PM should not be minimized. According to a recent META Group research study, in today’s $610 billion worldwide IT market, nearly $76 billion is simply wasted, severely underutilized, poorly implemented, or ineffectively deployed – and this figure is predicted to reach more than $90 billion this year.

Our research shows a proactive and automated application PM effort can yield impressive results related to cost cutting and improved IT effectiveness. The spectrum of benefits in companies with and without dedicated application PM professionals on staff closely mirror the importance of IT within an organization. In virtually all the cases we studied, sites that had a proactive application PM program in place also had upper management buy-in to the initiative and a recognized link between IT and corporate competitiveness.Automation plays a crucial role in exploiting IT value.

Every company can benefit from automated application PM. The key decision is how much of a dedicated staff effort to apply to the process. Our research shows that, typically, the more resources a company devotes to application PM, the greater the payback. Nonetheless, we believe the intensity of application PM should generally relate to the importance of IT to a company’s competitiveness. Moreover, by automating the process, companies can achieve effective results with fewer staff resources.

Overall, we found the following benefits of automated application performance management:

• Cost reductions related to deferring hardware upgrades

• Better service levels

• Improved batch performance

• Improved uptime

• Increased customer satisfaction

• Increased job satisfaction among IT professionals.

Notably, significant perceived barriers to using automated application PM tools include a sense of technical overload and lack of resources, a "hardware is cheap" mentality, and skepticism about the need (e.g., "we’ve already tuned our systems"). Automating the application PM process does not have to be overly complex and taxing on IT staff.

Although hardware costs are increasingly smaller components of IT budgets, hardware upgrades are expensive in real dollars, with impacts on software costs, and allocated staff time. We advise rethinking "brute-force" approaches to hardware upgrades and considering automated application PM as a complement.

Finally, although many companies assume they have tuned their systems to the fullest, isolating and addressing performance problems can be like finding a needle in a haystack. In one instance we researched, a large non-U.S. bank had a minor transaction that was consuming minimal CPU time. However, the transaction was executing 400,000 times per day and causing severe performance problems. Automated tools helped uncover the problem, and the company was able to avoid almost two upgrade cycles.

An emphasis on automated application performance management is fundamental to cutting costs and improving IT competitiveness. The politics and organizational nuances of application PM are problematic. To reduce their impact, IT management should focus on saving money by avoiding upgrades as an initial justification, consider the impacts on software cost, identify a champion within the organization to oversee PM effort, and ensure the champion has the authority to implement change.

About the Authors: David Vellante and David Floyer have 45 years of combined experience as consultants and business professionals in the IT industry. They have both held high-level positions at International Data Corporation, as well as other computer industry companies including, for Mr. Floyer, recent work at IBM.