In-Depth

Service Level Agreements Too IT-centric, Forrester Report Warns

IT must improve communications with its business users and develop business-centric measures.

• By James E. Powell
• 07/22/2008

Service level agreements (SLAs) are a popular tool for measuring how well IT is providing service to its customers, the organization's business users. A new Forrester Consulting report based on a survey of 389 global technology decision-makers who work with service management finds serious deficiencies with current SLAs. The study sheds light on how SLAs are currently being used, the degree of end-user dissatisfaction, and what IT can do to correct the problem.

At the heart of the matter is the IT-centric nature of the service level metrics, most of which have little or no relevance to business end users.

[T]here is one important question that IT executives should consider in these projects: Whose services are we talking about? Do people -- especially colleagues on the business side -- really care about how well IT performs their IT processes, such as fixing incidents, making configuration changes, and provisioning server environments? Isn't it much more relevant to discuss and measure the performance of the services that the business cares about -- such as processing orders, dealing with customers, and making and shipping products?

The report, Managing IT Services From The Outside In, commissioned by Compuware Corporation, explores the effectiveness of an organization's processes as well as what technologies they use. The survey asked about service management processes, how content for service-level agreements is developed, and how service quality is measured.

Although 81 percent of respondents are involved with formal SLAs, they meet those expectations only three quarters (74 percent) of the time. "The major reason cited for failure to meet the agreed-upon service levels was that 'The business had higher expectations than we could meet.'" Forrester says this is a "clear demonstration of poor business and IT alignment."

The majority of respondents use IT-centric metrics (including network and server availability, number of incidents, and number of failed transactions). Of the 321 respondents using SLAs, 56 percent establish a fresh metrics baseline and 33 percent use the previous year's performance at a guide. The baseline often involves only IT personnel.

Business users understand that these measures may be needed by IT, but Forrester points out that systems and networks are "extremely reliable" and "managing the applications portfolio is becoming increasingly complex with end users, inside and outside the enterprise, [as is] running their apps on different devices and over different networks."

Communication is Key

Communication and coordination between IT and business users are two areas Forrester says IT must examine; 41 percent of the respondents don't even "collect the right service quality data together for their IT and business executives, so they do not even attempt to provide these reports at all," the report notes.

Poor communications may also explain why SLAs may not be "agreements" at all.

They are not negotiated with the business partner because there is no dialogue process and because the business does not understand terms like "percentage availability" or "amount of storage." So when reviewing the IT service at the end of a budget period, business managers can say, "It was not good enough for me to do my business," even if the IT service levels were met on a mathematical basis from the IT point of view.

Forrester reports that the focus of metrics will move from IT to the business as communication increases between the two groups.

The survey found that although 87 percent of the respondents said they use tools to monitor the end-user experience (EUE) for some their business-critical apps, nearly two-thirds (64 percent) acknowledge that they learn about problems only when users contact the help desk.

There's no question about the impact of poor application performance: 57 percent say it increases business costs. Lost revenue and negative impacts to the satisfaction of external customers were each cited by 48 percent. Even so, not all organizations are using SLAs or believe they're needed at all. The study reports that 17 percent of respondents don't have formal SLAs, and half of these acknowledge the need but have not instituted a formal SLA program, and 37 percent of these respondents said "the business hadn't asked for SLAs." Nearly a quarter (24 percent) said SLAs were "overkill."

Two movements are affecting IT, according to Forrester. IT service management (ITSM) "supports the change of an IT organization from a support group into a service group." ITSM includes introducing ITIL or reorganizing IT operations into process groups (instead of tech silos), as well as "the introduction of account management, and even marketing concepts within the IT organization." ITSM is inward-looking and only indicates "how well the IT shop is being run." It does not address the need to know how the business is being run from the perspective of the business unit and the end user.

Business Service Management (BSM) looks at a different set of metrics, including the business transaction volume, transaction speed, and the processing backlog volume.

Report Recommendations

IT operations must establish business-centric SLA metrics, and to do that, IT must "understand their services from the outside in." Forrester recommends IT:

…spend time with business users and understand how they, the users, perceive the service and document what is important for business success or failure. Companies in this phase of service management usually focus on the quality of service around business applications. They need to analyze and document the dependencies between business applications and the details of IT infrastructure in order to better triage incidents or understand true business impact of pending performance or capacity issues.

Communication is key. The report also claims that many SLA reports lack information relevant to executives; 40 percent of survey respondents agreed. Forty-one percent of respondents said they don't provide regular reports to executives.

The report also strongly suggests that leveraging ITIL best practices should be the first methodology chosen because it specifically addresses IT processes. "ITIL is best for documenting and standardizing the IT processes themselves." It notes that quality-improvements programs (including Six Sigma) can help IT improve its processes. Together, the two methodologies provide "a powerful combination for continual IT service improvement."

Failure for IT to change its approach to SLAs may have serious consequences. The study notes that "IT organizations that consider their service management projects only at the IT service level will continue to disappoint their business users in the longer term, and the age-old CIO problem of being equipped to demonstrate value to the business will prevail."