In-Depth
Closing the Loop on SLA Shortcomings
By now, IT and its line-of-business customers should have perfected the art of service level management. The opposite is the case, however.
Service level agreements (SLA) are nothing new. IT has worked with them for several decades in one form or another, so you'd think IT and its line-of-business customers would have perfected the art of service level management by now.
According to a new study, however, they haven't. IT only meets its SLA obligations about three-fourths of the time.
There's plenty of blame to go around, says market-watcher Forrester Research, which conducted its service level management (SLM) study at the request of SLM and application performance specialist Compuware Corp. Forrester stresses that IT itself isn't strictly to blame. If blame of any kind is to be assigned, it should be meted out to both IT and the line of business, with absent executives -- who either aren't seeing the whole picture or who are sometimes left out of the loop entirely -- included for good measure.
SLM Is Not an IT-Centric Activity
One big problem, Forrester concludes, is that the preponderance of SLAs are defined in IT-centric terms. This isn't necessarily the fault of IT because the line of business has itself acceded to (and in some cases encouraged) such arrangements because it didn't have any choice or because it simply didn't know any better.
In addition, IT and the line of business aren't out front on enforcement, either. According to Forrester, 41 percent of respondents don't provide SLA information to their executives on a regular basis, while another 40 percent don't give their executives all of the requested information.
If organizations are going to close the loop on errant SLAs, Forrester argues, they're going to need to rethink how they identify, create, enforce, and manage such accords. That means doing away with IT-centric metrics such as network and server availability (or incident reports) which simply help perpetuate an unsustainable status quo, according to Forrester.
It isn't as if IT has always been mismanaging service levels, Forrester concedes. It's that the nature of service level management itself has changed -- drastically -- over the last decade or so.
"Instead of being responsible for monitoring hardware and networks and then fixing problems when they occur, the emphasis is now on managing the applications," says the Forrester report, Managing IT Services from the Outside In. "While systems and even networks are now extremely reliable, managing the applications portfolio is becoming increasingly complex, with end users, inside and outside the enterprise, running their apps on different devices and over different networks."
The Real-World Costs of SLA Shortcomings
There's a further wrinkle, according to Forrester. The art of SLM has changed on the IT side, and the expectations of business users have also evolved. Ever since the dot-com boom went bust -- and companies stopped pumping money into IT -- the line of business has had the upper hand, demanding a business-centric accounting from its information technology enablers.
One upshot is that line-of-business customers are less willing to overlook service-level shortcomings. "Business users' expectations of IT have also matured over time. Users understand the details much better -- but also expect IT to measure and report service quality using their business-oriented metrics," Forrester says. "They see IT as simply enabling business processes and will increasingly measure IT on how well it directly supports the business process using their terms."
What do SLA shortcomings cost? Does a 25 percent failure rate -- as reported by Forrester -- translate into a mostly benign real-world effect, or does IT's inability to meet its service-level obligations have more pernicious results? Not surprisingly, Forrester indicates, the latter scenario is closer to the truth. "More than half of survey respondents [52 percent] recognize that poor application performance negatively impacts customer service, and 48 percent say it results in lost revenue," the report indicates.
That's the intangible effect. In real-world dollars and cents, the results are more sobering. "In addition, 40 percent of the respondents reported that poor application performance has a 'severe financial impact' on their company, with 42 percent saying an hour of downtime costs the business at least $100,000," according to Forrester. Fully 10 percent of respondents say SLA shortcomings result in losses of $1 million or more.
When is an SLA Not an SLA?
When IT does meet its SLA obligations, line-of-business customers aren't always happy, Forrester points out. The culprit, again, is misalignment -- the chasm between what business users expect and what IT understands that it must deliver.
"When asked about the reasons for missing the agreed service level, the most-selected answer was 'The business had higher expectations than we could meet' -- a clear demonstration of poor business and IT alignment," Forrester reports.
The upshot, the market-watcher argues, is that many SLAs aren't actually service level agreements because IT and the line of business don't collaborate in any meaningful way to define service level parameters.
"[T]hese SLAs are often not true 'agreements.' They are not negotiated with the business partner because there is no dialogue process and because the business does not understand terms like 'percentage availability' or 'amount of storage.' So when reviewing the IT service at the end of a budget period, business managers can say, 'It was not good enough for me to do my business,' even if the IT service levels were met on a mathematical basis from the IT point of view."
In Praise of an Outside-in Perspective
Forrester's solution isn't surprising: IT must shift from a techno-centric to a business-centric understanding of the services it provides. Forrester says this amounts to understanding services "from the outside in."
"[IT] must spend time with the business users and understand how they, the users, perceive the service and document what is important for business success or failure," Forrester counsels. "Companies in this phase of service management usually focus on the quality of service around business applications. They need to analyze and document the dependencies between business applications and the details of IT infrastructure in order to better triage incidents or understand true business impact of pending performance or capacity issues."
To that end, Forrester champions what it calls "end user experience" (EUE) monitoring tools, of which Compuware -- the company that sponsored the study -- is one producer. Nevertheless, Forrester indicates, EUE tools help facilitate the outside-in perspective IT must develop if it's to adequately respond to the needs of business users.
"Most incidents submitted to a help desk are actually reported by end users. That may seem to be obvious and by design -- why else would one have a help desk? -- but it is actually a very bad indicator of service quality. It reflects a service management maturity that is purely reactive and often leads to a crisis in customer satisfaction," Forrester points out.
One problem is that the help desk itself can't always identify the source of the problem, in part because of buck-passing between and among groups of IT stakeholders -- e.g., server managers, network managers, database managers, and application managers. EUE tools -- which let IT experience application performance issues from the perspective of client end users -- can help mitigate this scenario.
"The ultimate judge of IT/business alignment and business service performance is the end user: If alignment is viewed as conformity to user expectations in terms of availability, performance, usability, and accuracy, then monitoring end user performance is the only way for IT to know that it is meeting these expectations," the Forrester report indicates.