In-Depth

Best Practices Report Tackles Metrics, KPIs

New TDWI report focuses on the importance of best practices for creating KPIs.

• By James E. Powell
• 01/07/2009

It’s difficult to manage what you don’t measure, and organizations still have a long way to go when it comes to developing key performance indicators (KPIs). The latest TDWI Best Practices Report, Performance Management Strategies: How to Create and Deploy Effective Metrics, notes that only one-third of survey respondents said they have either partially or fully deployed an initiative to develop KPIs. The report’s author, director of TDWI Research Wayne Eckerson, concludes that creating effective KPIs as part of a performance management initiative is still a new concept to many.

Based on a 25-question survey with 678 respondents in September 2008, as well as in-depth interviews with a dozen performance management practitioners and vendors, the report notes the power of performance metrics. "Executives can use such metrics to define and communicate strategic objectives tailored to every individual and role in the organization," Eckerson writes, noting a four-step cycle for creating strategies, building plans (including budgets and targets), monitoring plan execution, and adjusting activity to achieve strategic goals.

"In essence," Eckerson writes, "performance metrics distill an organization’s strategy to serve its stakeholders, linking strategy to processes. No wonder most organizations struggle to define performance metrics.”

The report explores the four maturity levels of an enterprise’s performance management capabilities and vision, and distinguishes between ordinary metrics and strategically-aligned metrics (KPIs). Dashboards -- a popular BI tool -- aren’t easy to develop, and it is difficult for developers to display the proper performance ranges as graphics. "Many organizations only display a handful of KPIs in an executive scorecard. As a result, those KPIs may be averages or aggregates based on multiple lower-level KPIs," Eckerson points out.

Even after they’re developed, KPIs should be straightforward to readers and easily understood, but that’s not always the case. Some employees don’t know how to read KPI displays or how to interpret KPIs. Furthermore, sets of KPIs can produce mixed signals, as in the case of a dashboard that displays a measure in red (indicating below-average performance) but moving upward (indicated in green).

To overcome such limitations, Eckerson offers an overview of seven attributes a good performance dashboard should possess, and explores the many types of KPIs and performance dashboards. Although much of BI looks at past performance, the report explains how organizations struggle to define accurate drivers for future performance.

Survey respondents note how business-unit employees have more KPIs to choose from (21.6 on average), as do departments and workgroups (24 on average). Executives have about 16 KPIs at their disposal. The disparity doesn’t surprise Eckerson: a "'hodge-podge' of dashboards may confuse and frustrate a chief performance officer, [but] it doesn’t faze the average mid-level manager who often uses multiple performance dashboards to manage various facets of their jobs."

TDWI”s report finds solid benefits to KPIs, though most organizations still need to improve. The survey shows that one-third (31 percent) of survey respondents with fully or partially deployed KPI initiatives say their KPIs have “changed behavior and improved performance” to a very high or high degree.” Almost half (47 percent) report that the impact of their KPIs has been moderate. The report explores in depth ten characteristics of high-impact KPIs.

Among the other survey results, when developing KPIs, over half of respondents (55 percent) review and prioritize metrics in existing reports, and most use interviews and joint-design sessions (51.3 percent). Senior executives (74.8 percent) and managers (55.6 percent) within an organization are responsible for defining KPIs, but implementation falls to IT departments (47.8 percent) and managers (47.4 percent).

- - -

Editor’s note: TDWI is hosting a Webinar on January 14 focused on the detailed results of the survey. For more information, visit http://www.tdwi.org/display.aspx?id=9114.