In-Depth

Study Finds Quality Improvement Projects Have Lower Failure Rate

Sometimes the smartest thing an organization can do is pull the plug on an ailing project—regardless of how much time and money it’s invested in it

• By Stephen Swoyer
• 09/13/2005

The relative success or failure of an IT project may depend on the exact meaning of the word “success.”

That’s one upshot of new research from consultancy Cutter Consortium, which found that IT projects premised on quality improvements—such as increased customer satisfaction, product quality, or staff productivity—were much less likely to fail than projects evaluated in terms of other objectives—such as whether they came in on time and under budget.

Cutter’s conclusions are based on a pair of large surveys. In the first, Cutter senior consultant E.M. Bennatan studied software project failures at more than 200 global organizations; in the second, senior consultant Khaled El Emam evaluated 232 completed software projects to determine whether project size had any bearing on project success.

In some respects, the research firm confirms what most IT professionals already know: the vast majority of software projects are neither completed on time nor under budget. Nearly half (44 percent) of all organizations say they have abandoned one or more software projects over the last three years.

Cutter also unearths a surprise or two. For example, not as many projects as you might think are severely delayed (exceeding their original schedule by more than 50 percent)—or prohibitively expensive (exceeding preliminary budgets by more than 50 percent). In fact, in spite of widespread experience with project abandonment, most software projects are eventually completed. Of course, as Cutter’s Bennatan notes, this isn’t always a good thing.

“So is this a serious problem? The results are certainly cause for some concern … but the data so far isn’t cause for major alarm,” Bennatan writes. “One of the reasons, of course, is that the data on cancelled projects is only part of the story.”

The "Better Late Than Never" Canard

Over the last few years, especially, large organizations have focused like a laser on bringing arrant projects to heel. To some extent, Bennatan concedes, these efforts have had an impact. “Some reports indicate that the rate of software project failure is declining,” he writes.

For example, among the 44 percent of organizations that have cancelled or abandoned projects, 34 percent did so with fewer than 10 percent of their projects, while another 7 percent did so with less than 25 percent of their projects. Most organizations are seeing projects through. “[E]ven if this is so, the problem is still quite significant. Most development organizations experience severe overruns, and many (nearly half) have cancelled or abandoned some of their projects,” he comments.

Why is it still a significant problem? For starters, whenever organizations are confronted with severe project delays or budget overruns, they typically opt to stay the course—without substantially revisiting or revising their initial project assumptions, the governance model they’re using to manage and track project progress, or the capabilities of their project development team.

This is a mistake, says Bennatan. “What lesson can we learn form the survey data? A key conclusion is that most software development organizations have let failing projects run their course at a significant cost in terms of time and budget."

The key, Bennatan writes, is to identify when a project is a catastrophe and when it’s not. For example, if a project is more than 50 percent behind schedule; if it’s more than 50 percent over budget; or if it’s plagued by serious quality problems, it’s objectively a catastrophic failure. This doesn’t mean it’s irredeemable, however: Even project catastrophes can still be salvaged, provided organizations take the necessary steps to do so.

These steps can include immediately suspending all project work; assigning an outside, unbiased auditor to assess project requirements, progress, and goals; evaluating what Bennatan calls the “true” project status (i.e., what has been achieved and what has not); evaluating the capabilities of the development team; defining new minimal goals and requirements with senior management and (line-of-business) customers; determining if revised goals can be achieved; rebuilding the project team; and several other steps.

Performing these steps won’t necessarily salvage an intractable project: Depending on just how far off track a project is, it could make more sense to give it a coup-de-grace than a coup-de-main. Organizations will have a better idea about how to proceed, says Bennatan.

Byte-Sized Project Development

In one respect, Cutter’s conclusions appear to validate an “agile” approach to software development, where teams of programmers participate in a rapid, test-driven application development effort that incorporates advanced features and functionality over time, in successive application builds. In a follow-up article, we’ll examine how agile project development methods can help organizations avoid project catastrophes in the first place.