Study Pegs Maintenance Costs at $1 Million per Application
In the first of a planned annual series of reports about global software quality trends, Cast, Inc. presents an eye-opening figure for all IT managers. The software analysis and measurement company's study estimates that the cost of fixing problems in production applications exceeds $1 million per application.
The analysis focused on "structural quality" -- the engineering soundness of an application's architecture, not whether the application actually met its functional requirements. Cast examined four characteristics -- security, performance, robustness, and changeability -- in a variety of industries and in both the public and private sectors. The firm analyzed source code for violations against the company's collection of industry standard best practices, then calculated a score "using an algorithm that weights the severity of each violation and its relevance to each individual quality characteristic."
The report notes that that "structural quality characteristics are important because they are difficult to detect through standard testing. However, structural quality problems are ... most likely to cause unforeseen operational problems such as outages, performance degradation, breaches by unauthorized users, or data corruption."
The study examined attributes that affect an application's ability to avert unauthorized intrusions, its responsiveness, its stability, and the ease and speed with which it can be changed (without introducing more problems).
COBOL programs achieved the best scores for security, "scoring 59 percent higher than .NET and 37 percent higher than Java EE technologies." Jay Sappidi is in charge of Cast Research Labs; he told me that's because COBOL is used to a great degree in the financial and insurance industries, a sector that is more focused on security. "Those industries have complex requirements, and the cost of failure is high. Furthermore, the applications have been in existence for so long that problems could have already been fixed over time. Finally, COBOL applications are typically not directly exposed externally, so they're obviously safer."
Government IT is spending considerably more on maintenance than private-sector firms as represented by the "changeability" score, which Cast says "is a good predictor of application maintenance costs." Sappidi says that Cast examined code complexity and the complexity of the database, how code is written, including the quality of documentation and how well such structures as loops are documented internally.
"With the government, outsourcing is high, and that equals high risk. Usually in the government sector, multiple vendors are used, and those vendors rotate frequently, which means that they're spending more money to keep the lights on rather than add functionality. In fact, 75 percent of government IT budgets are spent on maintenance; the figure in the private sector is closer to 50 to 60 percent. With only one-quarter of government budgets going into investments in new applications and new functionality, there's a big drain on their budgets."
Contrary to conventional wisdom, application size doesn't correlate to application quality -- as long as the application is (and stays) modular. "When an application is modular, its quality can be high even as it grows to a very large size."
COBOL applications are the exception. When size increases, quality falls. Sappidi says that .NET and Java EE applications have low complex objects (such as loops within loops within loops, and the deeper you go, the more complex the code). "In COBOL applications, 60 percent of components are complex, and the language is less modular."
The trade-off, however, is performance, which "tends to be far better in COBOL than in Java because most test platforms have built-in performance metrics that are run during QA. The down side is that the more modular the code, the greater the overhead."
The study examined 288 in-house-written applications, both onsite and outsourced -- 108 million lines of code in all -- from 75 organizations located mostly in North America, Europe, and India, which may be the largest application sample to be statistically analyzed. "The applications range from 10,000 to 5 million lines of code, with 26% containing less than 50,000 lines of code and 32% falling between 50,000 and 150,000 thousand lines of code," according to Cast. The report says that the average business application contains about 374,000 lines of code.
-- James E. Powell
Editorial Director, ESJ
Posted on 09/28/2010