Complexity Begets Complexity: Development Information Systems – A New Paradigm for Software Development
New computing capabilities mean more complex software is needed. More complex software leads to a more complicated development process. Development Information Systems (DIS) are a new class of software development applications -- modeled after approaches already used in manufacturing, human resources and finance. DIS are designed to provide all software developers and their managers with universal access to one, inclusive source of information.
The emergence of new computing capabilities, including languages, processing capabilities and communications increases software complexity. As a result, software becomes increasingly more difficult to preserve and maintenance costs spiral out of control. What was once well-structured, intuitive code now assumes the consistency and organizational characteristics of spaghetti: tangled, slippery and hard to get your hands around. According to Dan Stang, Research Analyst at Datapro, as much as 75 percent to 85 percent of software development is spent on re-engineering. As a result, new product development is left with minimal time and budget resources while its value and growth stagnate. To compound an organization’s software problems, development’s delivery schedule is often unpredictable and at the mercy of individual developers. Developers tend to work in seclusion, and are often forced to fend for themselves as opposed to acting as integrated team members. In addition, code is written by one, and rewritten by another; defects are fixed, only to have a new problems develop in unrelated areas; and development cycles are extended, increasing test cycles – all while scarce resources are wasted. If objections are raised, developers merely move on to the next position or project in the next company. Management has little or no control, and many processes are manual in nature. The entire process is out of control. Enter a new paradigm – a new class of software development applications – the Development Information System (DIS). Modeled after other successful information systems, from manufacturing (ERP and MRP) to human resources (HRIS) and finance, DIS are designed to provide all software developers and their managers with universal access to a one inclusive source of information. For years, executives have been investing in information systems for other areas of their business, often paying hundreds of thousands, even millions of dollars, to secure the management of their key assets. The time has come for software development to take advantage of the same theory and practice that it has created for its corporate siblings.
The Information Model – Controlling the Asset
At the heart of any information system is a central database that stores code entities, as well as capturing and recording their inter-relationships. This "Information Model" provides management with the fundamental capability to secure, catalog and codify its assets, in DIS’s case, the asset is software source code. Consistent with established software development processes (SEI, ISO9000, and Six Sigma), the Development Information System’s initial step is to gain control of the software asset. In order for a DIS to be effective, all elements of the software must be registered in a database, and the relationships between these elements must be captured. Source code, specifications, tests, documentation – literally everything associated with the software – must be included. If an element is excluded, the potential effectiveness of the system is jeopardized. Once the database is built, every information systems essentially performs three basic functions:1. The ability to query or navigate the data.2. The ability to automate change.3. The ability to generate reports, and establish control of assets. These three functions are the hallmark of "integrated" applications. In the case of ERP these functions are the bill of material, purchasing and master scheduling. Each application works off of the central database in a coordinated manner, and is able to take advantage of information that once was only privy to its creators and immediate users. Much like ERP, DIS uses common data and tailored applications to integrate disparate pieces and groups of information. The DIS uses this information to query source code, perform analysis of existing code and proposed changes, perform code reviews, optimize tests, automate documentation, component mining, packaging and generate reports. To continue the analogy, as technology advances, ERP’s information systems have become specialized for specific industry segments, like process or discreet manufacturing. In software development’s instance, customization includes different language capabilities, and the ability to perform specific projects, portability analysis and globalization.
Query – Comprehending the Asset
One of the most valuable functions a DIS provides is the ability to efficiently and accurately query its information model. Development Information Systems understand the relationships between individual parts of source code, as well as the location of related information, which facilitates its sophisticated query abilities. In addition, the query function of the DIS can provide detailed and "consumable" data both in textual and graphical formats. These familiar, descriptive reports represent the relationships between source code and documentation, tests, specifications and even software artifacts like defects. In conventional software systems, the process of identifying a portion of the code and its related support systems can be a challenge, even for the most experienced developers. For those developers who are new to the team, the task is essentially impossible. The DIS’s ability to perform queries helps bring new employees up-to-speed quickly, and allows veterans to efficiently navigate the system. In addition to its query functionality, the DIS’s heightened level of information access allows analysis to be performed, which extends the depth and value of source code comprehension. For example, the potential impact of a code change on software can be explored prior to the change, thus eliminating the "fix 1, break 2," syndrome of bug fixing. Also, dormant code and complex header structures can be identified in an effort to unwind the code’s complexity.
Change Automation – Improving the Asset
Once a Development Information System establishes detailed source code comprehension, initiating change can be a less daunting and more manageable process. The computer can automate code changes for en masse resolution, or more fastidious piecemeal operations, while providing the developer with an understanding of all impacts and necessary resolutions. A DIS can extract dormant code, simplify complex header files, and rectify problems, because it has the intelligence to recognize the relationships between the change and its context. The DIS’s information model automatically identifies the portions of the code structures affected by changes, and suggests the tests that need to be run. This insight significantly reduces test cycles, and can easily update software documentation. All applications utilize the speed and accuracy of the computer, while benefiting from an intrinsic understanding of key relationships captured by the DIS database.
Reporting and Control – Managing the Asset
The ability to generate reports represents management’s degree of asset control and accountability. In the past, software R&D managers did not have the ability to quantify or assess the status of their efforts. They were the only managers within the organization who lacked these important productivity assessors, and as a result, they were the only department that could not accurately predict or control business-affecting deliverables. Generating reports for productivity and status assessment within the DIS is simply a matter of documenting specific queries or analysis. Data on any quantifiable attribute of the information model like its total number of lines of code, or "Meyers complexity," are readily available. The DIS is also able to provide complex reports, such as the quantification of internal code quality as measured against company programming standards. These reports provide management with a method to track code quality prior to testing, enhancing their ability to provide accurate delivery timeframes. In addition, the DIS can process reports on the number of defects in selected subsections of code, which management can use to expedite changes. For example, defect reports help management know whether or not it is more efficient to rewrite or fix the offending body.
DIS vs. Standalone Tools
Today, vendors offer a variety of tools that address the individual and specific needs of developers. These tools are reasonably priced, and execute their specific tasks within their specific bounds. However, they are "Islands of Automation," similar to the islands that existed prior to the evolution of MRP. They are functional, remedial Band-Aids that provide temporary solutions to symptoms of greater problems. Within the big picture of software development, these tools lack the intelligence and the ability to understand relationships between disparate elements. And, while they do resolve their specific tasks and provide the user with valuable, albeit temporary resolutions, they operate within their own environment, in blissful ignorance. Some software development tool providers have begun to offer integrated families of tools, but these suites of solutions lack the central data model and are therefore "mute," disparate tools that are unable to communicate. Essentially, they are individual tools with a common user interface. User friendly? Yes. Truly integrated? No. Testing software executes without considering the impact of a defect, or its solution, on the entire system. Change Management systems offer check-in and check-out capabilities. Sometimes they even provide the ability to track these changes. However, they do not understand anything about the source code itself. CM systems treat source code as if it were a simple file with no history or impact on other parts of the code base other than the information supplied by the operator. It fails to consider implications and relationships. Project management systems track tasks to be performed and allow for planning, but they operate without the ability to consider what really happens to the source code during the project. Indeed, these systems can be valuable to project management, but without the ability to incorporate daily changes to source code, they are out of touch with their own projects. Without a central database, today’s applications function at a fraction of their potential. The very definition of the term "integrated" implies a central data repository, without which, "interfaced" is a more applicable term. With a central "Information Model," applications can truly be integrated, allowing access to all knowledge, when it is needed, by anyone who needs it. In fact, the "Information Model" can serve as the point of integration for all software development applications, including configuration management and project management, as well as specification generation, testing and development tools.Yes, it is possible to improve individual productivity rates by 10 percent to 20 percent through the use of query and mass change solutions. But, managers face more strategic problems than simply developers’ productivity. Tools are designed to help developers; Development Information Systems are design to help management. Developers write lines of code, they fix bugs and write documentation. Performing these tasks more efficiently is beneficial, but it doesn’t bridge the current islands of automation; it is not a strategic solution. In addition to resolving the symptoms of software development’s problems, DIS frees the developers to move beyond redundant efforts.
A Strategic Enterprise Solution
The implementation of CAD/CAM exemplifies a case where the benefits of a system are realized differently by users and by management. In the early days of CAD/CAM implementation (the information system for mechanical and electrical designers) user resistance was high. They saw their beautiful drawings replaced by sterile, but consistent, mechanically produced images. They did not understand, or didn’t care to appreciate, the value in the new system. Beyond the aesthetics of the CAD/CAM images, the CAD/CAM systems captured design intent while assuring fit and finish throughout the design and manufacturing process. From management’s point of view, the fact that the small effort invested up front assured downstream viability was enough to make CAD/CAM the new standard for design. CAD implementation was not a popular concept with already overworked design staffs, but they weren’t the ultimate decision-makers. Eventually, organizations recognized CAD’s value as a business tool. CAD delivered incredible payback to manufacturing, by improving both quality and organizational productivity. ERP installations illustrate a similar situation. Prior to ERP systems, purchasing agents hoarded and capitalized on entire file drawers of individualized supplier information. The introduction of ERP required that they put this information into the system for all to use. While they fought tooth and nail to maintain their realm of power, ultimately the purchasing agents lost their stronghold on individual information. They resentfully sacrificed their sense of empowerment to benefit organizational productivity. Companies viewed this as a small change in procedure, yet it was a significant change of process to the individual. Regardless of which perspective one takes, it resulted in the greater good of the company. The introduction of software Development Information Systems faces similar employee resistance. From an organization’s business perspective, there is enormous benefit to guaranteeing universal access to software information; the developer’s pain is small and temporary. To the developer, the change appears too significant and uncomfortable; this is a knee-jerk reaction to an unfamiliar concept. DIS prevents duplicated development; it negates the corruption of new functions by the resolution of prior errors; it supplants manual code reviews and ineffective tests and all resulting errors. But, perhaps DIS’s greatest benefit is derived from its impact on software quality. Can anyone place a price on lowered defect rates and solutions to previously unsolvable issues? The reduction of maintenance costs alone can be significant, and more importantly, can allow for the shift of resources from maintenance tasks to new product development. If even 10 percent of the current time spent on maintenance could be shifted to new product development, the impact on time-to-market and new feature creation would be tremendous. The concept of information systems is not new. However, harnessing the power of information systems technology for software development is revolutionary, and promises to turn the industry on it ear. The very nature of software Development Information Systems – the connectivity it provides to developers and the accountability it provides to management – will help information system executives realize the power and value in both the development process and the software systems themselves. DIS is a new paradigm for software development. And, frankly, it’s about time that the software development industry enjoys the same capabilities, and reaps the same rewards, that it has provided to its customers for decades.
Bruce Boes is the Vice President of Marketing at UPSPRING Software Inc. in Burlington, Mass.