In-Depth

Q&A: Arresting Bugs Earlier in Development Cycle Cuts Security Costs

How integrating security code testing into the development cycle saves time and dollars

• By Mathew Schwartz
• 12/17/2003

So much conversation about software bugs—that nice way of saying “coding errors” that lead to “vulnerabilities”—focuses on mitigating their effect on businesses, networks, and computer systems.

Yet the best way to deal with them is not create them in the first place.

Of course, creating that reality will be difficult. According to the United States Department of Defense and the Software Engineering Institute, on average, every 1,000 lines of code has 5 to 15 defects.

Companies, however, can do even marginally better and see immediate cost savings. For example, an IBM study looked at the cost of building security into an application. Relative to designing in security from the beginning, adding it during development cost 6.5 times as much; during testing 15 times; and after deployment and operations, 100 times. Though the study was conducted 20 years ago, experts say today’s numbers should be even more extreme, owing to such conditions as higher programmer costs and the greater interconnection of applications.

With cost on the line, Gartner analyst John Pescatore recommends companies consider as “standard due diligence” the integration of “security testing into development, QA [quality assurance], and testing.” Beyond cost savings, he notes, catching flaws early mitigates “identity theft, cyber crime, and business system downtime.”

Sounds easy, yet most developers, QA testers, and auditors aren’t security experts. One workaround, then, is giving them automated security testing tools. In general, the number of existing enterprise options is small. Sanctum’s new AppScan QA 4.0 for Mercury TestDirector 8.0, however, adds a QA testing tool to the mix.

Security Strategies spoke with Steve Orin, chief technology officer of Sanctum, and the company’s vice president of marketing, Diane Fraiman, about finding and fixing application vulnerabilities earlier.

Fraiman:

What problem is Sanctum addressing?

Our sole purpose in life is to stop, find, or detect any back-end manipulation of the data that occurs through the browser.

What’s the state of application security testing today?

Initially, the low-hanging fruit were the auditors—the folks living outside the walls such as Deloitte & Touche, PriceWaterhouseCoopers, then also folks inside the walls tasked with doing audits. That's where security testing stayed up until the last 9 to 12 months—at that audit level.

What’s changed with application security testing since then?

In the last couple of years, the application security markets have matured. Also [companies] face greater external pressures—for example the Gramm-Leach-Bliley Act and other regulatory compliance, [the California Breach Notification Act] SB 1386 in California, and the Putnam Bill right now in Congress. So … it's no longer just an issue for IT. Even more specifically, it's absolutely required that security testing be done and reports maintained in case of attack.

So more people worry now about their organization’s security?

It’s caused a whole new group inside the organization—from the general manager to the board of directors—to take a look at security. And they're spending millions of dollars per year on security; this goes down to the bottom line.

What can be done to improve current practices?

The earlier you find and fix these bugs and flaws—these defects—the less it's going to cost you. So we started to look at what's it going to cost to drive security into all areas of development … so you're not waiting to see if your application or site has application vulnerabilities.

What are best practices for integrating security into development?

You need to do security testing at each level of the development lifecycle. But [note that] we’re not trying to turn developers into security people. We do, however, begin to train them in real time as to the cause and effect of their efforts today, so they can make thoughtful [coding] decisions. With our tool, we've done a translation from security speak into developer speak.

Orin:

How does the new QA security testing tool work?

Well, QA doesn’t code the application or generate reports that go up to management levels. They're typically the feedback loop [to developers] … so we plug AppScan into their existing infrastructure and give them scripts [to run]. They get features such as reports, the ability to do a delta check—see what's changed since the last security test—plus do XML testing. AppScan integrates into TestDirector, a Mercury Interactive product, then adds a new test into that environment.

So this is just one more test running in their QA cycle?

Yes, because QA personnel aren't security experts. This is where the program helps test—without it being a whole other project for you to do.

Are companies generally open to integrating such QA security checking?

Historically, security testing has possibly been an IT function, and you're not going to get QA developers to learn a whole new set of tools without a whole new effort. They all want to do this, but they're already overbooked and understaffed.

How does the tool work?

It will crawl the code … then create custom tests, then validate those. Users see an advisory about how the test works, what does it mean—including cross-site scripting, injection attacks, buffer overflow, those kinds of things, but really also input validation, sanitation, state management—and [how to fix it]. Everything from the simple approach, such as “download a patch,” or more complex … if it’s a coding defect.

Can QA see problems in the code others wouldn’t have seen?

You'd be amazed sometimes at how the seams between applications are the problem area. A lot of times it's the integration where things fall apart—it worked great on the developer’s station, but put it together in QA and it falls apart.

What about checking the deltas? Is that to keep tabs on developers?

The typical product manager doesn't really know what cross-site scripting is, but by saying that this can lead to identity theft, they can place the priority on fixing the defects they need fixed. Then developers get information on it in their language as well.

So are companies getting the “fix your security sooner” message?

Quality in the last year has been expanded again to include security as part of quality—the application must be stable, run under a load, and be resilient against attack and misuse. To accomplish that, organizations need to test security earlier in the lifecycle, and the result is it reduces the cost, reduces the risk—not just from hackers, but now will I be the subject of a class action lawsuit, will the government come after me?