In-Depth

Seeking the Perfect Patch Process

To avoid the patch-and-pray cycle, IT organizations need to examine how to make vulnerabilities such as Sobig seem so small.

• By Mathew Schwartz
• 09/03/2003

The hard truth about vulnerabilities is that even with prior notice that critical vulnerabilities exist—in effect a warning of impending attacks—organizations must still install the patch on test machines before putting it into production. That takes time, during which worms still have time to attack.

Instead of simply reacting to the latest vulnerability of the moment, experts recommend security managers take a big step back, and work proactively to mitigate ongoing risks.

We discussed the state of vulnerability remediation today, and where its headed, with Stuart McClure, president and chief technology officer of Foundstone, and Steve Solomon, CEO of Citadel. They reveal the current procedures most security managers follow as well as the patch management issues organizations face in the future, and how better addressing them will improve overall network security.

What’s the state of the vulnerability remediation today?

McClure: There's really a whole spectrum of a vulnerability [that indicates] where there’s a problem. First there's an announcement, then a phase where you know you're vulnerable, then you apply remediation steps to try and fix it, then you patch the system. And from threat to remediation is a lifecycle problem. What [you need to do] is to take it all the way from threat to remediation much more quickly.

So it’s bad today?

Solomon: Today, security managers scanning a network is kind of like a deer in the headlights. People just can't believe how bad the network is.

How can an organization improve its approach?

Solomon: [Get] a more proactive approach. Establish more frequency of assessment and remediation. Right now what people don't do is enough assessment or frequent-enough remediation. So be proactive to reduce the security risks at the network level.

Are there any drivers for better scanning and patching?

Solomon: What's really pushing the scanning and remediation are the mandates—Sarbanes-Oxley, Gramm-Leach-Bliley. A lot of that is also pushing change control. And outside accounting firms are often checking in on your network to validate [what you’re doing] as well. Then of course if I have this knowledge and liability, the exposure is even greater, because I can be sued; corporations can be liable. That’s pushing organizations to adopt this more quickly.

Do most large firms already use automated vulnerability scanning tools?

McClure: It depends on the vertical. If you talk about banking or financial services, sure. They have for some time. If you talk pharmaceuticals or biotech, they might have a copy somewhere, but it's certainly not pervasive. It just depends.

We have some customers that use one or two types of technologies, we have other companies that are first-time customers with it. The important point is that folks are starting to see the problem with just scanning. You need to add a workflow and process around that to understand—are you fixing the problems, are you being responsible? So it's about fixing, providing a management structure for fixing the problem, and doing remediation.

Do you have examples of how being more proactive helps?

McClure: The DCOM worm is a good example. That worm’s foundations are built around a vulnerability, and from the moment the threat came out, weeks ago, to the point when there was actual exploit code released [a little while later], to the exploit starting, to the exploit code coming out, to the worm taking advantage of it … Proactive vulnerability remediation would compress that lifecycle—from threat to remediation on down—so you can fix the problem much more quickly.

How does vulnerability remediation integrate into an organization’s security program?

Solomon: Today it’s about more than just having a firewall and intrusion detection systems (IDS). [For example,] a lot of threats come from internal people—70 percent of vulnerabilities, by one account, happen inside the firewall. [Say] people make bad changes in servers, or worse. So they’re coming from the inside. Part of the functionality of a vulnerability scanner is identifying vulnerabilities, and then you have the opportunity to go ahead and remediate them as well. By contrast, today there’s too often a broken process around vulnerability remediation—it’s not quickly followed through from discovery to remediation.

What are organizations spending on patch management today?

Solomon: One Fortune 100 company, we were told, spent $30 million remediating some patches on their network for this last round of major vulnerabilities. That’s an amazing cost. But the thing we have to be so aware of is not just patching, but true vulnerabilities; patches only account for 20 to 30 percent of vulnerabilities. So you really need to focus on identifying everything that's in your scan of a high or medium threat level, things that require you fix them. You need to make sure you remediate and validate off of that scan. Foundstone is identifying it, then we're going out and fixing it.

What are Foundstone and Citadel doing together?

McClure: We've put a technology partnership together—a way to find, patch, and have an integrated approach to network security. What Foundstone is all about is providing an environment for finding, then fixing, the problem, and where Citadel comes in is actually putting the patches on.

But isn’t patch management just inherently difficult?

McClure: A lot of our customers are very large companies, and they can't exactly patch every moment of every day. Maybe they're on a weekly or monthly schedule, so you get caught, you might ask why can't you patch quicker than that, and the response is so often, 'We have too many systems and we don’t have time to patch everything.' That's where Citadel and Foundstone comes in, because we can provide you with priorities for patches, then with Citadel, we can provide a complete system for applying patches and fixing misconfigurations.

How rampant are misconfigurations versus other vulnerabilities?

McClure: CERT says 99 percent of attacks are attacks of known vulnerabilities. And from my experience, it's certainly a 60/40 or a 50/50 thing, misconfigurations versus other vulnerabilities.

Who in an organization tends to lead the vulnerability remediation charge?

Solomon: The CSO level. But it’s very interesting—today there are organizations that have separate assessment groups and remediation groups. That’s inherently a broken process. So today you typically have assessment and remediation teams that report up to the CSO, who then reports to the CFO, who’s the chief compliance officer. This comes back under the mandates of mandatory compliance.

So separate assessment and remediation teams don’t communicate effectively with each other?

It’s more that organizations have no validation or correlation of changes, so they can’t ensure a vulnerability has been fixed.

What can the average organization do better?

McClure: Just think about the complete lifecycle approach. If you think about what security is, it's what threats are present, how do you fix them, and when are they fixed? That lifecycle is not really being fixed. You have a lot of firewalls, IDS, but that doesn't prevent worms. Unless you think of security as a process, at the end of the day you're never going to impact risk. So the combination of vulnerability scanning and automated remediation really provides a crisp, clear vehicle for addressing the lifecycle of vulnerabilities.