In-Depth

Product Shootout: Intrusion Prevention

Third-party testing firm analyzes leading IDS products, encounters surprises.

• By Mathew Schwartz
• 01/28/2004

The main intrusion detection drawback has always been its rate of false alarms. Cry wolf too often—all early IDS products did, hence one reason for vendors changing the name—and before long, why would a harried security manager take the time to sort out fact from fiction?

What would it take to fix every potential IDS shortcoming? Beyond having precise focus only on real threats, the “must have” features common to any enterprise-scale network security product include ease of set-up and management (companies don’t want to hire another security manager to just administer the box), relief from the need to counteract protections a product would otherwise imbue (because it was too hard to set up); and high throughput so security doesn’t compromise availability.

Good news: Those features are now available, according to laborious new research from The NSS Group. It spent a year developing an IPS-testing methodology, then running each product through 750 different usability, performance, and security tests over a two-week period. It’s the sort of test bed, time, and rigorous analysis end users would love to be able to perform before ever purchasing any product.

NSS is independent. Vendors pay NSS an equal fee to participate in the study, then NSS charges for copies of its report.

According to the nearly 300-page report, a number of IPS products scored extremely well—not necessarily a common phenomenon. In particular, three products received the “NSS Approved” mark of excellence: Internet Security Systems (ISS) Proventia, Network Associates (NA) Intruvert and Entercept, and Top Layer Networks Attack Mitigator IPS. One product rated an outstanding “NSS Gold”: TippingPoint Technologies UnityOne.

Cisco, which submitted its Okena software for testing, later withdrew it.Of all the results, network degradation tests—the point, regardless of maximum capability, beyond which some anti-intrusion features might bog the network down—were especially interesting. True, not all companies need a product able to handle 1 Gbps throughput, but it’s interesting to see where everything really stands. NSS rated ISS Proventia to 200 Mbps throughput, Netscreen-IDP to 500 Mbps throughput, and the NAI, TipppingPoint and Top Layer products to 1 Gbps throughput.

The NSS Group report includes extensive evaluations of each product. For example, of the Top Layer Networks IPS 2400, Bob Walder, director of The NSS Group, says, “as an attack mitigation device [it’s] very impressive, combining almost flawless detection rates at gigabit wire speed with the lowest latency figures we’ve seen under normal traffic conditions.” In addition, he praised its reliability and said it didn’t block legitimate traffic or succumb “to common evasion techniques.”

Of the TippingPoint product, Walder says “performance at all levels of our load tests was impeccable, with 100 percent of all attacks being detected and blocked under all load conditions.” It also lauded the product’s “recommended settings” capability—a wizard for activating filters companies most commonly need.

The report will, of course, help companies seeking proof that IPS works. “When we go into a new customer environment, there are typically two major apprehensions,” says TippingPoint chief technology officer Marc Willebeek-LeMair:

—can the appliance handle the network throughput, or will it compromise network integrity?

—are you going to block legitimate traffic inadvertently?

To quell these fears, security managers now have “the most extensive intrusion detection report done,” he says. Also, “that there’s one product called out as gold speaks to the maturity of the technology.”

Vendors already promise even more for their next-generation products. “We’re anxious to get our new IPS 5500 product into NSS’ next round of testing to demonstrate that we’ve made an excellent product even better,” notes Mike Paquette, Top Layer’s vice president of product management.

That spirit of improvement, plus the at-hand results—qualitative comparisons between major IPS products on the market—should give security managers’ even better IPS options.

The NSS Group study link:http://www.nss.co.uk/ips