In-Depth
Putting IPS Claims to the Test
A neutral, third-party testing organization rates IPS performance, accuracy, and reliability.
Need to know how different intrusion prevention systems (IPS) stack up against each other? Look to the new, third edition of the “IPS Group Test” recently released by the NSS Group. Using over 800 different types of tests, products in the report were rated on performance, reliability, security effectiveness, and usability.
According to Bob Walder, head of the neutral NSS Group security testing lab in France, and the report’s author, the results “will give readers a complete perspective of the capabilities, maturity, and suitability for immediate deployment of each of the products tested.” Products from Cisco, Intoto, Juniper Networks, NFR, Radware, Symantec, and Westline are profiled in the report.
Not all IPS products submitted for testing were included. “Standards are very high, and only those appearing in this report have received NSS Approved awards,” notes Walder.
For the previous edition of the report, of the nine vendors who submitted products, four didn’t make the cut. “It was quite worrying to see such a high failure rate during the testing for edition two of the report. It left us wondering if vendors were rushing these products to market too quickly in order to jump on the latest security bandwagon,” says Walder.
This time, however, more products passed, which NSS says hints at a more-mature IPS market. “The quality of the products tested in this latest round is greatly improved,” notes Walder. In total, eight out of twelve products submitted passed and are in the report. They are the Cisco IPS-4255 V5.0(3), Cisco IPS-4240 V5.0(3), Intoto IntruPro V3.0, Juniper Networks IDP 600F V3.1, NFR Sentivist Smart Sensor 100C, Radware DefensePro-3000 V2.43, Symantec SNS 7160 V4.0.0.9, Westline Athena Aegis IPS 510L V2.1.
Beyond being an independent seal of approval, the report is also a useful reality check for vendors’ claimed IPS capabilities, and especially for judging differences between network-based IDS and true in-line IPS products. For example, “one thing to watch out for—don’t let the ‘reactive’ IDS vendors kid you into believing that they have intrusion prevention capabilities just because they can send TCP reset commands or reconfigure a firewall when they detect an attack,” cautions Walder.
When using network-based IDS, speed is of the essence, and reactive approaches may not react quickly enough. “Unless the attacker is operating on a 2400 baud modem,” writes Walder in the report, “the likelihood is that by the time the IDS has detected the offending packet, raised an alert, and transmitted the TCP resets—and especially by the time the two ends of the connection have received the reset packets and acted on them (or the firewall or router has had time to activate new rules to block the remainder of the flow)—the payload of the exploit has long since been delivered … game over!”
Rather, a true IPS sits in-line, Walder argues. “All the packets have to pass through it. Therefore, as soon as a suspicious packet has been detected—and before it is passed to the internal interface and on to the protected network—it can be dropped.” Furthermore, the IPS can start a paper trail: “all subsequent packets that are part of that session can also be dropped with very little additional processing.” To further help, some products can even send “TCP resets or ICMP unreachable messages to the attacking host.”
Given the need for IPS products to sit in-line, it’s worth reporting that actual—and not just claimed—throughput capabilities of many IPS continue to improve. “The IPS tests we conducted in 2004 saw top speeds of 1 to 2 Gbps (gigabits/second), whereas in 2005 we are now seeing devices that can handle significantly more throughput and actually look over-engineered for gigabit environments,” says Walder.
A forthcoming NSS report—testing will begin later this year—will examine multi-gigabit IPS.
Link to free copy of the report:
http://www.nss.co.uk/ips
Related Articles:
Q&A: Sorting Out Desktop Protection Technologies
http://www.esj.com/Security/article.aspx?EditorialsID=1401
Buyer Beware: Putting Intrusion Protection to the Test
http://www.esj.com/Security/article.aspx?EditorialsID=1290
About the Author
Mathew Schwartz is a Contributing Editor for Enterprise Systems and is its Security Strategies column, as well as being a long-time contributor to the company's print publications. Mr. Schwartz is also a security and technology freelance writer.