In-Depth

Bias-Free Security Testing

New security-risk management tools bridge the security/business gap

• By Mathew Schwartz
• 11/03/2004

Why protect $500 worth of data with a $5,000 firewall?

Security risk management means surrounding the highest-value assets with the best security, and according less protection to less-valuable assets. With unlimited resources, security spending wouldn’t matter. This being the real world, however, security experts recommend better protecting what attackers actually want to steal.

One way to assess the effectiveness of a risk management program is by using the Open Source Security Testing Methodology Manual (OSSTMM), released by the Institute for Security and Open Methodologies (ISECOM). In a nutshell, the OSSTMM is the only open-source tool available for bias-free security testing. Its users range from U.S. government agencies to large enterprises, including the Volkswagen Group’s Spanish IT subsidiary, Gedas Iberia.

Conducting an OSSTMM test “takes four to eight hours and you do some security measurements, and you get some answers, and the answers are factual—the machine responded or didn’t respond, the port was open or not. There’s no risk assessment, because there’s no opinion to it,” says Pete Herzog, managing director of ISECOM.

Instead of just making a list of which security tools are in place, the OSSTMM requires auditors to test security tool effectiveness. “We don’t care if you have a firewall. What we care is what’s accessible,” says Herzog. So the test first measures operational security. “If you’re doing business, you have to have certain things open, such as Web ports,” notes Herzog. Auditors simply count every potential risk vector, from accessible databases to Web applications.

Then there’s testing of actual security, which takes into consideration loss controls. “For example, a Web port could be open, but you have authentication or encryption. [Well] they aren’t actually secured, but there are loss controls, so data can’t be stolen or modified along the way.” In other words, someone might steal a database, but if it’s encrypted, “loss control says it would cost you more money and resources than the value of what you stole.”

The end result is a bias-free assessment of an organization’s security, as well as guidance on how to adjust it. “In the end, you not only have accurate benchmarks, but you can also verify if the percentage of what you spend on new security measures actually can be justified by increasing security or loss controls to the right assets at the right cost,” he says. The OSSTMM test is also a snapshot of an organization’s security, useful for measuring future progress.

While version three of the OSSTMM, which refines the testing process, is due for release shortly, with ISECOM’s blessing, a company called CIOview has already implemented it into a recently released tool for conducting security audits called SecurityNOW.

Questions, Then Monetary Answers

CIOview’s approach to creating its security tool mirrors that used for its other total cost of ownership (TCO) calculators. “A lot of us came out of the world of managed consulting,” says Dave Lauer, a senior analyst at CIOview. In a typical engagement, he notes, most of a consultant’s time might be spent on “doing the rote piece—the data collection piece,” which can quickly get boring. Instead, why not create a database of security products, price points, and what they do, so auditors could more quickly report results and make recommendations? “Our idea was to build a structured interview process that would guide an IT executive.”

The TCO tools employ a wizard that asks specific questions and uses your answers to make specific product- and dollar-related answers. “What we found was, as long as you keep the questions very focused on a particular technology area, you can do this very effectively,” says CIOview president Scott McCready. “So you could ask people a series of questions, concerning an ERP system, and very quickly tell them how much it’s going to cost them to deploy on Windows versus Unix.” The various calculators CIOview makes cover such things as servers, desktops, and storage, and include the questions and logic necessary to arrive at recommendations.

With SecurityNOW, “we’ve taken the same concept and applied it to security with the idea that we can very quickly determine what your specific risk is, and, importantly, what are the financial applications of that risk, and as a result, what security technology should you invest in to reduce that financial risk?” says McCready. While managers can use it, security auditors can also use the tool to present their findings.

Using the tool, people can assess not just what security they need, but how much it’s going to cost. The tool also generates business reports, including details on tool recommendations, and increases auditing speed. Via beta testers, “what we’ve found is we can cut the time for an audit from 30 days to 3 days,” says McCready. He hopes this will also help make security auditing a more-frequent activity. “What we’re trying to tell people is security is a daily affair. It degrades on a daily basis, and just running it on a quarterly basis doesn’t really tell you where you are.”

One early SecurityNOW user is Martin Dion, chief technology officer of Above Security, a Montreal-area company specializing in security management for networks and computer applications. He tested the product before its public release, and is also familiar with the OSSTMM. “OSSTMM is really how to diagnose the technical problems of the infrastructure,” he notes. As part of its consulting services, that’s just what Above Security does, and Dion relies upon the OSSTMM as his auditing methodology. “The advantage I think of OSSTMM is to bring something simple that anyone who’s computer literate can understand, and talk to managers who are less computer literate.”

After putting SecurityNOW to the test, Dion notes its “matrix” of security products is “quite extensive,” and includes dollar figures and perspective on what everything from a firewall to an authentication mechanism should do, expressed via security-risk-mitigation values.

Just having such a framework is a big start, he says. “I might not agree with all the values, but at least it’s something I can work with.” Too many methodologies today—especially from large consulting firms—are take it or leave it, he notes. “It’s a big problem in information security, from a methodological or process perspective,” since it makes discussing methodology or how results were arrived at difficult.

By contrast, this gives him an approach he can fine tune, plus a tool he can use to communicate his thinking, and results, to clients, not all of whom are necessarily information-security literate. CIOview includes a free, though less-featured, version of SecurityNOW precisely so auditors can share information with clients and allow them to see how adjusting security investments will alter their organization’s security posture. Sometimes the most-expensive firewall isn’t the optimal approach.

While Dion doesn’t think the tool will replace a full-scale security analysis conducted by an experienced auditor, he says it’s useful for smaller projects, and helps communicate the impact of security decisions to business people. “If you want a quick, simple, efficient tool to give more value to something that is more technical, SecurityNOW is great, because it will take the results, and put … business requirements into perspective … and [detail] how much those business investments will give you back.”

Related Article:

Q&A: Using Business Rules to Tackle Vulnerabilities
http://www.esj.com/Security/article.aspx?EditorialsID=1163