In-Depth
Executives Unhappy with Current Security Metrics
Faced with decreased security spending and executives who decry the state of security reporting, security managers need better report-writing skills.
Of how much value are the security reports you produce? Most executives answer, “Not much.”
Today, many organizations generate security reports manually, and half still deliver security reports on paper, though 56 percent do distribute at least some via e-mail, reports a recent survey. Meanwhile 60 percent of companies use or soon plan to reference trending data—changes over time—in their security reports, to demonstrate effectiveness.
Even so, 25 percent of executives slam the value of the security reports they currently receive. That finding should be cause for alarm for security managers, many of whom already face budget cuts.
No More Compliance Free Lunch
Security spending, of course, has been on the increase in many organizations, “driven by compliance initiatives and the desire to protect customers, patients, and the companies themselves from unnecessary risk,” notes Pamela Casale, chief marketing officer of Intellitactics, which commissioned the survey of 80 senior executives, which was conducted by research firm Frost & Sullivan, based in San Antonio, Texas.
Those same compliance initiatives and budget increases, however, also drive senior executives to pay more attention to security. “Executives want to ensure that their spending is having the required effect,” says Frost & Sullivan network security analyst Robert Ayoub.
For job security, security managers might want to hone their report-generating skills. The survey found 60 percent of companies measure security performance to justify their security spending. Of those, 80 percent say it is effective for demonstrating to other managers, and to executives, the value of information security and IT actions taken.
Method to Measurement
Metrics, once generated, may take on a life of their own. “One of the most interesting things we found,” says Ayoub, “was just the range of different items people were using the information in their dashboards and metrics for. You had plenty of people using it strictly for regulation compliance, then you had people using it to justify spending, then you had people using it as a security benchmark.”
Executives obviously aren’t disinterested parties. “Since 2003, regulatory compliance and increasing negative publicity surrounding internal breaches has expanded the responsibilities of the security team within most organizations,” he says. So it’s no surprise that at 75 percent of companies, the IT security team’s reports—the majority of which are created monthly—travel outside the IT department.
Even so, a quarter of executives are unhappy with the state of the security reports they receive, and that’s largely because they don’t see the kinds of information they want, he says. “To overcome this, security managers should provide information in a context that makes the information relevant and usable for decision-making and evaluation.”
Where Security Metrics Fall Short
Providing usable information first requires relaying the right data. Today, the leading security metrics companies report on (by their prevalence) are number of incidents (almost 60 percent), periodic measures of the risk posture of the enterprise (half of companies), business impact or cost of incidents (over 40 percent), and departmental achievement of security related behaviors (one-third).
Roughly 30 percent of companies also report on how the organization is doing vis-à-vis a “Top 10” vulnerabilities list (such as that maintained by the SANS Institute), as well as the productivity of security personnel and security technologies used. About 25 percent compute the financial losses suffered due to a security incident.
So are security managers relaying the right metrics to superiors? Unfortunately, no: too often security managers don’t translate the raw operational detail they so readily understand into business-speak. As a result, “senior management does not pay attention to security reports because to them the operational metrics just report status,” says Ayoub. For example, take the most typically generated report: the number of security incidents seen in a given period of time. While this report is relatively easy to generate—it often uses a single data source—frequently it means nothing to businesspeople.
By contrast, “reports that deliver high-value information for executive decision-making are less commonly used,” he says, and that may be simply because they’re more difficult to generate. Examples of such reports might include “measuring productivity gains from security investments, or the financial impact of incidents.”
As these examples suggest, the language barriers which often exist between security managers (who talk intrusions, AV, IPS, and IDS) and business managers (who speak ROI, balanced scorecard, and productivity) may be to blame. Indeed, while security managers often focus on how certain technology betters defenses, many executives’ chief concern isn’t what money is ultimately spent on, but rather just knowing return on investment and potential cost savings have been factored into every project.
Capture Sparingly
Security managers must also filter what their superiors see, and even before that, ensure they only collect information someone both cares about and takes responsibility for. (This goes both for metrics the information security department needs internally, as well as the metrics it shares with executives.)
“Each metric must have a defined recipient: someone who cares and will take action if a threshold is exceeded or an exception is identified,” Forrester analyst Khalid Kark notes in a recent report. “For example, if an organization collects and reports the number of spam messages blocked every day and no one in the company is really interested in this information, it should not be collected in the first place.”
Which business metrics rule? “Security managers need to easily position information in the context of risk policy, policy enforcement, and related business imperatives,” notes Intellitactics’ Casale.
Even before that, however, there’s an art to defining metrics. “Creating security metrics is a way to get your point of view across, rather than an objective in itself,” notes Kark. “Rather than trying to present a statistically correct picture right off the bat, make sure that your initial efforts at presenting metrics address areas you already know pose the biggest risks to the business.”
In other words, he says, “don’t let the tail wag the dog.”
Related Articles:
About the Author
Mathew Schwartz is a Contributing Editor for Enterprise Systems and is its Security Strategies column, as well as being a long-time contributor to the company's print publications. Mr. Schwartz is also a security and technology freelance writer.