In-Depth
Beyond Logs: Creating a Log-Management Program (Part 1 of 2)
Regulations are driving companies to audit their security logs. To help collect and analyze all that data, companies can turn to free syslog software and off-the-shelf security event management software. Which approach is right for you?
Does your organization spend time studying logs of system and network events?
Much of enterprise security today revolves around log analysis, which can reveal innumerable things, such as security incidents, incorrect device settings, policy violations, and suspicious activity. Computer forensics relies on logs, as does auditing software, and auditors increasingly look to log data to verify compliance with regulations, including the Gramm-Leach-Bliley Act, HIPAA, and Sarbanes-Oxley.
Yet while mining logs for events can identify or stop security or compliance breaches, given all the log data a typical enterprise generates daily, it’s increasingly difficult to manage it. Blame companies’ improved security posture, since practically every piece of security software or hardware—antivirus, personal firewalls, network firewalls, intrusion detection and prevention systems, anti-spyware, anti-spam, and many network devices and PC or server operating systems—generates relevant log data.
To better cope with logs, companies have three options: homegrown analysis tools, free utilities, or off-the-shelf security event management (SEM) software that automatically filters, correlates, and analyzes events. The latter is related to security information management (SIM) software, and both are sometimes grouped as security information and event management (SIEM) software. Gartner Group estimates 70 percent of companies that adopt SIEM software need both the SEM and SIM capabilities.
Approaches to Log Management
Determining which of these approaches is right for your organization depends upon what’s currently in place. According to Andrew Lark, chief marketing officer of LogLogic, “Our biggest competitor today is the homegrown solution, where they’ve got engineers sitting around, writing scripts to search on log files, and they’ve got pretty costly manual processes in place.” Large organizations can save time, and perhaps money, by adopting more automated log-management software.
Log management is not just a technology problem. Companies require security policies and IT procedures that detail proper log-management procedures to employees, including who’s responsible for what. While it should go without saying, someone in IT will also need to spend time every day to review log data.
According to a recent survey conducted by Check Point, the average enterprise spends between one and four hours per week on log management. Even so, log management is not a given. “A lot of times they’re not looking at logs,” notes Jane Goh, the product marketing manager for Check Point Software.
Other companies employ homegrown approaches of questionable security value. For example, “In one extreme case, we saw a customer printing their mainframe’s SMF [system management facilities] records, and if the pile was ever a lot thicker, on a monthly average, then they’d probably go take a look at the records,” says Marc van Zadelhoff, vice president of marketing and business development of Consul Risk Management. He notes the financial services company in question even went so far as to archive the printouts.
Implementing a Security Log Management Program
Several factors are driving companies to improve their log-management practices. “The quantity, volume, and variety of computer security logs have increased significantly, which has created a greater need for computer security log management—the process for generating, transmitting, storing, analyzing, and disposing of computer security log data,” notes the “Guide to Computer Security Log Management,” a draft report written by Murugiah Souppaya and Karen Kent and recently released by the National Institute of Standards and Technology (NIST).
To handle log data, companies need some kind of log-management software as well as a sufficient infrastructure. Given the need to maintain availability, plus the difficulties of normalizing data—reconciling logs in different formats—on the fly, the report’s authors recommend organizations “consider implementing a log-management infrastructure that includes centralized log servers and log-data storage.”
Don’t discount a security imperative here: log data is often sensitive data. Depending on how an organization employs logging, everything from the identity management software to corporate databases may be recording user names or even passwords in logs. Hence, organizations need proper procedures for securing and restricting access to log data, encrypting it in transit, securing it at rest, and ultimately disposing of it.
Using Syslog to Analyze Log Data
One software option for log management is syslog-based centralized logging software. With this approach, existing security products’ logs get pulled into a centralized repository in a standard syslog format, which most devices and applications either write natively or can easily be converted to.
According to the NIST report, common syslog products include Kiwi Syslog, Metalog, Modular Syslog (Msyslog), nsyslog, rsyslog, San Diego Supercomputer Center (SDSC), Secure Syslog, Syslog New Generation (Syslog-ng), and WinSyslog.
This approach may be best for IT managers who adopt a do-it-yourself philosophy and the extra time and resources needed to customize their own set-up. In particular, tweaking will be needed because the original syslog format isn’t secure enough for enterprise use. “Syslog was developed at a time when the security of logs was not a major consideration,” notes the NIST report. “Accordingly, syslog does not specify the use of basic security controls that would preserve the confidentiality, integrity, and availability of logs.” For example, using just syslog, organizations can’t guarantee the authenticity of logs, which means attackers could easily spoof logs or flood corporate networks with fake logs.
Newer versions of syslog, however, do add security capabilities, including encrypting data during transport; plus filtering, analysis, and event response. Since not all devices in your infrastructure may be compatible with newer versions of syslog, however, employing it may require additional integration work.
Security Event Management Software
The other option for analyzing logs is SEM (or variations SIM and SIEM) software. SEM software pulls information from security devices via agents or interfaces, normalizes the data, stores it in a centralized repository, and then analyzes it.
One driver for SEM software is automation. “A lot of folks are tired of trying to manually correlate the point products, so there is a lot of focus on trying to correlate logs and events from a variety of products,” says Rob Ayoub, an industry analyst for network security with Frost & Sullivan.
SEM software can also eliminate the need to retain specialists to help decode logs. Without SEM software, for example, “as a customer of mine put it, if I have Unix logs and Windows logs, I need to have a Unix and a Windows administrator to tell me what it means,” says Gavenraj Sodhi, a product manager at CA.
Such capabilities, of course, come at a cost, as the NIST report highlights. “Although SEM software typically offers more robust and broad log-management capabilities than syslog, SEM software is usually much more complicated and expensive to deploy than a centralized syslog implementation.” Also expect to implement additional infrastructure to support it. “SEM software is often more resource-intensive for individual hosts than syslog, because of the processing that agents perform.”
Buying software to decode what software you’ve already purchased is telling you may, of course, be a bitter pill. “The whole SIM space for me sort of points to a catastrophic failure on the part of the security vendors,” says Dan Farmer, co-founder and chief technology officer of Elemental Security. “Why do you need a tool aggregator to manage the tools? The fact that these systems that are out there are so difficult to extract value from—on a strategic level—is a real failing.”
Regardless, SEM software options abound. According to the NIST report, common SEM products—though this list is by no means definitive—include ArcSight Enterprise Security Manager from ArcSight, eTrust Security Command Center from CA, EventTracker from Prism Microsystems, GFi LANguard Security Event Log Monitor from GFI Software, LogCaster from RippleTech, NetIQ Security Manager from NetIQ, Security Management Center from OpenService, and Snare Server from InterSect Alliance. LogLogic, netForensics, and SenSage also offer eponymous products. Open Source Security Information Management (OSSIM) software is another option.
Drivers for SEM Shift
Recently, the users of SEM software, and its uses, have started to change. Historically, “early adopters of the technology tended to be very large organizations,” notes a Gartner Group report, owing to large organizations needing a better way to manage log data—and also being able to afford such software.
Now, however, outright event management is being supplanted by a need for more information. “SIM requirements—to support regulatory compliance initiatives—have replaced SEM as the primary driver for SIEM project funding,” says Gartner. “This means, fundamentally, that organizations are placing more emphasis on watching the actions of authorized users on servers.” Especially where regulations are concerned, companies must manage insiders’ access to sensitive information, and this need is driving much of today’s SEM adoption. “Auditors didn’t come to companies and say, ‘You need to start collecting your IDS and firewall logs and preventing more hackers from coming on the system,’” notes Consul’s van Zadelhoff. “They’d say your real risks are these privileged users that have god-like access to your system.”
Next week, in the conclusion of this two-part discussion, we examine how the security event management market is heading for a shakeout.
Related Articles: