15 Ways To Get More Value From Security Log And Event Data

[Excerpted from "15 Ways to Get More Value From Security Log and Event Data," a new report posted this week on Dark Reading's Security Monitoring Tech Center.]

Data is both a blessing and a curse for organizations today. The enormous amounts of log data generated by enterprise networks, servers, personal computing devices, and applications can be mined and analyzed to identify and even prevent threats to security, not to mention drive purposeful, strategic technology and business decisions. But all that data presents a huge challenge: In this sea of data, we often don’t know what’s important.

In other words, we are drowning in data but are thirsty for actionable information. There’s a growing push to solve this issue.

Trouble is, there is no universal attack detection “secret sauce,” nor a simple button you can push to help extract more value from your log files. In reality, the tasks that SIEM -- or security information and event management -- and log management tools are designed to help automate are complicated, and the scope of systems and data they are required to monitor makes detecting attacks and failures like finding the proverbial needle in a haystack.

In general, getting more out of your organization’s log and event data is really about doing more with what you already have. There is no dearth of tools for monitoring and analyzing the data generated by the dozens -- if not hundreds -- of IT systems in any organization. However, the resources and expertise to take full advantage of these tools can be elusive.

So how do you successfully mine, analyze, and apply information from log and event systems?

1. Analyze faster: It’s important to understand that few, if any, companies can block a targeted (that is, an APT) attack. At organizations under such an attack, the bad guys are likely to get in. So it’s equally important that your detection systems quickly identify threats and halt information from leaving the company.

To react to security events, you need real-time analysis. This means inspecting log data as it’s collected to detect policy violations. To do this, a platform must be able to support analysis at the endpoint, which means intelligent agents/collectors that sync enforcement rules from the central management server. Lots of platforms claim to have this capability, but not all of them actually do.

Any vendor that claims real-time analysis but runs all events through aggregation and correlation prior to analysis is missing the boat. Some policies make sense only once data has been correlated, but lots of events contain enough information to determine that an attack is taking place -- or at least that some suspicious activity is going on.

2. Filter the noise: With anywhere from 10 to 2,000 devices to monitor, each generating thousands of log events and/or alerts a day, there’s plenty of data for organizations to manage. Analysts must turn that data into actionable information, but with so much data to work with, analysts are likely to investigate only the most obvious attacks. Organizations must come up with a way (or ways) to reduce the number of alerts. Otherwise, there will be so many false positives to wade through that productivity will be impacted and effective coverage reduced.

Effective filtering is a combination of understanding threats and spotlighting relevant data. Take the time to understand which resources attackers will target and the likely paths they will take to get in, and assign a relative risk score to specific activity and write policies to detect suspect conditions. Visualization tools that highlight patterns and outliers are especially helpful when combing through reams of data, so take the time to integrate alerts and reports into these tools.

To read the other 13 tips on how to gain more from your security information, download the free report on how to get more from your security log and event information.

Have a comment on this story? Please click "Add a Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.