If we're to believe the movies, all it takes is a few keystrokes and, voila, those silver-screen hackers have total pwnage of their target. But this is real life. Most hackers don't look like Angelina Jolie and breaking into a corporate or government network is a long grind.

"There's a lot of failure that precedes actual success," says Michael Maloof, CTO at TriGeo Network Security.

For companies or government agencies that are the targets of these attacks, that's a good thing. Security pros often have enough time to stop an incident before critical information is damaged or stolen--if they're vigilant enough to spot the tell-tale signs that real-world hackers leave behind. Most IT systems, particularly security software, already gather large amounts of data and compile it into system logs that can offer valuable clues about activity in the infrastructure to those who know how to decipher the data. But, of course, in real life, there's always a catch.

In this case, the logs produce so much data and it's so scattered around the company that the task of going through it all and connecting the dots can quickly become overwhelming. In order to get the most out of their logs and effectively meet threats, IT organizations must efficiently manage the logs and correlate the data using a range of best practices and security information and event management tools. SIEM tools use advanced algorithms to analyze the avalanche of data coming from different devices, making it possible to see patterns in the way users and machines usually interact with the infrastructure, in order to pinpoint unusual behavior.

"In and of itself, a log-on failure is a meaningless event, no one cares about it--but 50 of them in 30 seconds at three in the morning trying to get onto a critical server, now that should get your attention," Maloof says. Hackers generate a lot of activity as they try to gain control, and IT will be completely oblivious to it if it isn't monitoring, preferably in real time and with tools that can correlate the activity, he says.

Use Compliance Dollars For Real Security

If effective log management were easy, everyone would be doing it. But that's not the case. Though most companies have log systems, less than a quarter of the IT market is doing a good job monitoring them, estimates John Burnham, VP of marketing at SIEM vendor Q1 Labs.

SIEM tools originally were developed to provide actionable information to protect critical infrastructure--particularly in government settings. Now regulators in many industries mandate their use. Unfortunately, most organizations don't do much more than what's required to prove that the logs have been collected, stored, and verified. In the scramble to comply, they forget the original intent of logging--security.

But since these systems are needed for compliance, they tend to be adequately funded. And now, the smartest companies are using this software for prevention, by monitoring in real time what's hitting networks, rather than just for after-the-fact, forensic analysis.

SIEM Success

Maximize Your Monitoring Investment
Become an InformationWeek Analytics subscriber and get our full report on how to maximize your SIEM monitoring investment.

This report provides :

Get This And All Our Reports

Know What Data You Need

Once companies start digging into log data, they often find that they aren't gathering enough to get a meaningful picture. "When you drill in, what you don't want is to get to some dead end and find you're missing data," says Rick Caccia, VP of product and channel marketing for ArcSight, Hewlett-Packard's security and compliance management arm. Erring on the side of collecting more data rather than less is best, Caccia says.

But collecting too much data has some pretty obvious downsides, too, if it creates a management nightmare. "It's like searching for a needle in a pile of needles," says Andrew Hay of The 451 Group, a research firm.

Figuring out which logs are critical for detecting threats in real time and doing forensic analysis requires a clear understanding of the threat landscape your company faces. Knowing who might be trying to break in, what information they're after, and why they want it is critical. In addition, you need to understand your infrastructure well enough to know which systems could show signs of malicious activity. This requires laying the groundwork with network and identity modeling.

"Having detailed, accurate network and physical infrastructure documentation is important in being able to act on information from these systems in a timely manner, before an attack has the chance to successfully compromise data," says John Sawyer in the InformationWeek Analytics/Dark Reading report "What's Going On? Monitor Networks to Thwart Intrusions."

If, for example, an attacker plugs a wireless access point into a network port in a conference room or a multifunction copy machine is used to scan the internal network, you'll need diagrams indicating where those physical network ports are located, so security can grab the system immediately for analysis, Sawyer says.

Changing Times

43% of companies ran log servers six years ago to support data collection; the majority said log collection was their biggest log management challenge. 89% use log servers today and collection is the biggest challenge for just one in 10 of them. 65% say analysis of reports and interpreting results is one of the biggest challenges today. 64% say searching the log reports is one of the biggest challenges today.

Centralize Your Logs

No matter how much data you collect, if it's scattered across the IT wastelands, it's not going to be of much use. Failing to centralize logs is one of the biggest mistakes IT security organizations make, Burnham says. In the worst cases, they leave log data scattered around the network. In other cases, they collect the logs for forensics or for compliance but then don't correlate them to all the other available data.

Taking advantage of logs starts with centralizing the data into one or two SIEM tools that can do the heavy-lifting analytics that would be impossible to do in a timely fashion otherwise. It sounds simple, but there are often cultural issues that prevent centralization.

"One of the key challenges our customers face is really getting all parts of the company to work together to actually make the connections to get the right scope of monitoring," says Joe Gottlieb, CEO of SIEM vendor SenSage. "And the things you want to monitor sit in different places within the organization and are controlled by different parts of the organization."

Establishing some semblance of executive buy-in and drawing in people from departments other than IT to assist with coordination are some of the best first steps to overcoming cultural issues.

Phased Deployment

None of these steps is meant to be carried out all in the same day--or even in the same month. Based on years of experience with SIEM installations, Bradford Nelson, a security professional at a large federal agency, suggests three stages for SIEM deployment.

"Keep the bar low at the beginning," Nelson says. Gather just the baseline information you need to establish what "normal" statistics and readings look like in order to find anomalies going forward. If you try to do everything at once--threat and anomaly detection, threat analysis, and response--you're going to fail, he says.

Start with a compliance-focused checklist, emphasizing the SIEM tool's security information management elements, primarily collecting logs for audit purposes. Nelson recommends spending about six months in this stage of deployment. Any more and you don't get enough payback from tools for the time you invest; any less and you risk a disaster.

Next is a growth stage during which you begin to use the security event management aspect of SIEM, utilizing real-time monitoring. Companies that evolve to the final, mature stage are those that can integrate SIEM analysis into their IT operations processes so that security becomes part of the overall IT framework. In this stage, you are taking advantage of external security data feeds, using on-board processes to log activity from new users and systems as they're added, analyzing business behavior, and utilizing business context to make security decisions. Companies that have mastered the art of system data correlation and analysis are able to extend the benefits outside the security group. The data can offer clues for infrastructure improvement and simplifying IT operations and business processes.

Moving through these SIEM stages requires starting with basic log groups and standardizing the analysis processes for them, SenSage's Gottlieb says. Then broaden the scope for more log subsets, using what you've learned from previous process development.

Companies and other organizations are definitely collecting more logs these days, but they have a long way to go when it comes to actually analyzing the data those logs provide in order to head off fraud and external attacks. Six years ago the biggest log management concern among security professionals was simply to collect logs from enough sources, according to the SANS Institute. That's now a concern for only about 10% of companies.

These days the difficulties lie in aggregating, analyzing, and searching log data. Companies that start building the right rule sets within their SIEM tools and tuning these tools according to their risk tolerance will find that they're able to block the bad guys before they do real damage to the infrastructure. It may not be just like the movies, but in real life that's as close to a happy ending as we get.

Sidebar: Where To Start With Logging

Log management and event correlation can be daunting for the uninitiated. Here are tips from log management expert Bill Roth, chief marketing officer at LogLogic, a security management vendor.

>> Begin With Basic Feeds: Prioritize the collection and analysis of Windows logs, syslogs, and Web server logs, and you'll have 90% of what you need, Roth says.

>> Look For The Abnormal: Spend time in the beginning watching stats from networking and firewall devices, to benchmark what's normal so you can spot aberrations later.

>> Watch For Privilege Changes: Attackers try to escalate their privileges to carry out attacks. Watch for this in key locations. One command in particular to be alert for, Roth says, is sudo, or "substitute user do," in Unix and Linux systems.

>> Check Error Messages: Watch for 404 messages in your Web server, distributed denial-of-service attacks, and too many attempts at access made by hackers gaining infrastructure information. --Ericka Chickowski

Dark Reading April 11, 2011 Issue

Download a free PDF of Dark Reading April Digital Issue
(registration required)