Users of security log data are collecting more needles than ever -- but they still feel as though they must comb the haystack to find them, according to a study released yesterday.

According to the "SANS 2011 Log Management Survey" (PDF) published this week, organizations are collecting and logging more security data than ever before. But they still have trouble finding and analyzing the specific information they need to diagnose potential security problems.

"When this survey started seven years ago, log collection was only being done by 43 percent of respondents, compared with 89 percent who indicated they collected logs this year," the report states. "Now, [enterprises are] also collecting logs for much more than detecting suspicious behavior and troubleshooting, as in the recent past. Over the past two years, more respondents are also collecting logs for use in forensic analysis and correlation and to meet/prove regulatory compliance."

Organizations are collecting more types of log data than ever before, adding physical systems such as HVAC and SCADA to the mix, the report says. "This means more log types to collect and analyze—each with their own data formats that can vary widely," the report says.

"The mechanics of collecting, storing and archiving the log data are no longer the challenge in today’s world of almost unlimited data storage," SANS says in the report. "The challenge now is extracting the needed information for monitoring, management, compliance and decision-making from what respondents say is upwards of 100,000 events recorded per day."

Real-time alerts are the most useful feature of log management tools, according to respondents. "However, they were less enthusiastic about their log management system’s ability to interface with third-party tools or larger SIEM environments," the report states. "Users also cited problems with correlation, searching and interfacing with heterogeneous systems, and difficulties locating information within logs."

The collection of log data on nontraditional IT systems is part of a larger trend toward managing all of the potential vulnerabilities in the enterprise, says Joe Gottlieb, CEO of SIEM vendor SenSage.

"It is no surprise to see more logging of SCADA, mobile, and POS devices, given the evolution of each over the last several years," Gottlieb says. "With more use of TCP/IP and general-purpose servers, modern HVAC/SCADA systems are less obscure and therefore more accessible via remote attacks, as was seen with Stuxnet. Mobile devices continue to become more prevalent business tools that access sensitive data, but remain underprotected relative to desktop and laptop PCs."

The problem is finding a uniform way to analyze security data on disparate systems and logs, Gottlieb notes. Both SANS and the new Open Security Intelligence forum are holding discussions on this issue, he observes.

Have a comment on this story? Please click "Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.