Security Intelligence Starts With Detecting The Weird

As companies try to make sense of a greater amount of information on their networks, anomaly detection becomes more difficult but more important as well
Companies need to get more focused in their attempts to detect anomalous behavior on their networks that may indicate a breach because attackers are quickly adapting to defensive technologies and becoming more stealthy, states a recent report.

In its 2012 Mid-Year Trend and Risk Report, IBM noted that attackers are getting more creative -- by necessity -- in getting around a target's defenses. Companies with a hardened perimeter have seen attackers try to breach a partner's systems in hopes of gaining easier access. Businesses that rely on signature-based security will face custom malware. And firms looking for communications to known botnet controllers may miss more surreptitious communications using, for example, DNS.

These sorts of tactics mean that companies need to have a better handle on the state of their networks, and what "weird" behaviors are happening, says Robert Freeman, research and development manager for IBM's X-Force.

"It's not necessarily about seeing that machines are talking at weird times of the days," he says. "A lot is about seeing weird activity within your network, where machines are talking to the wrong systems, moving large amounts of traffic."

Take the recently reported VOHO campaign: The cyberespionage attack used compromised websites frequented by targeted companies to infect the victims. Nearly 1,000 companies and organizations had machines infected by the attack, which installed a variant of the Gh0st remote access Trojan (RAT) on compromised machines. With custom-compressed malware and infection starting at a legitimate site, the attack easily evaded firms' perimeter defenses. Early detection would then require that companies have a good understanding of their network traffic patterns.

[ After a major breach, the University of Nebraska used logs from all of its databases, applications, networks, and security tools to piece together a picture of the attack within 48 hours. See Lessons In Campus Cybersecurity. ]

Detecting such campaigns requires that companies go beyond just focusing on coarse network patterns, says Tim Van Der Horst, a malware researcher with network- and Web-security provider Blue Coat Systems.

"The more granular that you can get, the better," he says. "You can look at the network as a whole and detect anomalies. It is better if you can look and see what individual users are doing and what individual devices are doing."

Anomaly detection depends on establishing a good baseline of network activity. If the model is too strict, then even slight changes in employee behavior will set off an alert. But if the anomaly detection system (ADS) allows too much misbehavior, then companies will miss attacks. It's a typical feedback loop, where a company needs to learn from alerts and tweak their systems, IBM's Freeman says.

"In reality, it is something of an ongoing process, where anomalies are no superficial things, such as connecting to IRC at 1 a.m.," he says. "It is seeing the entirety of the network."

In its Mid-Year report, IBM recommends that companies heavily monitor privileged users and access to sensitive data. Detecting and blocking strange transfers of large amounts of data can also prevent some attackers from exfiltrating information. Finally, companies should monitor and block access from countries where they don't do business. To help better inform defenses, businesses should collect additional data, say, from a threat intelligence service and store network flows for later analysis.

"Where all this is heading is probably toward big data analytic engines that are going to consume information from anomaly detection engines and other sources, and produce more than what SIEM [security information and event management] provides or IDS [intrusion detection system] provides," Freeman says. "Really we are at the beginning, the initial stages, of where this goes."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.