Vulnerabilities / Threats
03:50 PM
Connect Directly

Hunting Botnets On A Bigger Scale

Researchers build prototype botnet detection system that gathers a big-picture view of both known and unknown botnet activity

An international group of researchers has built a prototype system for detecting botnets on a large scale and that can sniff out previously undiscovered botnet command-and-control (C&C) servers.

Botnet hunters traditionally focus on inspecting individual botnets or botnet activity within organizations, for example, the researchers say. The new prototype, called Disclosure, expands the view of botnet activity to a wider scale and detects botnet C&C traffic in real-time, inspecting billions of flows of datasets each day, they say. It uses the NetFlow network protocol created by Cisco that gathers IP traffic data, plus some custom features they added that allow the tool to differentiate between C&C traffic and legitimate traffic based on flow size and behavior patterns of the clients, as well as time frames of the traffic. They also integrated it with some external reputation scoring services.

"I think the main contribution is that it's operating at such a large scale that you could have much broader [botnet] protection of the Internet at large," says William Robertson, assistant professor at the College of Computer and Information Science at Northeastern University, who, along with Engin Kirda of Northeastern, Leyla Bilge of Symantec Research Labs, Davide Balzarotti of Eurecom, and Christopher Kruegel of UC Santa Barbara, built and tested Disclosure.

"It's very efficient: It can process a day's worth of data in less than a day," Robertson says.

The prototype also was able to detect several botnet C&C servers that had been previously unknown, he says. "We manually verified those: We had some students probe those sites to discover if they were likely C&C servers or not."

Today's tools for botnet hunters provide them the ability to detect C&C channels between the botnet operator and the infected bots, or to detect botnets based on behavior among a group of machines that indicates they are bots, the researchers say.

"Once bots or, ideally, C&C servers have been identified, a number of actions can be performed, ranging from removal of infected endpoints from the network, to filtering C&C channels at edge routers, to orchestrated take-downs of the C&C servers themselves," the researchers wrote in their paper, which they will present in December at the Annual Computer Security Applications Conference in Orlando, Fla.

"Unfortunately, while previous botnet detection approaches are effective under certain circumstances, none of these approaches scales beyond a single administrative domain while retaining useful detection accuracy. This limitation restricts the application of automated botnet detection systems to those entities that are informed or motivated enough to deploy them," they wrote. "Thus, we have the current state of botnet mitigation, where small pockets of the Internet are fairly well protected against infection while the majority of endpoints remain vulnerable."

The prototype is not the first large-scale botnet protection approach, however: Damballa, for instance, offers DNS-based reputation filtering for protecting large customers such as ISPs.

Meanwhile, in tests of the tool in a university network and a Tier 1 ISP network, the researchers found that Disclosure spotted some 65 percent of known botnet C&C servers, with a 1 percent false-positive rate. It also caught new botnet C&C servers that weren't previously known.

NetFlow data is valuable in botnet detection, but NetFlow analysis alone has its limitations in an enterprise environment, where network address translation and IPSes can wreak havoc on detection there, security experts say. "But even in the ISP environment, flow-based systems have problems keeping up with the traffic. Therefore, as the authors of the paper discuss, they will have to do sampling of the overall NetFlow traffic. It is clear that by sampling the traffic, a large portion of the botnet traffic will not be observed due to the sampling," says Manos Antonakakis, principal scientist and director of academic sciences at Damballa. "Therefore, the particular flow-based botnet detection system will most likely detect quite noisy botnets" such as spam, DDoS, and peer-to-peer botnets, he says.

The researchers say their prototype is not meant to detect targeted attacks of mini-botnet C&C systems. "This approach is not for more targeted attacks. We are trying to look at characteristics of large-scale attacks," says Kirda, who is associate professor for information assurance at the College of Computer and Information Science and the Department of Electrical and Computer Engineering at Northeastern University. The researchers also previously had built a tool called Exposure that detects DNS anomalies.

[A large peer-to-peer botnet known for its resilience was spotted sniffing out potential victim voice-over-IP (VoIP) servers using an advanced stealth technique of camouflaging its efforts to recruit new bots. See Botnet Spotted Silently Scanning IPv4 Address Space For Vulnerable VoIP.]

Damballa's Antonakakis says Disclosure is yet another tool for botnet defenders. "New detection tools are useful in botnet research. I think research should focus more on how we can defend against emerging threats. To that extent, I consider this paper a step toward the right direction, however quite incremental, to already existing techniques," says Antonakakis, who while at Georgia Tech co-developed Notos (PDF), a dynamic reputation system for DNS traffic that helps spot botnet activity and that is used today by Damballa.

The Disclosure research paper is available here (PDF) for download.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.

Kelly Jackson Higgins is Executive Editor at She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Register for Dark Reading Newsletters
White Papers
Current Issue
Dark Reading Tech Digest, Dec. 19, 2014
Software-defined networking can be a net plus for security. The key: Work with the network team to implement gradually, test as you go, and take the opportunity to overhaul your security strategy.
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
Published: 2015-02-27
The seg_write_packet function in libavformat/segment.c in ffmpeg 2.1.4 and earlier does not free the correct memory location, which allows remote attackers to cause a denial of service ("invalid memory handler") and possibly execute arbitrary code via a crafted video that triggers a use after free.

Published: 2015-02-27
The dns-sync module before 0.1.1 for node.js allows context-dependent attackers to execute arbitrary commands via shell metacharacters in the first argument to the resolve API function.

Published: 2015-02-27
Cross-site scripting (XSS) vulnerability in Unified Web Interaction Manager in Cisco Unified Web and E-Mail Interaction Manager allows remote attackers to inject arbitrary web script or HTML via vectors related to a POST request, aka Bug ID CSCus74184.

Published: 2015-02-27
Unquoted Windows search path vulnerability in Toshiba Bluetooth Stack for Windows before 9.10.32(T) and Service Station before 2.2.14 allows local users to gain privileges via a Trojan horse application with a name composed of an initial substring of a path that contains a space character.

Published: 2015-02-27
checkpw 1.02 and earlier allows remote attackers to cause a denial of service (infinite loop) via a -- (dash dash) in a username.

Dark Reading Radio
Archived Dark Reading Radio
How can security professionals better engage with their peers, both in person and online? In this Dark Reading Radio show, we will talk to leaders at some of the security industry’s professional organizations about how security pros can get more involved – with their colleagues in the same industry, with their peers in other industries, and with the IT security community as a whole.