Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Vulnerabilities / Threats

03:50 PM
Connect Directly

Hunting Botnets On A Bigger Scale

Researchers build prototype botnet detection system that gathers a big-picture view of both known and unknown botnet activity

An international group of researchers has built a prototype system for detecting botnets on a large scale and that can sniff out previously undiscovered botnet command-and-control (C&C) servers.

Botnet hunters traditionally focus on inspecting individual botnets or botnet activity within organizations, for example, the researchers say. The new prototype, called Disclosure, expands the view of botnet activity to a wider scale and detects botnet C&C traffic in real-time, inspecting billions of flows of datasets each day, they say. It uses the NetFlow network protocol created by Cisco that gathers IP traffic data, plus some custom features they added that allow the tool to differentiate between C&C traffic and legitimate traffic based on flow size and behavior patterns of the clients, as well as time frames of the traffic. They also integrated it with some external reputation scoring services.

"I think the main contribution is that it's operating at such a large scale that you could have much broader [botnet] protection of the Internet at large," says William Robertson, assistant professor at the College of Computer and Information Science at Northeastern University, who, along with Engin Kirda of Northeastern, Leyla Bilge of Symantec Research Labs, Davide Balzarotti of Eurecom, and Christopher Kruegel of UC Santa Barbara, built and tested Disclosure.

"It's very efficient: It can process a day's worth of data in less than a day," Robertson says.

The prototype also was able to detect several botnet C&C servers that had been previously unknown, he says. "We manually verified those: We had some students probe those sites to discover if they were likely C&C servers or not."

Today's tools for botnet hunters provide them the ability to detect C&C channels between the botnet operator and the infected bots, or to detect botnets based on behavior among a group of machines that indicates they are bots, the researchers say.

"Once bots or, ideally, C&C servers have been identified, a number of actions can be performed, ranging from removal of infected endpoints from the network, to filtering C&C channels at edge routers, to orchestrated take-downs of the C&C servers themselves," the researchers wrote in their paper, which they will present in December at the Annual Computer Security Applications Conference in Orlando, Fla.

"Unfortunately, while previous botnet detection approaches are effective under certain circumstances, none of these approaches scales beyond a single administrative domain while retaining useful detection accuracy. This limitation restricts the application of automated botnet detection systems to those entities that are informed or motivated enough to deploy them," they wrote. "Thus, we have the current state of botnet mitigation, where small pockets of the Internet are fairly well protected against infection while the majority of endpoints remain vulnerable."

The prototype is not the first large-scale botnet protection approach, however: Damballa, for instance, offers DNS-based reputation filtering for protecting large customers such as ISPs.

Meanwhile, in tests of the tool in a university network and a Tier 1 ISP network, the researchers found that Disclosure spotted some 65 percent of known botnet C&C servers, with a 1 percent false-positive rate. It also caught new botnet C&C servers that weren't previously known.

NetFlow data is valuable in botnet detection, but NetFlow analysis alone has its limitations in an enterprise environment, where network address translation and IPSes can wreak havoc on detection there, security experts say. "But even in the ISP environment, flow-based systems have problems keeping up with the traffic. Therefore, as the authors of the paper discuss, they will have to do sampling of the overall NetFlow traffic. It is clear that by sampling the traffic, a large portion of the botnet traffic will not be observed due to the sampling," says Manos Antonakakis, principal scientist and director of academic sciences at Damballa. "Therefore, the particular flow-based botnet detection system will most likely detect quite noisy botnets" such as spam, DDoS, and peer-to-peer botnets, he says.

The researchers say their prototype is not meant to detect targeted attacks of mini-botnet C&C systems. "This approach is not for more targeted attacks. We are trying to look at characteristics of large-scale attacks," says Kirda, who is associate professor for information assurance at the College of Computer and Information Science and the Department of Electrical and Computer Engineering at Northeastern University. The researchers also previously had built a tool called Exposure that detects DNS anomalies.

[A large peer-to-peer botnet known for its resilience was spotted sniffing out potential victim voice-over-IP (VoIP) servers using an advanced stealth technique of camouflaging its efforts to recruit new bots. See Botnet Spotted Silently Scanning IPv4 Address Space For Vulnerable VoIP.]

Damballa's Antonakakis says Disclosure is yet another tool for botnet defenders. "New detection tools are useful in botnet research. I think research should focus more on how we can defend against emerging threats. To that extent, I consider this paper a step toward the right direction, however quite incremental, to already existing techniques," says Antonakakis, who while at Georgia Tech co-developed Notos (PDF), a dynamic reputation system for DNS traffic that helps spot botnet activity and that is used today by Damballa.

The Disclosure research paper is available here (PDF) for download.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.

Kelly Jackson Higgins is the Executive Editor of Dark Reading. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
US Formally Attributes SolarWinds Attack to Russian Intelligence Agency
Jai Vijayan, Contributing Writer,  4/15/2021
Dependency Problems Increase for Open Source Components
Robert Lemos, Contributing Writer,  4/14/2021
FBI Operation Remotely Removes Web Shells From Exchange Servers
Kelly Sheridan, Staff Editor, Dark Reading,  4/14/2021
Register for Dark Reading Newsletters
White Papers
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2021-04-20
An unsafe deserialization vulnerability in Bridgecrew Checkov by Prisma Cloud allows arbitrary code execution when processing a malicious terraform file. This issue impacts Checkov 2.0 versions earlier than Checkov 2.0.26. Checkov 1.0 versions are not impacted.
PUBLISHED: 2021-04-20
An information exposure through log file vulnerability exists in Palo Alto Networks PAN-OS software where secrets in PAN-OS XML API requests are logged in cleartext to the web server logs when the API is used incorrectly. This vulnerability applies only to PAN-OS appliances that are configured to us...
PUBLISHED: 2021-04-20
An information exposure through log file vulnerability exists in Palo Alto Networks PAN-OS software where the connection details for a scheduled configuration export are logged in system logs. Logged information includes the cleartext username, password, and IP address used to export the PAN-OS conf...
PUBLISHED: 2021-04-20
A denial-of-service (DoS) vulnerability in Palo Alto Networks GlobalProtect app on Windows systems allows a limited Windows user to send specifically-crafted input to the GlobalProtect app that results in a Windows blue screen of death (BSOD) error. This issue impacts: GlobalProtect app 5.1 versions...
PUBLISHED: 2021-04-19
An out-of-bounds (OOB) memory access flaw was found in fs/f2fs/node.c in the f2fs module in the Linux kernel in versions before 5.12.0-rc4. A bounds check failure allows a local attacker to gain access to out-of-bounds memory leading to a system crash or a leak of internal kernel information. The hi...