04:08 PM
Connect Directly

Hunting Botnets In The Cloud

Combining cloud, crowdsourcing, and big data to find and quash botnets on a larger scale

Comparing botnet command-and-control (C&C) traffic or malware within an organization to activity seen in other parts of the Internet isn't new. It's just that some security analysts are increasingly going there to gather better intelligence that they can use to quell an infection or help take down a botnet.

"The approach of using large bodies of data to identify botnets or malware, in general, has been going on for a long time. Now it's starting to become so widespread that startups are being galvanized by it ... making attention [be] paid to it," says Al Huger, vice president of development for the cloud technology group at Sourcefire and a co-founder of Immunet.

A group of researchers from Northeastern University, Symantec Research Labs, Eurecom, and UC Santa Barbara recently built a prototype system for detecting botnets on a large scale and for finding previously unknown botnet C&C servers. The so-called Disclosure tool uses the NetFlow protocol as well as custom features to spot botnet markers and to differentiate between C&C traffic and legitimate network traffic.

The breakthrough of the tool is that it spots botnet activity over the Internet as a whole, rather than just within an organization, the researchers say. And it ultimately can provide botnet protection "of the Internet at large," says William Robertson, assistant professor at the College of Computer and Information Science at Northeastern University, one of the developers of Disclosure. It's also a big-data type of tool that can process large amounts of data quickly, and can also spot previously unknown botnet servers operating out there, he says.

Some security vendors are expanding their botnet investigation into more cloud-based models: Seculert, for example, last month rolled out Seculert Sense, a cloud-based analysis engine that analyzes on-premise logs from an organization with its cloud-based botnet intelligence data. "Using the cloud as a technology enabler helps Seculert to better detect botnets and APTs, and therefore protect our customers," says Aviv Raff, co-founder and CTO at Seculert. "Only a cloud-based solution is capable of digesting a huge amount of data over a long enough period of time at an affordable cost in order to detect such persistent attacks."

So when Seculert detects a botnet infection in one organization, it can then spot the same attack on its other customers. "This is 'crowdsourcing' in order to battle the botnet and APT problem," Raff says. Seculert first spotted the Shamoon targeted attacks against Middle Eastern oil organizations, he says, with early versions of Seculert Sense.

At the heart of this cloud-based botnet-fighting model is "big data." And Seculert uses the Hadoop-based Amazon Elastic Map Reduce service in its offering. "It basically allows us to analyze huge amount of data using statistical analysis and machine-learning methodologies that consume large amount of CPU and large amount of storage for the logs," Raff says. "Therefore, we are able to see the bigger picture of the problem."

Incident response company Mandiant, meanwhile, recently quietly acquired Unveillance, a cloud-based botnet intelligence firm, and last month rolled out a new subscription cloud-based threat detection serviced based on Unveillance. "With its acquisition of Unveillance and its cloud-based botnet threat intelligence product, Mandiant can tell the enterprise whether it has any compromised hosts talking back to a criminal C2 infrastructure," writes Wendy Nather of The 451 Group.

[As companies try to make sense of a greater amount of information on their networks, anomaly detection becomes more difficult but more important as well. See Security Intelligence Starts With Detecting The Weird.]

It's crucial to have both an inside look at how a botnet has infected a particular organization, as well as external data on the larger operations and spread of the botnet, security experts say.

"Often it becomes remarkably simple to identify botnets, but getting your hands on good data is the challenge," Sourcefire's Huger says. "If you want to identify large-scale botnets, you need to get your hands on data that identifies them across multiple ISPs or millions of endpoints. Very few organizations are in a position to get their hands on that reliably and consistently."

That requires the ability to analyze botnet data from local and cloud-based sources in real time. "We collect actual big data amounts of information from" endpoints, he says, but that information in isolation is limited in value. "Seeing that endpoint go to a website ... and correlate that [behavior] with 30 other systems going there" in real time, you can get a better picture of the activity, he says.

Part of the problem of gathering good big data is competition among vendors that are hunting the botnets, he says. "The security industry doesn't generally play well together" when it comes to botnet information, for example, he says. "There are commercial competitors vying for customers."

It's not like in the antivirus sector, where malware sample-sharing is routine practice. Getting useful, global views of botnet activity can be difficult, he says. "You have to take large sets of data with seemingly innocuous data and marry them to come to broader conclusions."

Another challenge to beating botnets and APTs via the cloud: The bad guys are plenty organized and often better at sharing intelligence than the security industry, he says.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.

Kelly Jackson Higgins is Executive Editor at She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Register for Dark Reading Newsletters
White Papers
Current Issue
Five Emerging Security Threats - And What You Can Learn From Them
At Black Hat USA, researchers unveiled some nasty vulnerabilities. Is your organization ready?
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
Published: 2015-10-15
The Direct Rendering Manager (DRM) subsystem in the Linux kernel through 4.x mishandles requests for Graphics Execution Manager (GEM) objects, which allows context-dependent attackers to cause a denial of service (memory consumption) via an application that processes graphics data, as demonstrated b...

Published: 2015-10-15
netstat in IBM AIX 5.3, 6.1, and 7.1 and VIOS 2.2.x, when a fibre channel adapter is used, allows local users to gain privileges via unspecified vectors.

Published: 2015-10-15
Cross-site request forgery (CSRF) vulnerability in eXtplorer before 2.1.8 allows remote attackers to hijack the authentication of arbitrary users for requests that execute PHP code.

Published: 2015-10-15
Directory traversal vulnerability in QNAP QTS before 4.1.4 build 0910 and 4.2.x before 4.2.0 RC2 build 0910, when AFP is enabled, allows remote attackers to read or write to arbitrary files by leveraging access to an OS X (1) user or (2) guest account.

Published: 2015-10-15
Cisco Application Policy Infrastructure Controller (APIC) 1.1j allows local users to gain privileges via vectors involving addition of an SSH key, aka Bug ID CSCuw46076.

Dark Reading Radio
Archived Dark Reading Radio
According to industry estimates, about a million new IT security jobs will be created in the next two years but there aren't enough skilled professionals to fill them. On top of that, there isn't necessarily a clear path to a career in security. Dark Reading Executive Editor Kelly Jackson Higgins hosts guests Carson Sweet, co-founder and CTO of CloudPassage, which published a shocking study of the security gap in top US undergrad computer science programs, and Rodney Petersen, head of NIST's new National Initiative for Cybersecurity Education.