Analytics

11/7/2018
04:10 PM
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
50%
50%

Finding Gold in the Threat Intelligence Rush

Researchers sift through millions of threat intel observations to determine where to best find valuable threat data.

Threat intelligence feeds, sold for hundreds of thousands of dollars per year, are marketed on a specific premise: If an entity is seen acting maliciously in one place, it can be expected in others.

But that's not necessarily true, according to two researchers from SensePost SecureData. Founder and chief strategy officer Charl van der Walt and security analyst Sid Pillarisetty have spent six months analyzing the ability of threat intelligence to predict malicious activity. Their conclusion: There are both good and bad places and means to unearth reliable threat data on the Internet.

Van der Walt and Pillarisetty are part of a managed services team that conducts threat detection on behalf of UK customers. One of the issues they (and many security pros) deal with is detecting potentially harmful activity by IP addresses on customers' perimeters, van der Walt says. This includes people doing vulnerability scans, port scans, activity related to suspicious IP addresses, and anything that isn't obviously malicious but could warrant an investigation.

"The big question: How much effort does that sort of information warrant on behalf of enterprises?" he explains. "What should you be doing about it?"

Back in June, the duo began preliminary research on a relatively small dataset of threat indicators. They have since expanded their investigation to include more than 1 million online threat indicators and 1.3 billion correlations, or where suspicious events overlap.

At Black Hat Europe, in London this December, van der Walt and Pillarisetty will take the stage to share their findings in "Don't Eat Spaghetti with a Spoon: An Analysis of the Practical Value of Threat Intelligence." They hope to "move the needle along" in terms of understanding threat intelligence and equip other researchers with the data structures, tooling, methodology, and language to enable future research in the space, van der Walt says.

Different Companies Face Different Threats
In detecting malicious activity, the researchers have amassed indicators of compromise and IP addresses for several different customers. "What you end up having is threat intelligence, which we collect from one customer and is potentially applicable to another customer," van der Walt says.

This notion drives the business model of commercial threat feeds, which are sold to enterprises on the basis that they can drive intelligence-led security. Companies are told they can use feeds to pre-emptively block IP addresses that have appeared malicious for other customers.

These feeds are expensive in two ways, van der Walt explains. Businesses pay a lot of money to get them, for starters. When they do, the data demands attention and effort for security teams to respond. But in collecting and analyzing threats across companies, the researchers found that IP addresses that appear suspicious at one organization may not prove malicious at another.

For example, IP addresses that interact with honeypots prove malicious across businesses, they found. The duo set up a network of honeypots to correlate their observations of IP addresses and see how activity varied with the honeypot and with other networks. They learned the threat intelligence they collected via honeypots had a significantly higher fidelity than the threat data they directly gathered from customers' perimeters, van der Walt says.

Businesses would see a higher ROI by ingesting IP addresses from a honeypot and blocking those than by ingesting suspicious IP addresses from other feeds, Pillarisetty explains.

"What our initial research suggests – and we're trying to prove with a bigger dataset – is the proportion of suspicious IP addresses we observe at more than one customer is actually extremely low," van der Walt says. This implies companies relying on threat intelligence feeds spend a lot of time chasing shadows. "There's actually very little value in there," he adds.

At Black Hat Europe, the researchers also want to discuss whether certain processes need to be followed before the data they collect is actionable, Pillarisetty continues. They plan to investigate whether the IP addresses they get need to be processed further based on other factors in an environment.

"Only then can we say this is more malicious than other activity on your network," he says. It fits into the broader conversation of proposing better ways to gather threat intelligence.

Van der Walt says their research questions the underlying notion driving the threat intelligence business model. As consumers of threat feeds, he says, it changes how they view their value. Looking ahead, he anticipates they'll be able to verify some of the popular notions around the longevity of threat intelligence and the amount of time businesses have to respond to it.

In their initial study, van der Walt cites as an example, they observed multiple occurrences of the same IP address appearing in a two-day window. After that, the probability of seeing the same addresses "dropped off dramatically." In addition to analyzing the time frame of malicious IPs, he hopes they'll be able to determine other patterns. i.e., whether an IP seen at two companies will likely be seen at a third, or whether certain behavior indicates a reappearance of an IP address elsewhere.

Related Content:

 

 

 

Black Hat Europe returns to London Dec 3-6 2018  with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions and service providers in the Business Hall. Click for information on the conference and to register.

Kelly Sheridan is the Staff Editor at Dark Reading, where she focuses on cybersecurity news and analysis. She is a business technology journalist who previously reported for InformationWeek, where she covered Microsoft, and Insurance & Technology, where she covered financial ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
The Year in Security 2018
This Dark Reading Tech Digest explores the biggest news stories of 2018 that shaped the cybersecurity landscape.
Flash Poll
How Enterprises Are Attacking the Cybersecurity Problem
How Enterprises Are Attacking the Cybersecurity Problem
Data breach fears and the need to comply with regulations such as GDPR are two major drivers increased spending on security products and technologies. But other factors are contributing to the trend as well. Find out more about how enterprises are attacking the cybersecurity problem by reading our report today.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-6499
PUBLISHED: 2019-01-21
Teradata Viewpoint before 14.0 and 16.20.00.02-b80 contains a hardcoded password of TDv1i2e3w4 for the viewpoint database account (in viewpoint-portal\conf\server.xml) that could potentially be exploited by malicious users to compromise the affected system.
CVE-2019-6500
PUBLISHED: 2019-01-21
In Axway File Transfer Direct 2.7.1, an unauthenticated Directory Traversal vulnerability can be exploited by issuing a specially crafted HTTP GET request with %2e instead of '.' characters, as demonstrated by an initial /h2hdocumentation//%2e%2e/ substring.
CVE-2019-6498
PUBLISHED: 2019-01-21
GattLib 0.2 has a stack-based buffer over-read in gattlib_connect in dbus/gattlib.c because strncpy is misused.
CVE-2019-6497
PUBLISHED: 2019-01-20
Hotels_Server through 2018-11-05 has SQL Injection via the controller/fetchpwd.php username parameter.
CVE-2018-18908
PUBLISHED: 2019-01-20
The Sky Go Desktop application 1.0.19-1 through 1.0.23-1 for Windows performs several requests over cleartext HTTP. This makes the data submitted in these requests prone to Man in The Middle (MiTM) attacks, whereby an attacker would be able to obtain the data sent in these requests. Some of the requ...