Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Analytics

11/7/2018
04:10 PM
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
50%
50%

Finding Gold in the Threat Intelligence Rush

Researchers sift through millions of threat intel observations to determine where to best find valuable threat data.

Threat intelligence feeds, sold for hundreds of thousands of dollars per year, are marketed on a specific premise: If an entity is seen acting maliciously in one place, it can be expected in others.

But that's not necessarily true, according to two researchers from SensePost SecureData. Founder and chief strategy officer Charl van der Walt and security analyst Sid Pillarisetty have spent six months analyzing the ability of threat intelligence to predict malicious activity. Their conclusion: There are both good and bad places and means to unearth reliable threat data on the Internet.

Van der Walt and Pillarisetty are part of a managed services team that conducts threat detection on behalf of UK customers. One of the issues they (and many security pros) deal with is detecting potentially harmful activity by IP addresses on customers' perimeters, van der Walt says. This includes people doing vulnerability scans, port scans, activity related to suspicious IP addresses, and anything that isn't obviously malicious but could warrant an investigation.

"The big question: How much effort does that sort of information warrant on behalf of enterprises?" he explains. "What should you be doing about it?"

Back in June, the duo began preliminary research on a relatively small dataset of threat indicators. They have since expanded their investigation to include more than 1 million online threat indicators and 1.3 billion correlations, or where suspicious events overlap.

At Black Hat Europe, in London this December, van der Walt and Pillarisetty will take the stage to share their findings in "Don't Eat Spaghetti with a Spoon: An Analysis of the Practical Value of Threat Intelligence." They hope to "move the needle along" in terms of understanding threat intelligence and equip other researchers with the data structures, tooling, methodology, and language to enable future research in the space, van der Walt says.

Different Companies Face Different Threats
In detecting malicious activity, the researchers have amassed indicators of compromise and IP addresses for several different customers. "What you end up having is threat intelligence, which we collect from one customer and is potentially applicable to another customer," van der Walt says.

This notion drives the business model of commercial threat feeds, which are sold to enterprises on the basis that they can drive intelligence-led security. Companies are told they can use feeds to pre-emptively block IP addresses that have appeared malicious for other customers.

These feeds are expensive in two ways, van der Walt explains. Businesses pay a lot of money to get them, for starters. When they do, the data demands attention and effort for security teams to respond. But in collecting and analyzing threats across companies, the researchers found that IP addresses that appear suspicious at one organization may not prove malicious at another.

For example, IP addresses that interact with honeypots prove malicious across businesses, they found. The duo set up a network of honeypots to correlate their observations of IP addresses and see how activity varied with the honeypot and with other networks. They learned the threat intelligence they collected via honeypots had a significantly higher fidelity than the threat data they directly gathered from customers' perimeters, van der Walt says.

Businesses would see a higher ROI by ingesting IP addresses from a honeypot and blocking those than by ingesting suspicious IP addresses from other feeds, Pillarisetty explains.

"What our initial research suggests – and we're trying to prove with a bigger dataset – is the proportion of suspicious IP addresses we observe at more than one customer is actually extremely low," van der Walt says. This implies companies relying on threat intelligence feeds spend a lot of time chasing shadows. "There's actually very little value in there," he adds.

At Black Hat Europe, the researchers also want to discuss whether certain processes need to be followed before the data they collect is actionable, Pillarisetty continues. They plan to investigate whether the IP addresses they get need to be processed further based on other factors in an environment.

"Only then can we say this is more malicious than other activity on your network," he says. It fits into the broader conversation of proposing better ways to gather threat intelligence.

Van der Walt says their research questions the underlying notion driving the threat intelligence business model. As consumers of threat feeds, he says, it changes how they view their value. Looking ahead, he anticipates they'll be able to verify some of the popular notions around the longevity of threat intelligence and the amount of time businesses have to respond to it.

In their initial study, van der Walt cites as an example, they observed multiple occurrences of the same IP address appearing in a two-day window. After that, the probability of seeing the same addresses "dropped off dramatically." In addition to analyzing the time frame of malicious IPs, he hopes they'll be able to determine other patterns. i.e., whether an IP seen at two companies will likely be seen at a third, or whether certain behavior indicates a reappearance of an IP address elsewhere.

Related Content:

 

 

 

Black Hat Europe returns to London Dec 3-6 2018  with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions and service providers in the Business Hall. Click for information on the conference and to register.

Kelly Sheridan is the Staff Editor at Dark Reading, where she focuses on cybersecurity news and analysis. She is a business technology journalist who previously reported for InformationWeek, where she covered Microsoft, and Insurance & Technology, where she covered financial ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
For Cybersecurity to Be Proactive, Terrains Must Be Mapped
Craig Harber, Chief Technology Officer at Fidelis Cybersecurity,  10/8/2019
A Realistic Threat Model for the Masses
Lysa Myers, Security Researcher, ESET,  10/9/2019
USB Drive Security Still Lags
Dark Reading Staff 10/9/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
7 Threats & Disruptive Forces Changing the Face of Cybersecurity
This Dark Reading Tech Digest gives an in-depth look at the biggest emerging threats and disruptive forces that are changing the face of cybersecurity today.
Flash Poll
2019 Online Malware and Threats
2019 Online Malware and Threats
As cyberattacks become more frequent and more sophisticated, enterprise security teams are under unprecedented pressure to respond. Is your organization ready?
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-17552
PUBLISHED: 2019-10-14
An issue was discovered in idreamsoft iCMS v7.0.14. There is a spider_project.admincp.php SQL injection vulnerability in the 'upload spider project scheme' feature via a two-dimensional payload.
CVE-2019-17553
PUBLISHED: 2019-10-14
An issue was discovered in MetInfo v7.0.0 beta. There is SQL Injection via the admin/?n=tags&c=index&a=doSaveTags URI.
CVE-2019-17408
PUBLISHED: 2019-10-14
parserIfLabel in inc/zzz_template.php in ZZZCMS zzzphp 1.7.3 allows remote attackers to execute arbitrary code because the danger_key function can be bypassed via manipulations such as strtr.
CVE-2019-17545
PUBLISHED: 2019-10-14
GDAL through 3.0.1 has a poolDestroy double free in OGRExpatRealloc in ogr/ogr_expat.cpp when the 10MB threshold is exceeded.
CVE-2019-17546
PUBLISHED: 2019-10-14
tif_getimage.c in LibTIFF through 4.0.10, as used in GDAL through 3.0.1 and other products, has an integer overflow that potentially causes a heap-based buffer overflow via a crafted RGBA image, related to a "Negative-size-param" condition.