Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

8/14/2020
04:50 PM
50%
50%

Research Casts Doubt on Value of Threat Intel Feeds

Two commercial threat intelligence services and four open source feeds rarely provide the same information, raising questions about how security teams should gauge their utility.

Collect threat data from two of the largest threat intelligence providers, and the risk landscape they portray will be completely different — raising questions about the utility of threat intelligence feeds to organizations, a group of researchers said this week.

The researchers, from universities in the Netherlands and Germany, compared threat indicators from four open source threat intelligence feeds and two commercial feeds — which the researchers could not name — and found very little overlapping data between the services. On the commercial side, the larger Vendor 2 had 13% of the data covered by Vendor 1, while Vendor 1 only replicated 1.3% of the indicators from Vendor 2, said Xander Bouwman, a PhD candidate at Delft University of Technology and a primary author of the paper, in a presentation Wednesday.

"If two threat intelligence vendors are describing the same threats, you might expect that they are coming up with the same data," he said. "We find that this is not the case."

Even in tracking the same advanced persistent threat (APT) groups, threat intelligence vendors did not seem to collect the same data. Focusing on 22 threat groups that both vendors claimed to be tracking, the researchers found, at most, a 4% overlap in threat indicators, Bouwman said.

"This raises some questions about the coverage that these vendors are providing," he said. "If there is not so much overlap, what does that say about the visibility that these vendors are providing for the threat landscape as a whole?"

Threat intelligence includes open source threat intelligence, shared intelligence between organizations in the same industry, and commercial threat intelligence services. Open source threat intelligence often includes data from DNS blocklists, abuse feeds, malware hashes, and phishing lures. Shared intelligence is usually not available unless the organization joins a particular industry group. 

Commercial threat intelligence is often sold as a combination of reports to inform security teams and analysts and machine-readable indicators of compromise (IOCs) that be used to detect threats. A typical commercial feed, for example, could have dozens of threat reports and hundreds of IOCs every month. 

Unfortunately for potential customers, the uneven coverage means every threat intelligence provider's data set will be different, and there is little guarantee — or probability — that the threats will match what the customer will see. Without more information, the services are hard to evaluate, Bouwman said.

"This is what we refer to as a market with asymmetric information," he said. "The sellers know what they are selling, but the buyers don't know what they are buying."

The researchers compared the two commercial feeds with four open threat intelligence (OTI) feeds from Alienvault, Blocklist.de, CINScore, and EmergingThreats. While a few of the OTI feeds had significant overlap with other OTI sources, the commercial vendors had less than 1% overlap with any open threat intelligence feed. 

The lack of overlap raises questions about coverage and whether the services are providing a realistic picture of the threat landscape, Bouwman said.

Customers typically use threat intelligence for network detection, situational awareness, and prioritizing security operations centers' (SOCs) activities, the researchers found. Commercial feeds are better at providing context to users, according to a survey of 14 users of threat intelligence. Moreover, threat intelligence does not seem to be limited by cost, with only one in five in the survey citing cost as a factor. 

Unfortunately, customers are not very mature in terms of their knowledge of and skill in using threat intelligence, Bouwman said. Two respondents, for example, canceled their threat intelligence feeds because they were covering a sector unrelated to the organization's business.

"Customers do not seem to care about coverage, they are not optimizing for detection, and they are not talking about metrics," he said. "If they do mention metrics, it is almost always talking about false positives."

Overall, threat intelligence appears to be less about attaining insight into most threats and more about using the reports and IOCs as a way to understand the threat landscape, as well as occasionally for threat hunting. The most important factor may be whether the threat intelligence service helps save analyst time, the researchers stated.

Commercial vendors should help customers get the most productivity out of their feeds to justify their high cost, while customers need to require vendors to provide more information about the coverage the feeds provide, Bouwman said.

"In a market with asymmetric information, the willingness of consumers to pay might eventually go down because they cannot distinguish the good from the bad," he said.

Related Content:

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Oldest First  |  Newest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 10/27/2020
Modern Day Insider Threat: Network Bugs That Are Stealing Your Data
David Pearson, Principal Threat Researcher,  10/21/2020
Are You One COVID-19 Test Away From a Cybersecurity Disaster?
Alan Brill, Senior Managing Director, Cyber Risk Practice, Kroll,  10/21/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
Special Report: Computing's New Normal
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
How IT Security Organizations are Attacking the Cybersecurity Problem
How IT Security Organizations are Attacking the Cybersecurity Problem
The COVID-19 pandemic turned the world -- and enterprise computing -- on end. Here's a look at how cybersecurity teams are retrenching their defense strategies, rebuilding their teams, and selecting new technologies to stop the oncoming rise of online attacks.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-27956
PUBLISHED: 2020-10-28
An Arbitrary File Upload in the Upload Image component in SourceCodester Car Rental Management System 1.0 allows the user to conduct remote code execution via admin/index.php?page=manage_car because .php files can be uploaded to admin/assets/uploads/ (under the web root).
CVE-2020-27957
PUBLISHED: 2020-10-28
The RandomGameUnit extension for MediaWiki through 1.35 was not properly escaping various title-related data. When certain varieties of games were created within MediaWiki, their names or titles could be manipulated to generate stored XSS within the RandomGameUnit extension.
CVE-2020-16140
PUBLISHED: 2020-10-27
The search functionality of the Greenmart theme 2.4.2 for WordPress is vulnerable to XSS.
CVE-2020-9982
PUBLISHED: 2020-10-27
This issue was addressed with improved checks to prevent unauthorized actions. This issue is fixed in Apple Music 3.4.0 for Android. A malicious application may be able to leak a user's credentials.
CVE-2020-3855
PUBLISHED: 2020-10-27
An access issue was addressed with improved access restrictions. This issue is fixed in macOS Catalina 10.15.3, Security Update 2020-001 Mojave, Security Update 2020-001 High Sierra. A malicious application may be able to overwrite arbitrary files.