Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

8/14/2020
04:50 PM
50%
50%

Research Casts Doubt on Value of Threat Intel Feeds

Two commercial threat intelligence services and four open source feeds rarely provide the same information, raising questions about how security teams should gauge their utility.

Collect threat data from two of the largest threat intelligence providers, and the risk landscape they portray will be completely different — raising questions about the utility of threat intelligence feeds to organizations, a group of researchers said this week.

The researchers, from universities in the Netherlands and Germany, compared threat indicators from four open source threat intelligence feeds and two commercial feeds — which the researchers could not name — and found very little overlapping data between the services. On the commercial side, the larger Vendor 2 had 13% of the data covered by Vendor 1, while Vendor 1 only replicated 1.3% of the indicators from Vendor 2, said Xander Bouwman, a PhD candidate at Delft University of Technology and a primary author of the paper, in a presentation Wednesday.

"If two threat intelligence vendors are describing the same threats, you might expect that they are coming up with the same data," he said. "We find that this is not the case."

Even in tracking the same advanced persistent threat (APT) groups, threat intelligence vendors did not seem to collect the same data. Focusing on 22 threat groups that both vendors claimed to be tracking, the researchers found, at most, a 4% overlap in threat indicators, Bouwman said.

"This raises some questions about the coverage that these vendors are providing," he said. "If there is not so much overlap, what does that say about the visibility that these vendors are providing for the threat landscape as a whole?"

Threat intelligence includes open source threat intelligence, shared intelligence between organizations in the same industry, and commercial threat intelligence services. Open source threat intelligence often includes data from DNS blocklists, abuse feeds, malware hashes, and phishing lures. Shared intelligence is usually not available unless the organization joins a particular industry group. 

Commercial threat intelligence is often sold as a combination of reports to inform security teams and analysts and machine-readable indicators of compromise (IOCs) that be used to detect threats. A typical commercial feed, for example, could have dozens of threat reports and hundreds of IOCs every month. 

Unfortunately for potential customers, the uneven coverage means every threat intelligence provider's data set will be different, and there is little guarantee — or probability — that the threats will match what the customer will see. Without more information, the services are hard to evaluate, Bouwman said.

"This is what we refer to as a market with asymmetric information," he said. "The sellers know what they are selling, but the buyers don't know what they are buying."

The researchers compared the two commercial feeds with four open threat intelligence (OTI) feeds from Alienvault, Blocklist.de, CINScore, and EmergingThreats. While a few of the OTI feeds had significant overlap with other OTI sources, the commercial vendors had less than 1% overlap with any open threat intelligence feed. 

The lack of overlap raises questions about coverage and whether the services are providing a realistic picture of the threat landscape, Bouwman said.

Customers typically use threat intelligence for network detection, situational awareness, and prioritizing security operations centers' (SOCs) activities, the researchers found. Commercial feeds are better at providing context to users, according to a survey of 14 users of threat intelligence. Moreover, threat intelligence does not seem to be limited by cost, with only one in five in the survey citing cost as a factor. 

Unfortunately, customers are not very mature in terms of their knowledge of and skill in using threat intelligence, Bouwman said. Two respondents, for example, canceled their threat intelligence feeds because they were covering a sector unrelated to the organization's business.

"Customers do not seem to care about coverage, they are not optimizing for detection, and they are not talking about metrics," he said. "If they do mention metrics, it is almost always talking about false positives."

Overall, threat intelligence appears to be less about attaining insight into most threats and more about using the reports and IOCs as a way to understand the threat landscape, as well as occasionally for threat hunting. The most important factor may be whether the threat intelligence service helps save analyst time, the researchers stated.

Commercial vendors should help customers get the most productivity out of their feeds to justify their high cost, while customers need to require vendors to provide more information about the coverage the feeds provide, Bouwman said.

"In a market with asymmetric information, the willingness of consumers to pay might eventually go down because they cannot distinguish the good from the bad," he said.

Related Content:

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Oldest First  |  Newest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 10/23/2020
7 Tips for Choosing Security Metrics That Matter
Ericka Chickowski, Contributing Writer,  10/19/2020
Russian Military Officers Unmasked, Indicted for High-Profile Cyberattack Campaigns
Kelly Jackson Higgins, Executive Editor at Dark Reading,  10/19/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
Special Report: Computing's New Normal
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
How IT Security Organizations are Attacking the Cybersecurity Problem
How IT Security Organizations are Attacking the Cybersecurity Problem
The COVID-19 pandemic turned the world -- and enterprise computing -- on end. Here's a look at how cybersecurity teams are retrenching their defense strategies, rebuilding their teams, and selecting new technologies to stop the oncoming rise of online attacks.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-24847
PUBLISHED: 2020-10-23
A Cross-Site Request Forgery (CSRF) vulnerability is identified in FruityWifi through 2.4. Due to a lack of CSRF protection in page_config_adv.php, an unauthenticated attacker can lure the victim to visit his website by social engineering or another attack vector. Due to this issue, an unauthenticat...
CVE-2020-24848
PUBLISHED: 2020-10-23
FruityWifi through 2.4 has an unsafe Sudo configuration [(ALL : ALL) NOPASSWD: ALL]. This allows an attacker to perform a system-level (root) local privilege escalation, allowing an attacker to gain complete persistent access to the local system.
CVE-2020-5990
PUBLISHED: 2020-10-23
NVIDIA GeForce Experience, all versions prior to 3.20.5.70, contains a vulnerability in the ShadowPlay component which may lead to local privilege escalation, code execution, denial of service or information disclosure.
CVE-2020-25483
PUBLISHED: 2020-10-23
An arbitrary command execution vulnerability exists in the fopen() function of file writes of UCMS v1.4.8, where an attacker can gain access to the server.
CVE-2020-5977
PUBLISHED: 2020-10-23
NVIDIA GeForce Experience, all versions prior to 3.20.5.70, contains a vulnerability in NVIDIA Web Helper NodeJS Web Server in which an uncontrolled search path is used to load a node module, which may lead to code execution, denial of service, escalation of privileges, and information disclosure.