Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

8/14/2020
04:50 PM
50%
50%

Research Casts Doubt on Value of Threat Intel Feeds

Two commercial threat intelligence services and four open source feeds rarely provide the same information, raising questions about how security teams should gauge their utility.

Collect threat data from two of the largest threat intelligence providers, and the risk landscape they portray will be completely different — raising questions about the utility of threat intelligence feeds to organizations, a group of researchers said this week.

The researchers, from universities in the Netherlands and Germany, compared threat indicators from four open source threat intelligence feeds and two commercial feeds — which the researchers could not name — and found very little overlapping data between the services. On the commercial side, the larger Vendor 2 had 13% of the data covered by Vendor 1, while Vendor 1 only replicated 1.3% of the indicators from Vendor 2, said Xander Bouwman, a PhD candidate at Delft University of Technology and a primary author of the paper, in a presentation Wednesday.

"If two threat intelligence vendors are describing the same threats, you might expect that they are coming up with the same data," he said. "We find that this is not the case."

Even in tracking the same advanced persistent threat (APT) groups, threat intelligence vendors did not seem to collect the same data. Focusing on 22 threat groups that both vendors claimed to be tracking, the researchers found, at most, a 4% overlap in threat indicators, Bouwman said.

"This raises some questions about the coverage that these vendors are providing," he said. "If there is not so much overlap, what does that say about the visibility that these vendors are providing for the threat landscape as a whole?"

Threat intelligence includes open source threat intelligence, shared intelligence between organizations in the same industry, and commercial threat intelligence services. Open source threat intelligence often includes data from DNS blocklists, abuse feeds, malware hashes, and phishing lures. Shared intelligence is usually not available unless the organization joins a particular industry group. 

Commercial threat intelligence is often sold as a combination of reports to inform security teams and analysts and machine-readable indicators of compromise (IOCs) that be used to detect threats. A typical commercial feed, for example, could have dozens of threat reports and hundreds of IOCs every month. 

Unfortunately for potential customers, the uneven coverage means every threat intelligence provider's data set will be different, and there is little guarantee — or probability — that the threats will match what the customer will see. Without more information, the services are hard to evaluate, Bouwman said.

"This is what we refer to as a market with asymmetric information," he said. "The sellers know what they are selling, but the buyers don't know what they are buying."

The researchers compared the two commercial feeds with four open threat intelligence (OTI) feeds from Alienvault, Blocklist.de, CINScore, and EmergingThreats. While a few of the OTI feeds had significant overlap with other OTI sources, the commercial vendors had less than 1% overlap with any open threat intelligence feed. 

The lack of overlap raises questions about coverage and whether the services are providing a realistic picture of the threat landscape, Bouwman said.

Customers typically use threat intelligence for network detection, situational awareness, and prioritizing security operations centers' (SOCs) activities, the researchers found. Commercial feeds are better at providing context to users, according to a survey of 14 users of threat intelligence. Moreover, threat intelligence does not seem to be limited by cost, with only one in five in the survey citing cost as a factor. 

Unfortunately, customers are not very mature in terms of their knowledge of and skill in using threat intelligence, Bouwman said. Two respondents, for example, canceled their threat intelligence feeds because they were covering a sector unrelated to the organization's business.

"Customers do not seem to care about coverage, they are not optimizing for detection, and they are not talking about metrics," he said. "If they do mention metrics, it is almost always talking about false positives."

Overall, threat intelligence appears to be less about attaining insight into most threats and more about using the reports and IOCs as a way to understand the threat landscape, as well as occasionally for threat hunting. The most important factor may be whether the threat intelligence service helps save analyst time, the researchers stated.

Commercial vendors should help customers get the most productivity out of their feeds to justify their high cost, while customers need to require vendors to provide more information about the coverage the feeds provide, Bouwman said.

"In a market with asymmetric information, the willingness of consumers to pay might eventually go down because they cannot distinguish the good from the bad," he said.

Related Content:

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
News
Former CISA Director Chris Krebs Discusses Risk Management & Threat Intel
Kelly Sheridan, Staff Editor, Dark Reading,  2/23/2021
Edge-DRsplash-10-edge-articles
Security + Fraud Protection: Your One-Two Punch Against Cyberattacks
Joshua Goldfarb, Director of Product Management at F5,  2/23/2021
News
Cybercrime Groups More Prolific, Focus on Healthcare in 2020
Robert Lemos, Contributing Writer,  2/22/2021
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win an Amazon Gift Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
Building the SOC of the Future
Building the SOC of the Future
Digital transformation, cloud-focused attacks, and a worldwide pandemic. The past year has changed the way business works and the way security teams operate. There is no going back.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2021-21620
PUBLISHED: 2021-02-24
A cross-site request forgery (CSRF) vulnerability in Jenkins Claim Plugin 2.18.1 and earlier allows attackers to change claims.
CVE-2021-21621
PUBLISHED: 2021-02-24
Jenkins Support Core Plugin 2.72 and earlier provides the serialized user authentication as part of the "About user (basic authentication details only)" information, which can include the session ID of the user creating the support bundle in some configurations.
CVE-2021-21622
PUBLISHED: 2021-02-24
Jenkins Artifact Repository Parameter Plugin 1.0.0 and earlier does not escape parameter names and descriptions, resulting in a stored cross-site scripting (XSS) vulnerability exploitable by attackers with Job/Configure permission.
CVE-2020-28599
PUBLISHED: 2021-02-24
A stack-based buffer overflow vulnerability exists in the import_stl.cc:import_stl() functionality of Openscad openscad-2020.12-RC2. A specially crafted STL file can lead to code execution. An attacker can provide a malicious file to trigger this vulnerability.
CVE-2020-7846
PUBLISHED: 2021-02-24
Helpcom before v10.0 contains a file download and execution vulnerability caused by storing hardcoded cryptographic key. It finally leads to a file download and execution via access to crafted web page.