Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

8/14/2020
04:50 PM
50%
50%

Research Casts Doubt on Value of Threat Intel Feeds

Two commercial threat intelligence services and four open source feeds rarely provide the same information, raising questions about how security teams should gauge their utility.

Collect threat data from two of the largest threat intelligence providers, and the risk landscape they portray will be completely different — raising questions about the utility of threat intelligence feeds to organizations, a group of researchers said this week.

The researchers, from universities in the Netherlands and Germany, compared threat indicators from four open source threat intelligence feeds and two commercial feeds — which the researchers could not name — and found very little overlapping data between the services. On the commercial side, the larger Vendor 2 had 13% of the data covered by Vendor 1, while Vendor 1 only replicated 1.3% of the indicators from Vendor 2, said Xander Bouwman, a PhD candidate at Delft University of Technology and a primary author of the paper, in a presentation Wednesday.

"If two threat intelligence vendors are describing the same threats, you might expect that they are coming up with the same data," he said. "We find that this is not the case."

Even in tracking the same advanced persistent threat (APT) groups, threat intelligence vendors did not seem to collect the same data. Focusing on 22 threat groups that both vendors claimed to be tracking, the researchers found, at most, a 4% overlap in threat indicators, Bouwman said.

"This raises some questions about the coverage that these vendors are providing," he said. "If there is not so much overlap, what does that say about the visibility that these vendors are providing for the threat landscape as a whole?"

Threat intelligence includes open source threat intelligence, shared intelligence between organizations in the same industry, and commercial threat intelligence services. Open source threat intelligence often includes data from DNS blocklists, abuse feeds, malware hashes, and phishing lures. Shared intelligence is usually not available unless the organization joins a particular industry group. 

Commercial threat intelligence is often sold as a combination of reports to inform security teams and analysts and machine-readable indicators of compromise (IOCs) that be used to detect threats. A typical commercial feed, for example, could have dozens of threat reports and hundreds of IOCs every month. 

Unfortunately for potential customers, the uneven coverage means every threat intelligence provider's data set will be different, and there is little guarantee — or probability — that the threats will match what the customer will see. Without more information, the services are hard to evaluate, Bouwman said.

"This is what we refer to as a market with asymmetric information," he said. "The sellers know what they are selling, but the buyers don't know what they are buying."

The researchers compared the two commercial feeds with four open threat intelligence (OTI) feeds from Alienvault, Blocklist.de, CINScore, and EmergingThreats. While a few of the OTI feeds had significant overlap with other OTI sources, the commercial vendors had less than 1% overlap with any open threat intelligence feed. 

The lack of overlap raises questions about coverage and whether the services are providing a realistic picture of the threat landscape, Bouwman said.

Customers typically use threat intelligence for network detection, situational awareness, and prioritizing security operations centers' (SOCs) activities, the researchers found. Commercial feeds are better at providing context to users, according to a survey of 14 users of threat intelligence. Moreover, threat intelligence does not seem to be limited by cost, with only one in five in the survey citing cost as a factor. 

Unfortunately, customers are not very mature in terms of their knowledge of and skill in using threat intelligence, Bouwman said. Two respondents, for example, canceled their threat intelligence feeds because they were covering a sector unrelated to the organization's business.

"Customers do not seem to care about coverage, they are not optimizing for detection, and they are not talking about metrics," he said. "If they do mention metrics, it is almost always talking about false positives."

Overall, threat intelligence appears to be less about attaining insight into most threats and more about using the reports and IOCs as a way to understand the threat landscape, as well as occasionally for threat hunting. The most important factor may be whether the threat intelligence service helps save analyst time, the researchers stated.

Commercial vendors should help customers get the most productivity out of their feeds to justify their high cost, while customers need to require vendors to provide more information about the coverage the feeds provide, Bouwman said.

"In a market with asymmetric information, the willingness of consumers to pay might eventually go down because they cannot distinguish the good from the bad," he said.

Related Content:

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
More SolarWinds Attack Details Emerge
Kelly Jackson Higgins, Executive Editor at Dark Reading,  1/12/2021
Vulnerability Management Has a Data Problem
Tal Morgenstern, Co-Founder & Chief Product Officer, Vulcan Cyber,  1/14/2021
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
2020: The Year in Security
Download this Tech Digest for a look at the biggest security stories that - so far - have shaped a very strange and stressful year.
Flash Poll
Assessing Cybersecurity Risk in Today's Enterprises
Assessing Cybersecurity Risk in Today's Enterprises
COVID-19 has created a new IT paradigm in the enterprise -- and a new level of cybersecurity risk. This report offers a look at how enterprises are assessing and managing cyber-risk under the new normal.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2021-22850
PUBLISHED: 2021-01-19
HGiga EIP product lacks ineffective access control in certain pages that allow attackers to access database or perform privileged functions.
CVE-2021-22851
PUBLISHED: 2021-01-19
HGiga EIP product contains SQL Injection vulnerability. Attackers can inject SQL commands into specific URL parameter (document management page) to obtain database schema and data.
CVE-2021-22852
PUBLISHED: 2021-01-19
HGiga EIP product contains SQL Injection vulnerability. Attackers can inject SQL commands into specific URL parameter (online registration) to obtain database schema and data.
CVE-2021-3178
PUBLISHED: 2021-01-19
** DISPUTED ** fs/nfsd/nfs3xdr.c in the Linux kernel through 5.10.8, when there is an NFS export of a subdirectory of a filesystem, allows remote attackers to traverse to other parts of the filesystem via READDIRPLUS. NOTE: some parties argue that such a subdirectory export is not intended to preven...
CVE-2021-3177
PUBLISHED: 2021-01-19
Python 3.x through 3.9.1 has a buffer overflow in PyCArg_repr in _ctypes/callproc.c, which may lead to remote code execution in certain Python applications that accept floating-point numbers as untrusted input, as demonstrated by a 1e300 argument to c_double.from_param. This occurs because sprintf i...