Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Analytics

11/7/2018
04:10 PM
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
50%
50%

Finding Gold in the Threat Intelligence Rush

Researchers sift through millions of threat intel observations to determine where to best find valuable threat data.

Threat intelligence feeds, sold for hundreds of thousands of dollars per year, are marketed on a specific premise: If an entity is seen acting maliciously in one place, it can be expected in others.

But that's not necessarily true, according to two researchers from SensePost SecureData. Founder and chief strategy officer Charl van der Walt and security analyst Sid Pillarisetty have spent six months analyzing the ability of threat intelligence to predict malicious activity. Their conclusion: There are both good and bad places and means to unearth reliable threat data on the Internet.

Van der Walt and Pillarisetty are part of a managed services team that conducts threat detection on behalf of UK customers. One of the issues they (and many security pros) deal with is detecting potentially harmful activity by IP addresses on customers' perimeters, van der Walt says. This includes people doing vulnerability scans, port scans, activity related to suspicious IP addresses, and anything that isn't obviously malicious but could warrant an investigation.

"The big question: How much effort does that sort of information warrant on behalf of enterprises?" he explains. "What should you be doing about it?"

Back in June, the duo began preliminary research on a relatively small dataset of threat indicators. They have since expanded their investigation to include more than 1 million online threat indicators and 1.3 billion correlations, or where suspicious events overlap.

At Black Hat Europe, in London this December, van der Walt and Pillarisetty will take the stage to share their findings in "Don't Eat Spaghetti with a Spoon: An Analysis of the Practical Value of Threat Intelligence." They hope to "move the needle along" in terms of understanding threat intelligence and equip other researchers with the data structures, tooling, methodology, and language to enable future research in the space, van der Walt says.

Different Companies Face Different Threats
In detecting malicious activity, the researchers have amassed indicators of compromise and IP addresses for several different customers. "What you end up having is threat intelligence, which we collect from one customer and is potentially applicable to another customer," van der Walt says.

This notion drives the business model of commercial threat feeds, which are sold to enterprises on the basis that they can drive intelligence-led security. Companies are told they can use feeds to pre-emptively block IP addresses that have appeared malicious for other customers.

These feeds are expensive in two ways, van der Walt explains. Businesses pay a lot of money to get them, for starters. When they do, the data demands attention and effort for security teams to respond. But in collecting and analyzing threats across companies, the researchers found that IP addresses that appear suspicious at one organization may not prove malicious at another.

For example, IP addresses that interact with honeypots prove malicious across businesses, they found. The duo set up a network of honeypots to correlate their observations of IP addresses and see how activity varied with the honeypot and with other networks. They learned the threat intelligence they collected via honeypots had a significantly higher fidelity than the threat data they directly gathered from customers' perimeters, van der Walt says.

Businesses would see a higher ROI by ingesting IP addresses from a honeypot and blocking those than by ingesting suspicious IP addresses from other feeds, Pillarisetty explains.

"What our initial research suggests – and we're trying to prove with a bigger dataset – is the proportion of suspicious IP addresses we observe at more than one customer is actually extremely low," van der Walt says. This implies companies relying on threat intelligence feeds spend a lot of time chasing shadows. "There's actually very little value in there," he adds.

At Black Hat Europe, the researchers also want to discuss whether certain processes need to be followed before the data they collect is actionable, Pillarisetty continues. They plan to investigate whether the IP addresses they get need to be processed further based on other factors in an environment.

"Only then can we say this is more malicious than other activity on your network," he says. It fits into the broader conversation of proposing better ways to gather threat intelligence.

Van der Walt says their research questions the underlying notion driving the threat intelligence business model. As consumers of threat feeds, he says, it changes how they view their value. Looking ahead, he anticipates they'll be able to verify some of the popular notions around the longevity of threat intelligence and the amount of time businesses have to respond to it.

In their initial study, van der Walt cites as an example, they observed multiple occurrences of the same IP address appearing in a two-day window. After that, the probability of seeing the same addresses "dropped off dramatically." In addition to analyzing the time frame of malicious IPs, he hopes they'll be able to determine other patterns. i.e., whether an IP seen at two companies will likely be seen at a third, or whether certain behavior indicates a reappearance of an IP address elsewhere.

Related Content:

 

 

 

Black Hat Europe returns to London Dec 3-6 2018  with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions and service providers in the Business Hall. Click for information on the conference and to register.

Kelly Sheridan is the Staff Editor at Dark Reading, where she focuses on cybersecurity news and analysis. She is a business technology journalist who previously reported for InformationWeek, where she covered Microsoft, and Insurance & Technology, where she covered financial ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 9/17/2020
Cybersecurity Bounces Back, but Talent Still Absent
Simone Petrella, Chief Executive Officer, CyberVista,  9/16/2020
Meet the Computer Scientist Who Helped Push for Paper Ballots
Kelly Jackson Higgins, Executive Editor at Dark Reading,  9/16/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
Special Report: Computing's New Normal
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
How IT Security Organizations are Attacking the Cybersecurity Problem
How IT Security Organizations are Attacking the Cybersecurity Problem
The COVID-19 pandemic turned the world -- and enterprise computing -- on end. Here's a look at how cybersecurity teams are retrenching their defense strategies, rebuilding their teams, and selecting new technologies to stop the oncoming rise of online attacks.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-25789
PUBLISHED: 2020-09-19
An issue was discovered in Tiny Tiny RSS (aka tt-rss) before 2020-09-16. The cached_url feature mishandles JavaScript inside an SVG document.
CVE-2020-25790
PUBLISHED: 2020-09-19
** DISPUTED ** Typesetter CMS 5.x through 5.1 allows admins to upload and execute arbitrary PHP code via a .php file inside a ZIP archive. NOTE: the vendor disputes the significance of this report because "admins are considered trustworthy"; however, the behavior "contradicts our secu...
CVE-2020-25791
PUBLISHED: 2020-09-19
An issue was discovered in the sized-chunks crate through 0.6.2 for Rust. In the Chunk implementation, the array size is not checked when constructed with unit().
CVE-2020-25792
PUBLISHED: 2020-09-19
An issue was discovered in the sized-chunks crate through 0.6.2 for Rust. In the Chunk implementation, the array size is not checked when constructed with pair().
CVE-2020-25793
PUBLISHED: 2020-09-19
An issue was discovered in the sized-chunks crate through 0.6.2 for Rust. In the Chunk implementation, the array size is not checked when constructed with From<InlineArray<A, T>>.