Threat Intelligence
3/15/2016
04:00 PM
Connect Directly
Google+
Twitter
RSS
E-Mail
50%
50%

Threat Intelligence's Big Data Problem

Security teams are drowning in often useless threat intel data, but signs of maturity are emerging in what IT-Harvest predicts will be a $1.5 billion market by 2018.

First in a series on the evolution of threat intelligence

Something’s gotta give: nearly three-fourths of enterprises today say they ignore security events because they’re overwhelmed by the deluge of alerts. And that doesn’t even take into account the firehose of threat intelligence data they’re funneling today, a new report shows.

Mega-retailer Target was the poster child for security alert awareness gone bad—the needle in the haystack Target dismissed was actually the clue that it was under a major attack in the fall of 2013. Nearly three years after that epic data breach, security events, alerts, and threat intelligence feeds are exploding in many enterprises hungry for hints that they are in the bullseye. The tradeoff is that this deluge of data is drowning security teams who must sift, separate, and correlate the real threats from the false positives or irrelevant information.

Security event overload alone is causing some dramatic fallout: more than half of all security events get ignored by IT security pros due to the overload of information, according to a new Enterprise Strategy Group (ESG) report that surveyed 125 IT security pros on the state of incident response in their organizations. Around 30% of those organizations say they also have some 11 different threat intelligence feeds flowing in as well, the Phantom-commissioned report—published today--found.

Threat intelligence data is all about helping enterprises block or protect against the newest threats by providing in-the-wild attack and threat artifacts and intel that companies can compare and correlate with their security. But for many organizations, the deluge of this type of information isn’t much help if they can’t triage and apply it effectively. 

The threat intelligence market itself is booming, growing at a rapid clip at 84% annually, according to new data published today by IT-Harvest. The threat intel market—which was at $251 million in 2015—is expected to reach more than $460 million this year, says Richard Stiennon, chief research analyst for IT-Harvest.

Threat intelligence platform products such as those of ThreatConnect, ThreatStream (now Anomali), ThreatQuotient, and BrightPoint Security, made up $61 million of 2015’s total threat intel market revenues, according to IT-Harvest. The market is on track to hit $1.5 billion in 2018 at the current rate of growth, according to the report, which includes a look at more than 20 threat intelligence vendors, including FireEye’s iSIGHT Partners, Cyveillance+LookingGlass, Digital Shadows, and Flashpoint Intel.

“I expect a lot of churn and also a lot of startups,” Stiennon says of the threat intelligence space.

Signs of churn started to show in the past month, with Norse Corp.’s mass layoffs and executive shakeout. Security experts attributed Norse’s plight more to its own internal managerial problems and lack of a solid product as well some weak analysis reports, rather than as a bellwether of the threat intel space.

‘Threat’ Rebrand

Meanwhile, recent moves by other threat intel vendors show signs of a logical evolution of making threat intel more useful and manageable.

Late last month, ThreatStream dropped the “threat” moniker and rebranded itself as Anomali, now focusing on not just delivering threat intel, but also prioritizing and matching it for individual organizations. Threat intel has its own big data problem, according to executives at Anomali, which now is filtering down indicators of compromise (IOCs) and other threat intel for security event and information management (SIEM) systems, which it says weren’t built to process millions of IOCs.

“When we started [out], the volume of threat intelligence coming from feed vendors and open communities versus now was more manageable. There were hundreds of thousands of indicators of compromise, and now there are tens of millions,” says Hugh Njemanze, CEO of Anomali. “We expect this year to [reach] 100 million IoCs. There’s been an explosion.”

That kind of threat intel volume isn’t conducive for most in-house SIEM tools today. “Even the most robust SIEM is not able to ingest more than 1 million IOCs,” he says. Anomali’s new cloud-based products basically match event flows with IOCs, for example, and then feed contextual information about the incident to the SIEM.

“We’re taking on the burden of discovery and matching and letting the SIEM do what it’s good at: analyzing the millions of events they are collecting,” Njemanze explains. Security operations center teams need to know which IOCs are relevant, so that’s what Anomali is offering.

Anomali still offers ThreatStream Optic, its threat intel feed, in addition to its new Harmony Breach Analytics and Anomali Reports products. “We still see ourselves as a threat intelligence player, but we’re radically shifting how threat intel can be operationalized,” he says.

“I’m convinced TI platforms like ThreatStream’s [Anomali’s] have an opportunity. I haven’t seen anyone targeting dealing with the data. Building a distiller takes the good stuff out, and turns the SIEM into a log manager,” IT-Harvest’s Stiennan says.

ThreatConnect, meanwhile, has upgraded its ThreatConnect platform to better integrate a company’s security incidents with threat intelligence. “The goal of my platform is to bring the two together: every data set and correlate it with events and incidents that are unfolding so human beings don’t have to look at the noise. Instead, the most important things bubble up to the top, based on the underlying analytics,” says Adam Vincent, CEO of ThreatConnect.

ThreatConnect has partnered with Splunk, Palo Alto Networks, and others, to integrate threat intel with an organization’s incident detection and response processes. Version 4.0 of the ThreatConnect platform also lets companies customize reports for all levels of users, including C-level executives who want to see a map of which regions are targeting their company, for example, Vincent says.

Threat intelligence is about empowering decision-making, he says. “It’s not the end goal in itself.”

So rather than a retailer looking at 100 events in the order in which they occur, the threat intel platform would flag and prioritize events that appear to be connected or related to other attacks in the wild. “It would say this event is important because it looks coordinated, and it’s against equipment that has known vulnerabilities,” Vincent says. “And it looks at what type of techniques and tradecraft the [attacker] is using ... As the [company] investigates it, they are collecting additional information that is going to inform their decision-making.”

Most security vendors now offer some level of threat intelligence, and there are several open-source threat intel feeds as well. “The challenge right now is to tell high-quality threat intelligence from low-quality threat intelligence. It’s tough to distinguish, given the abundancy of options” out there, says Oliver Friedrichs, founder and CEO of startup Phantom.

“One of the biggest challenges is how to reconcile all the various feeds and how to actually make sense of them. The threat intelligence platform space is really striving to solve that,” says Friedrichs, whose firm offers an automation and “orchestration engine” for an organization’s security tools.

 

Related Content:

 

Interop 2016 Las VegasFind out more about security threats at Interop 2016, May 2-6, at the Mandalay Bay Convention Center, Las Vegas. Register today and receive an early bird discount of $200.

Kelly Jackson Higgins is Executive Editor at DarkReading.com. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
ACThomson
50%
50%
ACThomson,
User Rank: Author
3/25/2016 | 9:20:15 AM
An opportunity for artificial intelligence
There are quite a number of startups appearing that are using artificial intelligence techniques to help manage this overload, allowing the most important threats to be prioratised.
ecrutchlow
50%
50%
ecrutchlow,
User Rank: Apprentice
3/21/2016 | 12:38:59 PM
Target Example Completely Wrong
Your use of Target was completely wrong. This was Target's first excuse for why they didn't catch this early, but as the story evolved the true events came out resulting in the CIO and the CEO resigning.

The FireEye device did detect and alert the subcontractor team in India and FireEye saw the alert as well. The subcontractor team was contacted immediately by FireEye and informed of the significance of the breach. In turn, the subcontractor notified Target and they did not act on the information. For a good write-up on the events, please look up the SANS report titled, "Case Study: Critical Controls that Could Have Prevented Target Breach"

The article is correct regarding the amount of data organizations must review. It's really the story of the boy who cried wolf one too many times. Except we have systems that cry wolf thousands of times a day. Solutions that can reduce the noise is essential. Security operations centers should be focusing on real events and not on vetting false-positives.
MattDevost
50%
50%
MattDevost,
User Rank: Apprentice
3/17/2016 | 4:27:59 PM
An evolution, hopefully based on the fundamentals of intelligence
Look forward to tracking this new series.  The evolution of threat intelligence is an important topic as organizations migrate towards intelligence driven security programs.  Hopefully you spend a little time looking at the fundamentals of the intelligence process and how they need to be incorporated and evolved into the cyber security domain.  

Having built intel organizations at companies like iDEFENSE, iSIGHT, the Terrorism Research Center, and within 56 U.S. cities, this is a topic that I track closely.  Many organizations don't give enough attention to how they need to consume and act on intelligence to drive decisions.  It isn't just about how many feeds you can consume, but how those feeds fit into and drive an internal intelligence process that is iterative and has a robust feedback loop.
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
DNS Threats: What Every Enterprise Should Know
Domain Name System exploits could put your data at risk. Here's some advice on how to avoid them.
Flash Poll
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2013-7445
Published: 2015-10-15
The Direct Rendering Manager (DRM) subsystem in the Linux kernel through 4.x mishandles requests for Graphics Execution Manager (GEM) objects, which allows context-dependent attackers to cause a denial of service (memory consumption) via an application that processes graphics data, as demonstrated b...

CVE-2015-4948
Published: 2015-10-15
netstat in IBM AIX 5.3, 6.1, and 7.1 and VIOS 2.2.x, when a fibre channel adapter is used, allows local users to gain privileges via unspecified vectors.

CVE-2015-5660
Published: 2015-10-15
Cross-site request forgery (CSRF) vulnerability in eXtplorer before 2.1.8 allows remote attackers to hijack the authentication of arbitrary users for requests that execute PHP code.

CVE-2015-6003
Published: 2015-10-15
Directory traversal vulnerability in QNAP QTS before 4.1.4 build 0910 and 4.2.x before 4.2.0 RC2 build 0910, when AFP is enabled, allows remote attackers to read or write to arbitrary files by leveraging access to an OS X (1) user or (2) guest account.

CVE-2015-6333
Published: 2015-10-15
Cisco Application Policy Infrastructure Controller (APIC) 1.1j allows local users to gain privileges via vectors involving addition of an SSH key, aka Bug ID CSCuw46076.

Dark Reading Radio
Archived Dark Reading Radio
Tim Wilson speaks to two experts on vulnerability research – independent consultant Jeremiah Grossman and Black Duck Software’s Mike Pittenger – about the latest wave of vulnerabilities being exploited by online attackers