Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Risk

4/30/2019
10:00 AM
50%
50%

Threat Intelligence Firms Look to AI, but Still Require Humans

Machine learning and artificial intelligence are helping threat-intelligence firms cover a greater area of the darknet, but human analysts will always be necessary, experts say.

Threat intelligence firms are racing to expand their machine-learning capabilities to capture more of the un-indexed parts of the internet, but somewhat ironically, human analysts and experts remain critical to the effort. 

Last week, startup Resecurity announced its intent to index the entire dark web by 2020—petabytes of information. The effort would use machine learning and big data technologies, but Resecurity also specifically called out the necessity of human analysts.

The same week, rival threat intelligence firm Flashpoint announced an expansion to its service, adding more dark-web sources, underground marketplaces and encrypted forums. While the capabilities are fueled by advances in machine learning, human analysts remain key, says Flashpoint CEO Josh Lefkowitz.

"There is a tenuous and temporary nature to many of these sources, and that is what necessitates the agility of human involvement because the actors are constantly moving around," Lefkowitz says.

The amount of data created by companies has exploded as more enterprise infrastructure connects to the internet and produces logging information and other data. The availability of data on every facet of the business—and on consumers' habits—has powered the application of machine learning and artificial intelligence to a variety of new use cases.

Threat intelligence has quickly become a way for companies to keep abreast of vulnerabilities that impact their systems, breaches that affect their customers, and potential attacker interest. More than half of companies are using threat intelligence to improve their detection of attacks, according to a recent report from CyberEdge. A similar majority also used threat intelligence to help validate security alerts and signs of an attack and reduce the workload on their analysts, the same report stated.

"It is a two-pronged approach—you want to have a lot of tools in your toolbox," says Daniel Hatheway, senior technical analyst at digital intelligence firm Recorded Future. "It is more about creating transparency around what dangers the dark web does pose."

Most threat intelligence firms are focused on expanding access to the types of information which matter to clients. Flashpoint, for example, just announced that the company has added more robust vulnerability data, compromised account data from underground shops, and information from secured and encrypted forums.

To access that last category and refine the other data sets, they need human analysts, Lefkowitz says.

"If you just have automated scrapers that are spidering the entirety of the onion network—when the reality is that many of these environments are password-protected and they necessitate invite links that you receive because of your credibility and reputation in the community—you are missing so much if you are just relying on automation," he says.

Employees and threat analysts are, of course, expensive. So companies continue to look for ways to automate as much of the process as possible. Resecurity, for example, has focused on capturing as much of the dark web as possible for later inspection by clients and analysts.

"There have been attempts in the past to scan the dark web, but the tools that were available when those efforts were undertaken were extremely primitive," Gene Yoo, chief executive officer of Resecurity, said in a statement. "They generated a lot of false positives and noise—and not a lot of truly actionable intelligence. To deliver the maximum visibility into the dark web, to get to the point where we can associate a particular threat actor with his real identity, we need to apply the power of data science and big data."

Relying on humans for security is nothing new. The majority of companies heavily rely on employees to identify threats, with 85 percent of firms saying employees fill a significant role in their defense against cyber attacks, according to a survey conducted by security firm Glasswall Solutions. In fact, more than 40 percent of companies surveyed by Glasswall Solutions stated that they were "completely reliant on employees as their last line of defense," Glasswall stated in a release published along with the report.

It's unlikely that threat intelligence firms will ever be able to do away with human analysts. In fact, as companies' web scraping systems produce more potential alerts, connecting disparate pieces of information, the demand for analysts will likely increase.

"Part of this is recognizing that illicit actors continue to diversify where they are operating, necessitating an expansion in the aperture of our collections, of our capabilities," says Flashpoint's Lefkowitz. "It is that chess game, that cat-and-mouse game, … and we are only seeing that trend continue now as more technologies are at their fingertips that let them pursue their illicit goals."

 

Related Links

 

 

 

 

Join Dark Reading LIVE for two cybersecurity summits at Interop 2019. Learn from the industry's most knowledgeable IT security experts. Check out the Interop agenda here.

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
AI Is Everywhere, but Don't Ignore the Basics
Howie Xu, Vice President of AI and Machine Learning at Zscaler,  9/10/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
7 Threats & Disruptive Forces Changing the Face of Cybersecurity
This Dark Reading Tech Digest gives an in-depth look at the biggest emerging threats and disruptive forces that are changing the face of cybersecurity today.
Flash Poll
The State of IT Operations and Cybersecurity Operations
The State of IT Operations and Cybersecurity Operations
Your enterprise's cyber risk may depend upon the relationship between the IT team and the security team. Heres some insight on what's working and what isn't in the data center.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-16349
PUBLISHED: 2019-09-16
Bento4 1.5.1-628 has a NULL pointer dereference in AP4_ByteStream::ReadUI32 in Core/Ap4ByteStream.cpp when called from the AP4_TrunAtom class.
CVE-2019-16350
PUBLISHED: 2019-09-16
ffjpeg before 2019-08-18 has a NULL pointer dereference in idct2d8x8() at dct.c.
CVE-2019-16351
PUBLISHED: 2019-09-16
ffjpeg before 2019-08-18 has a NULL pointer dereference in huffman_decode_step() at huffman.c.
CVE-2019-16352
PUBLISHED: 2019-09-16
ffjpeg before 2019-08-21 has a heap-based buffer overflow in jfif_load() at jfif.c.
CVE-2016-10967
PUBLISHED: 2019-09-16
The real3d-flipbook-lite plugin 1.0 for WordPress has XSS via the wp-content/plugins/real3d-flipbook/includes/flipbooks.php bookId parameter.