Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Risk

4/30/2019
10:00 AM
50%
50%

Threat Intelligence Firms Look to AI, but Still Require Humans

Machine learning and artificial intelligence are helping threat-intelligence firms cover a greater area of the darknet, but human analysts will always be necessary, experts say.

Threat intelligence firms are racing to expand their machine-learning capabilities to capture more of the un-indexed parts of the internet, but somewhat ironically, human analysts and experts remain critical to the effort. 

Last week, startup Resecurity announced its intent to index the entire dark web by 2020—petabytes of information. The effort would use machine learning and big data technologies, but Resecurity also specifically called out the necessity of human analysts.

The same week, rival threat intelligence firm Flashpoint announced an expansion to its service, adding more dark-web sources, underground marketplaces and encrypted forums. While the capabilities are fueled by advances in machine learning, human analysts remain key, says Flashpoint CEO Josh Lefkowitz.

"There is a tenuous and temporary nature to many of these sources, and that is what necessitates the agility of human involvement because the actors are constantly moving around," Lefkowitz says.

The amount of data created by companies has exploded as more enterprise infrastructure connects to the internet and produces logging information and other data. The availability of data on every facet of the business—and on consumers' habits—has powered the application of machine learning and artificial intelligence to a variety of new use cases.

Threat intelligence has quickly become a way for companies to keep abreast of vulnerabilities that impact their systems, breaches that affect their customers, and potential attacker interest. More than half of companies are using threat intelligence to improve their detection of attacks, according to a recent report from CyberEdge. A similar majority also used threat intelligence to help validate security alerts and signs of an attack and reduce the workload on their analysts, the same report stated.

"It is a two-pronged approach—you want to have a lot of tools in your toolbox," says Daniel Hatheway, senior technical analyst at digital intelligence firm Recorded Future. "It is more about creating transparency around what dangers the dark web does pose."

Most threat intelligence firms are focused on expanding access to the types of information which matter to clients. Flashpoint, for example, just announced that the company has added more robust vulnerability data, compromised account data from underground shops, and information from secured and encrypted forums.

To access that last category and refine the other data sets, they need human analysts, Lefkowitz says.

"If you just have automated scrapers that are spidering the entirety of the onion network—when the reality is that many of these environments are password-protected and they necessitate invite links that you receive because of your credibility and reputation in the community—you are missing so much if you are just relying on automation," he says.

Employees and threat analysts are, of course, expensive. So companies continue to look for ways to automate as much of the process as possible. Resecurity, for example, has focused on capturing as much of the dark web as possible for later inspection by clients and analysts.

"There have been attempts in the past to scan the dark web, but the tools that were available when those efforts were undertaken were extremely primitive," Gene Yoo, chief executive officer of Resecurity, said in a statement. "They generated a lot of false positives and noise—and not a lot of truly actionable intelligence. To deliver the maximum visibility into the dark web, to get to the point where we can associate a particular threat actor with his real identity, we need to apply the power of data science and big data."

Relying on humans for security is nothing new. The majority of companies heavily rely on employees to identify threats, with 85 percent of firms saying employees fill a significant role in their defense against cyber attacks, according to a survey conducted by security firm Glasswall Solutions. In fact, more than 40 percent of companies surveyed by Glasswall Solutions stated that they were "completely reliant on employees as their last line of defense," Glasswall stated in a release published along with the report.

It's unlikely that threat intelligence firms will ever be able to do away with human analysts. In fact, as companies' web scraping systems produce more potential alerts, connecting disparate pieces of information, the demand for analysts will likely increase.

"Part of this is recognizing that illicit actors continue to diversify where they are operating, necessitating an expansion in the aperture of our collections, of our capabilities," says Flashpoint's Lefkowitz. "It is that chess game, that cat-and-mouse game, … and we are only seeing that trend continue now as more technologies are at their fingertips that let them pursue their illicit goals."

 

Related Links

 

 

 

 

Join Dark Reading LIVE for two cybersecurity summits at Interop 2019. Learn from the industry's most knowledgeable IT security experts. Check out the Interop agenda here.

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
When It Comes To Security Tools, More Isn't More
Lamont Orange, Chief Information Security Officer at Netskope,  1/11/2021
US Capitol Attack a Wake-up Call for the Integration of Physical & IT Security
Seth Rosenblatt, Contributing Writer,  1/11/2021
IoT Vendor Ubiquiti Suffers Data Breach
Dark Reading Staff 1/11/2021
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
2020: The Year in Security
Download this Tech Digest for a look at the biggest security stories that - so far - have shaped a very strange and stressful year.
Flash Poll
Assessing Cybersecurity Risk in Today's Enterprises
Assessing Cybersecurity Risk in Today's Enterprises
COVID-19 has created a new IT paradigm in the enterprise -- and a new level of cybersecurity risk. This report offers a look at how enterprises are assessing and managing cyber-risk under the new normal.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-25533
PUBLISHED: 2021-01-15
An issue was discovered in Malwarebytes before 4.0 on macOS. A malicious application was able to perform a privileged action within the Malwarebytes launch daemon. The privileged service improperly validated XPC connections by relying on the PID instead of the audit token. An attacker can construct ...
CVE-2021-3162
PUBLISHED: 2021-01-15
Docker Desktop Community before 2.5.0.0 on macOS mishandles certificate checking, leading to local privilege escalation.
CVE-2021-21242
PUBLISHED: 2021-01-15
OneDev is an all-in-one devops platform. In OneDev before version 4.0.3, there is a critical vulnerability which can lead to pre-auth remote code execution. AttachmentUploadServlet deserializes untrusted data from the `Attachment-Support` header. This Servlet does not enforce any authentication or a...
CVE-2021-21245
PUBLISHED: 2021-01-15
OneDev is an all-in-one devops platform. In OneDev before version 4.0.3, AttachmentUploadServlet also saves user controlled data (`request.getInputStream()`) to a user specified location (`request.getHeader("File-Name")`). This issue may lead to arbitrary file upload which can be used to u...
CVE-2021-21246
PUBLISHED: 2021-01-15
OneDev is an all-in-one devops platform. In OneDev before version 4.0.3, the REST UserResource endpoint performs a security check to make sure that only administrators can list user details. However for the `/users/` endpoint there are no security checks enforced so it is possible to retrieve ar...