Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

11/6/2019
10:00 AM
Anton Chuvakin
Anton Chuvakin
Commentary
Connect Directly
LinkedIn
Twitter
RSS
E-Mail vvv
50%
50%

The Uphill Battle of Triaging Alerts

Prioritizing alerts is foundational to security, but almost every organization struggles to manage this process efficiently. Here's what you can do about it.

As organizations increasingly shift focus to threat detection and response, there's one issue that seems to get worse over time: alert triage. Prioritizing security alerts has been a critical function for security teams since the early 1990s — so why does it remain such a challenge decades later?

Security teams are overwhelmed by daily alerts from security information and event management (SIEM), endpoint detection and response (EDR), and other detection tools, mostly due to the sheer volume of alerts generated, which is growing all the time, along with a rise in low-quality alerts and false positives. Many teams find they can only analyze a fraction of the thousands of alerts that come in each day, leaving threats that often go unnoticed for months.

This problem is believed to have played a part in Target's massive data breach in 2013. The company's security team reportedly failed to take action after detecting potentially malicious activity. Experts speculated that Target had been receiving hundreds of alerts every day that were mistakenly dismissed as false. 

Alert triaging has become even more challenging over the past decade as the attack surface has grown. While Windows and Linux server threats were once the main focus of attackers, cloud, mobile and Internet of Things technologies have given rise to their own threat detection needs, leading to new massive piles of alerts. Alert triaging is foundational, yet almost every organization struggles to manage it efficiently. 

What's standing in the way? Here are four issues:

Issue 1: Today's environments are more diverse.
In the past, when everyone used the same server operating system, one firm could create a flowchart for its alert response, target, and history, and it could apply to other organizations with the same system. This made it easier for security teams to address the volume of alerts that were similar. 

But these days, security tools and IT environments can range from advanced deception tools and machine learning, to traditional IT, cloud and industrial IoT technologies. One organization may still use on-premises mainframes, while another may be cloud-native, which makes having a unified approach impossible. 

Issue 2: Many advanced threats don't appear in alerts.
Even when you can successfully identify, analyze, and act on an alert, you must also account for the threats that you can't see. Ransomware, for example, clearly announces itself, but the process for other crimeware (and especially for stealthy state-sponsored threats) isn't as obvious. 

Sophisticated threat actors who can easily go undetected are on the rise. According to the Council on Foreign Relations (CFR), state-sponsored attacks have increased 122% from 2014 to 2018, and attackers have found ways to evade detection by hiding in encrypted traffic, which the Electronic Frontier Foundation now says comprises more than half of total traffic on the Web. These cybercriminals leverage SSL tunnels to sneak malware into the corporate network, hide commands, control traffic, and exfiltrate data. 

Even achieving alert triage perfection won't make detection and response capabilities truly world-class because attackers will always find ways to adapt to new detection tools and methods. This is forcing security teams to actively hunt for threats, rather than merely gathering (and then clearing) alerts.

Issue 3: Automation and machine learning tools have limits.
To create alert playbooks, some security teams lean on security orchestration, automation, and response (SOAR) technology. But too often, they buy these tools without understanding the prerequisites or even the reasoning behind their implementation. I've seen a security leader authorize the purchase of a SOAR tool to avoid coding a playbook, only to discover that the first task after implementation was to define his playbook in the tool via Python.

Furthermore, many detection technologies produce a litany of false positives and fail to completely cover monitored assets. I've worked with several security professionals who have grown frustrated with new anomaly-based systems that produce vague and untraceable warnings. In some cases, machine learning (ML)-based systems have actually produced more false positives than legacy tools do.

While security teams increasingly rely on ML technology to solve alert problems, I've also seen many reports that ML-produced alerts can be more difficult to confirm in real security operations, as they often lack transparency and context. For instance, an ML tool may spot a behavioral outlier, but it's unclear what the client should do next, where it should look, or who it should ask.  

Issue 4: If not robots, then what? 
"Security is a process, not a product." This quote from Bruce Schneier still stands. In theory, the solution is simple: Hire a small but skilled security team to define, refine, and implement mature alert response processes and customize tools and workflows to address alert overload. But in practice, the cost of assembling an expert team makes this option less feasible for many organizations. What we need is a combination of human brains confirming the alerts that are prepared by the machines in a way that's optimal for human decision.

To put this into practice, security teams should apply business context to alerts (such as the implications of an alert's severity), conduct alert triage steps outside of IT, and address the alerts presented by the system. Otherwise, you run the risk of missing important alerts and becoming the next massive data breach to dominate headlines. 

Related Content:

Check out The Edge, Dark Reading's new section for features, threat data, and in-depth perspectives. Today's top story: "Is Voting by Mobile App a Better Security Option or Just 'A Bad Idea'?"

Anton is a recognized security expert in the field of log management, SIEM and PCI DSS compliance. He is the author of several books and serves on advisory boards of several security startups. Before joining Chronicle, Anton was a research vice president and Distinguished ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
Major Brazilian Bank Tests Homomorphic Encryption on Financial Data
Kelly Sheridan, Staff Editor, Dark Reading,  1/10/2020
Exploits Released for As-Yet Unpatched Critical Citrix Flaw
Jai Vijayan, Contributing Writer,  1/13/2020
Microsoft Patches Windows Vuln Discovered by the NSA
Kelly Sheridan, Staff Editor, Dark Reading,  1/14/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Post a Comment
Current Issue
The Year in Security: 2019
This Tech Digest provides a wrap up and overview of the year's top cybersecurity news stories. It was a year of new twists on old threats, with fears of another WannaCry-type worm and of a possible botnet army of Wi-Fi routers. But 2019 also underscored the risk of firmware and trusted security tools harboring dangerous holes that cybercriminals and nation-state hackers could readily abuse. Read more.
Flash Poll
[Just Released] How Enterprises are Attacking the Cybersecurity Problem
[Just Released] How Enterprises are Attacking the Cybersecurity Problem
Organizations have invested in a sweeping array of security technologies to address challenges associated with the growing number of cybersecurity attacks. However, the complexity involved in managing these technologies is emerging as a major problem. Read this report to find out what your peers biggest security challenges are and the technologies they are using to address them.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-3683
PUBLISHED: 2020-01-17
The keystone-json-assignment package in SUSE Openstack Cloud 8 before commit d7888c75505465490250c00cc0ef4bb1af662f9f every user listed in the /etc/keystone/user-project-map.json was assigned full "member" role access to every project. This allowed these users to access, modify, create and...
CVE-2019-3682
PUBLISHED: 2020-01-17
The docker-kubic package in SUSE CaaS Platform 3.0 before 17.09.1_ce-7.6.1 provided access to an insecure API locally on the Kubernetes master node.
CVE-2019-17361
PUBLISHED: 2020-01-17
In SaltStack Salt through 2019.2.0, the salt-api NEST API with the ssh client enabled is vulnerable to command injection. This allows an unauthenticated attacker with network access to the API endpoint to execute arbitrary code on the salt-api host.
CVE-2019-19142
PUBLISHED: 2020-01-17
Intelbras WRN240 devices do not require authentication to replace the firmware via a POST request to the incoming/Firmware.cfg URI.
CVE-2019-19801
PUBLISHED: 2020-01-17
In Gallagher Command Centre Server versions of v8.10 prior to v8.10.1134(MR4), v8.00 prior to v8.00.1161(MR5), v7.90 prior to v7.90.991(MR5), v7.80 prior to v7.80.960(MR2) and v7.70 or earlier, an unprivileged but authenticated user is able to perform a backup of the Command Centre databases.