Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Analytics

8/9/2016
11:15 AM
Giora Engel
Giora Engel
Commentary
Connect Directly
Facebook
Twitter
LinkedIn
Google+
RSS
E-Mail vvv
100%
0%

Building A Detection Strategy With The Right Metrics

The tools used in detecting intrusions can lead to an overwhelming number of alerts, but they're a vital part of security.

Traditional security used to be focused exclusively on prevention and policy enforcement. While these mechanisms are still important, there is increasing attention paid to detection in order to be able to more rapidly spot potentially malicious activities that have circumvented preventative security.

The leading security teams are shifting their mission to detect a potential attack and respond appropriately and quickly. To achieve such a mission, organizations typically implement the following detection (or incident response) process:

  1. Start with an alert.
  2. Investigate the alert and determine if it is benign or unwanted.
  3. Make a proper decision and remediate or respond.

This detection process can be implemented at different levels of detail in various security organizations. I have seen implementations conducted by a single-person security team all the way to large security operations centers.

Of course, an effective detection strategy isn’t easy to achieve with a limited security budget and constrained staff. A major factor determining the effectiveness of the detection process is that security teams have to deal with the massive inefficiency and imprecision of security tools. For most organizations, the total alert volume is far beyond what any group can handle. Most alerts aren’t useful because either they don’t highlight anything of importance or are nonspecific or unsubstantiated, making an already impossible situation even worse. This situation is created by three main factors.

First, most tools are still primarily malware-oriented and based on technical artifacts, such as a signature, hash, domain, or a predefined list of software behaviors. This means that alerts will be firing all the time.

Second, many tools are designed to detect isolated “trees” and don’t understand that they are part of an overall “forest” of an orchestrated campaign being run by an attacker. It’s common to see a singular event — let’s say a scan inside the network — set off a hundreds or even thousands of individual alerts.

Third, many traditional tools are based on a do-it-yourself mentality. They’re created assuming a completely manual style of operational security. Part of this recognizes that most tools are rather low in fidelity and not intended to offload the work of a security professional by conducting some level of investigation or making a judgment call.

If organizations are going to be successful with detection, they need to measure and improve their efficiency. The following questions can help develop an effective detection strategy, or evaluate a specific new or existing detection tool:

  1. Detection coverage: If an active attacker were operating inside my network, would my systems see an operational activity and set off an alert?
  2. Detection quantity: Can my team investigate all the relevant alerts? Is it clear which are relevant?
  3. Detection quality: When an alert is investigated, can I reach a conclusion?

Detection coverage is determined by what is being sought — is a tool looking for malware? Reconnaissance? Lateral movement? The best approach here is to define a few use cases that you’d like to detect and make sure that they’re covered. Most data breaches are ultimately the result of an attacker that operates inside the internal network. The use cases can either be defined theoretically or tested live as an attack simulation or penetration test.

Detection quality and quantity can be measured in a very concrete way using standardized metrics:

  1. Detection quantity: efficiency — Count the total number of alerts per day and calculate how many alerts you have per 1,000 hosts.
  2. Detection quality: accuracy — Percentage of alerts handled by the team that are useful (not whitelisted or ignored) over the total number of alerts.

A Ponemon Institute study of 630 enterprises showed that the average number of alerts a security team receives per week from their security tools — firewalls, intrusion-detection systems, intrusion-prevention systems, sandboxing systems, security information and event management (SIEM) systems, etc. — was a shocking 172 per 1,000 hosts per day (2,420 alerts for an average 14,000 endpoints in a network). Where does one start with that volume of alerts, particularly given that the large majority of these are false-positives? The same study shows that, on average, two-thirds of the security staff’s time is wasted because their tools are grossly inefficient, so only 4% of alerts are typically investigated, leaving the possibility that somewhere around 96% of those untouched may be something important.

This is clearly an inefficient, inaccurate process because there is no good way to guarantee a focus on the right alerts. There are examples where signs of an attacker were hidden in a flood of alerts. In the Target breach, according to reported details, signals were detected by existing systems but buried under thousands of similar alerts without any way to know that this particular alert was important.

It’s not enough to have a haystack with some needles that can only be discovered with hindsight after it’s too late; we need a process that can get to the important needles to proactively detect attacks, before damage is done. Realistically, security teams can actively handle a handful of alerts per 1,000 hosts per day. This is based on the Ponemon data (4% of handled events equals 6.8 alerts/1,000 hosts/day) and also on my experience with different chief information security officers. If a process generates many more, they will be ignored.

Using metrics enables evaluation and comparison of different tools and methods, including overlapping products. For example, you can compare SIEM-based event correlation to network traffic analytics, sandbox-based detection, user and entity behavior analytics, endpoint detection, malware analysis, and so on, with these questions:

  1. Does the type of new alerts add significant coverage or attack detection capability?
  2. How many alerts per day does the system normally generate? Can my team investigate that amount?
  3. What percentage of the alerts can be handled and are relevant to handle?

With this perspective, we can ask questions such as, “Will this increase detection accuracy?” “How is this security tool making the team more efficient?” and “Can this security solution reduce staffing requirements?”

Related Content:

 

Giora Engel, vice president, product & strategy at LightCyber is a serial entrepreneur with many years of technological and managerial experience. For nearly a decade, he served as an officer in an elite technological unit in the Israel Defense Forces, where he initiated and ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 8/3/2020
Pen Testers Who Got Arrested Doing Their Jobs Tell All
Kelly Jackson Higgins, Executive Editor at Dark Reading,  8/5/2020
New 'Nanodegree' Program Provides Hands-On Cybersecurity Training
Nicole Ferraro, Contributing Writer,  8/3/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
Special Report: Computing's New Normal, a Dark Reading Perspective
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
The Changing Face of Threat Intelligence
The Changing Face of Threat Intelligence
This special report takes a look at how enterprises are using threat intelligence, as well as emerging best practices for integrating threat intel into security operations and incident response. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-15820
PUBLISHED: 2020-08-08
In JetBrains YouTrack before 2020.2.6881, the markdown parser could disclose hidden file existence.
CVE-2020-15821
PUBLISHED: 2020-08-08
In JetBrains YouTrack before 2020.2.6881, a user without permission is able to create an article draft.
CVE-2020-15823
PUBLISHED: 2020-08-08
JetBrains YouTrack before 2020.2.8873 is vulnerable to SSRF in the Workflow component.
CVE-2020-15824
PUBLISHED: 2020-08-08
In JetBrains Kotlin before 1.4.0, there is a script-cache privilege escalation vulnerability due to kotlin-main-kts cached scripts in the system temp directory, which is shared by all users by default.
CVE-2020-15825
PUBLISHED: 2020-08-08
In JetBrains TeamCity before 2020.1, users with the Modify Group permission can elevate other users' privileges.