Analytics

8/9/2016
11:15 AM
Giora Engel
Giora Engel
Commentary
Connect Directly
Facebook
Twitter
LinkedIn
Google+
RSS
E-Mail vvv
100%
0%

Building A Detection Strategy With The Right Metrics

The tools used in detecting intrusions can lead to an overwhelming number of alerts, but they're a vital part of security.

Traditional security used to be focused exclusively on prevention and policy enforcement. While these mechanisms are still important, there is increasing attention paid to detection in order to be able to more rapidly spot potentially malicious activities that have circumvented preventative security.

The leading security teams are shifting their mission to detect a potential attack and respond appropriately and quickly. To achieve such a mission, organizations typically implement the following detection (or incident response) process:

  1. Start with an alert.
  2. Investigate the alert and determine if it is benign or unwanted.
  3. Make a proper decision and remediate or respond.

This detection process can be implemented at different levels of detail in various security organizations. I have seen implementations conducted by a single-person security team all the way to large security operations centers.

Of course, an effective detection strategy isn’t easy to achieve with a limited security budget and constrained staff. A major factor determining the effectiveness of the detection process is that security teams have to deal with the massive inefficiency and imprecision of security tools. For most organizations, the total alert volume is far beyond what any group can handle. Most alerts aren’t useful because either they don’t highlight anything of importance or are nonspecific or unsubstantiated, making an already impossible situation even worse. This situation is created by three main factors.

First, most tools are still primarily malware-oriented and based on technical artifacts, such as a signature, hash, domain, or a predefined list of software behaviors. This means that alerts will be firing all the time.

Second, many tools are designed to detect isolated “trees” and don’t understand that they are part of an overall “forest” of an orchestrated campaign being run by an attacker. It’s common to see a singular event — let’s say a scan inside the network — set off a hundreds or even thousands of individual alerts.

Third, many traditional tools are based on a do-it-yourself mentality. They’re created assuming a completely manual style of operational security. Part of this recognizes that most tools are rather low in fidelity and not intended to offload the work of a security professional by conducting some level of investigation or making a judgment call.

If organizations are going to be successful with detection, they need to measure and improve their efficiency. The following questions can help develop an effective detection strategy, or evaluate a specific new or existing detection tool:

  1. Detection coverage: If an active attacker were operating inside my network, would my systems see an operational activity and set off an alert?
  2. Detection quantity: Can my team investigate all the relevant alerts? Is it clear which are relevant?
  3. Detection quality: When an alert is investigated, can I reach a conclusion?

Detection coverage is determined by what is being sought — is a tool looking for malware? Reconnaissance? Lateral movement? The best approach here is to define a few use cases that you’d like to detect and make sure that they’re covered. Most data breaches are ultimately the result of an attacker that operates inside the internal network. The use cases can either be defined theoretically or tested live as an attack simulation or penetration test.

Detection quality and quantity can be measured in a very concrete way using standardized metrics:

  1. Detection quantity: efficiency — Count the total number of alerts per day and calculate how many alerts you have per 1,000 hosts.
  2. Detection quality: accuracy — Percentage of alerts handled by the team that are useful (not whitelisted or ignored) over the total number of alerts.

A Ponemon Institute study of 630 enterprises showed that the average number of alerts a security team receives per week from their security tools — firewalls, intrusion-detection systems, intrusion-prevention systems, sandboxing systems, security information and event management (SIEM) systems, etc. — was a shocking 172 per 1,000 hosts per day (2,420 alerts for an average 14,000 endpoints in a network). Where does one start with that volume of alerts, particularly given that the large majority of these are false-positives? The same study shows that, on average, two-thirds of the security staff’s time is wasted because their tools are grossly inefficient, so only 4% of alerts are typically investigated, leaving the possibility that somewhere around 96% of those untouched may be something important.

This is clearly an inefficient, inaccurate process because there is no good way to guarantee a focus on the right alerts. There are examples where signs of an attacker were hidden in a flood of alerts. In the Target breach, according to reported details, signals were detected by existing systems but buried under thousands of similar alerts without any way to know that this particular alert was important.

It’s not enough to have a haystack with some needles that can only be discovered with hindsight after it’s too late; we need a process that can get to the important needles to proactively detect attacks, before damage is done. Realistically, security teams can actively handle a handful of alerts per 1,000 hosts per day. This is based on the Ponemon data (4% of handled events equals 6.8 alerts/1,000 hosts/day) and also on my experience with different chief information security officers. If a process generates many more, they will be ignored.

Using metrics enables evaluation and comparison of different tools and methods, including overlapping products. For example, you can compare SIEM-based event correlation to network traffic analytics, sandbox-based detection, user and entity behavior analytics, endpoint detection, malware analysis, and so on, with these questions:

  1. Does the type of new alerts add significant coverage or attack detection capability?
  2. How many alerts per day does the system normally generate? Can my team investigate that amount?
  3. What percentage of the alerts can be handled and are relevant to handle?

With this perspective, we can ask questions such as, “Will this increase detection accuracy?” “How is this security tool making the team more efficient?” and “Can this security solution reduce staffing requirements?”

Related Content:

 

Giora Engel, vice president, product & strategy at LightCyber is a serial entrepreneur with many years of technological and managerial experience. For nearly a decade, he served as an officer in an elite technological unit in the Israel Defense Forces, where he initiated and ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
New Bluetooth Hack Affects Millions of Vehicles
Dark Reading Staff 11/16/2018
Understanding Evil Twin AP Attacks and How to Prevent Them
Ryan Orsi, Director of Product Management for Wi-Fi at WatchGuard Technologies,  11/14/2018
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
Flash Poll
How Enterprises Are Attacking the Cybersecurity Problem
How Enterprises Are Attacking the Cybersecurity Problem
Data breach fears and the need to comply with regulations such as GDPR are two major drivers increased spending on security products and technologies. But other factors are contributing to the trend as well. Find out more about how enterprises are attacking the cybersecurity problem by reading our report today.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2018-19406
PUBLISHED: 2018-11-21
kvm_pv_send_ipi in arch/x86/kvm/lapic.c in the Linux kernel through 4.19.2 allows local users to cause a denial of service (NULL pointer dereference and BUG) via crafted system calls that reach a situation where the apic map is uninitialized.
CVE-2018-19407
PUBLISHED: 2018-11-21
The vcpu_scan_ioapic function in arch/x86/kvm/x86.c in the Linux kernel through 4.19.2 allows local users to cause a denial of service (NULL pointer dereference and BUG) via crafted system calls that reach a situation where ioapic is uninitialized.
CVE-2018-19404
PUBLISHED: 2018-11-21
In YXcms 1.4.7, protected/apps/appmanage/controller/indexController.php allow remote authenticated Administrators to execute any PHP code by creating a ZIP archive containing a config.php file, hosting the .zip file at an external URL, and visiting index.php?r=appmanage/index/onlineinstall&url= ...
CVE-2018-19387
PUBLISHED: 2018-11-20
format_cb_pane_tabs in format.c in tmux 2.7 through 2.8 might allow attackers to cause a denial of service (NULL Pointer Dereference and application crash) by arranging for a malloc failure.
CVE-2018-19388
PUBLISHED: 2018-11-20
FoxitReader.exe in Foxit Reader 9.3.0.10826 allows remote attackers to cause a denial of service (out-of-bounds read, access violation, and application crash) via TIFF data because of a ConvertToPDF_x86!ReleaseFXURLToHtml issue.