Analytics

8/9/2016
11:15 AM
Giora Engel
Giora Engel
Commentary
Connect Directly
Facebook
Twitter
LinkedIn
Google+
RSS
E-Mail vvv
100%
0%

Building A Detection Strategy With The Right Metrics

The tools used in detecting intrusions can lead to an overwhelming number of alerts, but they're a vital part of security.

Traditional security used to be focused exclusively on prevention and policy enforcement. While these mechanisms are still important, there is increasing attention paid to detection in order to be able to more rapidly spot potentially malicious activities that have circumvented preventative security.

The leading security teams are shifting their mission to detect a potential attack and respond appropriately and quickly. To achieve such a mission, organizations typically implement the following detection (or incident response) process:

  1. Start with an alert.
  2. Investigate the alert and determine if it is benign or unwanted.
  3. Make a proper decision and remediate or respond.

This detection process can be implemented at different levels of detail in various security organizations. I have seen implementations conducted by a single-person security team all the way to large security operations centers.

Of course, an effective detection strategy isn’t easy to achieve with a limited security budget and constrained staff. A major factor determining the effectiveness of the detection process is that security teams have to deal with the massive inefficiency and imprecision of security tools. For most organizations, the total alert volume is far beyond what any group can handle. Most alerts aren’t useful because either they don’t highlight anything of importance or are nonspecific or unsubstantiated, making an already impossible situation even worse. This situation is created by three main factors.

First, most tools are still primarily malware-oriented and based on technical artifacts, such as a signature, hash, domain, or a predefined list of software behaviors. This means that alerts will be firing all the time.

Second, many tools are designed to detect isolated “trees” and don’t understand that they are part of an overall “forest” of an orchestrated campaign being run by an attacker. It’s common to see a singular event — let’s say a scan inside the network — set off a hundreds or even thousands of individual alerts.

Third, many traditional tools are based on a do-it-yourself mentality. They’re created assuming a completely manual style of operational security. Part of this recognizes that most tools are rather low in fidelity and not intended to offload the work of a security professional by conducting some level of investigation or making a judgment call.

If organizations are going to be successful with detection, they need to measure and improve their efficiency. The following questions can help develop an effective detection strategy, or evaluate a specific new or existing detection tool:

  1. Detection coverage: If an active attacker were operating inside my network, would my systems see an operational activity and set off an alert?
  2. Detection quantity: Can my team investigate all the relevant alerts? Is it clear which are relevant?
  3. Detection quality: When an alert is investigated, can I reach a conclusion?

Detection coverage is determined by what is being sought — is a tool looking for malware? Reconnaissance? Lateral movement? The best approach here is to define a few use cases that you’d like to detect and make sure that they’re covered. Most data breaches are ultimately the result of an attacker that operates inside the internal network. The use cases can either be defined theoretically or tested live as an attack simulation or penetration test.

Detection quality and quantity can be measured in a very concrete way using standardized metrics:

  1. Detection quantity: efficiency — Count the total number of alerts per day and calculate how many alerts you have per 1,000 hosts.
  2. Detection quality: accuracy — Percentage of alerts handled by the team that are useful (not whitelisted or ignored) over the total number of alerts.

A Ponemon Institute study of 630 enterprises showed that the average number of alerts a security team receives per week from their security tools — firewalls, intrusion-detection systems, intrusion-prevention systems, sandboxing systems, security information and event management (SIEM) systems, etc. — was a shocking 172 per 1,000 hosts per day (2,420 alerts for an average 14,000 endpoints in a network). Where does one start with that volume of alerts, particularly given that the large majority of these are false-positives? The same study shows that, on average, two-thirds of the security staff’s time is wasted because their tools are grossly inefficient, so only 4% of alerts are typically investigated, leaving the possibility that somewhere around 96% of those untouched may be something important.

This is clearly an inefficient, inaccurate process because there is no good way to guarantee a focus on the right alerts. There are examples where signs of an attacker were hidden in a flood of alerts. In the Target breach, according to reported details, signals were detected by existing systems but buried under thousands of similar alerts without any way to know that this particular alert was important.

It’s not enough to have a haystack with some needles that can only be discovered with hindsight after it’s too late; we need a process that can get to the important needles to proactively detect attacks, before damage is done. Realistically, security teams can actively handle a handful of alerts per 1,000 hosts per day. This is based on the Ponemon data (4% of handled events equals 6.8 alerts/1,000 hosts/day) and also on my experience with different chief information security officers. If a process generates many more, they will be ignored.

Using metrics enables evaluation and comparison of different tools and methods, including overlapping products. For example, you can compare SIEM-based event correlation to network traffic analytics, sandbox-based detection, user and entity behavior analytics, endpoint detection, malware analysis, and so on, with these questions:

  1. Does the type of new alerts add significant coverage or attack detection capability?
  2. How many alerts per day does the system normally generate? Can my team investigate that amount?
  3. What percentage of the alerts can be handled and are relevant to handle?

With this perspective, we can ask questions such as, “Will this increase detection accuracy?” “How is this security tool making the team more efficient?” and “Can this security solution reduce staffing requirements?”

Related Content:

 

Giora Engel, vice president, product & strategy at LightCyber is a serial entrepreneur with many years of technological and managerial experience. For nearly a decade, he served as an officer in an elite technological unit in the Israel Defense Forces, where he initiated and ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Why Cybersecurity Must Be an International Effort
Kelly Sheridan, Associate Editor, Dark Reading,  12/6/2017
NIST Releases New Cybersecurity Framework Draft
Jai Vijayan, Freelance writer,  12/6/2017
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
The Year in Security: 2017
A look at the biggest news stories (so far) of 2017 that shaped the cybersecurity landscape -- from Russian hacking, ransomware's coming-out party, and voting machine vulnerabilities to the massive data breach of credit-monitoring firm Equifax.
Flash Poll
[Strategic Security Report] Assessing Cybersecurity Risk
[Strategic Security Report] Assessing Cybersecurity Risk
As cyber attackers become more sophisticated and enterprise defenses become more complex, many enterprises are faced with a complicated question: what is the risk of an IT security breach? This report delivers insight on how today's enterprises evaluate the risks they face. This report also offers a look at security professionals' concerns about a wide variety of threats, including cloud security, mobile security, and the Internet of Things.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2017-0290
Published: 2017-05-09
NScript in mpengine in Microsoft Malware Protection Engine with Engine Version before 1.1.13704.0, as used in Windows Defender and other products, allows remote attackers to execute arbitrary code or cause a denial of service (type confusion and application crash) via crafted JavaScript code within ...

CVE-2016-10369
Published: 2017-05-08
unixsocket.c in lxterminal through 0.3.0 insecurely uses /tmp for a socket file, allowing a local user to cause a denial of service (preventing terminal launch), or possibly have other impact (bypassing terminal access control).

CVE-2016-8202
Published: 2017-05-08
A privilege escalation vulnerability in Brocade Fibre Channel SAN products running Brocade Fabric OS (FOS) releases earlier than v7.4.1d and v8.0.1b could allow an authenticated attacker to elevate the privileges of user accounts accessing the system via command line interface. With affected version...

CVE-2016-8209
Published: 2017-05-08
Improper checks for unusual or exceptional conditions in Brocade NetIron 05.8.00 and later releases up to and including 06.1.00, when the Management Module is continuously scanned on port 22, may allow attackers to cause a denial of service (crash and reload) of the management module.

CVE-2017-0890
Published: 2017-05-08
Nextcloud Server before 11.0.3 is vulnerable to an inadequate escaping leading to a XSS vulnerability in the search module. To be exploitable a user has to write or paste malicious content into the search dialogue.