Analytics
4/22/2014
12:00 PM
Joshua Goldfarb
Joshua Goldfarb
Commentary
Connect Directly
Twitter
RSS
E-Mail
50%
50%

7 Tips To Improve 'Signal-to-Noise' In The SOC

When security analysts are desensitized to alerts because of sheer volume, they miss the true positives that can prevent a large-scale data breach. Here's how to up your game.

Reports on the recent Target and Neiman Marcus breaches have indicated that, in both cases, there were numerous alerts fired as a result of the intrusion activity. Yet, according to news accounts, the alerts were not properly handled, allowing system compromises to go undetected and giving attackers the crucial footholds needed to pull off large-scale breaches.

Though the exact reasons for these oversights remain unclear, it is likely that security operation centers (SOCs) responsible for monitoring retailers' networks had low "signal-to-noise ratios." By that, I mean that analysts in the SOC were adept at collecting vast amounts of security information, but they faced challenges in discerning the most severe, imminent threats -- and responding to them in an effective, timely manner.

A low signal-to-noise ratio means that, more than likely, these SOCs had daily work queues inundated with false positives, to the point where true positives were lost in the clutter of noise. Commonly in these scenarios, analysts attempt to review only the alerts of the highest priority. But because of the large volume of even the highest priority-flagged alerts, analysts are not able to successfully review all of them. When analysts become desensitized to alerts because of the sheer volume of false positives, they tend to miss the true positives.

What surprises me is not that organizations have a low signal-to-noise ratio, but rather, the fundamental assumption that it has to be this way. I know from experience that there is a better way. Here are seven tips that have worked well for me throughout my career:

Tip No. 1: Go for the "Money Shot." It's possible to detect attacks at multiple stages (e.g., exploit, payload delivery, command and control, etc.) using indicators of compromise (IoCs). Different stages of attack will have different false positive ratios, and a given attack may have multiple IoCs that can be used to identify it. Whenever possible, the most reliable IoCs with the lowest false positive rates should be used to benefit the signal-to-noise ratio.

Tip No. 2: Take a scalpel approach. Alerting technologies have tremendous potential to identify suspicious or malicious activity when used like a scalpel. Unfortunately, many organizations use them less like a scalpel and more like a hatchet. It pays to assess risk, security needs, operational needs, and business needs at each deployment location and focus alerting technologies selectively. Singling out only the most relevant alerts lowers the number of false positives and improves the signal-to-noise ratio.

Tip No. 3: Use correlation. Sometimes, an individual alert is not particularly interesting until observed in conjunction with one or more other alerts or activity of interest. In those cases, the alert should be sent to the work queue only when all the correlation criteria are met. This increases the fidelity of the alerts and reduces the number of false positives, both of which help the signal-to-noise ratio.

Tip No. 4: Write intelligent alerting. As the saying goes, "Ask a stupid question, get a stupid answer." Today's threats are sophisticated and require intelligent, targeted, incisive alert logic to extract activity of concern while minimizing false positives. In other words, today's threats require intelligent alert logic. Working to tighten this logic goes a long way toward an optimal signal-to-noise ratio.

Tip No. 5: Be picky with intelligence. Different intelligence sources will bring different fidelity, relevance, and value to security operations. Blindly integrating intelligence feeds without evaluating their fidelity and false positive rates can have a detrimental effect on security operations by significantly lowering the signal-to-noise ratio.

Tip No. 6: Prioritize appropriately. Prioritization is one of the greatest tools a security team can utilize. Alerts that are of higher fidelity and detect higher-risk activity can be assigned a higher priority, while alerts that are of lower fidelity and detect lower-risk activity can be assigned a lower priority. Analysts work the queue from highest priority to lowest priority, ensuring that the most reliable alerts covering the activity of the greatest risk are addressed first.

Tip No. 7: Review every alert. Why fill the queue with alerts that are never reviewed? The whole point is to draw the attention of an analyst for review. Using the techniques I've described, an organization can regain control of its queue with the ultimate goal that every alert will be reviewed. This ensures that nothing is overlooked.

(Source: Wikipedia)
(Source: Wikipedia)

To illustrate these concepts, consider the case of SOC A and SOC B. In SOC A, the daily work queue contains approximately 100 reliable, high-fidelity, usable alerts. Each one is reviewed by an analyst. If incident response is necessary for a given alert, it is performed.

In SOC B, the daily work queue contains approximately 100,000 alerts, almost all of which are false positives. Analysts attempt to review them according to priority. Because of the large volume (even for alerts of the highest priority), analysts cannot successfully review all of the highest-priority alerts. Additionally, because of the large number of false positives, SOC B's analysts become desensitized to alerts and do not take them particularly seriously.

One day, 10 additional alerts relating to payment-card stealing malware fire within a few minutes of one another.

In SOC A, where every alert is reviewed by an analyst, where the signal-to-noise ratio is high, and where 10 additional alerts seem like a lot, analysts successfully identify the breach in less than 24 hours. SOC A's team can perform analysis, containment, and remediation within the first 24 hours of the breach. The team can stop the bleeding before any payment card data is exfiltrated. Though there has been some damage, it can be controlled. The organization can assess the damage, respond appropriately, and return to normal business operations.

In SOC B, where an extremely small percentage of the alerts are reviewed by an analyst, where the signal-to-noise ratio is low, and where 10 additional alerts doesn't even raise an eyebrow, the breach remains undetected. Months later, SOC B will learn of the breach from a third party. The damage will be extensive, and full recovery will take months or years.

Which SOC would you rather have inside your organization?

Joshua Goldfarb is an experienced cyber security analyst with over a decade of experience building, operating, and running security operations centers (SOCs). Before joining nPulse Technologies as its Chief Security Officer, he worked as an independent consultant, applying ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
jg@npulsetech.com
50%
50%
jg@npulsetech.com,
User Rank: Apprentice
4/23/2014 | 9:58:54 AM
Re: Lowering the noise level -- gut feelings
Another excellent question Marilyn, thank you.  Gut feelings, or put another way, intuition, can certainly catalyze and inspire investigation and analysis.  Not every feeling or intuition is going to lead in a productive direction of course, and only experience can really help determine what may be a promising intuition vs. what may not be a good use of resources.  For example, it is reasonable to say "I don't think we have any legitimate business traffic to any .ce.ms domains."  Following this hypothesis, the organization should seek the ground truth that is contained in the data.  In my experience, what creates a dangerous feeling or intuition is when decisions are based solely on that feeling or intuition.  If decisions are made based upon investigation and analysis of the data catalyzed by those initial feelings or intuitions, then that can be a good thing in my opinion.
Marilyn Cohodas
50%
50%
Marilyn Cohodas,
User Rank: Strategist
4/23/2014 | 9:45:40 AM
Re: Lowering the noise level -- gut feelings
Phrases like "this feels like it's less actionable" or "I think this is always a false positive" can be dangerous.

How do you quantify those "gut feelings." And is there no place for them in the SOC?
jg@npulsetech.com
50%
50%
jg@npulsetech.com,
User Rank: Apprentice
4/22/2014 | 2:07:52 PM
Re: Lowering the noise level
Excellent question, Marilyn, and thank you.  In my experience, organizations too often take an "all or nothing" approach to managing the noise level.  For example, an organization may say "we get too many alerts from source X, so we are just going to ignore or turn off all alerts from source X."  I think a better approach would be "let's think about what business, operational, and security needs we can address through alerts from source X and tune source X delicately to address those needs."  Additionally, organizations will sometimes emote or intuit noise reduction.  Phrases like "this feels like it's less actionable" or "I think this is always a false positive" can be dangerous.  The best source for educated decisions is the data.  The data do not lie and form the basis for good security decision making (and the tips provided in the article of course).
Marilyn Cohodas
50%
50%
Marilyn Cohodas,
User Rank: Strategist
4/22/2014 | 12:43:17 PM
Lowering the noise level
Thanks for sharing you wisdom with the Dark Reading community Joshua. In your experience, what do you think are the biggest mistakes SOCs make in loweirng the noise level?
Register for Dark Reading Newsletters
White Papers
Cartoon
Current Issue
Dark Reading, January 2015
To find and fix exploits aimed directly at your business, stop waiting for alerts and become a proactive hunter.
Flash Poll
Threat Intel Today
Threat Intel Today
The 397 respondents to our new survey buy into using intel to stay ahead of attackers: 85% say threat intelligence plays some role in their IT security strategies, and many of them subscribe to two or more third-party feeds; 10% leverage five or more.
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2013-7402
Published: 2014-12-17
Multiple unspecified vulnerabilities in request.c in c-icap 0.2.x allow remote attackers to cause a denial of service (crash) via a crafted ICAP request.

CVE-2014-5437
Published: 2014-12-17
Multiple cross-site request forgery (CSRF) vulnerabilities in ARRIS Touchstone TG862G/CT Telephony Gateway with firmware 7.6.59S.CT and earlier allow remote attackers to hijack the authentication of administrators for requests that (1) enable remote management via a request to remote_management.php,...

CVE-2014-5438
Published: 2014-12-17
Cross-site scripting (XSS) vulnerability in ARRIS Touchstone TG862G/CT Telephony Gateway with firmware 7.6.59S.CT and earlier allows remote authenticated users to inject arbitrary web script or HTML via the computer_name parameter to connected_devices_computers_edit.php.

CVE-2014-7170
Published: 2014-12-17
Race condition in Puppet Server 0.2.0 allows local users to obtain sensitive information by accessing it in between package installation or upgrade and the start of the service.

CVE-2014-7285
Published: 2014-12-17
The management console on the Symantec Web Gateway (SWG) appliance before 5.2.2 allows remote authenticated users to execute arbitrary OS commands by injecting command strings into unspecified PHP scripts.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
Join us Wednesday, Dec. 17 at 1 p.m. Eastern Time to hear what employers are really looking for in a chief information security officer -- it may not be what you think.