Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Analytics

5/15/2014
12:00 PM
Levi Gundert
Levi Gundert
Commentary
Connect Directly
Twitter
LinkedIn
RSS
E-Mail vvv
50%
50%

Beware Cognitive Bias

Cognitive bias can compromise any profession. But when cognitive bias goes unrecognized in cyber security, far-reaching and serious consequences follow.

I distinctly remember the instructor’s mix of exasperation and inquisition. “He was acting suspicious?!” I was sitting in a Georgia classroom at the Federal Law Enforcement Training Center (FLETC) listening to a critique of my latest probable cause affidavit. I instantly realized my mistake. In the training scenario, the suspect was wearing warm clothing on a hot day, pacing, and avoiding eye contact during questioning. Instead of limiting the affidavit to my direct observations and detailing the suspect’s behavior, I inserted what I thought was a pithy conclusion.  

Cognitive bias affects everyone, and behavioral economists are continuously documenting its societal effects. Every profession likewise is influenced by cognitive bias. While the effects in criminal investigations are well documented, cyber security is a similar domain where the tendency to misinterpret data often leads to fallacious conclusions.  

A casual look at the list of cognitive biases should give you pause: anchoring, belief bias, confirmation bias, distinction bias, focusing effect, irrational escalation, and the list continues. We are all predisposed to these biases and tend to be overcritical of their effects in others, while minimizing their impact upon our own analytic faculties. For example, a few months ago I found myself examining a malware campaign that used multiple domains, all of which were registered with a Seychelles (an archipelago off of Africa’s Eastern coast) address. 

As I contemplated the Seychelles’ population size, I realized that I had recently observed additional malicious activity tied to Seychelles WHOIS registrant data. Similarly, I decided that those registering domains with Panamanian addresses also fit my evil-perpetrating model, based on prior knowledge and experience.

Thus with the help of my colleagues Jaeson Schultz and Andrew Tsonchev I collected all new domains registered with Seychelles or Panama addresses in the prior seven months and identified the incidence of customer Web blocks (Cloud Web Security). While I was confident we would find a block rate over 50%, the results did not support my assertion. Out of 19,557 Seychelles registrant domains, we blocked 337, which means less than 1% (.02%) were actually participating in malicious Web activity. The results were similar for Panama registrant domains. To be sure, we queried the same list of domains three months later to account for potential latency between domain registration and malicious use, and the results were consistent with our first query.

Now, data sources certainly matter. In this case the original domain lists may have been incomplete, and the domains may have been used for malicious campaigns in additional channels such as email. Regardless, I expected a high incidence of Web maliciousness based on a cognitive bias, specifically a confirmation bias.

In the realms of threat intelligence, incident response, and general network security monitoring, our profession suffers from cognitive biases just like any other profession. Yet the consequences of unrecognized cognitive biases in cyber security (and the resulting incorrect conclusions) may be more powerful and further reaching at this point in history.

How do companies compete with governments that are stealing intellectual property for economic competitive advantage? It’s a tough question, and before strategies are formed, corporate officers and board members first need to be able to answer with confidence the question: “How do we know who is behind this attack?” Sovereign nations have been asking the same question for millennia, but the Internet now facilitates a constant connection and higher degree of anonymity for talented and clever threat actors. Thus a centerpiece of foreign policy hinges on accurate conclusions driven by unbiased data analysis.

This is particularly true regarding attribution. Threat actors and cyber defenders operate in the context of a global Internet comprised of billions – soon to be trillions – of connected nodes. Identifying the person or party responsible for a specific cyber security event at a specific point in time is incredibly challenging, even for the most talented teams blessed with significant resources. This is true for every organization with an interest in identifying a deeper level of attribution, including geographic location and/or the individual or group responsible for a specific attack.

Last year Mandiant published the APT1 report -- a public watershed for cyber attack attribution -- which articulated the specific data and timeline that led to many of the report’s conclusions. Given the theme of the report, the supporting data was crucial to its credibility, and that data was not amassed overnight. If history is any indicator, successful attribution will continue to require prolonged time investments, sometimes even years.

Last year Sergio Caltagirone, Andrew Pendergast, and Christopher Betz released a paper entitled The Diamond Model of Intrusion Analysis. This remarkably succinct framework provides a consistent filter for malicious cyber event metadata. It is this type of framework that analysts must continually refer to while collecting and interpreting cyber attack data, in order to avoid unchecked cognitive bias.

Decision makers desperately need finished intelligence and logical assertions to plot the future course of military action, corporate policy, and foreign policy. Operationally this equates to domains, IP addresses, infrastructure owners, malicious code, etc., and the facts should perform a report’s summation. As analysts, we should not be inserting conjecture masquerading as fact into reports, because it is damaging to our industry and it impedes our ability to work toward a more secure Internet. If we fail to articulate the facts around a malicious cyber event properly, avoidable conflicts may ensue, and ultimately our entire industry loses trust and credibility.

Cognitive bias is rarely intentional, but hopefully we can continue to look for and confront our own analytical mistakes -- assisted by a reliable framework -- in order to produce a better security product (in any form). Industry and government decision makers and the general public will benefit, which should lead to improved education and efforts around the cyberthreat landscape we are daily confronting.

Levi Gundert is the vice president of intelligence and risk at Recorded Future where he leads the continuous effort to measurably decrease operational risk for customers. Levi has spent the past 20 years in both government and the private sector, defending networks, ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
cumulonimbus
50%
50%
cumulonimbus,
User Rank: Apprentice
5/21/2014 | 1:43:13 PM
Re: know your enemy
You make some good points about timing and involvement. I believe much more needs to be done to protect IP and other valuable data, particularly with offshoring and cloud. All too often (and all too late) companies find themselves vulnerable and are left with an extremely vexing problem. These problems should not occur, or at least be very rare, and are somewhat symptomatic of the C21 M.O., both in the private and public sectors. In some ways, it behooves us to think like a hacker. Ironically, in eastern philosophy our adversary is also our master.
levigundert
50%
50%
levigundert,
User Rank: Guru
5/21/2014 | 11:18:13 AM
Re: know your enemy
Certainly, though the accompanying Interpol Red Notice means that these suspects will be extradited if they ever travel.
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
5/21/2014 | 9:23:54 AM
Re: know your enemy
It appears the DOJ indictiment has opened the floodgates for more naming and shaming. The bad news is that many of the defendants will never be prosecuted, but the good news is that putting faces to the attacks raises awareness among businesses and the general public. 
levigundert
50%
50%
levigundert,
User Rank: Guru
5/21/2014 | 12:04:42 AM
Re: know your enemy
Thanks @kjhiggins. Beyond what I stated in the article, given the latest DOJ indictments of the five Chinese PLA employees, I hope there are new incentives to pursue attribution in conjunction with law enforcement.

I agree though that businesses are still struggling with the appropriate response after breaches, specifically around the decision (and timing) to involve law enforcement.
levigundert
50%
50%
levigundert,
User Rank: Guru
5/20/2014 | 11:47:33 PM
Re: know your enemy
Thank you for taking the time to read and comment, I appreciate the feedback!
levigundert
50%
50%
levigundert,
User Rank: Guru
5/20/2014 | 11:45:20 PM
Re: Keep it local
Thanks for taking the time to comment. I agree that objectivity does require an incredible amount of self discipline.
cumulonimbus
50%
50%
cumulonimbus,
User Rank: Apprentice
5/19/2014 | 8:26:09 AM
Keep it local
Cognitive bias, indeed any bias. is the natural order. It is how we think and how we came to be; a natural product of evolution. Objective observation requires an overwhelming act of self discipline. In the field of IT we are constantly dealing with the threat of cyber crime, thus trust in our IT personnel is paramount.

All that stands between us (our data) and them (the dark side of human behavior) is a false sense of security; an electronic barrier that ultimately cannot withstand penetration by a persistent and highly informed attack. The attraction of this kind of act is the anonymity and obscurity provided by the worldwide interconnection that is the internet. Notwithstanding the fact that over the shoulder attacks are probably the most frequent, our best defense lies in multi-factor authentication, personal representation (local and accountable human resources), multi-layer boundaries, and constant vigilance. Other than that, what is offline is, for the most part, no longer a target.
Bprince
50%
50%
Bprince,
User Rank: Ninja
5/18/2014 | 12:39:38 AM
Re: know your enemy
I would say that knowing the who is important, especially if we are talking about a national security issue/attack on the defense industry. I agree with the overall point of the article 100 percent. If there are going to be assertions made about who is responsible for an attack, the proof needs to be carefully vetted.

BP
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
5/16/2014 | 11:12:31 AM
know your enemy
Interesting piece, @LeviGundert. There are mixed perspectives among security vendors on how important it is to know the *who* (threat group/region) behind the attack versus the attackers' M.O. and what they are after. I wonder if that clouds the issue for enterprises trying to map out their security strategies and tools.
Data Privacy Protections for the Most Vulnerable -- Children
Dimitri Sirota, Founder & CEO of BigID,  10/17/2019
Sodinokibi Ransomware: Where Attackers' Money Goes
Kelly Sheridan, Staff Editor, Dark Reading,  10/15/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
7 Threats & Disruptive Forces Changing the Face of Cybersecurity
This Dark Reading Tech Digest gives an in-depth look at the biggest emerging threats and disruptive forces that are changing the face of cybersecurity today.
Flash Poll
2019 Online Malware and Threats
2019 Online Malware and Threats
As cyberattacks become more frequent and more sophisticated, enterprise security teams are under unprecedented pressure to respond. Is your organization ready?
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-18216
PUBLISHED: 2019-10-20
** DISPUTED ** The BIOS configuration design on ASUS ROG Zephyrus M GM501GS laptops with BIOS 313 relies on the main battery instead of using a CMOS battery, which reduces the value of a protection mechanism in which booting from a USB device is prohibited. Attackers who have physical laptop access ...
CVE-2019-18214
PUBLISHED: 2019-10-19
The Video_Converter app 0.1.0 for Nextcloud allows denial of service (CPU and memory consumption) via multiple concurrent conversions because many FFmpeg processes may be running at once. (The workload is not queued for serial execution.)
CVE-2019-18202
PUBLISHED: 2019-10-19
Information Disclosure is possible on WAGO Series PFC100 and PFC200 devices before FW12 due to improper access control. A remote attacker can check for the existence of paths and file names via crafted HTTP requests.
CVE-2019-18209
PUBLISHED: 2019-10-19
templates/pad.html in Etherpad-Lite 1.7.5 has XSS when the browser does not encode the path of the URL, as demonstrated by Internet Explorer.
CVE-2019-18198
PUBLISHED: 2019-10-18
In the Linux kernel before 5.3.4, a reference count usage error in the fib6_rule_suppress() function in the fib6 suppression feature of net/ipv6/fib6_rules.c, when handling the FIB_LOOKUP_NOREF flag, can be exploited by a local attacker to corrupt memory, aka CID-ca7a03c41753.