Analytics
5/15/2014
12:00 PM
Levi Gundert
Levi Gundert
Commentary
Connect Directly
Twitter
LinkedIn
RSS
E-Mail
50%
50%

Beware Cognitive Bias

Cognitive bias can compromise any profession. But when cognitive bias goes unrecognized in cyber security, far-reaching and serious consequences follow.

I distinctly remember the instructor’s mix of exasperation and inquisition. “He was acting suspicious?!” I was sitting in a Georgia classroom at the Federal Law Enforcement Training Center (FLETC) listening to a critique of my latest probable cause affidavit. I instantly realized my mistake. In the training scenario, the suspect was wearing warm clothing on a hot day, pacing, and avoiding eye contact during questioning. Instead of limiting the affidavit to my direct observations and detailing the suspect’s behavior, I inserted what I thought was a pithy conclusion.  

Cognitive bias affects everyone, and behavioral economists are continuously documenting its societal effects. Every profession likewise is influenced by cognitive bias. While the effects in criminal investigations are well documented, cyber security is a similar domain where the tendency to misinterpret data often leads to fallacious conclusions.  

A casual look at the list of cognitive biases should give you pause: anchoring, belief bias, confirmation bias, distinction bias, focusing effect, irrational escalation, and the list continues. We are all predisposed to these biases and tend to be overcritical of their effects in others, while minimizing their impact upon our own analytic faculties. For example, a few months ago I found myself examining a malware campaign that used multiple domains, all of which were registered with a Seychelles (an archipelago off of Africa’s Eastern coast) address. 

As I contemplated the Seychelles’ population size, I realized that I had recently observed additional malicious activity tied to Seychelles WHOIS registrant data. Similarly, I decided that those registering domains with Panamanian addresses also fit my evil-perpetrating model, based on prior knowledge and experience.

Thus with the help of my colleagues Jaeson Schultz and Andrew Tsonchev I collected all new domains registered with Seychelles or Panama addresses in the prior seven months and identified the incidence of customer Web blocks (Cloud Web Security). While I was confident we would find a block rate over 50%, the results did not support my assertion. Out of 19,557 Seychelles registrant domains, we blocked 337, which means less than 1% (.02%) were actually participating in malicious Web activity. The results were similar for Panama registrant domains. To be sure, we queried the same list of domains three months later to account for potential latency between domain registration and malicious use, and the results were consistent with our first query.

Now, data sources certainly matter. In this case the original domain lists may have been incomplete, and the domains may have been used for malicious campaigns in additional channels such as email. Regardless, I expected a high incidence of Web maliciousness based on a cognitive bias, specifically a confirmation bias.

In the realms of threat intelligence, incident response, and general network security monitoring, our profession suffers from cognitive biases just like any other profession. Yet the consequences of unrecognized cognitive biases in cyber security (and the resulting incorrect conclusions) may be more powerful and further reaching at this point in history.

How do companies compete with governments that are stealing intellectual property for economic competitive advantage? It’s a tough question, and before strategies are formed, corporate officers and board members first need to be able to answer with confidence the question: “How do we know who is behind this attack?” Sovereign nations have been asking the same question for millennia, but the Internet now facilitates a constant connection and higher degree of anonymity for talented and clever threat actors. Thus a centerpiece of foreign policy hinges on accurate conclusions driven by unbiased data analysis.

This is particularly true regarding attribution. Threat actors and cyber defenders operate in the context of a global Internet comprised of billions – soon to be trillions – of connected nodes. Identifying the person or party responsible for a specific cyber security event at a specific point in time is incredibly challenging, even for the most talented teams blessed with significant resources. This is true for every organization with an interest in identifying a deeper level of attribution, including geographic location and/or the individual or group responsible for a specific attack.

Last year Mandiant published the APT1 report -- a public watershed for cyber attack attribution -- which articulated the specific data and timeline that led to many of the report’s conclusions. Given the theme of the report, the supporting data was crucial to its credibility, and that data was not amassed overnight. If history is any indicator, successful attribution will continue to require prolonged time investments, sometimes even years.

Last year Sergio Caltagirone, Andrew Pendergast, and Christopher Betz released a paper entitled The Diamond Model of Intrusion Analysis. This remarkably succinct framework provides a consistent filter for malicious cyber event metadata. It is this type of framework that analysts must continually refer to while collecting and interpreting cyber attack data, in order to avoid unchecked cognitive bias.

Decision makers desperately need finished intelligence and logical assertions to plot the future course of military action, corporate policy, and foreign policy. Operationally this equates to domains, IP addresses, infrastructure owners, malicious code, etc., and the facts should perform a report’s summation. As analysts, we should not be inserting conjecture masquerading as fact into reports, because it is damaging to our industry and it impedes our ability to work toward a more secure Internet. If we fail to articulate the facts around a malicious cyber event properly, avoidable conflicts may ensue, and ultimately our entire industry loses trust and credibility.

Cognitive bias is rarely intentional, but hopefully we can continue to look for and confront our own analytical mistakes -- assisted by a reliable framework -- in order to produce a better security product (in any form). Industry and government decision makers and the general public will benefit, which should lead to improved education and efforts around the cyberthreat landscape we are daily confronting.

Levi Gundert is an internationally recognized information security and risk management leader and a cybersecurity advisor to leading corporations.  In his role with TRAC, he identifies and analyzes threats and shares cybersecurity information with industry, government, ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
cumulonimbus
50%
50%
cumulonimbus,
User Rank: Apprentice
5/21/2014 | 1:43:13 PM
Re: know your enemy
You make some good points about timing and involvement. I believe much more needs to be done to protect IP and other valuable data, particularly with offshoring and cloud. All too often (and all too late) companies find themselves vulnerable and are left with an extremely vexing problem. These problems should not occur, or at least be very rare, and are somewhat symptomatic of the C21 M.O., both in the private and public sectors. In some ways, it behooves us to think like a hacker. Ironically, in eastern philosophy our adversary is also our master.
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/21/2014 | 11:18:13 AM
Re: know your enemy
Certainly, though the accompanying Interpol Red Notice means that these suspects will be extradited if they ever travel.
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
5/21/2014 | 9:23:54 AM
Re: know your enemy
It appears the DOJ indictiment has opened the floodgates for more naming and shaming. The bad news is that many of the defendants will never be prosecuted, but the good news is that putting faces to the attacks raises awareness among businesses and the general public. 
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/21/2014 | 12:04:42 AM
Re: know your enemy
Thanks @kjhiggins. Beyond what I stated in the article, given the latest DOJ indictments of the five Chinese PLA employees, I hope there are new incentives to pursue attribution in conjunction with law enforcement.

I agree though that businesses are still struggling with the appropriate response after breaches, specifically around the decision (and timing) to involve law enforcement.
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/20/2014 | 11:47:33 PM
Re: know your enemy
Thank you for taking the time to read and comment, I appreciate the feedback!
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/20/2014 | 11:45:20 PM
Re: Keep it local
Thanks for taking the time to comment. I agree that objectivity does require an incredible amount of self discipline.
cumulonimbus
50%
50%
cumulonimbus,
User Rank: Apprentice
5/19/2014 | 8:26:09 AM
Keep it local
Cognitive bias, indeed any bias. is the natural order. It is how we think and how we came to be; a natural product of evolution. Objective observation requires an overwhelming act of self discipline. In the field of IT we are constantly dealing with the threat of cyber crime, thus trust in our IT personnel is paramount.

All that stands between us (our data) and them (the dark side of human behavior) is a false sense of security; an electronic barrier that ultimately cannot withstand penetration by a persistent and highly informed attack. The attraction of this kind of act is the anonymity and obscurity provided by the worldwide interconnection that is the internet. Notwithstanding the fact that over the shoulder attacks are probably the most frequent, our best defense lies in multi-factor authentication, personal representation (local and accountable human resources), multi-layer boundaries, and constant vigilance. Other than that, what is offline is, for the most part, no longer a target.
Bprince
50%
50%
Bprince,
User Rank: Ninja
5/18/2014 | 12:39:38 AM
Re: know your enemy
I would say that knowing the who is important, especially if we are talking about a national security issue/attack on the defense industry. I agree with the overall point of the article 100 percent. If there are going to be assertions made about who is responsible for an attack, the proof needs to be carefully vetted.

BP
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
5/16/2014 | 11:12:31 AM
know your enemy
Interesting piece, @LeviGundert. There are mixed perspectives among security vendors on how important it is to know the *who* (threat group/region) behind the attack versus the attackers' M.O. and what they are after. I wonder if that clouds the issue for enterprises trying to map out their security strategies and tools.
Register for Dark Reading Newsletters
White Papers
Cartoon
Current Issue
Dark Reading, September 16, 2014
Malicious software is morphing to be more targeted, stealthy, and destructive. Are you prepared to stop it?
Flash Poll
Threat Intel Today
Threat Intel Today
The 397 respondents to our new survey buy into using intel to stay ahead of attackers: 85% say threat intelligence plays some role in their IT security strategies, and many of them subscribe to two or more third-party feeds; 10% leverage five or more.
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2014-0985
Published: 2014-09-20
Stack-based buffer overflow in Advantech WebAccess (formerly BroadWin WebAccess) 7.2 allows remote attackers to execute arbitrary code via the NodeName parameter.

CVE-2014-0986
Published: 2014-09-20
Stack-based buffer overflow in Advantech WebAccess (formerly BroadWin WebAccess) 7.2 allows remote attackers to execute arbitrary code via the GotoCmd parameter.

CVE-2014-0987
Published: 2014-09-20
Stack-based buffer overflow in Advantech WebAccess (formerly BroadWin WebAccess) 7.2 allows remote attackers to execute arbitrary code via the NodeName2 parameter.

CVE-2014-0988
Published: 2014-09-20
Stack-based buffer overflow in Advantech WebAccess (formerly BroadWin WebAccess) 7.2 allows remote attackers to execute arbitrary code via the AccessCode parameter.

CVE-2014-0989
Published: 2014-09-20
Stack-based buffer overflow in Advantech WebAccess (formerly BroadWin WebAccess) 7.2 allows remote attackers to execute arbitrary code via the AccessCode2 parameter.

Best of the Web
Dark Reading Radio