Analytics
5/15/2014
12:00 PM
Levi Gundert
Levi Gundert
Commentary
Connect Directly
Twitter
LinkedIn
RSS
E-Mail
50%
50%

Beware Cognitive Bias

Cognitive bias can compromise any profession. But when cognitive bias goes unrecognized in cyber security, far-reaching and serious consequences follow.

I distinctly remember the instructor’s mix of exasperation and inquisition. “He was acting suspicious?!” I was sitting in a Georgia classroom at the Federal Law Enforcement Training Center (FLETC) listening to a critique of my latest probable cause affidavit. I instantly realized my mistake. In the training scenario, the suspect was wearing warm clothing on a hot day, pacing, and avoiding eye contact during questioning. Instead of limiting the affidavit to my direct observations and detailing the suspect’s behavior, I inserted what I thought was a pithy conclusion.  

Cognitive bias affects everyone, and behavioral economists are continuously documenting its societal effects. Every profession likewise is influenced by cognitive bias. While the effects in criminal investigations are well documented, cyber security is a similar domain where the tendency to misinterpret data often leads to fallacious conclusions.  

A casual look at the list of cognitive biases should give you pause: anchoring, belief bias, confirmation bias, distinction bias, focusing effect, irrational escalation, and the list continues. We are all predisposed to these biases and tend to be overcritical of their effects in others, while minimizing their impact upon our own analytic faculties. For example, a few months ago I found myself examining a malware campaign that used multiple domains, all of which were registered with a Seychelles (an archipelago off of Africa’s Eastern coast) address. 

As I contemplated the Seychelles’ population size, I realized that I had recently observed additional malicious activity tied to Seychelles WHOIS registrant data. Similarly, I decided that those registering domains with Panamanian addresses also fit my evil-perpetrating model, based on prior knowledge and experience.

Thus with the help of my colleagues Jaeson Schultz and Andrew Tsonchev I collected all new domains registered with Seychelles or Panama addresses in the prior seven months and identified the incidence of customer Web blocks (Cloud Web Security). While I was confident we would find a block rate over 50%, the results did not support my assertion. Out of 19,557 Seychelles registrant domains, we blocked 337, which means less than 1% (.02%) were actually participating in malicious Web activity. The results were similar for Panama registrant domains. To be sure, we queried the same list of domains three months later to account for potential latency between domain registration and malicious use, and the results were consistent with our first query.

Now, data sources certainly matter. In this case the original domain lists may have been incomplete, and the domains may have been used for malicious campaigns in additional channels such as email. Regardless, I expected a high incidence of Web maliciousness based on a cognitive bias, specifically a confirmation bias.

In the realms of threat intelligence, incident response, and general network security monitoring, our profession suffers from cognitive biases just like any other profession. Yet the consequences of unrecognized cognitive biases in cyber security (and the resulting incorrect conclusions) may be more powerful and further reaching at this point in history.

How do companies compete with governments that are stealing intellectual property for economic competitive advantage? It’s a tough question, and before strategies are formed, corporate officers and board members first need to be able to answer with confidence the question: “How do we know who is behind this attack?” Sovereign nations have been asking the same question for millennia, but the Internet now facilitates a constant connection and higher degree of anonymity for talented and clever threat actors. Thus a centerpiece of foreign policy hinges on accurate conclusions driven by unbiased data analysis.

This is particularly true regarding attribution. Threat actors and cyber defenders operate in the context of a global Internet comprised of billions – soon to be trillions – of connected nodes. Identifying the person or party responsible for a specific cyber security event at a specific point in time is incredibly challenging, even for the most talented teams blessed with significant resources. This is true for every organization with an interest in identifying a deeper level of attribution, including geographic location and/or the individual or group responsible for a specific attack.

Last year Mandiant published the APT1 report -- a public watershed for cyber attack attribution -- which articulated the specific data and timeline that led to many of the report’s conclusions. Given the theme of the report, the supporting data was crucial to its credibility, and that data was not amassed overnight. If history is any indicator, successful attribution will continue to require prolonged time investments, sometimes even years.

Last year Sergio Caltagirone, Andrew Pendergast, and Christopher Betz released a paper entitled The Diamond Model of Intrusion Analysis. This remarkably succinct framework provides a consistent filter for malicious cyber event metadata. It is this type of framework that analysts must continually refer to while collecting and interpreting cyber attack data, in order to avoid unchecked cognitive bias.

Decision makers desperately need finished intelligence and logical assertions to plot the future course of military action, corporate policy, and foreign policy. Operationally this equates to domains, IP addresses, infrastructure owners, malicious code, etc., and the facts should perform a report’s summation. As analysts, we should not be inserting conjecture masquerading as fact into reports, because it is damaging to our industry and it impedes our ability to work toward a more secure Internet. If we fail to articulate the facts around a malicious cyber event properly, avoidable conflicts may ensue, and ultimately our entire industry loses trust and credibility.

Cognitive bias is rarely intentional, but hopefully we can continue to look for and confront our own analytical mistakes -- assisted by a reliable framework -- in order to produce a better security product (in any form). Industry and government decision makers and the general public will benefit, which should lead to improved education and efforts around the cyberthreat landscape we are daily confronting.

Levi Gundert is an internationally recognized information security and risk management leader and a cybersecurity advisor to leading corporations.  In his role with TRAC, he identifies and analyzes threats and shares cybersecurity information with industry, government, ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
cumulonimbus
50%
50%
cumulonimbus,
User Rank: Apprentice
5/21/2014 | 1:43:13 PM
Re: know your enemy
You make some good points about timing and involvement. I believe much more needs to be done to protect IP and other valuable data, particularly with offshoring and cloud. All too often (and all too late) companies find themselves vulnerable and are left with an extremely vexing problem. These problems should not occur, or at least be very rare, and are somewhat symptomatic of the C21 M.O., both in the private and public sectors. In some ways, it behooves us to think like a hacker. Ironically, in eastern philosophy our adversary is also our master.
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/21/2014 | 11:18:13 AM
Re: know your enemy
Certainly, though the accompanying Interpol Red Notice means that these suspects will be extradited if they ever travel.
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
5/21/2014 | 9:23:54 AM
Re: know your enemy
It appears the DOJ indictiment has opened the floodgates for more naming and shaming. The bad news is that many of the defendants will never be prosecuted, but the good news is that putting faces to the attacks raises awareness among businesses and the general public. 
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/21/2014 | 12:04:42 AM
Re: know your enemy
Thanks @kjhiggins. Beyond what I stated in the article, given the latest DOJ indictments of the five Chinese PLA employees, I hope there are new incentives to pursue attribution in conjunction with law enforcement.

I agree though that businesses are still struggling with the appropriate response after breaches, specifically around the decision (and timing) to involve law enforcement.
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/20/2014 | 11:47:33 PM
Re: know your enemy
Thank you for taking the time to read and comment, I appreciate the feedback!
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/20/2014 | 11:45:20 PM
Re: Keep it local
Thanks for taking the time to comment. I agree that objectivity does require an incredible amount of self discipline.
cumulonimbus
50%
50%
cumulonimbus,
User Rank: Apprentice
5/19/2014 | 8:26:09 AM
Keep it local
Cognitive bias, indeed any bias. is the natural order. It is how we think and how we came to be; a natural product of evolution. Objective observation requires an overwhelming act of self discipline. In the field of IT we are constantly dealing with the threat of cyber crime, thus trust in our IT personnel is paramount.

All that stands between us (our data) and them (the dark side of human behavior) is a false sense of security; an electronic barrier that ultimately cannot withstand penetration by a persistent and highly informed attack. The attraction of this kind of act is the anonymity and obscurity provided by the worldwide interconnection that is the internet. Notwithstanding the fact that over the shoulder attacks are probably the most frequent, our best defense lies in multi-factor authentication, personal representation (local and accountable human resources), multi-layer boundaries, and constant vigilance. Other than that, what is offline is, for the most part, no longer a target.
Bprince
50%
50%
Bprince,
User Rank: Ninja
5/18/2014 | 12:39:38 AM
Re: know your enemy
I would say that knowing the who is important, especially if we are talking about a national security issue/attack on the defense industry. I agree with the overall point of the article 100 percent. If there are going to be assertions made about who is responsible for an attack, the proof needs to be carefully vetted.

BP
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
5/16/2014 | 11:12:31 AM
know your enemy
Interesting piece, @LeviGundert. There are mixed perspectives among security vendors on how important it is to know the *who* (threat group/region) behind the attack versus the attackers' M.O. and what they are after. I wonder if that clouds the issue for enterprises trying to map out their security strategies and tools.
More Blogs from Commentary
Phishing: What Once Was Old Is New Again
I used to think the heyday of phishing had passed. But as Symantec notes in its 2014 Internet Security Threat Report, I was wrong!
Dark Reading Radio: Data Loss Prevention (DLP) Fail
Learn about newly found vulnerabilities in commercial and open-source DLP software in our latest episode of Dark Reading Radio with security researchers Zach Lanier and Kelly Lum.
The Perfect InfoSec Mindset: Paranoia + Skepticism
A little skeptical paranoia will ensure that you have the impulse to react quickly to new threats while retaining the logic to separate fact from fiction.
Weak Password Advice From Microsoft
Tempting as it may seem to do away with strong passwords for low-risk websites, password reuse is still a significant threat to both users and business.
Internet of Things: 4 Security Tips From The Military
The military has been connecting mobile command posts, unmanned vehicles, and wearable computers for decades. It’s time to take a page from their battle plan.
Register for Dark Reading Newsletters
White Papers
Flash Poll
Current Issue
Cartoon
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2014-2970
Published: 2014-07-31
** REJECT ** DO NOT USE THIS CANDIDATE NUMBER. ConsultIDs: CVE-2014-5139. Reason: This candidate is a duplicate of CVE-2014-5139, and has also been used to refer to an unrelated topic that is currently outside the scope of CVE. This unrelated topic is a LibreSSL code change adding functionality ...

CVE-2014-0914
Published: 2014-07-30
Cross-site scripting (XSS) vulnerability in IBM Maximo Asset Management 6.2 through 6.2.8 and 6.x and 7.x through 7.5.0.6, Maximo Asset Management 7.5 through 7.5.0.3 and 7.5.1 through 7.5.1.2 for SmartCloud Control Desk, and Maximo Asset Management 6.2 through 6.2.8 for Tivoli IT Asset Management f...

CVE-2014-0915
Published: 2014-07-30
Multiple cross-site scripting (XSS) vulnerabilities in IBM Maximo Asset Management 6.2 through 6.2.8, 6.x and 7.1 through 7.1.1.2, and 7.5 through 7.5.0.6; Maximo Asset Management 7.5 through 7.5.0.3 and 7.5.1 through 7.5.1.2 for SmartCloud Control Desk; and Maximo Asset Management 6.2 through 6.2.8...

CVE-2014-0947
Published: 2014-07-30
Unspecified vulnerability in the server in IBM Rational Software Architect Design Manager 4.0.6 allows remote authenticated users to execute arbitrary code via a crafted update site.

CVE-2014-0948
Published: 2014-07-30
Unspecified vulnerability in IBM Rational Software Architect Design Manager and Rational Rhapsody Design Manager 3.x and 4.x before 4.0.7 allows remote authenticated users to execute arbitrary code via a crafted ZIP archive.

Best of the Web
Dark Reading Radio