Analytics
5/15/2014
12:00 PM
Levi Gundert
Levi Gundert
Commentary
Connect Directly
Twitter
LinkedIn
RSS
E-Mail
50%
50%

Beware Cognitive Bias

Cognitive bias can compromise any profession. But when cognitive bias goes unrecognized in cyber security, far-reaching and serious consequences follow.

I distinctly remember the instructor’s mix of exasperation and inquisition. “He was acting suspicious?!” I was sitting in a Georgia classroom at the Federal Law Enforcement Training Center (FLETC) listening to a critique of my latest probable cause affidavit. I instantly realized my mistake. In the training scenario, the suspect was wearing warm clothing on a hot day, pacing, and avoiding eye contact during questioning. Instead of limiting the affidavit to my direct observations and detailing the suspect’s behavior, I inserted what I thought was a pithy conclusion.  

Cognitive bias affects everyone, and behavioral economists are continuously documenting its societal effects. Every profession likewise is influenced by cognitive bias. While the effects in criminal investigations are well documented, cyber security is a similar domain where the tendency to misinterpret data often leads to fallacious conclusions.  

A casual look at the list of cognitive biases should give you pause: anchoring, belief bias, confirmation bias, distinction bias, focusing effect, irrational escalation, and the list continues. We are all predisposed to these biases and tend to be overcritical of their effects in others, while minimizing their impact upon our own analytic faculties. For example, a few months ago I found myself examining a malware campaign that used multiple domains, all of which were registered with a Seychelles (an archipelago off of Africa’s Eastern coast) address. 

As I contemplated the Seychelles’ population size, I realized that I had recently observed additional malicious activity tied to Seychelles WHOIS registrant data. Similarly, I decided that those registering domains with Panamanian addresses also fit my evil-perpetrating model, based on prior knowledge and experience.

Thus with the help of my colleagues Jaeson Schultz and Andrew Tsonchev I collected all new domains registered with Seychelles or Panama addresses in the prior seven months and identified the incidence of customer Web blocks (Cloud Web Security). While I was confident we would find a block rate over 50%, the results did not support my assertion. Out of 19,557 Seychelles registrant domains, we blocked 337, which means less than 1% (.02%) were actually participating in malicious Web activity. The results were similar for Panama registrant domains. To be sure, we queried the same list of domains three months later to account for potential latency between domain registration and malicious use, and the results were consistent with our first query.

Now, data sources certainly matter. In this case the original domain lists may have been incomplete, and the domains may have been used for malicious campaigns in additional channels such as email. Regardless, I expected a high incidence of Web maliciousness based on a cognitive bias, specifically a confirmation bias.

In the realms of threat intelligence, incident response, and general network security monitoring, our profession suffers from cognitive biases just like any other profession. Yet the consequences of unrecognized cognitive biases in cyber security (and the resulting incorrect conclusions) may be more powerful and further reaching at this point in history.

How do companies compete with governments that are stealing intellectual property for economic competitive advantage? It’s a tough question, and before strategies are formed, corporate officers and board members first need to be able to answer with confidence the question: “How do we know who is behind this attack?” Sovereign nations have been asking the same question for millennia, but the Internet now facilitates a constant connection and higher degree of anonymity for talented and clever threat actors. Thus a centerpiece of foreign policy hinges on accurate conclusions driven by unbiased data analysis.

This is particularly true regarding attribution. Threat actors and cyber defenders operate in the context of a global Internet comprised of billions – soon to be trillions – of connected nodes. Identifying the person or party responsible for a specific cyber security event at a specific point in time is incredibly challenging, even for the most talented teams blessed with significant resources. This is true for every organization with an interest in identifying a deeper level of attribution, including geographic location and/or the individual or group responsible for a specific attack.

Last year Mandiant published the APT1 report -- a public watershed for cyber attack attribution -- which articulated the specific data and timeline that led to many of the report’s conclusions. Given the theme of the report, the supporting data was crucial to its credibility, and that data was not amassed overnight. If history is any indicator, successful attribution will continue to require prolonged time investments, sometimes even years.

Last year Sergio Caltagirone, Andrew Pendergast, and Christopher Betz released a paper entitled The Diamond Model of Intrusion Analysis. This remarkably succinct framework provides a consistent filter for malicious cyber event metadata. It is this type of framework that analysts must continually refer to while collecting and interpreting cyber attack data, in order to avoid unchecked cognitive bias.

Decision makers desperately need finished intelligence and logical assertions to plot the future course of military action, corporate policy, and foreign policy. Operationally this equates to domains, IP addresses, infrastructure owners, malicious code, etc., and the facts should perform a report’s summation. As analysts, we should not be inserting conjecture masquerading as fact into reports, because it is damaging to our industry and it impedes our ability to work toward a more secure Internet. If we fail to articulate the facts around a malicious cyber event properly, avoidable conflicts may ensue, and ultimately our entire industry loses trust and credibility.

Cognitive bias is rarely intentional, but hopefully we can continue to look for and confront our own analytical mistakes -- assisted by a reliable framework -- in order to produce a better security product (in any form). Industry and government decision makers and the general public will benefit, which should lead to improved education and efforts around the cyberthreat landscape we are daily confronting.

Levi Gundert is an internationally recognized information security and risk management leader and a cybersecurity advisor to leading corporations.  In his role with TRAC, he identifies and analyzes threats and shares cybersecurity information with industry, government, ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
cumulonimbus
50%
50%
cumulonimbus,
User Rank: Apprentice
5/21/2014 | 1:43:13 PM
Re: know your enemy
You make some good points about timing and involvement. I believe much more needs to be done to protect IP and other valuable data, particularly with offshoring and cloud. All too often (and all too late) companies find themselves vulnerable and are left with an extremely vexing problem. These problems should not occur, or at least be very rare, and are somewhat symptomatic of the C21 M.O., both in the private and public sectors. In some ways, it behooves us to think like a hacker. Ironically, in eastern philosophy our adversary is also our master.
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/21/2014 | 11:18:13 AM
Re: know your enemy
Certainly, though the accompanying Interpol Red Notice means that these suspects will be extradited if they ever travel.
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
5/21/2014 | 9:23:54 AM
Re: know your enemy
It appears the DOJ indictiment has opened the floodgates for more naming and shaming. The bad news is that many of the defendants will never be prosecuted, but the good news is that putting faces to the attacks raises awareness among businesses and the general public. 
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/21/2014 | 12:04:42 AM
Re: know your enemy
Thanks @kjhiggins. Beyond what I stated in the article, given the latest DOJ indictments of the five Chinese PLA employees, I hope there are new incentives to pursue attribution in conjunction with law enforcement.

I agree though that businesses are still struggling with the appropriate response after breaches, specifically around the decision (and timing) to involve law enforcement.
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/20/2014 | 11:47:33 PM
Re: know your enemy
Thank you for taking the time to read and comment, I appreciate the feedback!
levigundert
50%
50%
levigundert,
User Rank: Apprentice
5/20/2014 | 11:45:20 PM
Re: Keep it local
Thanks for taking the time to comment. I agree that objectivity does require an incredible amount of self discipline.
cumulonimbus
50%
50%
cumulonimbus,
User Rank: Apprentice
5/19/2014 | 8:26:09 AM
Keep it local
Cognitive bias, indeed any bias. is the natural order. It is how we think and how we came to be; a natural product of evolution. Objective observation requires an overwhelming act of self discipline. In the field of IT we are constantly dealing with the threat of cyber crime, thus trust in our IT personnel is paramount.

All that stands between us (our data) and them (the dark side of human behavior) is a false sense of security; an electronic barrier that ultimately cannot withstand penetration by a persistent and highly informed attack. The attraction of this kind of act is the anonymity and obscurity provided by the worldwide interconnection that is the internet. Notwithstanding the fact that over the shoulder attacks are probably the most frequent, our best defense lies in multi-factor authentication, personal representation (local and accountable human resources), multi-layer boundaries, and constant vigilance. Other than that, what is offline is, for the most part, no longer a target.
Bprince
50%
50%
Bprince,
User Rank: Ninja
5/18/2014 | 12:39:38 AM
Re: know your enemy
I would say that knowing the who is important, especially if we are talking about a national security issue/attack on the defense industry. I agree with the overall point of the article 100 percent. If there are going to be assertions made about who is responsible for an attack, the proof needs to be carefully vetted.

BP
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
5/16/2014 | 11:12:31 AM
know your enemy
Interesting piece, @LeviGundert. There are mixed perspectives among security vendors on how important it is to know the *who* (threat group/region) behind the attack versus the attackers' M.O. and what they are after. I wonder if that clouds the issue for enterprises trying to map out their security strategies and tools.
More Blogs from Commentary
Infographic: With BYOD, Mobile Is The New Desktop
Security teams have no choice but to embrace the rapid proliferation of BYO devices, apps, and cloud services. To ignore it is to put your head in the sand.
Internet of Things: Security For A World Of Ubiquitous Computing
Endpoint security is hardly dead, and claiming that it is oversimplifies the challenges corporations face now and in the not-very-distant future.
CEO Report Card: Low Grades for Risk Management
Dark Reading's latest community poll shows a stunning lack of confidence in chief execs' commitment to cyber security.
A New Age in Cyber Security: Public Cyberhealth
The cleanup aimed at disrupting GameOver Zeus and CryptoLocker offers an instructive template for managing mass cyber infections.
Passwords & The Future Of Identity: Payment Networks?
The solution to the omnipresent and enduring password problem may be closer than you think.
Register for Dark Reading Newsletters
White Papers
Flash Poll
Current Issue
Cartoon
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2013-7392
Published: 2014-07-22
Gitlist allows remote attackers to execute arbitrary commands via shell metacharacters in a file name to Source/.

CVE-2014-2385
Published: 2014-07-22
Multiple cross-site scripting (XSS) vulnerabilities in the web UI in Sophos Anti-Virus for Linux before 9.6.1 allow local users to inject arbitrary web script or HTML via the (1) newListList:ExcludeFileOnExpression, (2) newListList:ExcludeFilesystems, or (3) newListList:ExcludeMountPaths parameter t...

CVE-2014-3518
Published: 2014-07-22
jmx-remoting.sar in JBoss Remoting, as used in Red Hat JBoss Enterprise Application Platform (JEAP) 5.2.0, Red Hat JBoss BRMS 5.3.1, Red Hat JBoss Portal Platform 5.2.2, and Red Hat JBoss SOA Platform 5.3.1, does not properly implement the JSR 160 specification, which allows remote attackers to exec...

CVE-2014-3530
Published: 2014-07-22
The org.picketlink.common.util.DocumentUtil.getDocumentBuilderFactory method in PicketLink, as used in Red Hat JBoss Enterprise Application Platform (JBEAP) 5.2.0 and 6.2.4, expands entity references, which allows remote attackers to read arbitrary code and possibly have other unspecified impact via...

CVE-2014-4326
Published: 2014-07-22
Elasticsearch Logstash 1.0.14 through 1.4.x before 1.4.2 allows remote attackers to execute arbitrary commands via a crafted event in (1) zabbix.rb or (2) nagios_nsca.rb in outputs/.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
Where do information security startups come from? More important, how can I tell a good one from a flash in the pan? Learn how to separate ITSec wheat from chaff in this episode.