Threat Intelligence
4/20/2017
10:30 AM
Jeff Schilling
Jeff Schilling
Commentary
Connect Directly
Facebook
Twitter
LinkedIn
RSS
E-Mail vvv
100%
0%

Cutting through the Noise: Is It AI or Pattern Matching?

Many vendors are fudging terms when trying to sell their artificial intelligence security systems. Here's what you need to know when you buy.

At any recent security conference lately, you probably have heard hundreds of vendors repeating the words "We have the best artificial intelligence (AI) and machine learning." If you happened to be in one of those conversations and asked "What does that mean?," you probably got a blank stare. Many security consumers are frustrated when marketing pitches don't clearly articulate what AI does in a product to help protect an environment better.

There are several dilemmas facing security companies that keep them from being more up-front about how they use AI and machine learning. For some, the concepts are a marketing statement only, and what they call AI and machine learning is actually pattern matching. Also, machine learning relies on a tremendous volume of data to be effective, and there are very few vendors that possess enough of it to be successful in its implementation.

To avoid a wasted investment in this technology, it's essential to understand the basics of what AI and machine learning provide in security tools. My goal is for you to be equipped to ask the right questions when a vendor proclaims "We have the best AI!"

What Is AI and What Does It Do?
There are many definitions of AI; the definition I use is that it's a system that can monitor behavior, learn, and then adapt and problem solve. The problem solving is where the machine learning component of AI comes into effect. An example of an AI use case is when a machine plays chess. Because there are so many permutations of options and movements in chess, it requires an AI system to observe adversary behavior, learn the consequence of these behaviors, and then formulate actions that result in a strategy to defeat a human opponent.

In contrast, a good example of pattern matching is playing checkers. Any simple computer program can run the limited number of moves and counter-moves an adversary would use in checkers based on the pattern of movement that the human establishes.

So, the first question a vendor should be asked is: "Is your AI doing pattern matching or problem-solving?" A good indicator that a security tool is just doing pattern matching is if a vendor tells you that it works right out of the box and doesn't need to learn the environment. Pattern matching isn't necessarily bad, but it isn't adaptable to ever-changing threats. This technique constantly requires vendor updates to stay alongside threats while never getting ahead of them.  

[Check out the two-day Dark Reading Cybersecurity Crash Course at Interop ITX, May 15 & 16, where Dark Reading editors and some of the industry's top cybersecurity experts will share the latest data security trends and best practices.]

Another question to ask vendors is: "What type of machine learning does your system use?" Decision tree learning is common in what I'd refer to as pattern-matching systems. Basically, decision tree learning consists of mapping observations to predefined conclusions or actions. For example, "if I see this C2 domain in packet capture and detect this hash value in network intrusion monitoring, then I likely have this particular threat actor in my environment." This approach is great for lowering the amount of human touch or repetitive tasks your security team must do on a daily basis. 

Another common machine learning strategy in security tools is a Bayesian model. This is where you transition between algebra with a decision tree to calculus with a Bayesian approach and, in my opinion, move from pattern matching to true AI. Essentially, the Bayesian model observes the state of many variables across your environment and maps them based on whether they're true or false into a data table that allows the AI system to determine the probability or confidence level that a particular event has happened. This approach doesn't have predefined conclusions on these observations; instead, it informs you of anomalous activity, based on the observation or status of many variables, that there's a high probability of malicious activity. The more data points you can process through a Bayesian model, the more accurate it becomes.

My favorite machine learning technique is clustering or k-means clustering. This is a machine learning system that plots a graph of what's expected from the telemetry of your environment in a clustered model. If you have a million dots in your cluster, and you see a dot that's plotted outside of the cluster, that's an anomaly that should be investigated. This learning system doesn't need a pattern; it's basically mapping what normal behavior looks like and identifies outliers. 

Bringing It All Together
A good AI system will have elements of a decision tree (for known patterns), Bayesian analysis for anomaly detection, and clustering for baseline monitoring. If the AI system you want to buy can seamlessly integrate these three machine learning techniques, you're probably on the right track.

But you're not done yet. Next, you have to ask yourself: "What data do I need to provide to these tools, and do I collect the data needed to make these tools work the way they're designed?" Billions of dollars are wasted on strategies and tools because the users who buy them get wowed by a sales pitch and can't implement the tools when they get them.

Here's my advice for selecting an AI solution to enhance your security program:

1. Don't fall for the marketing pitch, and ask the critical questions.

2. Make sure you understand the machine learning strategies leveraged by the vendor and that they make sense for the data you have in your environment

3. Decide if you are playing checkers or chess with your adversary.

Understanding these points will help you make an informed decision that avoids wasted investments, time, and resources. While AI and machine learning create efficiencies that can't be duplicated by humans, the human element still must be in place to make sense of the information and process it properly so it can be used to achieve organizational objectives.

Related Content:

Jeff Schilling, a retired U.S. Army colonel, is Armor's chief security officer. He is responsible for the cyber and physical security programs for the corporate environment and customer-focused capabilities. His areas of focus include cloud operations, client services, ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
Security Operations and IT Operations: Finding the Path to Collaboration
A wide gulf has emerged between SOC and NOC teams that's keeping both of them from assuring the confidentiality, integrity, and availability of IT systems. Here's how experts think it should be bridged.
Flash Poll
New Best Practices for Secure App Development
New Best Practices for Secure App Development
The transition from DevOps to SecDevOps is combining with the move toward cloud computing to create new challenges - and new opportunities - for the information security team. Download this report, to learn about the new best practices for secure application development.
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2013-7445
Published: 2015-10-15
The Direct Rendering Manager (DRM) subsystem in the Linux kernel through 4.x mishandles requests for Graphics Execution Manager (GEM) objects, which allows context-dependent attackers to cause a denial of service (memory consumption) via an application that processes graphics data, as demonstrated b...

CVE-2015-4948
Published: 2015-10-15
netstat in IBM AIX 5.3, 6.1, and 7.1 and VIOS 2.2.x, when a fibre channel adapter is used, allows local users to gain privileges via unspecified vectors.

CVE-2015-5660
Published: 2015-10-15
Cross-site request forgery (CSRF) vulnerability in eXtplorer before 2.1.8 allows remote attackers to hijack the authentication of arbitrary users for requests that execute PHP code.

CVE-2015-6003
Published: 2015-10-15
Directory traversal vulnerability in QNAP QTS before 4.1.4 build 0910 and 4.2.x before 4.2.0 RC2 build 0910, when AFP is enabled, allows remote attackers to read or write to arbitrary files by leveraging access to an OS X (1) user or (2) guest account.

CVE-2015-6333
Published: 2015-10-15
Cisco Application Policy Infrastructure Controller (APIC) 1.1j allows local users to gain privileges via vectors involving addition of an SSH key, aka Bug ID CSCuw46076.

Dark Reading Radio
Archived Dark Reading Radio
In past years, security researchers have discovered ways to hack cars, medical devices, automated teller machines, and many other targets. Dark Reading Executive Editor Kelly Jackson Higgins hosts researcher Samy Kamkar and Levi Gundert, vice president of threat intelligence at Recorded Future, to discuss some of 2016's most unusual and creative hacks by white hats, and what these new vulnerabilities might mean for the coming year.