Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Risk

10/6/2015
10:30 AM
Jay Jacobs
Jay Jacobs
Commentary
Connect Directly
Twitter
LinkedIn
RSS
E-Mail vvv
50%
50%

Dont Be Fooled: In Cybersecurity Big Data Is Not The Goal

In other words, the skills to be a security expert do not translate to being able to understand and extract meaning from security data.

With any hyped-up topic, there is a disconnect between what people think can be accomplished and what can actually be accomplished.  The latest example is “Big Data.” There’s something different about big data because when executed properly, modern data analysis can seem like nothing short of magic to outsiders.  It’s this perception of magic that is so attractive to businesses looking to pull the next rabbit out of a hat. 

What’s also unique about “big data” is the way the term is perceived and used in the security industry, which is much different than the approach of  data scientists. This blog is an attempt to bridge the divide and talk about “big data” behind the scenes because when it comes down to it, big data is not a goal, it is logistics, and many of security’s big data problems are being solved by small data solutions.

Big data is just hiding the small data

In many instances, “Big Data Analytics” is an embellishment, a little white lie with good intentions.  In reality, the primary task done directly with big data is figuring out how to turn it into small data. In many cases, big data will be reduced down by counting, comparing or some other aggregation.  Another technique is to pull a small subset or sample from the large data set to get a more manageable data set. Either way, many of the big data jobs are done to produce small data. 

For example, if you are working with log data from thousands of systems, you may want to produce a small data file where each system is reduced to a single line with counts of sources and login statuses (or whatever is being measured).  Don’t be fooled by the use of “small” here, the output may still be in the megabyte or even gigabyte range, which may be just small enough to load into memory on a laptop for analysis. 

If that deflated the mystique of big data for you, don’t worry, there is a small set of big data implementations that will perform analytics at scale, for example, if you need to build a unique and specific model on each application across all of your servers (This is a technique we use to develop security ratings.) While each application may represent small data, doing analysis across all applications represents a challenge.  And finally there is a very small sliver of analytics doing complex computations at scale, but these are rare and chances are your problems are just not that special. 

Big data enables [over]confidence

While much of the big data analysis is being done on small data, that doesn’t mean it’s the same old small data analysis.  Big data is ushering in its own sets of challenges because most of the classic statistics were developed using pencil and paper with just a few dozen samples, perhaps going into the hundreds.  The good news is that most of the techniques are actually improved with more data. Analyses can find ever more subtleties as more data is used. Smaller differences can be discovered and more and more nuanced patterns can be detected.  This is where some of that magic comes from: there can be big gain from small advantages.  However this is not without side effects.  One of the classic measures of statistical significance, the p-value, is often irrelevant on large samples.  Because analysis can uncover smaller and smaller differences in big data, in reverse, smaller differences now become significant.  The effect is that big data can fool those unaware of this effect. 

Big data, small samples

Even though big data still has that new-hype smell about it, underlying that hype are techniques that can be traced back centuries -- and all point to a single constant: good data analysis.  It’s a hard lesson to learn, but good data analysis is a different skill than domain expertise.  In other words, the skills to be a security expert (for example) do not translate to being able to understand and extract meaning from security data.  Neither is good data analysis an intuitive skill; it is not picked up by proximity to data day after day.  It is only learned by intentional study of statistics and related fields.

As an example, in the age of big data where a million data points is labelled as small, a sample of a few hundred and even a few thousand seems meager and perhaps pathetic to the uninitiated.  I’ve seen people dismiss research results because they thought the samples were too small (and they were over a thousand).  We must keep in mind that data analysis is not intuitive and even though we have a lot of data, it may not take a lot of data to provide insight or support a business decision.  So shake off this notion that big data is a goal and get down to the business of learning from data.

Jay Jacobs has over 15 years of experience within IT and information security with a focus on cryptography, risk, and data analysis. Most recently, he has joined BitSight Technologies, the Standard in Security Ratings, as their Senior Data Scientist. Previously, he was a Data ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Windows 10 Migration: Getting It Right
Kevin Alexandra, Principal Solutions Engineer at BeyondTrust,  5/15/2019
Baltimore Ransomware Attack Takes Strange Twist
Kelly Jackson Higgins, Executive Editor at Dark Reading,  5/14/2019
When Older Windows Systems Won't Die
Kelly Sheridan, Staff Editor, Dark Reading,  5/17/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
Building and Managing an IT Security Operations Program
As cyber threats grow, many organizations are building security operations centers (SOCs) to improve their defenses. In this Tech Digest you will learn tips on how to get the most out of a SOC in your organization - and what to do if you can't afford to build one.
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-12184
PUBLISHED: 2019-05-19
There is XSS in browser/components/MarkdownPreview.js in BoostIO Boostnote 0.11.15 via a label named flowchart, sequence, gallery, or chart, as demonstrated by a crafted SRC attribute of an IFRAME element, a different vulnerability than CVE-2019-12136.
CVE-2019-12173
PUBLISHED: 2019-05-18
MacDown 0.7.1 (870) allows remote code execution via a file:\\\ URI, with a .app pathname, in the HREF attribute of an A element. This is different from CVE-2019-12138.
CVE-2019-12172
PUBLISHED: 2019-05-17
Typora 0.9.9.21.1 (1913) allows arbitrary code execution via a modified file: URL syntax in the HREF attribute of an AREA element, as demonstrated by file:\\\ on macOS or Linux, or file://C| on Windows. This is different from CVE-2019-12137.
CVE-2019-12168
PUBLISHED: 2019-05-17
Four-Faith Wireless Mobile Router F3x24 v1.0 devices allow remote code execution via the Command Shell (aka Administration > Commands) screen.
CVE-2019-12170
PUBLISHED: 2019-05-17
ATutor through 2.2.4 is vulnerable to arbitrary file uploads via the mods/_core/backups/upload.php (aka backup) component. This may result in remote command execution. An attacker can use the instructor account to fully compromise the system using a crafted backup ZIP archive. This will allow for PH...