Application Security
11/10/2014
11:30 AM
Jeff Williams
Jeff Williams
Commentary
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
50%
50%

The Staggering Complexity of Application Security

During the past few decades of high-speed coding we have automated our businesses so fast that we are now incapable of securing what we have built.

First, a few facts.

A typical, midsized financial organization has a portfolio of over a thousand applications. The largest firms exceed ten thousand applications. Each of these applications, on average, has hundreds of thousands of lines of custom code, and the largest can have over ten million lines. In addition, each application also includes anywhere from dozens to hundreds of software libraries, frameworks, and components that typically total over ten times the size of the custom code. And this portfolio is growing rapidly -- over 20% of these applications have new and updated code each year.

By comparison, consider the US Federal Tax Code, which has also grown dramatically over the years. Currently, the tax code totals just 4.4 million lines of “code” – roughly equivalent to just a handful of applications. As a security researcher I’ve discovered thousands of vulnerabilities in code. But as a former CEO, I’ve also analyzed a ton of legal contracts for loopholes. What’s interesting is that whether I’m scrutinizing software code or reviewing legal language, the analysis is not as different as you might think. Both require a detailed understanding of specialized language and a solid understanding of the underlying business.

So after two decades of high speed coding, a typical large financial organization has a pile of code as large as 2,000 copies of the entire 73,954 pages in the US Federal Tax Code -- almost 10 billion lines of code.

The current security situation
After the details of JP Morgan’s breach were revealed recently, a former employee told The New York Times that it was as if “[the attackers] stole the schematics to the Capitol -- [JP Morgan] can’t just switch out every single door and window pane overnight.” Actually, I think it’s considerably worse than that. It took decades to create the software infrastructure at JP Morgan, and there is no practical way to just change it out. To me, it’s more like discovering that an entire city has been built with asbestos insulation, lead pipes, and unshielded power lines. And they all need to be swapped out without disrupting service.

Now, according to experts at Aspect Security, the typical enterprise web application has an eye-opening 22.4 serious vulnerabilities. They aren’t always easy to find, and they vary in likelihood and severity of exploit. But combining this level of vulnerability with an increasingly sophisticated threat has led to a marked increase in breaches. The breaches just this year have been sobering.

Click on this link  for an interactive view of the Word Cloud by David McCandless.
Click on this link for an interactive view of the Word Cloud by David McCandless.

The limits of traditional AppSec
In the good old days, we did manual penetration testing and code review to find vulnerabilities. These reviews got pretty good coverage and developers had time to fix problems before the code went into production. But recent advances in software development, including widespread use of libraries and components, high-speed development methodologies, complex frameworks, and inscrutable protocols have made manual analysis too slow for all but the most critical applications.

Many industries reach a point when production speeds are maximized but quality can’t keep up. The automobile industry went through many difficult years retooling for increased quality. The Agile and DevOps communities have been successful at using faster iterations to keep software projects from straying too far afield. So we are building code faster and faster, but security hasn’t evolved. We simply have to find some new technologies and techniques for gaining assurance at DevOps speed and portfolio scale.

Refactoring AppSec for speed and scale
First, we need to abandon the notion of security exceptionalism, and learn from other industries. We can instrument the entire software development process -- design, development, test, and even production -- so that applications continuously test themselves and provide real time security feedback as they run. Essentially, we have to make security testing, intrusion detection and response, and runtime protection a part of every application.

Organizations like Etsy, Netflix, Twitter, and Yelp have already realized the problem and have started to deploy new security tools that work differently. These tools are not legacy tools that run at the end of the development process. These tools instrument the software development process, gathering security information in real time as applications are built, integrated, tested, and deployed. Most importantly, these tools, like continuous integration and continuous delivery tools, don’t disrupt the normal software delivery process. Security tools that interfere with or slow down software delivery don’t get used. Zane Lackey from Signal Sciences put it best, “development organizations view delays as damage, and route around them.”

How can we get there? It’s not as hard as you might think. You can start today by creating a script, test case, or tool that performs a single simple security test and distributing it across your application portfolio. Or use the free Contrast for Eclipse plugin that I wrote about last month. Much of the infrastructure you’ll need has probably already been created by Agile and DevOps teams.

We need to reimagine all of our security testing techniques so that they make sense in a continuous environment. We also need our security experts to become coaches and toolsmiths rather than the ones to chase down every vulnerability -- because that will never scale.

A pioneer in application security, Jeff Williams is the founder and CTO of Contrast Security, a revolutionary application security product that enhances software with the power to defend itself, check itself for vulnerabilities, and join a security command and control ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
af_bugcrowd
50%
50%
af_bugcrowd,
User Rank: Author
11/12/2014 | 6:31:52 PM
Re: Devils you know
It's only going to be worse in mobile vulnerabilities once the eyeballs and money on them catches up to usage. There are so many different devices that are going to be unique creases in many of them.  
Marilyn Cohodas
50%
50%
Marilyn Cohodas,
User Rank: Strategist
11/11/2014 | 9:39:44 AM
Re: Devils you know
@Stratustician It's hard for me to imagine that DevOps can still be so tone deaf when it comes to appsec when vulnerabilities like Heartbleed and POODLE have become so commonplace in security news reports. 
Marilyn Cohodas
50%
50%
Marilyn Cohodas,
User Rank: Strategist
11/10/2014 | 3:43:24 PM
Re: What can you control?
It's  hard for me to imagine getting 100 percent compliance with anything. So anything that moves the needle in the right direction would seem to make sense to me..
TerryB
100%
0%
TerryB,
User Rank: Ninja
11/10/2014 | 1:27:34 PM
What can you control?
Great article and insight. But doesn't this only work if EVERYONE in the stack does it? Your testing may show that your product is fine but holes in o/s or browser still expose you. Maybe I'm just naive but were the old mainframes running MVS-370 as vulnerable as Windows/browser you run apps on today?
Stratustician
100%
0%
Stratustician,
User Rank: Moderator
11/10/2014 | 1:27:02 PM
Re: Devils you know
So true.  For me I find it's usually one of 2 issues that cause these security issues. Often DevOps teams don't have good security input early enough in the design stage, or don't keep up with testing prior to release. The other option is they bring security in way too late and then plug their ears when vulnerabilities in the code are detected by the security team after release, and then it's much harder to pull the app or change it without causing downtime. 

That or overall folks just don't feel that app security is that important to the overall security posture of an organization, which sadly is why we might be in this position in the first place.
Whoopty
100%
0%
Whoopty,
User Rank: Ninja
11/10/2014 | 12:19:33 PM
Devils you know
I know a few people who do a little freelance tech security work and several of them have pointed out major bugs to corporations about their sites or apps and often times, the flaw isn't fixed because it would require such an upheavel. It's worrying and makes you wonder when a big breach occurs, whether the company knew about it and chose to do nothing.

The delemna of course is that those firms will be damned if they interrupt service for a fix, and damned if they get caught out later. It's a difficult situation to be in. 
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
Five Emerging Security Threats - And What You Can Learn From Them
At Black Hat USA, researchers unveiled some nasty vulnerabilities. Is your organization ready?
Flash Poll
DevOps Impact on Application Security
DevOps Impact on Application Security
Managing the interdependency between software and infrastructure is a thorny challenge. Often, its a developers are from Mars, systems engineers are from Venus situation.
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2013-7445
Published: 2015-10-15
The Direct Rendering Manager (DRM) subsystem in the Linux kernel through 4.x mishandles requests for Graphics Execution Manager (GEM) objects, which allows context-dependent attackers to cause a denial of service (memory consumption) via an application that processes graphics data, as demonstrated b...

CVE-2015-4948
Published: 2015-10-15
netstat in IBM AIX 5.3, 6.1, and 7.1 and VIOS 2.2.x, when a fibre channel adapter is used, allows local users to gain privileges via unspecified vectors.

CVE-2015-5660
Published: 2015-10-15
Cross-site request forgery (CSRF) vulnerability in eXtplorer before 2.1.8 allows remote attackers to hijack the authentication of arbitrary users for requests that execute PHP code.

CVE-2015-6003
Published: 2015-10-15
Directory traversal vulnerability in QNAP QTS before 4.1.4 build 0910 and 4.2.x before 4.2.0 RC2 build 0910, when AFP is enabled, allows remote attackers to read or write to arbitrary files by leveraging access to an OS X (1) user or (2) guest account.

CVE-2015-6333
Published: 2015-10-15
Cisco Application Policy Infrastructure Controller (APIC) 1.1j allows local users to gain privileges via vectors involving addition of an SSH key, aka Bug ID CSCuw46076.

Dark Reading Radio
Archived Dark Reading Radio
Join Dark Reading community editor Marilyn Cohodas and her guest, David Shearer, (ISC)2 Chief Executive Officer, as they discuss issues that keep IT security professionals up at night, including results from the recent 2016 Black Hat Attendee Survey.