Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Application Security

11:30 AM
Jeff Williams
Jeff Williams
Connect Directly
E-Mail vvv

The Staggering Complexity of Application Security

During the past few decades of high-speed coding we have automated our businesses so fast that we are now incapable of securing what we have built.

First, a few facts.

A typical, midsized financial organization has a portfolio of over a thousand applications. The largest firms exceed ten thousand applications. Each of these applications, on average, has hundreds of thousands of lines of custom code, and the largest can have over ten million lines. In addition, each application also includes anywhere from dozens to hundreds of software libraries, frameworks, and components that typically total over ten times the size of the custom code. And this portfolio is growing rapidly -- over 20% of these applications have new and updated code each year.

By comparison, consider the US Federal Tax Code, which has also grown dramatically over the years. Currently, the tax code totals just 4.4 million lines of “code” – roughly equivalent to just a handful of applications. As a security researcher I’ve discovered thousands of vulnerabilities in code. But as a former CEO, I’ve also analyzed a ton of legal contracts for loopholes. What’s interesting is that whether I’m scrutinizing software code or reviewing legal language, the analysis is not as different as you might think. Both require a detailed understanding of specialized language and a solid understanding of the underlying business.

So after two decades of high speed coding, a typical large financial organization has a pile of code as large as 2,000 copies of the entire 73,954 pages in the US Federal Tax Code -- almost 10 billion lines of code.

The current security situation
After the details of JP Morgan’s breach were revealed recently, a former employee told The New York Times that it was as if “[the attackers] stole the schematics to the Capitol -- [JP Morgan] can’t just switch out every single door and window pane overnight.” Actually, I think it’s considerably worse than that. It took decades to create the software infrastructure at JP Morgan, and there is no practical way to just change it out. To me, it’s more like discovering that an entire city has been built with asbestos insulation, lead pipes, and unshielded power lines. And they all need to be swapped out without disrupting service.

Now, according to experts at Aspect Security, the typical enterprise web application has an eye-opening 22.4 serious vulnerabilities. They aren’t always easy to find, and they vary in likelihood and severity of exploit. But combining this level of vulnerability with an increasingly sophisticated threat has led to a marked increase in breaches. The breaches just this year have been sobering.

Click on this link  for an interactive view of the Word Cloud by David McCandless.
Click on this link for an interactive view of the Word Cloud by David McCandless.

The limits of traditional AppSec
In the good old days, we did manual penetration testing and code review to find vulnerabilities. These reviews got pretty good coverage and developers had time to fix problems before the code went into production. But recent advances in software development, including widespread use of libraries and components, high-speed development methodologies, complex frameworks, and inscrutable protocols have made manual analysis too slow for all but the most critical applications.

Many industries reach a point when production speeds are maximized but quality can’t keep up. The automobile industry went through many difficult years retooling for increased quality. The Agile and DevOps communities have been successful at using faster iterations to keep software projects from straying too far afield. So we are building code faster and faster, but security hasn’t evolved. We simply have to find some new technologies and techniques for gaining assurance at DevOps speed and portfolio scale.

Refactoring AppSec for speed and scale
First, we need to abandon the notion of security exceptionalism, and learn from other industries. We can instrument the entire software development process -- design, development, test, and even production -- so that applications continuously test themselves and provide real time security feedback as they run. Essentially, we have to make security testing, intrusion detection and response, and runtime protection a part of every application.

Organizations like Etsy, Netflix, Twitter, and Yelp have already realized the problem and have started to deploy new security tools that work differently. These tools are not legacy tools that run at the end of the development process. These tools instrument the software development process, gathering security information in real time as applications are built, integrated, tested, and deployed. Most importantly, these tools, like continuous integration and continuous delivery tools, don’t disrupt the normal software delivery process. Security tools that interfere with or slow down software delivery don’t get used. Zane Lackey from Signal Sciences put it best, “development organizations view delays as damage, and route around them.”

How can we get there? It’s not as hard as you might think. You can start today by creating a script, test case, or tool that performs a single simple security test and distributing it across your application portfolio. Or use the free Contrast for Eclipse plugin that I wrote about last month. Much of the infrastructure you’ll need has probably already been created by Agile and DevOps teams.

We need to reimagine all of our security testing techniques so that they make sense in a continuous environment. We also need our security experts to become coaches and toolsmiths rather than the ones to chase down every vulnerability -- because that will never scale.

A pioneer in application security, Jeff Williams is the founder and CTO of Contrast Security, a revolutionary application security product that enhances software with the power to defend itself, check itself for vulnerabilities, and join a security command and control ... View Full Bio
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
11/12/2014 | 6:31:52 PM
Re: Devils you know
It's only going to be worse in mobile vulnerabilities once the eyeballs and money on them catches up to usage. There are so many different devices that are going to be unique creases in many of them.  
Marilyn Cohodas
Marilyn Cohodas,
User Rank: Strategist
11/11/2014 | 9:39:44 AM
Re: Devils you know
@Stratustician It's hard for me to imagine that DevOps can still be so tone deaf when it comes to appsec when vulnerabilities like Heartbleed and POODLE have become so commonplace in security news reports. 
Marilyn Cohodas
Marilyn Cohodas,
User Rank: Strategist
11/10/2014 | 3:43:24 PM
Re: What can you control?
It's  hard for me to imagine getting 100 percent compliance with anything. So anything that moves the needle in the right direction would seem to make sense to me..
User Rank: Ninja
11/10/2014 | 1:27:34 PM
What can you control?
Great article and insight. But doesn't this only work if EVERYONE in the stack does it? Your testing may show that your product is fine but holes in o/s or browser still expose you. Maybe I'm just naive but were the old mainframes running MVS-370 as vulnerable as Windows/browser you run apps on today?
User Rank: Moderator
11/10/2014 | 1:27:02 PM
Re: Devils you know
So true.  For me I find it's usually one of 2 issues that cause these security issues. Often DevOps teams don't have good security input early enough in the design stage, or don't keep up with testing prior to release. The other option is they bring security in way too late and then plug their ears when vulnerabilities in the code are detected by the security team after release, and then it's much harder to pull the app or change it without causing downtime. 

That or overall folks just don't feel that app security is that important to the overall security posture of an organization, which sadly is why we might be in this position in the first place.
User Rank: Ninja
11/10/2014 | 12:19:33 PM
Devils you know
I know a few people who do a little freelance tech security work and several of them have pointed out major bugs to corporations about their sites or apps and often times, the flaw isn't fixed because it would require such an upheavel. It's worrying and makes you wonder when a big breach occurs, whether the company knew about it and chose to do nothing.

The delemna of course is that those firms will be damned if they interrupt service for a fix, and damned if they get caught out later. It's a difficult situation to be in. 
7 Old IT Things Every New InfoSec Pro Should Know
Joan Goodchild, Staff Editor,  4/20/2021
Cloud-Native Businesses Struggle With Security
Robert Lemos, Contributing Writer,  5/6/2021
Defending Against Web Scraping Attacks
Rob Simon, Principal Security Consultant at TrustedSec,  5/7/2021
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Write a Caption, Win an Amazon Gift Card! Click Here
Latest Comment: What Virtual Reality phishing attacks will look like in 2030.
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2021-05-11
A cross-site request forgery (CSRF) vulnerability in Jenkins Xray - Test Management for Jira Plugin 2.4.0 and earlier allows attackers to connect to an attacker-specified URL using attacker-specified credentials IDs obtained through another method, capturing credentials stored in Jenkins.
PUBLISHED: 2021-05-11
Jenkins Xray - Test Management for Jira Plugin 2.4.0 and earlier does not perform a permission check in an HTTP endpoint, allowing with Overall/Read permission to enumerate credentials IDs of credentials stored in Jenkins.
PUBLISHED: 2021-05-11
Jenkins P4 Plugin 1.11.4 and earlier does not perform permission checks in multiple HTTP endpoints, allowing attackers with Overall/Read permission to connect to an attacker-specified Perforce server using attacker-specified username and password.
PUBLISHED: 2021-05-11
A cross-site request forgery (CSRF) vulnerability in Jenkins P4 Plugin 1.11.4 and earlier allows attackers to connect to an attacker-specified Perforce server using attacker-specified username and password.
PUBLISHED: 2021-05-11
Jenkins Xcode integration Plugin 2.0.14 and earlier does not configure its XML parser to prevent XML external entity (XXE) attacks.