Getting The Most From Web Application Testing Results

While most organizations today have implemented some level of Web application vulnerability testing tools, relatively few have managed to reap the kind of benefits they may have initially expected when they first deployed. The messy truth to it all is that the tools are only the first step to improving the internal code base, and most organizations haven't found a way to bridge the gap between security and developer teams.

"The real challenge with application security is the security folks can run scans and identify problems, and they understand the risk in a lot of cases, but they don't have the background to go in and actually change the code," says Dan Cornell, principal of Denim Group, an application security consulting group.

That means the results of those scans must be communicated to developers. And that's where things often fall apart. Security teams don't speak the developer lingo, they don't try to offer suggestions through tools and workflows that developers use every day, and they don't package fix requests in a way that would make it easier for developers to satisfy them. What's more, security teams offer very little guidance on the root causes of the vulnerabilities in the code so that developers can make long-term changes to their practices in order to reduce the number of vulnerabilities they're producing in the long term.

Here's how security teams can start making it easier for developers to make fixes.

[How does HTML5 increase risk? See Beware of HTML5 Development Risks.]

1. View Vulns As Software Defects
If you really want to speak in developers' language, then step one is to stop referring to vulnerabilities as vulnerabilities.

"A lot of security folks will go to the developers and be like, 'Well, you guys must be really stupid because you wrote code with all these vulnerabilities, and here's a 300-page PDF report that you guys have to work off of to fix these problems,'" Cornell says.

Not only is that a terrible interpersonal interaction, he says, it's not a very effective way to play to a developer's technical strengths, either. To a developer, it's either a software defect or it isn't. Rather than simply feeding developers raw vulnerability data, security teams should be thinking about how to translate that into a software defect that can be concretely addressed through coding changes.

"Security teams need to be able to understand how to talk to developers; developers speak in relation to features and bugs," says Dan Kuykendall, CTO of application scanning vendor NT OBJECTives. "Their tools and their schedules and everything is built around bugs and features. So stop making it a 'security issue.' Then you can actually have a conversation."

2. Make Defects Information Available In Existing Developer Tools
Most security teams expect developers to either learn a new tool or do the translation necessary to import defects into the processes and bug-tracking tools developers already have in place to manage their existing workload. According to Cornell, if security would like to see better developer cooperation, then they should provide that information ready-made for how the developers do their everyday work.

The way developers change code in their world, they receive a task request or a defect in their bug-tracking system, it gets assigned to a specific developer, they fix it, check it in, QA verifies it, and it is pushed out, he says.

"If security teams want to see developers taking action, there's a whole lot less friction involved if they're able to meet the developers with the systems they're already using and the processes and constructs they're used to using to manage their workload," says Cornell. "Frankly, to approach them in a different way is a little bit arrogant on the part of the security folks and disrespectful to what the developers have done to create the systems that they're already using to manage their workloads."

3. Package Defects For Less Dev Admin Time
Not only should security organizations consider ensuring that data is consumable within a developer's bug-tracking tool, they should also try to make it easily consumable. As things stand, says Cornell, security scanners are built for thoroughness, often pumping out hundreds of vulnerabilities at a time, particularly early on in a scanning program.

But that doesn't mean there should necessarily be a one-to-one mapping in most cases to those vulnerabilities becoming software defects each needing a fix within a bug-tracker tool.

"If you run a scanner and find a hundred cross-site scripting vulnerabilities and 200 SQL injection vulnerabilities, and then take each of those vulnerabilities and turn that into a defect, you are going to get murdered in the parking lot by your QA manager," Cornell says.

It is critical that security teams "slice and dice" the vulnerability data and package it in a way that makes sense for developers. For example, they could group all of the vulnerabilities on a specific page or in a specific subdirectory into a single defect header so that they're all assigned to the developer who is responsible for the functionality on that page or in that directory.

"You may have a list of 500 things that need to get done, but at the end of the day that might need to be 20 or 25 bugs to go over to the software developers," Cornell says. "You want to package these vulnerabilities up in a way that they look like the software defects that developers are expecting."

Doing so reduces the amount of time development leads need to take just sifting through the data to assign out vulnerabilities so that they can focus on the coding fixes themselves.

Similarly, during that packaging process, security teams should aim to reduce the amount of duplicative results from multiple scanning tools.

"Sometimes one tool will find something that another tool doesn't, but a lot of times they overlap considerably," says Kuykendall, who says security teams will endear themselves to developers if they do the deduplication on the front end rather than making the developers scratch their heads over the same vulnerabilities found by different tools.

4. Offer Guidance Along With The Submitted Defect
Security teams will earn special brownie points with development teams if they not only package and translate vulnerabilities for their tools, but also include guidance with those submitted defects about how to make the fixes or the root causes behind the problems.

"Not that anybody is going to write anything that's revolutionary about how to fix cross-site scripting or SQL injection, but it's really about getting developers -- especially developers writing in different languages -- links to external resources and basic descriptions and recommendations to help them fix the problems," Kuykendall says. "A lot of times when you're merging results, that gets lost. It's key to make sure that information gets carried over."

5. Center Scanning Around Processes, Not Product
"We see a lot of organizations invest in great tools; they take the tool, they benchmark it, get the license, and scan the heck out of everything the week they get it because it's a new toy," Cornell says. "But then the challenge is how do you make sure that you maximize the value that you get out of the investment in these things, and what that comes down to is really having a program in place."

Having the technology is important, but really getting ROI out of it demands using it on a defined schedule with checkpoints in the development process, Cornell says.

Developing a scanning program around processes -- specifically existing QA and code fixing processes -- rather than products will also help developers deliver better code in the long run, Kuykendall says.

"When security teams can plug into these processes and not worry so much about which product is used, there's less resistance that way," he says.

Not only that, say Cornell and Kuykendall, this kind of process also makes it easier to take bug fixing to the next level. Companies that take a programmatic approach can start looking at analytics, looking for vulnerability prevalence and trends that can be fed back into the development process so that the number of vulnerabilities being produced in the first place is reduced over time.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.