Erase The Line Between QA Defects And Security Flaws?

When it comes time to test applications for flaws, when is a bug a security defect, and when is it a quality defect? According to some developer experts, during preproduction the line can be so blurry that the industry would do well to quit trying to draw it and instead endeavor to do testing that reduces overall defect rates so that the code quality and, consequently, security increase across the board.

"The way we view it is that for developers, a defect is a defect. We believe the overlap in quality and security is huge, and you really shouldn't distinguish between a security defect and a quality defect," says Zack Samocha, project director for Coverity Scan at testing vendor Coverity. [Why does SQL injection linger? See 10 Reasons SQL Injection Still Works.]

Amid a flurry of new reports released about the state of secure application development, a bit of a silver lining peeped through the clouds this week with the release of the "2012 Coverity Scan Open Source Report," which showed that the overall code quality improved within a set of 450 million lines of open-source code examined by Coverity in 2012. Among the common types of code defects fixed were some of those most likely to cause security vulnerabilities, including control flow issues, null pointer dereferences, and resource leaks.

From a security angle, Samocha believes that the takeaway from the report is that if an organization can push to get developers to improve the overall code quality, then security will see commensurate improvements in the process.

"You need to think about your code quality and code practice, in general," he says. "To be successful with security, you need to have security as a part of developer requirements, part of their defects to fix, unrelated to security or quality."

The logistics behind this rising tide to flaw discovery could put more urgency behind remediation of security problems during preproductions, if done right. As things stand, security issues are often left unfound or unfixed in the rush to ship code. Defects with security implications would be more likely to trip the development "circuit breaker" during QA if policies mandate higher levels of quality before code is made live.

"One of the things that is a truism in software development is if you actually put a requirement or something at the end, and it stops them from shipping something, the software producers, in general, are really attuned to that," says Mike Armistead, vice president and general manager of HP Fortify. "When you do put that bar there, the right thing starts to happen."

But getting rid of the distinction between quality and security defects could mean asking developers and QA testers to test for flaws they may not have experience with. According to Dan Cornell, principal of Denim Group, getting these populations on board for this kind of testing will require education and clear directions from management.

"Developers and testers already have jobs to do, and, prior to your current initiative, those jobs probably didn't require them to do security testing," he wrote recently. "Assuming you want to developers and testers to adopt new practices and new tools at any sort of scale you need to show them what they need to do and be as specific and prescriptive as possible so that you minimize the impact on all the other stuff the developers and QA testers are supposed to do."

However, taking the tack of putting development defects all in the same bucket during developer testing doesn't necessarily mean security gets subsumed by QA and developers. It just improves the refinement process prior to bringing in secure development experts.

"If you do it well, then later in the cycle if security audit is testing something, hopefully they will just find the things only they can find, rather than all of the defects a developer could have fixed earlier in the cycle," Samocha says.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.