Risk
1/30/2009
12:53 PM
Connect Directly
Google+
Twitter
RSS
E-Mail
50%
50%

Tech Insight: How to Pick The Right Web Application Vulnerability Scanner

There's more to a "black box" scanner than the number of vulnerabilities it can detect

The mistake most people make when they first buy a Web application vulnerability scanner is to assume it's a simple point-and-click tool.

"It's not like network scanning where you go to an IP address and scan the network," says Danny Allan, director of security research for IBM Rational Software, which sells the AppScan vulnerability scanner. "This is not just a point-and-click product."

Web application vulnerability scanning -- also known as "black box" testing (as opposed to source-code scanning, or white-box testing) -- touches on various levels, transactions, and interactions associated with a Web application. And it requires an experienced hand to run it in order to get the most out of the process of detecting security flaws in Web applications, security experts say.

"The people who are running the scanner matter a lot more than the scanner itself. These are not simple hammers anyone can use. They require the operator to have a significant level of Web security knowledge," says Jeremiah Grossman, CTO of WhiteHat Security, a Web security services firm.

Another misconception about these devices is that the more vulnerabilities they find, the better they are. "Many people go by vuln counts in Web scanners, which is incorrect," notes Caleb Sima, CTO of the application security center at HP, which sells the WebInspect Web app scanner. That's because some products lump together multiple iterations of a specific vulnerability. If one scanner finds 12 SQL injection flaws, and another finds five, it doesn't mean the second one is necessarily missing more bugs, Sima says.

Consider, then, how a scanner counts vulnerabilities rather than how many times it finds SQL injection bugs. Eve more important, IBM's Allan says, is the underlying coding problem that caused the vulnerability. "You may have one cross-site scripting vulnerability and 80 different ways to exploit it," he says. "You need to focus not on the [vulnerability] issue, but on why it happened...that helps prevent security issues from happening again in the future."

That said, it's not so simple to compare these products head-to-head. A European security researcher's recently released test results comparing three major Web app scanners highlighted those challenges given the differences in their approaches. Anantasec performed vulnerability scans against several applications using Acunetix WVS version 6.0, IBM Rational AppScan Version 7.7.620 Service Pack 2, and HP WebInspect Version 7.7.869. He concluded that Acunetix performed the best overall, but as a second layer of analysis, he also used Acunetix's AcuSensor, which looks at source code using a form of white-box testing. So it wasn't actually an apples-to-apples comparison, experts say.

The three products posted mixed results in finding specific vulnerabilities among different applications in the tests; for instance, in some apps one tool would miss XSS flaws, while in others it would find most of them. "Web applications can be very complex, and there are a lot of reasons that would cause a scanner to miss a vulnerability," Anantasec says. "Some of the reasons are poor crawling capabilities, bad JavaScript parsing, inconsistent scanning, or just bugs."

So what should you look for when selecting a Web application vulnerability scanner? IBM's Allan says to first look at how well they test for known vulnerabilities, conceding that most products are fairly equal in the regard. "Most products have similar capabilities in the testing," he says.

Another important feature is the ability to maintain your login state during a scan so that if the person running the scan gets logged out during a test, he doesn't have to start all over again. "The ability to login and maintain login state effectively is hugely important because if the scanner cannot [do so], the scan is invalid because the functionality [would remain] untested," WhiteHat's Grossman says.

In addition, look for whether the product supports JavaScript and Flash so you can scan for flaws in these application types. That's key given the growth of Web 2.0-based apps, experts say, but is still a weak area for scanners. "While these scanners can technically assess XML Web services and identify vulnerabilities in Flash-related software, no product has proved to be anywhere close to comprehensive," says Grossman, who cautions not to blindly trust the results of that part of the scan.

It helps to know the features and functions of the scanner before you test it so you can get the most out of your test -- and prevent any problems. Don't just rely on a demo test the vendor provides, either. Test it against your own Website apps.

Not all products have all of their features on by default, so learn about the options and don't just enable them all without considering the impact on your specific application. "Turning on all security tests and scan configuration can be damaging," IBM's Allan notes.

A retailer testing IBM's AppScan, for example, turned on all of its options -- including "execute JavaScript" -- which inadvertently crashed its email server. "They executed JavaScript...[The application] had a 'mail to' form and sent thousands of requests to the mail server and crashed it," he says. "You need to see if these are the right options [for scanning your application]. Thankfully, it was a fairly minor incident, but it shows that scan configuration needs to be considered carefully -- especially in production systems."

Meanwhile, HP's Sima says many of his company's enterprise customers are going to the next level -- performing recurring Web scanning. "This is where the majority of scanning is done on an after-production scale," he says. "This is a scan of a high-level policy on all Web-facing properties that hold the company's branding, including off-site. It allows the company to identify its risk and quickly identify glaring holes in things it may or may not control, but can cause damage from a branding perspective -- or worse."

One aspect security experts agree on is that black-box testing alone isn't enough. "Black-box testing should be used together with a sensor technology like AcuSensor, and you should also add source-code review into the mix," Anantasec says.

Nick Selby, vice president and research director with The 451 Group, says to look out for false positives and false negatives with these tools, as well.

"At their most basic, Web application vulnerability assessment tools can give you a basic snapshot of known vulnerabilities in Web applications, and some kind of explanation of what it finds, what the severity of a given vuln is, and tips and suggestions on how to fix them," Selby says. "Because this is a very fluid environment, though, false positives and false negatives abound. That's why we suggest vulnerability scanning services in addition to the software -- many firms including White Hat and Cenzic offer these [services]."

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message Kelly Jackson Higgins is Executive Editor at DarkReading.com. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Register for Dark Reading Newsletters
Partner Perspectives
What's This?
In a digital world inundated with advanced security threats, Intel Security seeks to transform how we live and work to keep our information secure. Through hardware and software development, Intel Security delivers robust solutions that integrate security into every layer of every digital device. In combining the security expertise of McAfee with the innovation, performance, and trust of Intel, this vision becomes a reality.

As we rely on technology to enhance our everyday and business life, we must too consider the security of the intellectual property and confidential data that is housed on these devices. As we increase the number of devices we use, we increase the number of gateways and opportunity for security threats. Intel Security takes the “security connected” approach to ensure that every device is secure, and that all security solutions are seamlessly integrated.
Featured Writers
White Papers
Cartoon
Current Issue
Dark Reading's October Tech Digest
Fast data analysis can stymie attacks and strengthen enterprise security. Does your team have the data smarts?
Flash Poll
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2013-3304
Published: 2014-10-30
Directory traversal vulnerability in Dell EqualLogic PS4000 with firmware 6.0 allows remote attackers to read arbitrary files via a .. (dot dot) in the default URI.

CVE-2013-7409
Published: 2014-10-30
Buffer overflow in ALLPlayer 5.6.2 through 5.8.1 allows remote attackers to cause a denial of service (crash) and possibly execute arbitrary code via a long string in a .m3u (playlist) file.

CVE-2014-3446
Published: 2014-10-30
SQL injection vulnerability in wcm/system/pages/admin/getnode.aspx in BSS Continuity CMS 4.2.22640.0 allows remote attackers to execute arbitrary SQL commands via the nodeid parameter.

CVE-2014-3584
Published: 2014-10-30
The SamlHeaderInHandler in Apache CXF before 2.6.11, 2.7.x before 2.7.8, and 3.0.x before 3.0.1 allows remote attackers to cause a denial of service (infinite loop) via a crafted SAML token in the authorization header of a request to a JAX-RS service.

CVE-2014-3623
Published: 2014-10-30
Apache WSS4J before 1.6.17 and 2.x before 2.0.2, as used in Apache CXF 2.7.x before 2.7.13 and 3.0.x before 3.0.2, when using TransportBinding, does properly enforce the SAML SubjectConfirmation method security semantics, which allows remote attackers to conduct spoofing attacks via unspecified vect...

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
Follow Dark Reading editors into the field as they talk with noted experts from the security world.