Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


04:43 PM
Connect Directly

Test: Most Web Application Scanners Missed Nearly Half Of Vulnerabilities

Tools were scanning their own test Websites

Most Web application scanning tools miss vulnerabilities and generate false positives on their own public testing sites, according to a recent test of some of these products.

Larry Suto, an application security consultant, tested the Web app scanners for accuracy and false positives as well as the time it took with each to get the best possible results, including running, reviewing, and supplementing the results from the scans. He tested Acunetix, IBM's AppScan, Portswigger.net BurpSuitePro, Cenzic's Hailstorm, HP's WebInspect, NT Objectives' NTOSpider, and Qualys' managed scanning service.

Suto says what surprised him most about what he found in the tests was how the tools didn't catch vulnerabilities and threw false positives when scanning their own test Websites. "I think the report shows that while these tools are very helpful, one should not rely on them exclusively for security," he says.

He found that, overall, the tools missed nearly half of the vulnerabilities on the sites -- even when they were fully "trained," or tuned, rather than set to point and scan the sites. And the big-name products weren't necessarily the most accurate: NTOSpider fared the best overall, with a 94 percent accuracy rating after being trained, followed by Hailstorm, with 62 percent, AppScan (55 percent), Acunetix (47 percent), BurpSuitePro (36 percent), and WebInspect (33 percent). Qualys' service caught 28.5 percent, but Suto notes that it's pure "point and shoot" scanning.

The test (PDF) counted specific types of vulnerabilities: authentication bypass or brute forcing, SQL injection/blind SQL injection, cross-site scripting/persistent cross-site scripting, command injection, XPath injection, SOAP/AJAX attacks, CSRF/HTTP response splitting, arbitrary file upload attacks, remote file include (PHP code injection), and application errors.

Jeremiah Grossman, founder and CTO of WhiteHat Security, says the culprit with accuracy is the scanner's crawling engine. "The crawling engine is most important," Grossman says. "When scanners miss vulnerabilities, it's almost always the case that they didn't find the link they had to test that one vulnerability."

Suto found "moderate" improvements in accuracy when the tools were "trained," except for the case of Hailstorm, which went from 39.6 percent accuracy with "point and shoot" to 62 percent after being trained.

The fastest scanners (in order) were: BurpSuitePro, Qualys, WebInspect, NTOSpider, Appscan, Hailstorm, and Acunetix. When factoring in man-hours, BurpSuitePro and Qualys also came out on top.

Suto's findings in his Web app scanner tests reflect the challenges facing Web application security, as Web applications are often riddled with flaws. "This is a lot harder problem than network scanning. These results should cause security professionals to have significant reason for concern if they are relying on one of the less accurate tools. There is a good chance that they are missing a significant number of vulnerabilities," he wrote in the report.

Wolfgang Kandek, CTO at Qualys, says while a network application scanner detects an old version of an app and then pushes a patch, Web application scanning is much more complicated: "With a Web application scan, you have to go back to your developer and tell him or her, 'Here's the problem,' " he says. There's the element of re-education that comes in the wake of a Web vulnerability discovery, he says.

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.

Kelly Jackson Higgins is the Executive Editor of Dark Reading. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Ransomware Is Not the Problem
Adam Shostack, Consultant, Entrepreneur, Technologist, Game Designer,  6/9/2021
How Can I Test the Security of My Home-Office Employees' Routers?
John Bock, Senior Research Scientist,  6/7/2021
New Ransomware Group Claiming Connection to REvil Gang Surfaces
Jai Vijayan, Contributing Writer,  6/10/2021
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Write a Caption, Win an Amazon Gift Card! Click Here
Latest Comment: Who knew face masks could also prevent the PII from spreading
Current Issue
The State of Cybersecurity Incident Response
In this report learn how enterprises are building their incident response teams and processes, how they research potential compromises, how they respond to new breaches, and what tools and processes they use to remediate problems and improve their cyber defenses for the future.
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2021-06-15
Apache HTTP Server protocol handler for the HTTP/2 protocol checks received request headers against the size limitations as configured for the server and used for the HTTP/1 protocol as well. On violation of these restrictions and HTTP response is sent to the client with a status code indicating why...
PUBLISHED: 2021-06-14
A buffer overflow vulnerability in SonicOS allows a remote attacker to cause a Denial of Service (DoS) by sending a specially crafted request. This vulnerability affects SonicOS Gen5, Gen6, Gen7 platforms, and SonicOSv virtual firewalls.
PUBLISHED: 2021-06-14
magento-scripts contains scripts and configuration used by Create Magento App, a zero-configuration tool-chain which allows one to deploy Magento 2. In versions 1.5.1 and 1.5.2, after changing the function from synchronous to asynchronous there wasn't implemented handler in the start, stop, exec, an...
PUBLISHED: 2021-06-14
net/can/bcm.c in the Linux kernel through 5.12.10 allows local users to obtain sensitive information from kernel stack memory because parts of a data structure are uninitialized.
PUBLISHED: 2021-06-14
Cross-site Scripting (XSS) vulnerability in the main dashboard of Ellipse APM versions allows an authenticated user or integrated application to inject malicious data into the application that can then be executed in a victim’s browser. This issue affects: Hitachi ABB Power Grids ...