Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Perimeter

3/22/2010
05:19 PM
John H. Sawyer
John H. Sawyer
Commentary
50%
50%

Automated Web Scanners Bring The Noise

One fish, two fish, red fish, skipfish...huh? That was my initial thought. Skipfish is definitely an interesting name for a Web application security scanner. It sounds like it came straight out of a Dr. Seuss book, but instead it's an awesome new tool from Michal Zalewski and Google.

One fish, two fish, red fish, skipfish...huh? That was my initial thought. Skipfish is definitely an interesting name for a Web application security scanner. It sounds like it came straight out of a Dr. Seuss book, but instead it's an awesome new tool from Michal Zalewski and Google.Web security tools aren't anything new to Michal, who previously wrote ratproxy, a passive semiautomated Web application security-auditing tool that acts as a proxy to your Web browser while performing checks. Initial reports state that skipfish is fast but noisy. After some testing of my own, it is screaming fast. And as you might expect from an automated scanner, it is also very noisy.

So here's my question to you: Isn't that what you'd expect from an automated Web application scanner? I've used numerous open-source Web scanning tools from simple scanners, like Nikto, to more powerful and complex tools, like w3af. I even use a commercial solution. Whenever you're scanning and using brute-force file and directory discovery, it's going to be noisy, with tens to hundreds of thousands of 404 (file missing) error messages as it's attempting to discovery interesting locations not linked from the main site.

After reading through some of the comments in blogs and on Twitter about Skipfish's "noisiness," I started comparing a couple of other tools that have brute-force file and directory discovery features. Nikto scans for more than 6100 dangerous files/CGIs and other checks. The documentation even states that it is not designed to be overly stealthy. No big surprise there.

In looking at the "dictionaries" included with skipfish, the complete config includes 1,802 "words" that will get combined with 92 "extensions" to become one monster list of files and directories that get probed.

DirBuster is another well-known discovery brute-forcing tool. I've only used it once in Web penetration testing training session so I forgot just how cool it was because of the way the directory lists are created. As the tool's overview page says, "tools of this nature are often as only good as the directory and file list they come with. A different approach was taken to generating this. The list was generated from scratch, by crawling the Internet and collecting the directory and files that are actually used by developers!"

That just rocks, and it gives DirBuster the largest list I've come across in a Web scanner. My next test will be to integrate the two and see if I can make Skipfish's brute-forcing capability even better, although at some point I can foresee it getting possibly too long making the scan unbearingly long.

Skipfish is definitely a cool tool with a bright future, but don't forget what it is and what it does. Its inherent nature is going to cause it to be a noisy tool because it scans a Web app looking for certain files and all those requests will (or should) be logged, which can generate some hefty log files.

Quick update: The documentation for the dictionaries included with Skipfish discusses the DirBuster approach at the very bottom of the document and why Skipfish doesn't use it. Basically, "keywords related to popular websites and brands are heavily overrepresented," and "some of the most interesting security-related keywords are not commonly indexed." So, I likely won't bother testing DirBuster's list with Skipfish and will trust Michals's design decision. :)

John H. Sawyer is a senior security engineer on the IT Security Team at the University of Florida. The views and opinions expressed in this blog are his own and do not represent the views and opinions of the UF IT Security Team or the University of Florida. When John's not fighting flaming, malware-infested machines or performing autopsies on blitzed boxes, he can usually be found hanging with his family, bouncing a baby on one knee and balancing a laptop on the other. Special to Dark Reading.

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 8/10/2020
Researcher Finds New Office Macro Attacks for MacOS
Curtis Franklin Jr., Senior Editor at Dark Reading,  8/7/2020
Hacking It as a CISO: Advice for Security Leadership
Kelly Sheridan, Staff Editor, Dark Reading,  8/10/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
Special Report: Computing's New Normal, a Dark Reading Perspective
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
The Changing Face of Threat Intelligence
The Changing Face of Threat Intelligence
This special report takes a look at how enterprises are using threat intelligence, as well as emerging best practices for integrating threat intel into security operations and incident response. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-8720
PUBLISHED: 2020-08-13
Buffer overflow in a subsystem for some Intel(R) Server Boards, Server Systems and Compute Modules before version 1.59 may allow a privileged user to potentially enable denial of service via local access.
CVE-2020-12300
PUBLISHED: 2020-08-13
Uninitialized pointer in BIOS firmware for Intel(R) Server Board Families S2600CW, S2600KP, S2600TP, and S2600WT may allow a privileged user to potentially enable escalation of privilege via local access.
CVE-2020-12301
PUBLISHED: 2020-08-13
Improper initialization in BIOS firmware for Intel(R) Server Board Families S2600ST, S2600BP and S2600WF may allow a privileged user to potentially enable escalation of privilege via local access.
CVE-2020-7307
PUBLISHED: 2020-08-13
Unprotected Storage of Credentials vulnerability in McAfee Data Loss Prevention (DLP) for Mac prior to 11.5.2 allows local users to gain access to the RiskDB username and password via unprotected log files containing plain text credentials.
CVE-2020-8679
PUBLISHED: 2020-08-13
Out-of-bounds write in Kernel Mode Driver for some Intel(R) Graphics Drivers before version 26.20.100.7755 may allow an authenticated user to potentially enable denial of service via local access.