Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Comments
The Common Core Of Application Security
Newest First  |  Oldest First  |  Threaded View
DaveWichers
50%
50%
DaveWichers,
User Rank: Apprentice
9/25/2015 | 5:49:01 PM
Re: OWASP Benchmark Clarifications
Yes. That's me. And your concern is fair and you aren't the first to bring it up. We are addressing this by making everything free and open and reproducible by anyone, and more importantly, getting lots more people involved, so we can eliminate any potential for bias. We already have a number of open source projects contributing to the project, and a bunch of commercial vendors and even some non-vendors approached me at this week's AppSec USA conference and asked to get involved, which I welcome whole heartedly. We are going to expand the team to as many who want to participate, and ensure there are many eyes and many contributors to the work we produce. The OWASP Board has expressed their support for this project, as it's exactly the kind of thing OWASP should be doing. This project is really getting some momentum and together we can all make it great. Please contribute if it's of interest to you.
BigJim2
50%
50%
BigJim2,
User Rank: Apprentice
9/25/2015 | 9:57:08 AM
Re: OWASP Benchmark Clarifications
Is this the same Dave Wichers that is a co-founder of Aspect Security, the company that created Constrast? It seems like a conflict of interest for someone from a vendor to create a benchmark that will be used to grade their competition. I'm even more surprised that a government agency (DHS) would sponsor that activity.
DaveWichers
50%
50%
DaveWichers,
User Rank: Apprentice
9/23/2015 | 3:42:42 PM
OWASP Benchmark Clarifications
Jason, this discussion is great and I'm thrilled that the OWASP Benchmark is driving improvements in application vulnerability detection tools. But I did want to add a few clarifications on how the Benchmark works.

In your Benchmark results table, you indicate: "True Positives detected by Fortify SCA, and declared Secure by Benchmark" - 9206. While its great that Fortify found all these additional vulnerabilities in the Benchmark, the Benchmark makes no claims there are no other vulnerabilities in it beyond the ones specifically tested for and scored. Any such results found by any tool are simply ignored in the Benchmark scoring system, so they have no effect on the score one way or the other. So, saying that Fortify found a bunch of issues the project wasn't aware of and other tools did not find simply isn't accurate. Most of the tools we tested found a bunch of additional issues, just like Fortify did.

As part of our 1.2 effort, we have eliminated a number of unintended vulnerabilities of the type tested for in the Benchmark, particularly XSS. This is an ongoing effort and we have more work to do there. In fact, if you can send us your results, we'll be happy to use them to help us track down and eliminate more of them. That said, these 'extra' vulnerabilities are, and should be, ignored as they simply aren't measured/scored.

You also mention: "False Positives reported by Fortify SCA" - 4,852.  In the Benchmark v1.1, there are 9206 True Negative test cases, Meaning 9206 test cases that are safe, and do not possess the type of vulnerability they are testing for. And Fortify reported 4,852 of the as actual vulnerabilities (False Positives as you said). The Benchmark project scores that as 4,852 out of 9,206, which is a 52.7% False Positive rate. So if your True Positive rate is actually 100% as you claim, the Benchmark would produce an average score for Fortify as 100% - 52.7% which equals 47.3%. This average score for Fortify is higher than the scores the project has seen with the results we were able to generate so we are pleased to see that your team's efforts have improved its score against the Benchmark and that your customer's will ultimately benefit from these improvements.

I think discussions like this are incredibly healthy and hope lots of vendors for both commercial and free tools will get involved to make both the OWASP Benchmark project and their tools better for the community we both serve.  And given the amount of discussions I'm having with project participants at OWASP, the discussions are just getting started, and many tools, including Fortify, are getting better already. And in fact, I'm going to talk about exactly that at my OWASP AppSec USA talk on the Benchmark project tomorrow afternoon at 4. If any of you are around, please come by!!

Dave Wichers

OWASP Benchmark Project Lead


7 Tips for Infosec Pros Considering A Lateral Career Move
Kelly Sheridan, Staff Editor, Dark Reading,  1/21/2020
For Mismanaged SOCs, The Price Is Not Right
Kelly Sheridan, Staff Editor, Dark Reading,  1/22/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
IT 2020: A Look Ahead
Are you ready for the critical changes that will occur in 2020? We've compiled editor insights from the best of our network (Dark Reading, Data Center Knowledge, InformationWeek, ITPro Today and Network Computing) to deliver to you a look at the trends, technologies, and threats that are emerging in the coming year. Download it today!
Flash Poll
How Enterprises are Attacking the Cybersecurity Problem
How Enterprises are Attacking the Cybersecurity Problem
Organizations have invested in a sweeping array of security technologies to address challenges associated with the growing number of cybersecurity attacks. However, the complexity involved in managing these technologies is emerging as a major problem. Read this report to find out what your peers biggest security challenges are and the technologies they are using to address them.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2015-3154
PUBLISHED: 2020-01-27
CRLF injection vulnerability in Zend\Mail (Zend_Mail) in Zend Framework before 1.12.12, 2.x before 2.3.8, and 2.4.x before 2.4.1 allows remote attackers to inject arbitrary HTTP headers and conduct HTTP response splitting attacks via CRLF sequences in the header of an email.
CVE-2019-17190
PUBLISHED: 2020-01-27
A Local Privilege Escalation issue was discovered in Avast Secure Browser 76.0.1659.101. The vulnerability is due to an insecure ACL set by the AvastBrowserUpdate.exe (which is running as NT AUTHORITY\SYSTEM) when AvastSecureBrowser.exe checks for new updates. When the update check is triggered, the...
CVE-2014-8161
PUBLISHED: 2020-01-27
PostgreSQL before 9.0.19, 9.1.x before 9.1.15, 9.2.x before 9.2.10, 9.3.x before 9.3.6, and 9.4.x before 9.4.1 allows remote authenticated users to obtain sensitive column values by triggering constraint violation and then reading the error message.
CVE-2014-9481
PUBLISHED: 2020-01-27
The Scribunto extension for MediaWiki allows remote attackers to obtain the rollback token and possibly other sensitive information via a crafted module, related to unstripping special page HTML.
CVE-2015-0241
PUBLISHED: 2020-01-27
The to_char function in PostgreSQL before 9.0.19, 9.1.x before 9.1.15, 9.2.x before 9.2.10, 9.3.x before 9.3.6, and 9.4.x before 9.4.1 allows remote authenticated users to cause a denial of service (crash) or possibly execute arbitrary code via a (1) large number of digits when processing a numeric ...