Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Comments
The Common Core Of Application Security
Oldest First  |  Newest First  |  Threaded View
DaveWichers
50%
50%
DaveWichers,
User Rank: Apprentice
9/23/2015 | 3:42:42 PM
OWASP Benchmark Clarifications
Jason, this discussion is great and I'm thrilled that the OWASP Benchmark is driving improvements in application vulnerability detection tools. But I did want to add a few clarifications on how the Benchmark works.

In your Benchmark results table, you indicate: "True Positives detected by Fortify SCA, and declared Secure by Benchmark" - 9206. While its great that Fortify found all these additional vulnerabilities in the Benchmark, the Benchmark makes no claims there are no other vulnerabilities in it beyond the ones specifically tested for and scored. Any such results found by any tool are simply ignored in the Benchmark scoring system, so they have no effect on the score one way or the other. So, saying that Fortify found a bunch of issues the project wasn't aware of and other tools did not find simply isn't accurate. Most of the tools we tested found a bunch of additional issues, just like Fortify did.

As part of our 1.2 effort, we have eliminated a number of unintended vulnerabilities of the type tested for in the Benchmark, particularly XSS. This is an ongoing effort and we have more work to do there. In fact, if you can send us your results, we'll be happy to use them to help us track down and eliminate more of them. That said, these 'extra' vulnerabilities are, and should be, ignored as they simply aren't measured/scored.

You also mention: "False Positives reported by Fortify SCA" - 4,852.  In the Benchmark v1.1, there are 9206 True Negative test cases, Meaning 9206 test cases that are safe, and do not possess the type of vulnerability they are testing for. And Fortify reported 4,852 of the as actual vulnerabilities (False Positives as you said). The Benchmark project scores that as 4,852 out of 9,206, which is a 52.7% False Positive rate. So if your True Positive rate is actually 100% as you claim, the Benchmark would produce an average score for Fortify as 100% - 52.7% which equals 47.3%. This average score for Fortify is higher than the scores the project has seen with the results we were able to generate so we are pleased to see that your team's efforts have improved its score against the Benchmark and that your customer's will ultimately benefit from these improvements.

I think discussions like this are incredibly healthy and hope lots of vendors for both commercial and free tools will get involved to make both the OWASP Benchmark project and their tools better for the community we both serve.  And given the amount of discussions I'm having with project participants at OWASP, the discussions are just getting started, and many tools, including Fortify, are getting better already. And in fact, I'm going to talk about exactly that at my OWASP AppSec USA talk on the Benchmark project tomorrow afternoon at 4. If any of you are around, please come by!!

Dave Wichers

OWASP Benchmark Project Lead
BigJim2
50%
50%
BigJim2,
User Rank: Apprentice
9/25/2015 | 9:57:08 AM
Re: OWASP Benchmark Clarifications
Is this the same Dave Wichers that is a co-founder of Aspect Security, the company that created Constrast? It seems like a conflict of interest for someone from a vendor to create a benchmark that will be used to grade their competition. I'm even more surprised that a government agency (DHS) would sponsor that activity.
DaveWichers
50%
50%
DaveWichers,
User Rank: Apprentice
9/25/2015 | 5:49:01 PM
Re: OWASP Benchmark Clarifications
Yes. That's me. And your concern is fair and you aren't the first to bring it up. We are addressing this by making everything free and open and reproducible by anyone, and more importantly, getting lots more people involved, so we can eliminate any potential for bias. We already have a number of open source projects contributing to the project, and a bunch of commercial vendors and even some non-vendors approached me at this week's AppSec USA conference and asked to get involved, which I welcome whole heartedly. We are going to expand the team to as many who want to participate, and ensure there are many eyes and many contributors to the work we produce. The OWASP Board has expressed their support for this project, as it's exactly the kind of thing OWASP should be doing. This project is really getting some momentum and together we can all make it great. Please contribute if it's of interest to you.


Overcoming the Challenge of Shorter Certificate Lifespans
Mike Cooper, Founder & CEO of Revocent,  10/15/2020
7 Tips for Choosing Security Metrics That Matter
Ericka Chickowski, Contributing Writer,  10/19/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
Special Report: Computing's New Normal
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
How IT Security Organizations are Attacking the Cybersecurity Problem
How IT Security Organizations are Attacking the Cybersecurity Problem
The COVID-19 pandemic turned the world -- and enterprise computing -- on end. Here's a look at how cybersecurity teams are retrenching their defense strategies, rebuilding their teams, and selecting new technologies to stop the oncoming rise of online attacks.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-27605
PUBLISHED: 2020-10-21
BigBlueButton through 2.2.8 uses Ghostscript for processing of uploaded EPS documents, and consequently may be subject to attacks related to a "schwache Sandbox."
CVE-2020-27606
PUBLISHED: 2020-10-21
BigBlueButton before 2.2.8 (or earlier) does not set the secure flag for the session cookie in an https session, which makes it easier for remote attackers to capture this cookie by intercepting its transmission within an http session.
CVE-2020-27607
PUBLISHED: 2020-10-21
In BigBlueButton before 2.2.8 (or earlier), the client-side Mute button only signifies that the server should stop accepting audio data from the client. It does not directly configure the client to stop sending audio data to the server, and thus a modified server could store the audio data and/or tr...
CVE-2020-27608
PUBLISHED: 2020-10-21
In BigBlueButton before 2.2.8 (or earlier), uploaded presentations are sent to clients without a Content-Type header, which allows XSS, as demonstrated by a .png file extension for an HTML document.
CVE-2020-27609
PUBLISHED: 2020-10-21
BigBlueButton through 2.2.8 records a video meeting despite the deactivation of video recording in the user interface. This may result in data storage beyond what is authorized for a specific meeting topic or participant.