Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Comments
The Common Core Of Application Security
Threaded  |  Newest First  |  Oldest First
DaveWichers
50%
50%
DaveWichers,
User Rank: Apprentice
9/23/2015 | 3:42:42 PM
OWASP Benchmark Clarifications
Jason, this discussion is great and I'm thrilled that the OWASP Benchmark is driving improvements in application vulnerability detection tools. But I did want to add a few clarifications on how the Benchmark works.

In your Benchmark results table, you indicate: "True Positives detected by Fortify SCA, and declared Secure by Benchmark" - 9206. While its great that Fortify found all these additional vulnerabilities in the Benchmark, the Benchmark makes no claims there are no other vulnerabilities in it beyond the ones specifically tested for and scored. Any such results found by any tool are simply ignored in the Benchmark scoring system, so they have no effect on the score one way or the other. So, saying that Fortify found a bunch of issues the project wasn't aware of and other tools did not find simply isn't accurate. Most of the tools we tested found a bunch of additional issues, just like Fortify did.

As part of our 1.2 effort, we have eliminated a number of unintended vulnerabilities of the type tested for in the Benchmark, particularly XSS. This is an ongoing effort and we have more work to do there. In fact, if you can send us your results, we'll be happy to use them to help us track down and eliminate more of them. That said, these 'extra' vulnerabilities are, and should be, ignored as they simply aren't measured/scored.

You also mention: "False Positives reported by Fortify SCA" - 4,852.  In the Benchmark v1.1, there are 9206 True Negative test cases, Meaning 9206 test cases that are safe, and do not possess the type of vulnerability they are testing for. And Fortify reported 4,852 of the as actual vulnerabilities (False Positives as you said). The Benchmark project scores that as 4,852 out of 9,206, which is a 52.7% False Positive rate. So if your True Positive rate is actually 100% as you claim, the Benchmark would produce an average score for Fortify as 100% - 52.7% which equals 47.3%. This average score for Fortify is higher than the scores the project has seen with the results we were able to generate so we are pleased to see that your team's efforts have improved its score against the Benchmark and that your customer's will ultimately benefit from these improvements.

I think discussions like this are incredibly healthy and hope lots of vendors for both commercial and free tools will get involved to make both the OWASP Benchmark project and their tools better for the community we both serve.  And given the amount of discussions I'm having with project participants at OWASP, the discussions are just getting started, and many tools, including Fortify, are getting better already. And in fact, I'm going to talk about exactly that at my OWASP AppSec USA talk on the Benchmark project tomorrow afternoon at 4. If any of you are around, please come by!!

Dave Wichers

OWASP Benchmark Project Lead
BigJim2
50%
50%
BigJim2,
User Rank: Apprentice
9/25/2015 | 9:57:08 AM
Re: OWASP Benchmark Clarifications
Is this the same Dave Wichers that is a co-founder of Aspect Security, the company that created Constrast? It seems like a conflict of interest for someone from a vendor to create a benchmark that will be used to grade their competition. I'm even more surprised that a government agency (DHS) would sponsor that activity.
DaveWichers
50%
50%
DaveWichers,
User Rank: Apprentice
9/25/2015 | 5:49:01 PM
Re: OWASP Benchmark Clarifications
Yes. That's me. And your concern is fair and you aren't the first to bring it up. We are addressing this by making everything free and open and reproducible by anyone, and more importantly, getting lots more people involved, so we can eliminate any potential for bias. We already have a number of open source projects contributing to the project, and a bunch of commercial vendors and even some non-vendors approached me at this week's AppSec USA conference and asked to get involved, which I welcome whole heartedly. We are going to expand the team to as many who want to participate, and ensure there are many eyes and many contributors to the work we produce. The OWASP Board has expressed their support for this project, as it's exactly the kind of thing OWASP should be doing. This project is really getting some momentum and together we can all make it great. Please contribute if it's of interest to you.


COVID-19: Latest Security News & Commentary
Dark Reading Staff 10/27/2020
Are You One COVID-19 Test Away From a Cybersecurity Disaster?
Alan Brill, Senior Managing Director, Cyber Risk Practice, Kroll,  10/21/2020
Modern Day Insider Threat: Network Bugs That Are Stealing Your Data
David Pearson, Principal Threat Researcher,  10/21/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
Special Report: Computing's New Normal
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
How IT Security Organizations are Attacking the Cybersecurity Problem
How IT Security Organizations are Attacking the Cybersecurity Problem
The COVID-19 pandemic turned the world -- and enterprise computing -- on end. Here's a look at how cybersecurity teams are retrenching their defense strategies, rebuilding their teams, and selecting new technologies to stop the oncoming rise of online attacks.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-11854
PUBLISHED: 2020-10-27
Arbitrary code execution vlnerability in Operation bridge Manager, Application Performance Management and Operations Bridge (containerized) vulnerability in Micro Focus products products Operation Bridge Manager, Operation Bridge (containerized) and Application Performance Management. The vulneravil...
CVE-2020-11858
PUBLISHED: 2020-10-27
Code execution with escalated privileges vulnerability in Micro Focus products Operation Bridge Manager and Operation Bridge (containerized). The vulneravility affects: 1.) Operation Bridge Manager versions: 2020.05, 2019.11, 2019.05, 2018.11, 2018.05, 10.63,10.62, 10.61, 10.60, 10.12, 10.11, 10.10 ...
CVE-2020-23945
PUBLISHED: 2020-10-27
A SQL injection vulnerability exists in Victor CMS V1.0 in the cat_id parameter of the category.php file. This parameter can be used by sqlmap to obtain data information in the database.
CVE-2020-7754
PUBLISHED: 2020-10-27
This affects the package npm-user-validate before 1.0.1. The regex that validates user emails took exponentially longer to process long input strings beginning with @ characters.
CVE-2020-6023
PUBLISHED: 2020-10-27
Check Point ZoneAlarm before version 15.8.139.18543 allows a local actor to escalate privileges while restoring files in Anti-Ransomware.