Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Endpoint

5/24/2019
10:30 AM
Connect Directly
Google+
Twitter
RSS
E-Mail
50%
50%

NSS Labs Admits Its Test of CrowdStrike Falcon Was 'Inaccurate'

CrowdStrike, NSS Labs reach confidential settlement over 2017 endpoint product testing dispute.

[This article was updated on 5/25/2019 with updated and corrected information on the status of NSS Labs' lawsuit over the AMTSO testing protocol]

NSS Labs has retracted its 2017 publicly reported and disputed test results of CrowdStrike's Falcon endpoint security product as part of a confidential settlement reached with the security vendor over a pair of lawsuits.

The February 2017 advanced endpoint protection test report, which graded Falcon poorly, was challenged in court by CrowdStrike in a lawsuit, which alleged that the testing was incomplete and conducted using illegally obtained Falcon software, and defied CrowdStrike's request for NSS Labs to halt the testing.

In a statement posted on its website this week, NSS Labs said that its 2017 test results of CrowdStrike Falcon were inaccurate and had been retracted.

"NSS's testing of the CrowdStrike Falcon platform was incomplete and the product was not properly configured with prevention capabilities enabled. In addition to the results having already been acknowledged as partially incomplete, we now acknowledge they are not accurate and confirm that they do not meet our standards for publication," NSS Labs said in the statement, which also included an apology to CrowdStrike for the "inaccurate" test results.

NSS Labs released the full AEP test report, including the flawed results of Falcon, during the 2017 RSA Conference. CrowdStrike had requested a temporary restraining order and preliminary injunction against NSS Labs to halt the report's publication, but the court dismissed that request and the report went out.

George Kurtz, president and CEO of CrowdStrike, at that time said the tests were run using incomplete and incorrect information, and run improperly. CrowdStrike had hired NSS Labs in 2016 to perform private testing of Falcon, but later dropped the testing deal after the concerns over the quality of tests, which detected legitimate applications like Adobe and Skype as malicious, for example.

NSS Labs, however, continued to perform public tests on Falcon using software it acquired via a reseller.

The testing organization has been no stranger to controversy and conflict with security vendors. It's currently embroiled in another lawsuit: in September of 2018, NSS Labs filed an antitrust lawsuit against CrowdStrike, ESET, and Symantec as well as the Anti-Malware Testing Standards Organization (AMTSO), over a vendor-backed testing protocol. The nonprofit AMTSO adopted a testing protocol standard that its members had voted for and plan to adopt.

With the recent settlement between NSS Labs and CrowdStrike, CrowdStrike is no longer a party in that case. 

NSS Labs accused AMTSO and the three security vendors of unfairly allowing their products to be tested only by organizations that comply with the AMTSO. CrowdStrike at the time dismissed the suit as groundless, stating: "NSS is a for-profit, pay-to-play testing organization that obtains products through fraudulent means and is desperate to defend its business model from open and transparent testing."

Related Content:

Kelly Jackson Higgins is the Executive Editor of Dark Reading. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Dr.T
50%
50%
Dr.T,
User Rank: Ninja
5/29/2019 | 8:22:59 AM
NSS
NSS Labs accused AMTSO and the three security vendors of unfairly allowing their products to be tested only by organizations that comply with the AMTSO. Lawsuit after lawsuit. Do the right thing to avoid problems.
Dr.T
50%
50%
Dr.T,
User Rank: Ninja
5/29/2019 | 8:20:27 AM
Re: Wrong side of history
Open, transparent, standards a la AMTSO and NetSecOPEN, are the way forward. I would agree. Open standards less trouble in long terms.
Dr.T
50%
50%
Dr.T,
User Rank: Ninja
5/29/2019 | 8:19:23 AM
Re: Wrong side of history
Credibility duly - severely - dented. Good way to put it. I like it.
Dr.T
50%
50%
Dr.T,
User Rank: Ninja
5/29/2019 | 8:18:44 AM
Re: Wrong side of history
To publish a vendor's test results at RSA and stand by those results steadfastly for more than two years Agree. Damage is already made. Nothing will change.
Dr.T
50%
50%
Dr.T,
User Rank: Ninja
5/29/2019 | 8:17:53 AM
Illegally?
in a lawsuit, which alleged that the testing was incomplete and conducted using illegally obtained Falcon software, If it was obtained illegally then that is where the problem starts.
HardenStance
67%
33%
HardenStance,
User Rank: Strategist
5/26/2019 | 12:01:49 PM
Wrong side of history
To publish a vendor's test results at RSA and stand by those results steadfastly for more than two years before finally walking back and apologizing for them being inaccurate is a pretty high octane error.

Credibility duly - severely - dented.

As a one-off, it ought to be recoverable.

However at a far broader level - witness the ongoing lawsuit against AMTSO - NSS Labs is just on the wrong, wrong side of cyber security testing history.

Open, transparent, standards a la AMTSO and NetSecOPEN, are the way forward.

Seems to me that NSS Labs is committing John Chambers' cardinal sin of missing - worse, trying to resist - a market transition.
7 Tips for Infosec Pros Considering A Lateral Career Move
Kelly Sheridan, Staff Editor, Dark Reading,  1/21/2020
For Mismanaged SOCs, The Price Is Not Right
Kelly Sheridan, Staff Editor, Dark Reading,  1/22/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
IT 2020: A Look Ahead
Are you ready for the critical changes that will occur in 2020? We've compiled editor insights from the best of our network (Dark Reading, Data Center Knowledge, InformationWeek, ITPro Today and Network Computing) to deliver to you a look at the trends, technologies, and threats that are emerging in the coming year. Download it today!
Flash Poll
How Enterprises are Attacking the Cybersecurity Problem
How Enterprises are Attacking the Cybersecurity Problem
Organizations have invested in a sweeping array of security technologies to address challenges associated with the growing number of cybersecurity attacks. However, the complexity involved in managing these technologies is emerging as a major problem. Read this report to find out what your peers biggest security challenges are and the technologies they are using to address them.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-8003
PUBLISHED: 2020-01-27
A double-free vulnerability in vrend_renderer.c in virglrenderer through 0.8.1 allows attackers to cause a denial of service by triggering texture allocation failure, because vrend_renderer_resource_allocated_texture is not an appropriate place for a free.
CVE-2019-20427
PUBLISHED: 2020-01-27
In the Lustre file system before 2.12.3, the ptlrpc module has a buffer overflow and panic, and possibly remote code execution, due to the lack of validation for specific fields of packets sent by a client. Interaction between req_capsule_get_size and tgt_brw_write leads to a tgt_shortio2pages integ...
CVE-2019-20428
PUBLISHED: 2020-01-27
In the Lustre file system before 2.12.3, the ptlrpc module has an out-of-bounds read and panic due to the lack of validation for specific fields of packets sent by a client. The ldl_request_cancel function mishandles a large lock_count parameter.
CVE-2019-20429
PUBLISHED: 2020-01-27
In the Lustre file system before 2.12.3, the ptlrpc module has an out-of-bounds read and panic (via a modified lm_bufcount field) due to the lack of validation for specific fields of packets sent by a client. This is caused by interaction between sptlrpc_svc_unwrap_request and lustre_msg_hdr_size_v2...
CVE-2019-20430
PUBLISHED: 2020-01-27
In the Lustre file system before 2.12.3, the mdt module has an LBUG panic (via a large MDT Body eadatasize field) due to the lack of validation for specific fields of packets sent by a client.