Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Endpoint

9/19/2018
07:00 PM
Connect Directly
Google+
Twitter
RSS
E-Mail
50%
50%

NSS Labs Files Antitrust Suit Against Symantec, CrowdStrike, ESET, AMTSO

Suit underscores longtime battle between vendors and labs over control of security testing protocols.

UPDATED 9/20 with comments from ESET, AV-Test, and CrowdStrike

Security product testing firm NSS Labs today filed an antitrust lawsuit against cybersecurity vendors CrowdStrike, ESET, and Symantec as well as the Anti-Malware Testing Standards Organization (AMTSO) over a vendor-backed testing protocol.

The lawsuit accuses the three security vendors and the nonprofit AMTSO, of which they and other endpoint security vendors are members, of unfairly allowing their products to be tested only by organizations that comply with AMTSO's testing protocol standard. NSS Labs, which also is a member of AMTSO, earlier this year voted against adoption of the standard and says it has no plans to comply with it.

A majority of AMTSO members voted in favor of the standard in May of this year, and most plan to adopt the protocol.

Friction between security vendors and independent testing labs is nothing new. Vendors and labs traditionally have had an uneasy and sometimes contentious relationship over control of the testing process and parameters. NSS Labs' suit appears to represent an escalation of that age-old conflict, security experts say.

NSS Labs is calling foul in its lawsuit: "NSS Labs has suffered antitrust injury as a result of the acts herein alleged because it is the direct and principal target of the concerted refusal to deal/group boycott" any testing organizations that don't adopt ATMSO's testing standard, the lawsuit says.

In an interview with Dark Reading, Jason Brvenik, chief technology officer at NSS Labs, said the ATMSO standard falls short. "Our fundamental focus is that if a product is good enough to sell, it's good enough to test," and NSS Labs shouldn't be forced to comply with ATMSO's standard, he says. "It should be an independent test."

Brvenik says the AMTSO standard does not support independent testing. "It's driven by vendors to create a picture of capabilities that are not true," for example, he says. "The standard is more like guidelines to interact with than a standard, and it doesn't make things better for products" or the way they are tested, he says.

According to the NSS Labs suit, other vendors that spoke out against the adoption of AMTSO's standards included AVComparatives, AV-Test, and SKD LABS. None of those vendors are named as parties in NSS Labs' case. Efforts to reach AVComparatives, and SKD Labs were unsuccessful as of this posting.

AV-Test CEO Andreas Marx said he had just heard about the suit and is unable to comment at this time.

CrowdStrike called the suit groundless: "NSS is a for-profit, pay-to-play testing organization that obtains products through fraudulent means and is desperate to defend its business model from open and transparent testing. We believe their lawsuit is baseless," the company said in a statement.

"CrowdStrike supports independent and standards-based testing—including public testing—for our products and for the industry. We have undergone independent testing with AV-Comparatives, SE Labs, and MITRE and you can find information on that testing here," the statement said. "We applaud AMTSO’s efforts to promote clear, consistent, and transparent testing standards."

ESET said it had not been officially contacted about the suit, but that it refutes the allegations. "We are aware of the allegations stated in the blog post from NSS Labs, however, we have yet to receive official legal communication. As legal proceedings appear to have been initiated, we are unable to say more at this time, beyond the statement that we categorically deny the allegations," an ESET spokesperson said. "Our customers should be reassured that ESET’s products have been rigorously tested by many independent third-party reviewers around the world, received numerous awards for their level of protection of end users over many years, and are widely praised by industry-leading specialists." 

Symantec would not comment on the case, and efforts to reach AMTSO  were unsuccessful as of this posting.

In a blog post earlier this month, ATMSO president Dennis Batchelder wrote that the protocol is a voluntary framework for testing anti-malware software "fairly and transparently." 

For enterprises, there aren't many options for vetting security software. Most don't have the resources to perform their own in-house testing of security products, so they rely on consulting firms' recommendations, third-party testing organizations — or the claims of their vendor.

Jon Oltsik, senior principal analyst with consulting firm Enterprise Strategy Group, says he's seen enterprises struggle with the testing dilemma. "Customers don't know how to test the efficacy of next-generation endpoint security products," he says. "No one trusts vendors to test their own product."

The concept of a vetted product testing standard is a "very good idea," says Oltsik, who notes that he has not specifically studied ATMSO's protocol.

Bottom Line
NSS Labs meantime argues that the AMTSO and its standard are anti-competitive. "They claim to try to improve testing but what they're actually doing is actively preventing unbiased testing. Further, vendors are openly exerting control and collectively boycotting testing organizations that don't comply with their AMTSO standards — even going so far as to block the independent purchase and testing of their products," Vikram Phatak, CEO of NSS Labs wrote in a blog post today announcing the suit.

Meanwhile, NSS Labs claims in its lawsuit that AMTSO's efforts have hurt its bottom line. "NSS Labs has lost sales and profits from the sale and license of its public testing reports, including its AEP Group Test reports, because it cannot charge customers who purchase reports that do not include all market participants as much as it could charge for reports that included all market participants." 

 

Black Hat Europe returns to London Dec. 3-6, 2018, with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions, and service providers in the Business Hall. Click for information on the conference and to register.

Kelly Jackson Higgins is Executive Editor at DarkReading.com. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
pdcz
50%
50%
pdcz,
User Rank: Apprentice
12/7/2018 | 6:36:46 AM
Re: NSS Antitrust Law Suit
Exactly. I do not see any problem. And if there IS some problem with standard (which really is actually just guideline) then why they do not publish openly what are the problematic parts and how they would like to change them. As somebody said alrerady this brings questions what NSS has to hide by not accepting this open "standard".
HardenStance
50%
50%
HardenStance,
User Rank: Strategist
9/21/2018 | 6:57:24 AM
It's important that the NSS Labs Suit should not succeed.
 

Can't say I'm in the least bit surprised.

It's important that the NSS Labs suit against AMTSO should not succeed, in my view.

What AMTSO is trying to achieve is self-evidently a 'good thing'. Its goals - as supported by a who's-who of pretty much anybody who's anybody in endpoint security - are entirely consistent with bringing more open, transparent, security testing standards to consumers and businesses.

Has AMTSO somewhere in its articles of incorporation or operating practices left some kind of miniscule loophole - some missing word, some un-crossed 't' or some upside down full-stop - that a lawyer can seize upon as a 'gotcha' for their client? I haven't the faintest idea. You have to hope not.

Something else. IT security professionals have long complained about lack of transparency in endpoint security testing. Many look at independent test house results based on proprietary test methods and the first thing the eyes do is look to the clouds (if they don't glaze over first).

You get the politicians you deserve? I don't altogether go along with that. But do enterprise IT organizations get the security testing standards they deserve? Yeah, they do. AMTSO needs support from its lawyers and its members, right now. But it needs a lot more support from enterprise users too.
tdsan
50%
50%
tdsan,
User Rank: Ninja
9/20/2018 | 5:17:18 PM
Thank you for your insight
I appreciate the background information you provided, it seems that NSS is going after a number of companies (not just Crowdstrike), this seems to be backlash from other encounters they have had outside of this specific incident:

→ Cybersecurity vendors CrowdStrike, ESET, and Symantec

However, the question still remains, if Crowdstrike has nothing to hide and if they have confidence in their product, then why is there such resistance, seems to me that Crowdstrike has something to hide (also, per the recommendation from the courts). NSS labs uses tools from the wild that will more resemble attacks as opposed to running simple tests that don't really give the user a clear indication of how their product will fair when it is attacked using surreptious and/or zero-day attack methods.

In addition, I would like to know which methods they thought did not meet their approval, in the wild, there will be numerous things that don't meet their approval, especially after spending $150K.

I am curious what the courts will rule, again, time will tell.

Todd
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
9/20/2018 | 3:20:33 PM
Re: NSS Antitrust Law Suit
@tdsan,

That's a good question and there a few things going on here. NSS Labs maintains that the AMTSO is skewed to the vendors and not independent, and that its own testing is more independent than one approved by vendors (AMTSO's). There also appears to be some bad blood between NSS Labs and CrowdStrike, publicly dating back to 2017 when CrowdStrike attempted (and failed) to get a restraining order to prevent NSS from publishing test results. According to CrowdStrike, it found NSS's testing methods questionable and the results based on incorrect and incomplete information. So it pulled out of the test, and then NSS accessed CrowdStrike's Falcon product via reseller in a violation of CrowdStrike's EULA. CrowdStrike said the test didn't have all of the product's features enabled, so it wasn't an accurate asssessment: https://www.darkreading.com/endpoint/crowdstrike-fails-in-bid-to-stop-nss-labs-from-publishing-test-results-at-rsa/d/d-id/1328154

CrowdStrike called NSS Labs' antitrust suit "baseless" this morning, and now NSS is firing back and calling out CrowdStrike's "smear tactics" and dismissing AMTSO as a "pay to play" thing. 

So...I really don't think NSS Labs has any plans to go AMTSO. 
tdsan
50%
50%
tdsan,
User Rank: Ninja
9/20/2018 | 2:35:15 PM
NSS Antitrust Law Suit
What I am confused about is as follows:

→ Why can't NSS Adopt the ATMSO protocol/methodology as part of their overall testing strategy?

→ Why don't they provide ATMSO and their independent testing concept and compare the two?

If they did this, then they would be the defacto standard when it comes to security pentesting.

I think it is about pride and money as opposed to anything else.

Oh well, time will tell.

Todd
10 Ways to Keep a Rogue RasPi From Wrecking Your Network
Curtis Franklin Jr., Senior Editor at Dark Reading,  7/10/2019
The Security of Cloud Applications
Hillel Solow, CTO and Co-founder, Protego,  7/11/2019
Where Businesses Waste Endpoint Security Budgets
Kelly Sheridan, Staff Editor, Dark Reading,  7/15/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: "Jim, stop pretending you're drowning in tickets."
Current Issue
Building and Managing an IT Security Operations Program
As cyber threats grow, many organizations are building security operations centers (SOCs) to improve their defenses. In this Tech Digest you will learn tips on how to get the most out of a SOC in your organization - and what to do if you can't afford to build one.
Flash Poll
The State of IT Operations and Cybersecurity Operations
The State of IT Operations and Cybersecurity Operations
Your enterprise's cyber risk may depend upon the relationship between the IT team and the security team. Heres some insight on what's working and what isn't in the data center.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-13360
PUBLISHED: 2019-07-16
In CentOS-WebPanel.com (aka CWP) CentOS Web Panel 0.9.8.836, remote attackers can bypass authentication in the login process by leveraging knowledge of a valid username.
CVE-2019-13383
PUBLISHED: 2019-07-16
In CentOS-WebPanel.com (aka CWP) CentOS Web Panel 0.9.8.846, the Login process allows attackers to check whether a username is valid by reading the HTTP response.
CVE-2019-13603
PUBLISHED: 2019-07-16
An issue was discovered in the HID Global DigitalPersona (formerly Crossmatch) U.are.U 4500 Fingerprint Reader Windows Biometric Framework driver 5.0.0.5. It has a statically coded initialization vector to encrypt a user's fingerprint image, resulting in weak encryption of that. This, in combination...
CVE-2019-13605
PUBLISHED: 2019-07-16
In CentOS-WebPanel.com (aka CWP) CentOS Web Panel 0.9.8.838 to 0.9.8.846, remote attackers can bypass authentication in the login process by leveraging the knowledge of a valid username. The attacker must defeat an encoding that is not equivalent to base64, and thus this is different from CVE-2019-1...
CVE-2019-13615
PUBLISHED: 2019-07-16
VideoLAN VLC media player 3.0.7.1 has a heap-based buffer over-read in mkv::demux_sys_t::FreeUnused() in modules/demux/mkv/demux.cpp when called from mkv::Open in modules/demux/mkv/mkv.cpp.