Endpoint

9/19/2018
07:00 PM
Connect Directly
Google+
Twitter
RSS
E-Mail
50%
50%

NSS Labs Files Antitrust Suit Against Symantec, CrowdStrike, ESET, AMTSO

Suit underscores longtime battle between vendors and labs over control of security testing protocols.

UPDATED 9/20 with comments from ESET, AV-Test, and CrowdStrike

Security product testing firm NSS Labs today filed an antitrust lawsuit against cybersecurity vendors CrowdStrike, ESET, and Symantec as well as the Anti-Malware Testing Standards Organization (AMTSO) over a vendor-backed testing protocol.

The lawsuit accuses the three security vendors and the nonprofit AMTSO, of which they and other endpoint security vendors are members, of unfairly allowing their products to be tested only by organizations that comply with AMTSO's testing protocol standard. NSS Labs, which also is a member of AMTSO, earlier this year voted against adoption of the standard and says it has no plans to comply with it.

A majority of AMTSO members voted in favor of the standard in May of this year, and most plan to adopt the protocol.

Friction between security vendors and independent testing labs is nothing new. Vendors and labs traditionally have had an uneasy and sometimes contentious relationship over control of the testing process and parameters. NSS Labs' suit appears to represent an escalation of that age-old conflict, security experts say.

NSS Labs is calling foul in its lawsuit: "NSS Labs has suffered antitrust injury as a result of the acts herein alleged because it is the direct and principal target of the concerted refusal to deal/group boycott" any testing organizations that don't adopt ATMSO's testing standard, the lawsuit says.

In an interview with Dark Reading, Jason Brvenik, chief technology officer at NSS Labs, said the ATMSO standard falls short. "Our fundamental focus is that if a product is good enough to sell, it's good enough to test," and NSS Labs shouldn't be forced to comply with ATMSO's standard, he says. "It should be an independent test."

Brvenik says the AMTSO standard does not support independent testing. "It's driven by vendors to create a picture of capabilities that are not true," for example, he says. "The standard is more like guidelines to interact with than a standard, and it doesn't make things better for products" or the way they are tested, he says.

According to the NSS Labs suit, other vendors that spoke out against the adoption of AMTSO's standards included AVComparatives, AV-Test, and SKD LABS. None of those vendors are named as parties in NSS Labs' case. Efforts to reach AVComparatives, and SKD Labs were unsuccessful as of this posting.

AV-Test CEO Andreas Marx said he had just heard about the suit and is unable to comment at this time.

CrowdStrike called the suit groundless: "NSS is a for-profit, pay-to-play testing organization that obtains products through fraudulent means and is desperate to defend its business model from open and transparent testing. We believe their lawsuit is baseless," the company said in a statement.

"CrowdStrike supports independent and standards-based testing—including public testing—for our products and for the industry. We have undergone independent testing with AV-Comparatives, SE Labs, and MITRE and you can find information on that testing here," the statement said. "We applaud AMTSO’s efforts to promote clear, consistent, and transparent testing standards."

ESET said it had not been officially contacted about the suit, but that it refutes the allegations. "We are aware of the allegations stated in the blog post from NSS Labs, however, we have yet to receive official legal communication. As legal proceedings appear to have been initiated, we are unable to say more at this time, beyond the statement that we categorically deny the allegations," an ESET spokesperson said. "Our customers should be reassured that ESET’s products have been rigorously tested by many independent third-party reviewers around the world, received numerous awards for their level of protection of end users over many years, and are widely praised by industry-leading specialists." 

Symantec would not comment on the case, and efforts to reach AMTSO  were unsuccessful as of this posting.

In a blog post earlier this month, ATMSO president Dennis Batchelder wrote that the protocol is a voluntary framework for testing anti-malware software "fairly and transparently." 

For enterprises, there aren't many options for vetting security software. Most don't have the resources to perform their own in-house testing of security products, so they rely on consulting firms' recommendations, third-party testing organizations — or the claims of their vendor.

Jon Oltsik, senior principal analyst with consulting firm Enterprise Strategy Group, says he's seen enterprises struggle with the testing dilemma. "Customers don't know how to test the efficacy of next-generation endpoint security products," he says. "No one trusts vendors to test their own product."

The concept of a vetted product testing standard is a "very good idea," says Oltsik, who notes that he has not specifically studied ATMSO's protocol.

Bottom Line
NSS Labs meantime argues that the AMTSO and its standard are anti-competitive. "They claim to try to improve testing but what they're actually doing is actively preventing unbiased testing. Further, vendors are openly exerting control and collectively boycotting testing organizations that don't comply with their AMTSO standards — even going so far as to block the independent purchase and testing of their products," Vikram Phatak, CEO of NSS Labs wrote in a blog post today announcing the suit.

Meanwhile, NSS Labs claims in its lawsuit that AMTSO's efforts have hurt its bottom line. "NSS Labs has lost sales and profits from the sale and license of its public testing reports, including its AEP Group Test reports, because it cannot charge customers who purchase reports that do not include all market participants as much as it could charge for reports that included all market participants." 

 

Black Hat Europe returns to London Dec. 3-6, 2018, with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions, and service providers in the Business Hall. Click for information on the conference and to register.

Kelly Jackson Higgins is Executive Editor at DarkReading.com. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
pdcz
50%
50%
pdcz,
User Rank: Apprentice
12/7/2018 | 6:36:46 AM
Re: NSS Antitrust Law Suit
Exactly. I do not see any problem. And if there IS some problem with standard (which really is actually just guideline) then why they do not publish openly what are the problematic parts and how they would like to change them. As somebody said alrerady this brings questions what NSS has to hide by not accepting this open "standard".
HardenStance
50%
50%
HardenStance,
User Rank: Strategist
9/21/2018 | 6:57:24 AM
It's important that the NSS Labs Suit should not succeed.
 

Can't say I'm in the least bit surprised.

It's important that the NSS Labs suit against AMTSO should not succeed, in my view.

What AMTSO is trying to achieve is self-evidently a 'good thing'. Its goals - as supported by a who's-who of pretty much anybody who's anybody in endpoint security - are entirely consistent with bringing more open, transparent, security testing standards to consumers and businesses.

Has AMTSO somewhere in its articles of incorporation or operating practices left some kind of miniscule loophole - some missing word, some un-crossed 't' or some upside down full-stop - that a lawyer can seize upon as a 'gotcha' for their client? I haven't the faintest idea. You have to hope not.

Something else. IT security professionals have long complained about lack of transparency in endpoint security testing. Many look at independent test house results based on proprietary test methods and the first thing the eyes do is look to the clouds (if they don't glaze over first).

You get the politicians you deserve? I don't altogether go along with that. But do enterprise IT organizations get the security testing standards they deserve? Yeah, they do. AMTSO needs support from its lawyers and its members, right now. But it needs a lot more support from enterprise users too.
tdsan
50%
50%
tdsan,
User Rank: Strategist
9/20/2018 | 5:17:18 PM
Thank you for your insight
I appreciate the background information you provided, it seems that NSS is going after a number of companies (not just Crowdstrike), this seems to be backlash from other encounters they have had outside of this specific incident:

→ Cybersecurity vendors CrowdStrike, ESET, and Symantec

However, the question still remains, if Crowdstrike has nothing to hide and if they have confidence in their product, then why is there such resistance, seems to me that Crowdstrike has something to hide (also, per the recommendation from the courts). NSS labs uses tools from the wild that will more resemble attacks as opposed to running simple tests that don't really give the user a clear indication of how their product will fair when it is attacked using surreptious and/or zero-day attack methods.

In addition, I would like to know which methods they thought did not meet their approval, in the wild, there will be numerous things that don't meet their approval, especially after spending $150K.

I am curious what the courts will rule, again, time will tell.

Todd
Kelly Jackson Higgins
50%
50%
Kelly Jackson Higgins,
User Rank: Strategist
9/20/2018 | 3:20:33 PM
Re: NSS Antitrust Law Suit
@tdsan,

That's a good question and there a few things going on here. NSS Labs maintains that the AMTSO is skewed to the vendors and not independent, and that its own testing is more independent than one approved by vendors (AMTSO's). There also appears to be some bad blood between NSS Labs and CrowdStrike, publicly dating back to 2017 when CrowdStrike attempted (and failed) to get a restraining order to prevent NSS from publishing test results. According to CrowdStrike, it found NSS's testing methods questionable and the results based on incorrect and incomplete information. So it pulled out of the test, and then NSS accessed CrowdStrike's Falcon product via reseller in a violation of CrowdStrike's EULA. CrowdStrike said the test didn't have all of the product's features enabled, so it wasn't an accurate asssessment: https://www.darkreading.com/endpoint/crowdstrike-fails-in-bid-to-stop-nss-labs-from-publishing-test-results-at-rsa/d/d-id/1328154

CrowdStrike called NSS Labs' antitrust suit "baseless" this morning, and now NSS is firing back and calling out CrowdStrike's "smear tactics" and dismissing AMTSO as a "pay to play" thing. 

So...I really don't think NSS Labs has any plans to go AMTSO. 
tdsan
50%
50%
tdsan,
User Rank: Strategist
9/20/2018 | 2:35:15 PM
NSS Antitrust Law Suit
What I am confused about is as follows:

→ Why can't NSS Adopt the ATMSO protocol/methodology as part of their overall testing strategy?

→ Why don't they provide ATMSO and their independent testing concept and compare the two?

If they did this, then they would be the defacto standard when it comes to security pentesting.

I think it is about pride and money as opposed to anything else.

Oh well, time will tell.

Todd
'PowerSnitch' Hacks Androids via Power Banks
Kelly Jackson Higgins, Executive Editor at Dark Reading,  12/8/2018
The Case for a Human Security Officer
Ira Winkler, CISSP, President, Secure Mentem,  12/5/2018
Windows 10 Security Questions Prove Easy for Attackers to Exploit
Kelly Sheridan, Staff Editor, Dark Reading,  12/5/2018
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
10 Best Practices That Could Reshape Your IT Security Department
This Dark Reading Tech Digest, explores ten best practices that could reshape IT security departments.
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2018-8651
PUBLISHED: 2018-12-12
A cross site scripting vulnerability exists when Microsoft Dynamics NAV does not properly sanitize a specially crafted web request to an affected Dynamics NAV server, aka "Microsoft Dynamics NAV Cross Site Scripting Vulnerability." This affects Microsoft Dynamics NAV.
CVE-2018-8652
PUBLISHED: 2018-12-12
A Cross-site Scripting (XSS) vulnerability exists when Windows Azure Pack does not properly sanitize user-provided input, aka "Windows Azure Pack Cross Site Scripting Vulnerability." This affects Windows Azure Pack Rollup 13.1.
CVE-2018-8617
PUBLISHED: 2018-12-12
A remote code execution vulnerability exists in the way that the Chakra scripting engine handles objects in memory in Microsoft Edge, aka "Chakra Scripting Engine Memory Corruption Vulnerability." This affects Microsoft Edge, ChakraCore. This CVE ID is unique from CVE-2018-8583, CVE-2018-8...
CVE-2018-8618
PUBLISHED: 2018-12-12
A remote code execution vulnerability exists in the way that the Chakra scripting engine handles objects in memory in Microsoft Edge, aka "Chakra Scripting Engine Memory Corruption Vulnerability." This affects Microsoft Edge, ChakraCore. This CVE ID is unique from CVE-2018-8583, CVE-2018-8...
CVE-2018-8619
PUBLISHED: 2018-12-12
A remote code execution vulnerability exists when the Internet Explorer VBScript execution policy does not properly restrict VBScript under specific conditions, aka "Internet Explorer Remote Code Execution Vulnerability." This affects Internet Explorer 9, Internet Explorer 11, Internet Exp...