Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
3/3/2020
12:35 PM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

NSS Labs Revises Endpoint Security Test Model

New product ratings system comes amid growing shift in the testing market toward more "open and transparent" evaluation of security tools.

[3/5/2020 This story was updated with a correction that NSS Labs will continue to use its "Recommended," "Neutral," and "Caution" notices with endpoint products, in addition to the new ratings system, and with additional detail that its nonprofit will cover consumer security products, not just IoT.]

Cybersecurity testing company NSS Labs, which was quietly acquired by a private equity firm late last fall, has launched both a new ratings system for endpoint security product testing for that product category, and a new nonprofit testing organization for consumers for security and Internet of Things (IoT) products.

NSS Labs in October 2019 was purchased for an undisclosed figure by private equity firm Consecutive Inc., a move that was not publicly announced by the companies but which they later confirmed. Multiple sources close to NSS Labs described the merger as a fire sale of sorts to restructure the company amid financial woes, but NSS Labs CEO Jason Brvenik tells Dark Reading that the deal represents a reorganization by the company in order to better focus its resources.

According to Brvenik, the previous venture capital (VC) model was not a fit for NSS Labs or the testing market, mainly due to VC focus on growth and product. NSS Labs was under pressure from investors to sell a security-as-a-service threat intelligence offering for exploits, but the now-defunct Cyber Advanced Warning System (CAWS) service failed to gather steam among enterprises. CAWS, which was developed by NSS and had integrated with various threat intel vendors in that space, alerted customers on active exploits in the wild. NSS Labs since has folded its CAWS technology into its testing as a bundled service offering, according to the company.

"What we heard from the market was they didn't want more work from us [with the service]; they wanted answers and not data that makes them do more work," Brvenik says.

"We're now back to focusing on what we are really good at and what we're known for," he says. "It allowed us to look more at what we deliver to market and to make pivots to the cloud" and other areas, he noted.

NSS Labs announced the new initiatives of new test rankings and a new nonprofit testing arm for consumer security and IoT products during the RSA Conference in San Francisco last week. The new product ratings method, which the testing firm has first launched for endpoint protection products, rates vendor tools based on the criteria of management, false-positive rate, resistance to evasion, total cost of ownership, and their block rate of malware, exploits, and targeted attacks. NSS said it will also will continue to flag products as "Recommended," "Neutral," or "Caution," as well as now rate the products on a grading scale of AAA as the highest to D as the lowest.

The Testing Conundrum
The new moves by NSS Labs come at a time when traditional security product testing is undergoing a slow but welcome transformation. Vendors and test labs long have had an uneasy and often contentious relationship over control of the testing parameters and process, and NSS Labs at times has been at the center of that battle: The company in May 2019 retracted and apologized for a 2017 publicly released endpoint protection test report on CrowdStrike's Falcon, which CrowdStrike in turn challenged in a lawsuit alleging that the test was incomplete and used illegally obtained Falcon software.

CrowdStrike had hired NSS Labs the year before to conduct private testing of Falcon, but later terminated the testing engagement over concerns over the quality of the tests after it detected legitimate apps as malicious. NSS Labs continued to publicly test Falcon, using software it had acquired through a reseller.

In September 2018, NSS Labs filed an antitrust lawsuit against cybersecurity vendors CrowdStrike, ESET, and Symantec, as well as the nonprofit Anti-Malware Testing Standards Organization (AMTSO), over a vendor-backed testing protocol it deemed as unfair and vendor-centric. AMTSO's testing protocol aims for transparency between testers and vendors.

But NSS Labs dropped the lawsuit in December 2019, citing "progress" in how AMTSO and vendors were working with test labs. That doesn't mean that NSS Labs is now all-in for the AMTSO testing protocol, however: Brvenik says NSS Labs has no plans to adopt the AMTSO protocol for its testing programs. "We have not seen sufficient evolution there," he says. "It remains a vendor-driven environment."

Meanwhile, enterprises — at which testing is aimed — have been caught in the middle of such spats and faced with an often opaque testing model that critics have described as a vendor pay-to-play. Most don't have the resources to conduct their own in-house testing of security products, so they are left with recommendations from consulting firms, third-party testing organizations, or just claims of the vendors.

Brian Monkman — executive director of NetSecOPEN, an industry organization that coordinates network security performance testing based on its Internet Engineering Task Force standard-based process — says enterprises should be able to get open and transparent security testing from a neutral third-party testing organization.

"When enterprises are looking at testing results to help them decide what security products get short listed, they need to look at how the testing was done and the level of detail, and what level of detail security product vendors are prepared to provide," Monkman says. "An open and transparent nature is starting to emerge in the endpoint testing market."

Take Mitre's commercial testing of endpoint security products, which it launched in late 2018. The nonprofit evaluates the products against its ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge) model, using well-documented attack methods and techniques employed by nation-state and other advanced threat groups. Mitre's tests are based on open standards and methods, and the vendors perform live defenses with their products. It's a more collaborative test environment, many security vendors are embracing it, and the results are made public.

Meanwhile, the goal of NSS Labs' new product ratings scale is to ensure that organizations can choose the product or technology that best fits their needs, which may not be the most leading-edge product, notes Brvenik. Scoring endpoint products with a percentage grade is not necessarily representative of just how good a product is, he says.

"If you have five products and four of them are at 99.9% and one is at 99.5%, it's going to look like it [stinks] in a 2D [two-dimensional] axis, even though it's a great product. That model didn't fit well in that space," he says.

Chester Wisniewski, a principal research scientist with security vendor Sophos, says customers are demanding more transparency from security vendors. But there are plenty of challenges with endpoint testing, including that vendors can block only the threats they know about, he notes. "There's no way to test nation-state stuff" with today's tests, he says.

The underlying issue is that more attacks today are launched by humans behind a keyboard using stolen credentials. "The human will just keep changing the malware until they get through," Wisniewski says, a scenario that's difficult to simulate in most test environments.

IoT
The details of NSS Labs' new nonprofit are still being ironed out, but the organization will use NSS Labs' test infrastructure to put consumer security and IoT products under the security microscope and publish the results for the public. One concern is the intersection between enterprise networks and their users when they go home to smart devices and their Wi-Fi networks.

NSS Labs isn't the first to take on consumer IoT security testing: There's the Cyber Independent Testing Lab (CITL), a nonprofit led by Peiter "Mudge" Zatko and Sarah Zatko, which recently teamed up with Consumer Reports on a digital standard for consumer privacy, for instance. "They are doing cool consumer stuff, and [looking to conduct] cybersecurity testing in a rigorous" environment, says security expert Bruce Schneier.

Schneier also points to a security and privacy consumer labeling project underway at Carnegie Mellon University's CyLab, which is building a prototype Privacy and Security Label akin to a nutrition label that could be affixed to an IoT product's box. The goal is to help inform consumers about an IoT product's security and privacy features — or lack thereof — including how data it collects is used and whether or how it requires authentication, for example.

Related Content:

Check out The Edge, Dark Reading's new section for features, threat data, and in-depth perspectives. Today's featured story: "How to Prevent an AWS Cloud Bucket Data Leak."

Kelly Jackson Higgins is the Executive Editor of Dark Reading. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Where Are the 'Great Exits' in the Data Security Market?
Dave Cole, Cofounder and CEO, Open Raven,  10/13/2020
Overcoming the Challenge of Shorter Certificate Lifespans
Mike Cooper, Founder & CEO of Revocent,  10/15/2020
US Counterintelligence Director & Fmr. Europol Leader Talk Election Security
Kelly Sheridan, Staff Editor, Dark Reading,  10/16/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
Special Report: Computing's New Normal
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
How IT Security Organizations are Attacking the Cybersecurity Problem
How IT Security Organizations are Attacking the Cybersecurity Problem
The COVID-19 pandemic turned the world -- and enterprise computing -- on end. Here's a look at how cybersecurity teams are retrenching their defense strategies, rebuilding their teams, and selecting new technologies to stop the oncoming rise of online attacks.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-4564
PUBLISHED: 2020-10-20
IBM Sterling B2B Integrator Standard Edition 5.2.0.0 through 6.0.3.1 and IBM Sterling File Gateway 2.2.0.0 through 6.0.3.1 are vulnerable to cross-site scripting. This vulnerability allows users to embed arbitrary JavaScript code in the Web UI thus altering the intended functionality potentially lea...
CVE-2020-4748
PUBLISHED: 2020-10-20
IBM Spectrum Scale 5.0.0 through 5.0.5.2 is vulnerable to cross-site scripting. This vulnerability allows users to embed arbitrary JavaScript code in the Web UI thus altering the intended functionality potentially leading to credentials disclosure within a trusted session. IBM X-Force ID: 188517.
CVE-2020-4749
PUBLISHED: 2020-10-20
IBM Spectrum Scale 5.0.0 through 5.0.5.2 does not set the secure attribute on authorization tokens or session cookies. Attackers may be able to get the cookie values by sending a http:// link to a user or by planting this link in a site the user goes to. The cookie will be sent to the insecure link ...
CVE-2020-4755
PUBLISHED: 2020-10-20
IBM Spectrum Scale 5.0.0 through 5.0.5.2 is vulnerable to cross-site scripting. This vulnerability allows users to embed arbitrary JavaScript code in the Web UI thus altering the intended functionality potentially leading to credentials disclosure within a trusted session. IBM X-Force ID: 188595.
CVE-2020-4756
PUBLISHED: 2020-10-20
IBM Spectrum Scale V4.2.0.0 through V4.2.3.23 and V5.0.0.0 through V5.0.5.2 as well as IBM Elastic Storage System 6.0.0 through 6.0.1.0 could allow a local attacker to invoke a subset of ioctls on the device with invalid arguments that could crash the keneral and cause a denial of service. IBM X-For...