Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Vulnerabilities / Threats

2/5/2019
10:30 AM
Brian Monkman
Brian Monkman
Commentary
Connect Directly
LinkedIn
RSS
E-Mail vvv
100%
0%

Taming the Wild, West World of Security Product Testing

The industry has long needed an open, industry-standard testing framework. NetSecOPEN is working to make that happen.

Deciding what products can improve an organization's network security is a complex process. You must weigh a number of factors as part of the purchase decision, one of the most crucial of which is the impact of the product on network performance. However, given the current state of security product testing, it is virtually impossible to perform an accurate "apples-to-apples" product comparison. Proprietary testing methods conducted under uniquely optimized conditions create a chaotic scenario in which everyone plays by their own rules and customers are left struggling to sort it all out. NetSecOPEN is working to solve this problem by developing an open, industry-standard testing framework.

Wild, Wild West
Other industries have established standards, with which all companies must comply, and for good reason. When different companies use the same terms and claim to use the same metrics but define the terms and calculate the metrics entirely differently, it creates chaos for customers.

For example, years ago there was no standard for determining miles per gallon for vehicles. Automobile manufacturers had their own proprietary definitions and methods for calculating miles per gallon. Two vehicles that both got 25 mpg according to the manufacturer might have wildly different mileage results in the real world. The National Highway Traffic Safety Administration and the Environment Protection Agency stepped in and established standardized definitions and requirements for fuel economy, enabling consumers to use miles-per-gallon ratings to evaluate automobile performance with confidence.

There are many other industries that could benefit from standardized methodologies. Laptop manufacturers cite battery life as a key feature of their devices, but the battery life results customers experience rarely — if ever — live up to the claims. Vendors test battery life in very specific conditions with highly customized configurations. The result is that there is no accurate way to compare battery life claims from one vendor to the next.

Cybersecurity is critical for organizations, and it generally represents a very significant investment. It is not feasible for a company to implement and test a wide variety of solutions to determine which works best. Even when an organization is able to narrow down the options and conduct pilot tests in the organization's own environment, vendors can, and often do, place strict limits and constraints on how the pilot test is configured and managed

I previously worked in the technology testing field and have firsthand experience with some of the challenges of traditional testing methodologies. Vendors frequently impose specific test requirements that highlight the performance aspects on which they want to focus — which more or less invalidates the purpose of testing in the first place. Ultimately, such an approach threatens the integrity of testing in general.

Standardizing Network Security Product Testing
There are currently no up-to-date, relevant open test standards for network security performance testing. In the last decade, networks have evolved from 80% unencrypted HTTP — in many enterprises, over 80% of the perimeter traffic is now encrypted with HTTPS and modern secure cipher suites. In other words, network traffic has evolved, changing significantly over the last 10 years, but testing standards and methodologies have not been updated or adapted to account for these changes.

One result of these rapid changes and the absence of universal test standards is that to determine the performance of their network security solutions, testing groups have developed proprietary methods. We have reached a critical point, however, where we need to close the gap between proprietary test performance metrics and observed real-world performance. Otherwise, the tests themselves may become meaningless.

What is needed is greater transparency and standardization of testing methodology, with real-world factors integrated into the testing scenarios. Leading cybersecurity tool vendors and testing labs recognize these requirements, which is why momentum is building for developing and implementing standardized testing methodologies.

Role of NetSecOPEN
NetSecOPEN, a nonprofit, membership-driven organization, was formed in 2017 with the goal of developing open standards for testing network security products. Founding members include leading security vendors, test equipment vendors, and testing laboratories, including Check Point, Cisco, Fortinet, Palo Alto Networks, SonicWall, Sophos, and WatchGuard; test solution and services vendors Spirent and Ixia/Keysight; and testing labs European Advanced Networking Test Center (EANTC), and the University of New Hampshire InterOperability Lab (UNH-IOL).

The organization exists to overcome the current situation — competing and confusing testing methodologies — and establish a new way of designing tests that are open, transparent, and created collaboratively. NetSecOPEN's testing methodology was developed in consultation with the current membership and will continue to evolve as new members join and as a new generation of security products come to market.

The effort to standardize is backed by significant collaboration and momentum. The intent is not to compete with or replace today's testing labs. In fact, the industry's premier testing labs support the effort and are collaborating to improve and standardize network security performance testing. Testing organizations and vendors alike recognize that apples-to-apples performance tests that realistically portray the impact of a network security product on network performance are essential, and they are cooperating to make that happen.

Related Content:

Brian Monkman is executive director of NetSecOPEN, a nonprofit, membership-driven organization with a goal of developing open standards for testing network security products. A 25-year network security veteran, he has extensive experience in technical support, sales ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
ThomasMaloney
50%
50%
ThomasMaloney,
User Rank: Apprentice
2/14/2019 | 12:23:12 AM
First things first
You know what I feel about consumers using their digital devices? I think that many of them really don't care about security because they only worry about the tangible and immediate things like performance and speed. Things like security is secondary to them because they don't see the immediate threat.
EdwardThirlwall
50%
50%
EdwardThirlwall,
User Rank: Moderator
2/11/2019 | 2:09:23 AM
Guidelines needed
It would indeed cause chaos to erupt amongst the end users. People would eventually get confused as to what are the true terms and conditions that they can fall back on when doubts arise. There is no benchmark being set which can be the guideline that they all can refer to to lead them to a solution.
SOC 2s & Third-Party Assessments: How to Prevent Them from Being Used in a Data Breach Lawsuit
Beth Burgin Waller, Chair, Cybersecurity & Data Privacy Practice , Woods Rogers PLC,  12/5/2019
Deliver a Deadly Counterpunch to Ransomware Attacks: 4 Steps
Mathew Newfield, Chief Information Security Officer at Unisys,  12/10/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: Our Endpoint Protection system is a little outdated... 
Current Issue
Navigating the Deluge of Security Data
In this Tech Digest, Dark Reading shares the experiences of some top security practitioners as they navigate volumes of security data. We examine some examples of how enterprises can cull this data to find the clues they need.
Flash Poll
Rethinking Enterprise Data Defense
Rethinking Enterprise Data Defense
Frustrated with recurring intrusions and breaches, cybersecurity professionals are questioning some of the industrys conventional wisdom. Heres a look at what theyre thinking about.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-19604
PUBLISHED: 2019-12-11
Arbitrary command execution is possible in Git before 2.20.2, 2.21.x before 2.21.1, 2.22.x before 2.22.2, 2.23.x before 2.23.1, and 2.24.x before 2.24.1 because a "git submodule update" operation can run commands found in the .gitmodules file of a malicious repository.
CVE-2019-14861
PUBLISHED: 2019-12-10
All Samba versions 4.x.x before 4.9.17, 4.10.x before 4.10.11 and 4.11.x before 4.11.3 have an issue, where the (poorly named) dnsserver RPC pipe provides administrative facilities to modify DNS records and zones. Samba, when acting as an AD DC, stores DNS records in LDAP. In AD, the default permiss...
CVE-2019-14870
PUBLISHED: 2019-12-10
All Samba versions 4.x.x before 4.9.17, 4.10.x before 4.10.11 and 4.11.x before 4.11.3 have an issue, where the S4U (MS-SFU) Kerberos delegation model includes a feature allowing for a subset of clients to be opted out of constrained delegation in any way, either S4U2Self or regular Kerberos authent...
CVE-2019-14889
PUBLISHED: 2019-12-10
A flaw was found with the libssh API function ssh_scp_new() in versions before 0.9.3 and before 0.8.8. When the libssh SCP client connects to a server, the scp command, which includes a user-provided path, is executed on the server-side. In case the library is used in a way where users can influence...
CVE-2019-1484
PUBLISHED: 2019-12-10
A remote code execution vulnerability exists when Microsoft Windows OLE fails to properly validate user input, aka 'Windows OLE Remote Code Execution Vulnerability'.