Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Risk

5/13/2019
10:30 AM
Timothy Winters
Timothy Winters
Commentary
Connect Directly
Twitter
RSS
E-Mail vvv
50%
50%

How Open Testing Standards Can Improve Security

When creating security metrics, it's critical that test methodologies cover multiple scenarios to ensure that devices perform as expected in all environments.

Networks are a complex collection of components defined by many different standards. These standards help solve network problems ranging from security to performance and usability.

An open standard is a publicly available standard that can be consumed in a variety of ways for deploying a secure solution for a network. Readers of open security standards use them to understand how a technology might be useful to solve security on the network. Implementers of open standards can create solutions to address documented security issues. Network operators read standards to understand how the different implementations work together to make a complete security solution.

These network solutions often come from different sources, which leads to the creation of a variety of testing procedures and methodologies to ensure that network components support all the security and performance requirements of the network users. Since the majority of standards are also open, it would make sense that the methods for testing are also open. But often this isn't the case, and I think it should be.

The Case for Open Security Testing Standards
The argument I often hear against open testing standards is because network component engineers can see the test and create a solution based on the known criteria. This, to use a grade school analogy, seems like cheating since the test questions are known in advance, making it possible for a network operator to engineer their products to pass the test. If the tests have full coverage for the security features that a network operator wants, then it doesn't matter if they know what is being tested. The outcome of the testing will be a network component that shows compliance to the full coverage of test cases. By creating an open testing environment, network component engineers can build a solution that will meet the network operators' requirements.

When creating security metrics, it's critical that test methodologies cover multiple scenarios to ensure that devices perform as expected in all environments. For security test methodologies, it may be necessary to randomize input parameters to cover all use cases in order to detect devices that have tuned device performance to meet test case needs rather than the needs of real use cases. For example, when measuring if a firewall detects CVEs, it's important to run a traffic mix with vulnerabilities to ensure the device detects and blocks attacks under a variety of conditions.

Another advantage of open testing standards is that they give users and network operators the ability to see what security testing is performed and how testing is performed. Knowing what security test cases are being performed allows the operator to confirm that the test meets specific requirements. If not, they can add additional tests.

Creating a Feedback Loop
If there is an organization responsible for maintaining the standard, operators can feed that information back to cover missing areas so that in the future the network operator won't have to run additional testing. Knowing how network components are tested also lets network operators and users better understand the meaning of results because results alone often don't give enough context about the testing conditions of the network component. For example, it's important to understand if a device passes security tests when there is no load but doesn't detect attacks when it's under load.

It's also important to compare security results from different networking providers as a means of increasing transparency into testing methodologies, which also leads to better decision-making processes. In other words, open testing standards provide an "apples to apples" comparison opportunity. In security performance testing, for example, the results of a bandwidth test on a firewall can change greatly based on the security features that are enabled. If no open standard exists to specify that information, a user might be looking at results for two different implementations and not understand that the results differ depending on what features are enabled.

Implementers of security standards are aided by having open testing standards offering better visibility into what network operators are interested in validating. Network operators are aided by open standards testing to allow them to achieve comparisons that make network decisions easier.

Related Content:

 

 

 Join Dark Reading LIVE for two cybersecurity summits at Interop 2019. Learn from the industry's most knowledgeable IT security experts. Check out the Interop agenda here.

Timothy Winters is a senior manager at the University of New Hampshire InterOperability Laboratory (UNH-IOL). He works with companies from all over the world to develop broad-based, flexible testing strategies to cost-effectively meet network interoperability ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
AI Is Everywhere, but Don't Ignore the Basics
Howie Xu, Vice President of AI and Machine Learning at Zscaler,  9/10/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
7 Threats & Disruptive Forces Changing the Face of Cybersecurity
This Dark Reading Tech Digest gives an in-depth look at the biggest emerging threats and disruptive forces that are changing the face of cybersecurity today.
Flash Poll
The State of IT Operations and Cybersecurity Operations
The State of IT Operations and Cybersecurity Operations
Your enterprise's cyber risk may depend upon the relationship between the IT team and the security team. Heres some insight on what's working and what isn't in the data center.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-16349
PUBLISHED: 2019-09-16
Bento4 1.5.1-628 has a NULL pointer dereference in AP4_ByteStream::ReadUI32 in Core/Ap4ByteStream.cpp when called from the AP4_TrunAtom class.
CVE-2019-16350
PUBLISHED: 2019-09-16
ffjpeg before 2019-08-18 has a NULL pointer dereference in idct2d8x8() at dct.c.
CVE-2019-16351
PUBLISHED: 2019-09-16
ffjpeg before 2019-08-18 has a NULL pointer dereference in huffman_decode_step() at huffman.c.
CVE-2019-16352
PUBLISHED: 2019-09-16
ffjpeg before 2019-08-21 has a heap-based buffer overflow in jfif_load() at jfif.c.
CVE-2016-10967
PUBLISHED: 2019-09-16
The real3d-flipbook-lite plugin 1.0 for WordPress has XSS via the wp-content/plugins/real3d-flipbook/includes/flipbooks.php bookId parameter.