Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Risk

5/21/2020
02:00 PM
Henry Harrison
Henry Harrison
Commentary
Connect Directly
LinkedIn
RSS
E-Mail vvv
50%
50%

How an Industry Consortium Can Reinvent Security Solution Testing

By committing to independent testing to determine value, vendors will ensure that their products do what they say they do.

The challenge with proof-of-concepts (PoCs) for cybersecurity solutions is that they primarily tell chief information security officers (CISOs) and their teams that a product will be quick to integrate and has a strong user interface. These things are easy to measure. But whether the solutions actually work in terms of defeating attacks and mitigating risk? That is a much more difficult capability to assess. Unfortunately, when the PoC fails to prevent exposure, CISOs are too often caught in the middle after a crippling attack.

Why do PoCs fall short? It's because the cost of pursuing in-depth testing remains prohibitive for many organizations. Cybersecurity vendors take full advantage because they have no incentive to do much more than simply measure user interface and ease of integration. That's why it's past time to tear down and rebuild how we conduct solution evaluations.

Note the use of the word "we" here. We are doomed to continue spinning our wheels unless we unite as an industry.

While vendors have introduced some notable initiatives, such as NetSecOpen, the industry can't totally rely on vendors to provide a plan or framework for a more standardized approach for assessing and then testing new solutions. Enterprises must take the lead if we want to see real change. Now more than ever during our COVID-19 existence, we need an industry consortium to empower enterprises to better assess products, especially for organizations that do not have the margins to oversee effective PoCs on their own.

In some cases, a logical first step is to rely on the assessments already carried out by governments. On the other hand, while it's important to consider that there are significant differences between governments and commercial entities, it is not always clear that their security requirements are the same. In addition, each government has their own approach and opinion – a real challenge for global enterprises.

A Fragmented Market
The cybersecurity solutions and services market has grown increasingly fragmented, and the beleaguered CISO is under immense pressure to demonstrate ROI. Thus, a consortium – one that brings together multiple buyers of a tool to collaborate on its true value before purchase and implementation – proves critical. The consortium would readily resolve the cost issues, as companies would collectively pool their PoC budgeting to fund for more thorough white box and black box testing.

It is true that these businesses compete against each other in the marketplace. But they are also peers. It's vastly better to collaborate with peers/competitors to know what is unknown before spending countless dollars on a tool that could very well fail them in short time. Besides, it's not as if one consortium member would gain competitive edge over another in combining resources for testing; everyone is on an even playing field.

For this to work, the participating organizations must designate independent testers as an indispensable component. The testers would serve as unquestioned truth-seekers with no skin in the game. They are strictly the "home inspectors" here, not the buyers or sellers of the house.

As for methodology, independent testers must ensure the products protect against both existing threats and future, as-of-yet-unknown ones. They should require vendors to reveal complete details about their designs, implementation, and engineering practices - without allowing them to hide behind "commercial confidentiality" as means to avoid disclosure.

Intensive Assessments and Reviews
With this, testers will be able to conduct intensive assessments of detailed design and implementation documentation, along with source code reviews. They then can proceed with comprehensive white box testing against known attacks, as well as potential future ones on the assumption that attackers will eventually be armed with full knowledge of design and implementation. Of course, such a level of extensive testing will amount to an expensive proposition. But, again, consortium members would share the cost burden by pooling together their available funding.

What's more, should this optimal level of evaluation emerge as the norm, it will force vendors to make changes on their own. In the interest of pure survival, they will budget for even more rigorous internal vetting of what they intend to bring to the market.

Enterprises and their CISOs are dealing with a broad spectrum of cyber-risk mitigation activities and operational issues that make it difficult for them to pay attention to the available product assessment and testing options. Enterprises with like-minded goals can trust each other to determine the right testing they collectively require. Power in numbers and bigger dollar pools will enable the consortium plan to drive real change.

We cannot delay. The current state of PoC processes, product assessment, and testing won't tell us enough of what we need to know. But by committing to a consortium with independent testing to determine true value, vendors will have to ensure the products do what they say they do – or risk extinction.

Related Content:

Check out The Edge, Dark Reading's new section for features, threat data, and in-depth perspectives. Today's featured story: "Biometrics in the Great Beyond."

Henry Harrison is co-founder and CTO at Garrison, and a seasoned IT industry executive, serial entrepreneur and the brain behind Garrison's core technologies. Henry has a background in leading the development of innovation in cyber security and Garrison was founded to create ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
HardenStance
50%
50%
HardenStance,
User Rank: Strategist
5/21/2020 | 2:53:40 PM
Nothing wrong with the sentiment but...
There's nothing wrong with the sentiment here.

Quite the contrary.

It's as objectionable as motherhood and apple pie. 

But just as useful, practically speaking.

The challenge is to scrabble together a sustained commitment and supporting resources for something like this.

Enterprise CISOs expect their team members to play for their team and to be able to correlate any external collaboration closely to demonstrable near term advantage for their organizations.

For this type of effort to succeed requires multi-party funding. That means business organizations (e.g Chambers of Commerce); the cyber security industry (vendors) and (yes) government (national cyber security agencies, Industry & Economy Ministries etc)

Complex and challenging but I can't see how anything happens without it.

 
AndrewRGravett
50%
50%
AndrewRGravett,
User Rank: Apprentice
5/22/2020 | 4:20:43 AM
pros and cons
I can definitely see the benefits of this approach to both parties i.e. transparency of results, reduction of effort but also some issues the first issue I would consider is overall independent governance of a consortium so that there is a level playing field, rigorous test methodology etc, second  that even within the same industry or market segment consortium "no two environments are the same" for example networking hardware age, firmware, configuration, traffic flows etc so a result in a standardized PoC test lab would still need replicating and testing in "my environment" so effectively doubling the effort..
COVID-19: Latest Security News & Commentary
Dark Reading Staff 6/4/2020
Abandoned Apps May Pose Security Risk to Mobile Devices
Robert Lemos, Contributing Writer,  5/29/2020
How AI and Automation Can Help Bridge the Cybersecurity Talent Gap
Peter Barker, Chief Product Officer at ForgeRock,  6/1/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: What? IT said I needed virus protection!
Current Issue
How Cybersecurity Incident Response Programs Work (and Why Some Don't)
This Tech Digest takes a look at the vital role cybersecurity incident response (IR) plays in managing cyber-risk within organizations. Download the Tech Digest today to find out how well-planned IR programs can detect intrusions, contain breaches, and help an organization restore normal operations.
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-13768
PUBLISHED: 2020-06-04
In MiniShare before 1.4.2, there is a stack-based buffer overflow via an HTTP PUT request, which allows an attacker to achieve arbitrary code execution, a similar issue to CVE-2018-19861, CVE-2018-19862, and CVE-2019-17601. NOTE: this product is discontinued.
CVE-2020-13849
PUBLISHED: 2020-06-04
The MQTT protocol 3.1.1 requires a server to set a timeout value of 1.5 times the Keep-Alive value specified by a client, which allows remote attackers to cause a denial of service (loss of the ability to establish new connections), as demonstrated by SlowITe.
CVE-2020-13848
PUBLISHED: 2020-06-04
Portable UPnP SDK (aka libupnp) 1.12.1 and earlier allows remote attackers to cause a denial of service (crash) via a crafted SSDP message due to a NULL pointer dereference in the functions FindServiceControlURLPath and FindServiceEventURLPath in genlib/service_table/service_table.c.
CVE-2020-11682
PUBLISHED: 2020-06-04
Castel NextGen DVR v1.0.0 is vulnerable to CSRF in all state-changing request. A __RequestVerificationToken is set by the web interface, and included in requests sent by web interface. However, this token is not verified by the application: the token can be removed from all requests and the request ...
CVE-2020-12847
PUBLISHED: 2020-06-04
Pydio Cells 2.0.4 web application offers an administrative console named “Cells Console� that is available to users with an administrator role. This console provides an administrator user with the possibility of changing several settings, including the applicat...