NSS Labs Announces 2017 Breach Detection Systems Group Test ResultsFour Products Receive Recommended Rating, other products receive a neutral rating, security recommended rating, and caution rating.
AUSTIN, Texas - NSS Labs, a source for independent fact-based cybersecurity guidance, today announced the results of its Breach Detection Systems (BDS) Group Test. For the 2017 BDS Group Test, NSS Labs expanded test techniques to include new evasions. Evasion techniques are a means of disguising and modifying attacks in order to avoid detection and blocking by security products. These methods are increasingly selected by threat actors to circumvent security controls. Test results indicate that most breach detection systems are unable to stop all evasions, which is a major concern.
According to the Verizon 2017 Data Breach Investigations Report, 75% of data breaches are perpetrated by outsiders. BDS products are now more widely adopted in enterprise security deployments to catch attacks that bypass other perimeter defenses such as next-generation firewalls. The NSS Labs 2017 Enterprise Security Architecture Study indicates that 44.1% of US enterprises deploy BDS products. These products provide enhanced detection of advanced malware, zero-day attacks, and targeted attacks. Data obtained by BDS threat analysis helps to accelerate investigation and remediation, thereby reducing the business impact of threats.
The BDS Group Test reports offer insights into which infection vectors were effective and how long it took for products to detect them. When evaluating breach detection systems, time-to-detect metrics are critical as they directly impact ROI for enterprises.
Seven products from six market-leading security vendors were tested for security effectiveness, performance, stability and reliability, and total cost of ownership (TCO). Of the seven products tested, four received a Recommended rating and two scored 100% for security effectiveness.
Key findings from the test:
- Five out of the seven products tested missed evasions.
- Overall Security Effectiveness ranged between 80.2% and 100.0%.
- The average Security Effectiveness rating was 93.2%; five products received a Security Effectiveness rating above the average, and two received a Security Effectiveness rating below the average.
- False positive rates ranged from 0% to 0.36%.
- TCO per Protected Mbps ranged between US$16 and US$128, with most tested products costing less than US$44 per protected Mbps.
- The average TCO per Protected Mbps (Value) was US$48.82; five products demonstrated value above the average and two demonstrated value below the average.
“The 2017 BDS Group Test revealed that most products were able to detect the majority of breaches within 60 minutes, but some products took several hours to detect the same breaches,” said Jason Brvenik, Chief Technology Officer at NSS Labs. “To reduce the frequency and impact of security incidents, enterprises must consider the detection rate and the time a product takes to detect attacks.”
The following products were tested:
- Check Point Software Technologies 15600 Next Generation Threat Prevention & SandBlast™ (NGTX) Appliance R77.30
- Cisco FirePower 8120 v.6 & Cisco AMP v.22.214.171.12430
- FireEye Network Security NX 10450 v7.9.2 & EX 8400 v7.9.0
- FireEye Network Security 6500NXES-VA v7.9.2
- Fortinet FortiSandbox-2000E v.FSA 2.4.1 & FortiClient (APT Agent) v.126.96.36.1995
- Lastline Enterprise v7.25
- Trend Micro Deep Discovery Inspector Model 4000 v3.8 SP5 & OfficeScan (OSCE) v.12.0.1807
NSS Labs is committed to providing empirical data and objective group test results that enable organizations to make educated decisions about purchasing and optimizing security infrastructure products and services. As with all NSS Labs group tests, there is no fee for participation, and the test methodology is available in the public domain to provide transparency and to help enterprises understand the factors behind test results. Click here for more information about our group test policies.
Click here for more information on this test, and the test methodology used, or to purchase the individual Test Reports. Click here to download the Security Value Map™, which provides a graphic comparison of Security Effectiveness and TCO across the tested products.