Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


10:00 AM
Zane Lackey
Zane Lackey
Connect Directly
E-Mail vvv

For Real Security, Don't Let Failure Be Your Measure of Success

For too long, we've focused almost exclusively on keeping out the bad guys rather than what to do when they get in (and they will).

Maintaining security is a critical mission for every business today, so you'd expect there to be a commonly accepted way to measure its success. But while other areas of IT are evaluated by increasingly precise and detailed metrics — availability, mean time to repair, and so on — for security, it typically comes down to a crude binary. If we got breached, we failed. If we haven't been breached, we're succeeding. But note the missing word in that last sentence: yet. In reality, every organization will be breached at some point. The only question is whether they'll know about it. To make a breach less likely to happen and less damaging when it does, we need a better way to know how well we're doing along the way.

When Pass-Fail Security Fails
The most obvious problem with the binary pass-fail approach to measuring cybersecurity success is that there's no way to use it to drive incremental improvement. More fundamentally, it's based on the false idea of perfect prevention as a goal. For years, both vendors and compliance-driven security programs have acted as if the right combination of firewalls and antivirus protection can keep companies entirely safe. As a result, they haven't focused enough on what happens when an attacker makes it through these perimeter-focused defenses. At that point, with all the emphasis having been on prevention and not detection, it's often game over for the company as the attacker moves undetected through the environment.

Assume Penetration — and Act Accordingly
Cybersecurity professionals would do well to consider the lessons learned and innovation in the residential security market. In the old days, people counted on locking their windows and doors to protect their homes, and yet burglaries remained common. It only took one forgotten lock and a burglar would soon be carrying away his loot. Perimeter alarms were similarly vulnerable to human error and brute force — until motion detectors enabled alerts no matter how intruders got in. Now, smart home technologies make it possible for people to get instant visibility of a possible home breach, see what's happening inside and around the house, lock doors and windows remotely, and call the police while there's still time.

For too long, cybersecurity has been all about preventative controls like the lock while completely missing visibility and detection controls like the camera. But even the best lock is a tactic, not a strategy. What happens when the lock fails to stop a burglar, or when an attacker bypasses the installed antivirus? At that point, the focus must shift to quick detection and reaction in order to limit the damage — and the idea of more and more locks is no longer relevant.

Make Hackers Hate You
If no product can completely prevent a breach, we must consider other metrics of success for security. A three-step approach offers a starting point for this assessment:

1. What are the most common successful attack vectors for my type of company and environment?

2. How likely would we be to detect such an attack should it occur?

3. Can we make this type of attack harder and more expensive for the hacker?

For example: For years, we all put antivirus protection on our laptops, but how would we know if it has been bypassed or if an attack were succeeding against it? If off-the-shelf commands are being run in a malicious manner on the laptop, even if there's no "malware" in a classic sense, you can be fairly sure there's an attacker involved — but first, you must be set up to detect this. What other types of attacks pose the greatest risk for your business? What signs could tip you off that they're underway, and are you able to detect them? These sorts of questions led to a fundamental transformation of the antivirus industry, from legacy players to today's thriving modern endpoint protection market.

Assume Nothing
Once you have this visibility, the next step is to create a feedback loop to continuously evaluate and improve your countermeasures. Bug bounties and attack simulations by modern penetration test firms will tell you what's working and what isn't. Returning to our home security analogy, homeowners often concentrate resources such as cameras and motion detectors around the TV, but a simulation could show that a burglar's most likely target is the jewelry in the bedroom. To make the most of finite cybersecurity resources, the most effective strategy is to deploy detection and visibility throughout your environment, run simulations to see where what a successful attack will look like, and then use this knowledge to deploy defenses where they'll do the most good.

Visibility, detection, and the continuous improvement they enable can make it harder to breach your environment. To make it more expensive for attackers as well, you also want to pepper your environment with detection tools that instantly phone home if accessed by an attacker. Historically this was done via hard-to-maintain tools like honeypots, but today it's easily accomplished via modern products (for instance, the Thinkst Canary).

By shifting from a model of "100% focus on preventative controls with a compliance mindset" to "obtaining visibility and using feedback loops to give us data on where to better allocate our security resources," we can define and measure cybersecurity success in a way that's both more realistic and more useful for driving improvement. Are you detecting the attackers that matter most? How quickly and accurately? Strengthen and shift your resources based on what you've learned. Repeat.

Related Content:


Black Hat USA returns to Las Vegas with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions and service providers in the Business Hall. Click for information on the conference and to register.

Zane Lackey is the co-founder and CSO at Signal Sciences and the author of Building a Modern Security Program (O'Reilly Media). He serves on multiple advisory boards, including the National Technology Security Coalition, the Internet Bug Bounty Program, and the US State ... View Full Bio

Recommended Reading:

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Ninja
7/19/2019 | 7:53:18 PM
Excellent point about obtaining visibility
  1. What are the most common successful attack vectors for my type of company and environment?
    • Email, Web, RDP and SSH
  2. How likely would we be to detect such an attack should it occur?
    • Extremely likely, we review logs everyday on external systems, we have SIEM, HIDS, NIDS monitoring essential and non-essential systems, notifications are sent and collected using Logwatch | crontab provides extensive information
  3. Can we make this type of attack harder and more expensive for the hacker?
    • Yes, using Web (NGFW, place in DMZ) and Email (Proofpoint)
    • RDP - use hardening mechanisms provided by DISA Stigs, only allow local subnets
    • SSH - use keys, remove root access, only allow certain ranges of IP to access systems
    • IPv6 - implement IPv6 and move off of IPv4 (most attacks come from IPv4, we have identified that on our AWS and GCP servers), configure VPN ESP/AH AES256 IPSec tunnels
    • Utilize cloud service to monitor servers and access
    • Countries that are not relevant to the business remove access, insert rules to block countries (PaloAlto, SonicWall, Juniper, and others do a good job), but also employ the blocking mechanism on outbound ACLs
    • Encrypt data at rest and in transit, utilize IPSec from IPv6
    • Configure MPLS VPN rd1:1 connections to remote sites, think about IS-IS configuration for remote connections, think about Route Bridges and TRILL, move away from OSPF (no self-healing properties)

This is what I can think about off the top of my head, other items will come.

How SolarWinds Busted Up Our Assumptions About Code Signing
Dr. Jethro Beekman, Technical Director,  3/3/2021
'ObliqueRAT' Now Hides Behind Images on Compromised Websites
Jai Vijayan, Contributing Writer,  3/2/2021
Attackers Turn Struggling Software Projects Into Trojan Horses
Robert Lemos, Contributing Writer,  2/26/2021
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Write a Caption, Win an Amazon Gift Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2021-03-04
Xerox AltaLink B8045/B8055/B8065/B8075/B8090 and C8030/C8035/C8045/C8055/C8070 multifunction printers with software releases before 101.00x.099.28200 allow an attacker to execute an unwanted binary during a exploited clone install. This requires creating a clone file and signing that file with a com...
PUBLISHED: 2021-03-04
Xerox AltaLink B8045/B8055/B8065/B8075/B8090 and C8030/C8035/C8045/C8055/C8070 multifunction printers with software releases before 101.00x.099.28200 allow a user with administrative privileges to turn off data encryption on the device, thus leaving it open to potential cryptographic information dis...
PUBLISHED: 2021-03-03
The Java client for the Datadog API before version 1.0.0-beta.9 has a local information disclosure of sensitive information downloaded via the API using the API Client. The Datadog API is executed on a unix-like system with multiple users. The API is used to download a file containing sensitive info...
PUBLISHED: 2021-03-03
resources/public/js/orchestrator.js in openark orchestrator before 3.2.4 allows XSS via the orchestrator-msg parameter.
PUBLISHED: 2021-03-03
GLPI is open source software which stands for Gestionnaire Libre de Parc Informatique and it is a Free Asset and IT Management Software package. In GLPI before verison 9.5.4, there is a vulnerability within the document upload function (Home > Management > Documents > Add, or /front/documen...