Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

10:00 AM
Joshua Goldfarb
Joshua Goldfarb
Connect Directly
E-Mail vvv

Keeping a Strong Security Metrics Framework Strong

Don't just report metrics -- analyze, understand, monitor, and adjust them. These 10 tips will show you how.

It takes a significant effort by security teams to build robust metrics that serve the organization well and add value. But keeping that framework strong over time is also an area that requires strategic investment. Unfortunately, it's an area that is often overlooked.

Here are 10 tips to help you maintain the value of your security metrics framework.

Tip 1: Check in with your audience. Metrics are developed to provide important information to the security organization's audiences, not for the sake of the metrics themselves. As such, it's critical to ensure that what you report addresses your audiences' information needs, questions, and concerns. Check in with stakeholders regularly to solicit, accept, and incorporate their feedback. Your audiences aren't a drag on your metrics, they're the reason for them.

Tip 2: Stay alert and attuned. Don't just report metrics — analyze, understand, monitor, and adjust them. If you see that one or more metrics are trending in an uncomfortable manner, dig deeper to understand why that's the case and what the ramifications are for the business. When you monitor metrics on a continuous basis, you will ensure that the risk those metrics measure does not rise to unacceptable levels. If risk levels do rise too high, you can course correct to effectively manage that risk.

Tip 3: Ensure data accuracy. A framework is only as good as the data underlying it. You may have the most relevant and timely metrics, but if the data used to calculate them is inaccurate, inconsistent, and/or flawed, the metrics will be as well. Reliable data serves as an input to reliable metrics while unreliable data, by default, produces unreliable metrics.

Tip 4: Experiment with different models and aggregations. Maybe the way you modeled your framework and aggregated your metrics worked well for you last year. But perhaps things have changed since then and that approach will no longer work. If you've built your metrics modularly, you'll be able to leverage them across a variety of different models and aggregations. Find the one that works for your present-day business environment.

Tip 5: Keep after controls. A mature metrics framework includes proper mapping back to controls. Keep after this mapping. Over time, controls may change in substance, importance, and/or priority. Further, mappings may evolve to be incorrect. Ensuring accurate mapping between controls and metrics allows the security team to continually assess and measure the efficacy of controls to the overall security posture of the business.

Tip 6: Keep after risk. Risk is not static or distinct. It is continuous, dynamic, and fluid. Keeping an eye on the changing risk landscape allows an organization to focus on mitigating the organization's most important and relevant risks, while reducing time and resources spent on less important and relevant issues. This allows finite security resources to be applied to the maximal risk mitigation.

Tip 7: Mind your ranges. When a metric is designed and measured, it creates a data point. Usually that data point is a number or a percentage, which, in and of itself, tells very little of the overall picture and offers no context. To add important context to the risk equation, you need to set an acceptable range and acceptable deviations from that range. Over time, those ranges may require adjustment to reflect changes in the evolution of the business environment and the threat landscape which will affect the tolerance level for the various data points that you measure. Minding your ranges will ensure that your tolerances are in line with acceptable risk levels.

Tip 8: Leverage intelligence. In addition to aiding and informing preventative and detective capabilities, intelligence can also inform metrics. Good intelligence can help you stay informed of existing threats and become aware of new threats. This in turn helps you to continually assess whether or not your metrics have addressed the right set of threats to your organization.

Tip 9: Stay connected. Peer organizations, industry groups, and experts can help an organization see where it lies relative to other organizations of similar size, industry, and geography. These connections can provide essential information that will keep your metrics framework strong.

Tip 10: Be efficient. No metrics framework is sustainable if the process of putting together and reporting the metrics is a headache in and of itself. In order for metrics to be practical and to provide value on a continual basis, they need to be scalable. Consolidate data required for metrics into as few systems as possible. Leverage automated reporting and dashboards to simplify the process of generating metrics when required, ideally automatically and in near real-time. This ensures that metrics will always be fresh. It also reduces your investment in to creating, designing, developing, and generating new metrics, which will, in turn, encourage innovation, creativity, and forward-thinking.

Related Content:

Check out The Edge, Dark Reading's new section for features, threat data, and in-depth perspectives. Today's top story: "C-Level & Studying for the CISSP."

Josh (Twitter: @ananalytical) is currently Director of Product Management at F5.  Previously, Josh served as VP, CTO - Emerging Technologies at FireEye and as Chief Security Officer for nPulse Technologies until its acquisition by FireEye.  Prior to joining nPulse, ... View Full Bio

Recommended Reading:

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
9/22/2020 | 4:13:37 PM
Tip 3 crucial
Tip 3 Ensuring Data accuracy is fundamental.  The tools can only be as good as is permitted by the underlying data.  So if data is missing, compromised or there is too much irrelievant data overwhleming the tools, the metrics will provide a false sense of security and lead in the wrong direction.  This  defeats the entire purpose of a security framework and sets up for failure.   Many companies miss this crucial part and undercut their substantial investment from the very start.  Incomplete or faulty metrics only supercharge your problems,  need to make sure you are using the right information foundation for your framework. 
User Rank: Ninja
3/31/2020 | 9:42:09 AM
Great post and valid points
Tip 2: Stay alert and attuned. Don't just report metrics — analyze, understand, monitor, and adjust them

I do think this is the most important aspect of the post (I do like the areas where you indirectly referred to staying vigiliant and ready). This is a sliding scale that should be adjusted all the time and carefully looking at the metrics to help with this process is essential (basically using a SIEM or brain to capture information and format the data in a way where it makes sense).

But I have to be the bearer of bad news, companies are doing that (again, not all but a large majority of organizations, those who have been hacked - https://www.csis.org/programs/technology-policy-program/significant-cyber-incidents. There is something called Actionable intelligence where the metrics are used to determine if there is a problem and determines what can be done but we need to start looking beyond the numbers and metrics and look into "Prescriptive" security methods where the data is prioritized, listed and categorized where the SIEM solution or some other automated tool can run programs to modify parameters in a networked device. We need to be able to reach that level of potential before the threat becomes a reality.

Industry 4.0 Innovation Map Reveals Emerging Technologies & Startups


We need to be able to tie in all of these sources of data to create a "Sentinel" to identify, analyze, prioritize, react, and learn.

User Rank: Author
2/12/2020 | 4:32:43 PM
Using Metrics to Assess Controls
Using metrics to help assess the effectiveness of controls is a great tip for auditors. As auditors, we can sometimes face difficult conversations when control owners are having trouble processing shortcomings of a control(s). By matching the control to a metric, we maybe able to better help them understand our view. 
Inside the Ransomware Campaigns Targeting Exchange Servers
Kelly Sheridan, Staff Editor, Dark Reading,  4/2/2021
Beyond MITRE ATT&CK: The Case for a New Cyber Kill Chain
Rik Turner, Principal Analyst, Infrastructure Solutions, Omdia,  3/30/2021
Register for Dark Reading Newsletters
White Papers
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2021-04-15
Exposure of Sensitive Information in the web interface in McAfee Advanced Threat Defense (ATD) prior to 4.12.2 allows remote authenticated users to view sensitive unencrypted information via a carefully crafted HTTP request parameter. The risk is partially mitigated if your ATD instances are deploye...
PUBLISHED: 2021-04-15
Cleartext Transmission of Sensitive Information between McAfee Endpoint Security (ENS) for Windows prior to 10.7.0 February 2021 Update and McAfee Global Threat Intelligence (GTI) servers using DNS allows a remote attacker to view the requests from ENS and responses from GTI over DNS. By gaining con...
PUBLISHED: 2021-04-15
Cleartext Transmission of Sensitive Information vulnerability in the ePO Extension of McAfee Content Security Reporter (CSR) prior to 2.8.0 allows an ePO administrator to view the unencrypted password of the McAfee Web Gateway (MWG) or the password of the McAfee Web Gateway Cloud Server (MWGCS) read...
PUBLISHED: 2021-04-15
Denial of Service vulnerability in McAfee Data Loss Prevention (DLP) Endpoint for Windows prior to 11.6.100 allows a local, low privileged, attacker to cause a BSoD through suspending a process, modifying the processes memory and restarting it. This is triggered by the hdlphook driver reading invali...
PUBLISHED: 2021-04-15
Privilege Escalation vulnerability in McAfee Data Loss Prevention (DLP) Endpoint for Windows prior to 11.6.100 allows a local, low privileged, attacker to write to arbitrary controlled kernel addresses. This is achieved by launching applications, suspending them, modifying the memory and restarting ...