Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

10:00 AM
Joshua Goldfarb
Joshua Goldfarb
Connect Directly
E-Mail vvv

Keeping a Strong Security Metrics Framework Strong

Don't just report metrics -- analyze, understand, monitor, and adjust them. These 10 tips will show you how.

It takes a significant effort by security teams to build robust metrics that serve the organization well and add value. But keeping that framework strong over time is also an area that requires strategic investment. Unfortunately, it's an area that is often overlooked.

Here are 10 tips to help you maintain the value of your security metrics framework.

Tip 1: Check in with your audience. Metrics are developed to provide important information to the security organization's audiences, not for the sake of the metrics themselves. As such, it's critical to ensure that what you report addresses your audiences' information needs, questions, and concerns. Check in with stakeholders regularly to solicit, accept, and incorporate their feedback. Your audiences aren't a drag on your metrics, they're the reason for them.

Tip 2: Stay alert and attuned. Don't just report metrics — analyze, understand, monitor, and adjust them. If you see that one or more metrics are trending in an uncomfortable manner, dig deeper to understand why that's the case and what the ramifications are for the business. When you monitor metrics on a continuous basis, you will ensure that the risk those metrics measure does not rise to unacceptable levels. If risk levels do rise too high, you can course correct to effectively manage that risk.

Tip 3: Ensure data accuracy. A framework is only as good as the data underlying it. You may have the most relevant and timely metrics, but if the data used to calculate them is inaccurate, inconsistent, and/or flawed, the metrics will be as well. Reliable data serves as an input to reliable metrics while unreliable data, by default, produces unreliable metrics.

Tip 4: Experiment with different models and aggregations. Maybe the way you modeled your framework and aggregated your metrics worked well for you last year. But perhaps things have changed since then and that approach will no longer work. If you've built your metrics modularly, you'll be able to leverage them across a variety of different models and aggregations. Find the one that works for your present-day business environment.

Tip 5: Keep after controls. A mature metrics framework includes proper mapping back to controls. Keep after this mapping. Over time, controls may change in substance, importance, and/or priority. Further, mappings may evolve to be incorrect. Ensuring accurate mapping between controls and metrics allows the security team to continually assess and measure the efficacy of controls to the overall security posture of the business.

Tip 6: Keep after risk. Risk is not static or distinct. It is continuous, dynamic, and fluid. Keeping an eye on the changing risk landscape allows an organization to focus on mitigating the organization's most important and relevant risks, while reducing time and resources spent on less important and relevant issues. This allows finite security resources to be applied to the maximal risk mitigation.

Tip 7: Mind your ranges. When a metric is designed and measured, it creates a data point. Usually that data point is a number or a percentage, which, in and of itself, tells very little of the overall picture and offers no context. To add important context to the risk equation, you need to set an acceptable range and acceptable deviations from that range. Over time, those ranges may require adjustment to reflect changes in the evolution of the business environment and the threat landscape which will affect the tolerance level for the various data points that you measure. Minding your ranges will ensure that your tolerances are in line with acceptable risk levels.

Tip 8: Leverage intelligence. In addition to aiding and informing preventative and detective capabilities, intelligence can also inform metrics. Good intelligence can help you stay informed of existing threats and become aware of new threats. This in turn helps you to continually assess whether or not your metrics have addressed the right set of threats to your organization.

Tip 9: Stay connected. Peer organizations, industry groups, and experts can help an organization see where it lies relative to other organizations of similar size, industry, and geography. These connections can provide essential information that will keep your metrics framework strong.

Tip 10: Be efficient. No metrics framework is sustainable if the process of putting together and reporting the metrics is a headache in and of itself. In order for metrics to be practical and to provide value on a continual basis, they need to be scalable. Consolidate data required for metrics into as few systems as possible. Leverage automated reporting and dashboards to simplify the process of generating metrics when required, ideally automatically and in near real-time. This ensures that metrics will always be fresh. It also reduces your investment in to creating, designing, developing, and generating new metrics, which will, in turn, encourage innovation, creativity, and forward-thinking.

Related Content:

Check out The Edge, Dark Reading's new section for features, threat data, and in-depth perspectives. Today's top story: "C-Level & Studying for the CISSP."

Josh (Twitter: @ananalytical) is currently Director of Product Management at F5.  Previously, Josh served as VP, CTO - Emerging Technologies at FireEye and as Chief Security Officer for nPulse Technologies until its acquisition by FireEye.  Prior to joining nPulse, ... View Full Bio

Recommended Reading:

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
9/22/2020 | 4:13:37 PM
Tip 3 crucial
Tip 3 Ensuring Data accuracy is fundamental.  The tools can only be as good as is permitted by the underlying data.  So if data is missing, compromised or there is too much irrelievant data overwhleming the tools, the metrics will provide a false sense of security and lead in the wrong direction.  This  defeats the entire purpose of a security framework and sets up for failure.   Many companies miss this crucial part and undercut their substantial investment from the very start.  Incomplete or faulty metrics only supercharge your problems,  need to make sure you are using the right information foundation for your framework. 
User Rank: Ninja
3/31/2020 | 9:42:09 AM
Great post and valid points
Tip 2: Stay alert and attuned. Don't just report metrics — analyze, understand, monitor, and adjust them

I do think this is the most important aspect of the post (I do like the areas where you indirectly referred to staying vigiliant and ready). This is a sliding scale that should be adjusted all the time and carefully looking at the metrics to help with this process is essential (basically using a SIEM or brain to capture information and format the data in a way where it makes sense).

But I have to be the bearer of bad news, companies are doing that (again, not all but a large majority of organizations, those who have been hacked - https://www.csis.org/programs/technology-policy-program/significant-cyber-incidents. There is something called Actionable intelligence where the metrics are used to determine if there is a problem and determines what can be done but we need to start looking beyond the numbers and metrics and look into "Prescriptive" security methods where the data is prioritized, listed and categorized where the SIEM solution or some other automated tool can run programs to modify parameters in a networked device. We need to be able to reach that level of potential before the threat becomes a reality.

Industry 4.0 Innovation Map Reveals Emerging Technologies & Startups


We need to be able to tie in all of these sources of data to create a "Sentinel" to identify, analyze, prioritize, react, and learn.

User Rank: Author
2/12/2020 | 4:32:43 PM
Using Metrics to Assess Controls
Using metrics to help assess the effectiveness of controls is a great tip for auditors. As auditors, we can sometimes face difficult conversations when control owners are having trouble processing shortcomings of a control(s). By matching the control to a metric, we maybe able to better help them understand our view. 
More SolarWinds Attack Details Emerge
Kelly Jackson Higgins, Executive Editor at Dark Reading,  1/12/2021
Vulnerability Management Has a Data Problem
Tal Morgenstern, Co-Founder & Chief Product Officer, Vulcan Cyber,  1/14/2021
Register for Dark Reading Newsletters
White Papers
Current Issue
2020: The Year in Security
Download this Tech Digest for a look at the biggest security stories that - so far - have shaped a very strange and stressful year.
Flash Poll
Assessing Cybersecurity Risk in Today's Enterprises
Assessing Cybersecurity Risk in Today's Enterprises
COVID-19 has created a new IT paradigm in the enterprise -- and a new level of cybersecurity risk. This report offers a look at how enterprises are assessing and managing cyber-risk under the new normal.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2021-01-19
Cross-site scripting vulnerability in GROWI (v4.2 Series) versions prior to v4.2.3 allows remote attackers to inject an arbitrary script via unspecified vectors.
PUBLISHED: 2021-01-19
Affected versions of Atlassian Confluence Server and Data Center allow remote attackers to impact the application's availability via a Denial of Service (DoS) vulnerability in the avatar upload feature. The affected versions are before version 7.2.0.
PUBLISHED: 2021-01-18
An issue was discovered in the Source Integration plugin before 2.4.1 for MantisBT. An attacker can gain access to the Summary field of private Issues (either marked as Private, or part of a private Project), if they are attached to an existing Changeset. The information is visible on the view.php p...
PUBLISHED: 2021-01-18
Tar.php in Archive_Tar through 1.4.11 allows write operations with Directory Traversal due to inadequate checking of symbolic links, a related issue to CVE-2020-28948.
PUBLISHED: 2021-01-18
Missing Authorization vulnerability in McAfee Agent (MA) for Windows prior to 5.7.1 allows local users to block McAfee product updates by manipulating a directory used by MA for temporary files. The product would continue to function with out-of-date detection files.