Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

10:00 AM
Joshua Goldfarb
Joshua Goldfarb
Connect Directly
E-Mail vvv

Keeping a Strong Security Metrics Framework Strong

Don't just report metrics -- analyze, understand, monitor, and adjust them. These 10 tips will show you how.

It takes a significant effort by security teams to build robust metrics that serve the organization well and add value. But keeping that framework strong over time is also an area that requires strategic investment. Unfortunately, it's an area that is often overlooked.

Here are 10 tips to help you maintain the value of your security metrics framework.

Tip 1: Check in with your audience. Metrics are developed to provide important information to the security organization's audiences, not for the sake of the metrics themselves. As such, it's critical to ensure that what you report addresses your audiences' information needs, questions, and concerns. Check in with stakeholders regularly to solicit, accept, and incorporate their feedback. Your audiences aren't a drag on your metrics, they're the reason for them.

Tip 2: Stay alert and attuned. Don't just report metrics — analyze, understand, monitor, and adjust them. If you see that one or more metrics are trending in an uncomfortable manner, dig deeper to understand why that's the case and what the ramifications are for the business. When you monitor metrics on a continuous basis, you will ensure that the risk those metrics measure does not rise to unacceptable levels. If risk levels do rise too high, you can course correct to effectively manage that risk.

Tip 3: Ensure data accuracy. A framework is only as good as the data underlying it. You may have the most relevant and timely metrics, but if the data used to calculate them is inaccurate, inconsistent, and/or flawed, the metrics will be as well. Reliable data serves as an input to reliable metrics while unreliable data, by default, produces unreliable metrics.

Tip 4: Experiment with different models and aggregations. Maybe the way you modeled your framework and aggregated your metrics worked well for you last year. But perhaps things have changed since then and that approach will no longer work. If you've built your metrics modularly, you'll be able to leverage them across a variety of different models and aggregations. Find the one that works for your present-day business environment.

Tip 5: Keep after controls. A mature metrics framework includes proper mapping back to controls. Keep after this mapping. Over time, controls may change in substance, importance, and/or priority. Further, mappings may evolve to be incorrect. Ensuring accurate mapping between controls and metrics allows the security team to continually assess and measure the efficacy of controls to the overall security posture of the business.

Tip 6: Keep after risk. Risk is not static or distinct. It is continuous, dynamic, and fluid. Keeping an eye on the changing risk landscape allows an organization to focus on mitigating the organization's most important and relevant risks, while reducing time and resources spent on less important and relevant issues. This allows finite security resources to be applied to the maximal risk mitigation.

Tip 7: Mind your ranges. When a metric is designed and measured, it creates a data point. Usually that data point is a number or a percentage, which, in and of itself, tells very little of the overall picture and offers no context. To add important context to the risk equation, you need to set an acceptable range and acceptable deviations from that range. Over time, those ranges may require adjustment to reflect changes in the evolution of the business environment and the threat landscape which will affect the tolerance level for the various data points that you measure. Minding your ranges will ensure that your tolerances are in line with acceptable risk levels.

Tip 8: Leverage intelligence. In addition to aiding and informing preventative and detective capabilities, intelligence can also inform metrics. Good intelligence can help you stay informed of existing threats and become aware of new threats. This in turn helps you to continually assess whether or not your metrics have addressed the right set of threats to your organization.

Tip 9: Stay connected. Peer organizations, industry groups, and experts can help an organization see where it lies relative to other organizations of similar size, industry, and geography. These connections can provide essential information that will keep your metrics framework strong.

Tip 10: Be efficient. No metrics framework is sustainable if the process of putting together and reporting the metrics is a headache in and of itself. In order for metrics to be practical and to provide value on a continual basis, they need to be scalable. Consolidate data required for metrics into as few systems as possible. Leverage automated reporting and dashboards to simplify the process of generating metrics when required, ideally automatically and in near real-time. This ensures that metrics will always be fresh. It also reduces your investment in to creating, designing, developing, and generating new metrics, which will, in turn, encourage innovation, creativity, and forward-thinking.

Related Content:

Check out The Edge, Dark Reading's new section for features, threat data, and in-depth perspectives. Today's top story: "C-Level & Studying for the CISSP."

Josh (Twitter: @ananalytical) is an experienced information security leader who works with enterprises to mature and improve their enterprise security programs.  Previously, Josh served as VP, CTO - Emerging Technologies at FireEye and as Chief Security Officer for ... View Full Bio
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
2/12/2020 | 4:32:43 PM
Using Metrics to Assess Controls
Using metrics to help assess the effectiveness of controls is a great tip for auditors. As auditors, we can sometimes face difficult conversations when control owners are having trouble processing shortcomings of a control(s). By matching the control to a metric, we maybe able to better help them understand our view. 
Cybersecurity Industry: It's Time to Stop the Victim Blame Game
Jessica Smith, Senior Vice President, The Crypsis Group,  2/25/2020
Google Adds More Security Features Via Chronicle Division
Robert Lemos, Contributing Writer,  2/25/2020
Register for Dark Reading Newsletters
White Papers
Current Issue
6 Emerging Cyber Threats That Enterprises Face in 2020
This Tech Digest gives an in-depth look at six emerging cyber threats that enterprises could face in 2020. Download your copy today!
Flash Poll
How Enterprises Are Developing and Maintaining Secure Applications
How Enterprises Are Developing and Maintaining Secure Applications
The concept of application security is well known, but application security testing and remediation processes remain unbalanced. Most organizations are confident in their approach to AppSec, although others seem to have no approach at all. Read this report to find out more.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2020-02-27
In Wireshark 3.2.0 to 3.2.1, 3.0.0 to 3.0.8, and 2.6.0 to 2.6.14, the LTE RRC dissector could leak memory. This was addressed in epan/dissectors/packet-lte-rrc.c by adjusting certain append operations.
PUBLISHED: 2020-02-27
openssl_x509_check_host in lua-openssl 0.7.7-1 mishandles X.509 certificate validation because it uses lua_pushboolean for certain non-boolean return values.
PUBLISHED: 2020-02-27
openssl_x509_check_email in lua-openssl 0.7.7-1 mishandles X.509 certificate validation because it uses lua_pushboolean for certain non-boolean return values.
PUBLISHED: 2020-02-27
openssl_x509_check_ip_asc in lua-openssl 0.7.7-1 mishandles X.509 certificate validation because it uses lua_pushboolean for certain non-boolean return values.
PUBLISHED: 2020-02-27
Type confusion in V8 in Google Chrome prior to 80.0.3987.116 allowed a remote attacker to potentially exploit heap corruption via a crafted HTML page.