Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Perimeter

5/22/2019
02:30 PM
Ira Winkler
Ira Winkler
Commentary
Connect Directly
Twitter
LinkedIn
RSS
E-Mail vvv
50%
50%

Proving the Value of Security Awareness with Metrics that 'Deserve More'

Without metrics that matter to the business, awareness programs will continue to be the bastard child of security.

Awareness programs are a staple of any good security program, yet they rarely get the resources that they require to accomplish their goal. The reason seems to be, "You get the budget that you deserve, not the budget that you need."

Awareness programs generally run phishing campaigns and push out computer-based training (CBT). Phishing has some potential demonstration of value. CBT proves people view training. There might be some collateral advantage as well. However, the only tangible metrics most awareness programs provide is proof of compliance, and potentially showing that people become less susceptible to simulated phishing. 

To "deserve more" you need to determine metrics that show true business value. To that end, I've broken awareness-relevant metrics into four categories: Compliance, Engagement, Tangible Return on Investment (ROI), and Intangible Benefits.

Compliance Metrics
Compliance metrics involve ensuring that an organization satisfies third-party compliance needs, which, in general, means that you ensure employees complete awareness training. The minimal amount of budget required to provide training to your employees is justified at a minimum level.

Engagement Metrics
This is the most common category of metrics that I see used by most awareness practitioners. These metrics involve the general use and acceptance of the awareness components. Completion metrics, which overlap with compliance metrics, is one form. Sometimes, people are surveyed about the likability of the materials. Metrics may also include voluntary engagement with supplemental materials, such as if a user voluntarily takes extra awareness training, attends an event, etc.

There are two caveats with engagement metrics. They don't indicate effectiveness, and remember that the more engagement people have, the less the time they are performing their jobs.

Tangible Behavioral Improvement and ROI
This is the metric that shows employees are actually raising their security awareness. In this case, you need to measure actual security behaviors or the indications of those behaviors. It is irrelevant if people can recite the criteria for a good password. Do they have a secure password? It doesn't matter if people know they should secure sensitive information. Do they leave sensitive information vulnerable when they leave their desk?

Simulated phishing attacks do not demonstrate behavioral change. The issue is that while simulated phishing attacks could indicate how people might respond to real attacks, it is too easy to manipulate phishing simulations to return any level response that you want. On the other hand, fewer malware incidents on the network are a relevant metric.

Ideally, you want to attach a financial value to behavioral changes. For example, if malware incidents are decreased by 25%, and there is an average cost associated to malware incidents, you can calculate an estimated ROI. Likewise, if there are fewer incidents associated with compromised credentials, you can calculate an estimated ROI. To do so, you need to collect metrics that are related to potential awareness failings, and track those metrics.

Intangible Benefits
Clearly, you want to demonstrate a financial return; however, there are other potential points of value to an organization. For example, if an awareness program creates goodwill toward the security department, users may be more likely to report incidents and cooperate with other efforts. If users believe the awareness program generates value to them, there might be better employee retention. If employees are less susceptible to personal attacks, they might spend less time at work mitigating stolen identities.

"Deserve More"
While compliance requirements will mean that there will always be an awareness program, it is up to a security awareness practitioner to demonstrate that their efforts deserve more than the minimum funding required to achieve compliance. The way to do this is to focus on the periodic collection of metrics that demonstrate a ROI well beyond standard engagement metrics. This is how you deserve more.

Related Content:

Ira Winkler is president of Secure Mentem and author of Advanced Persistent Security. View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Sodinokibi Ransomware: Where Attackers' Money Goes
Kelly Sheridan, Staff Editor, Dark Reading,  10/15/2019
Data Privacy Protections for the Most Vulnerable -- Children
Dimitri Sirota, Founder & CEO of BigID,  10/17/2019
7 SMB Security Tips That Will Keep Your Company Safe
Steve Zurier, Contributing Writer,  10/11/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: The old using of sock puppets for Shoulder Surfing technique. 
Current Issue
7 Threats & Disruptive Forces Changing the Face of Cybersecurity
This Dark Reading Tech Digest gives an in-depth look at the biggest emerging threats and disruptive forces that are changing the face of cybersecurity today.
Flash Poll
2019 Online Malware and Threats
2019 Online Malware and Threats
As cyberattacks become more frequent and more sophisticated, enterprise security teams are under unprecedented pressure to respond. Is your organization ready?
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-17513
PUBLISHED: 2019-10-18
An issue was discovered in Ratpack before 1.7.5. Due to a misuse of the Netty library class DefaultHttpHeaders, there is no validation that headers lack HTTP control characters. Thus, if untrusted data is used to construct HTTP headers with Ratpack, HTTP Response Splitting can occur.
CVE-2019-8216
PUBLISHED: 2019-10-17
Adobe Acrobat and Reader versions , 2019.012.20040 and earlier, 2017.011.30148 and earlier, 2017.011.30148 and earlier, 2015.006.30503 and earlier, and 2015.006.30503 and earlier have an out-of-bounds read vulnerability. Successful exploitation could lead to information disclosure .
CVE-2019-8217
PUBLISHED: 2019-10-17
Adobe Acrobat and Reader versions , 2019.012.20040 and earlier, 2017.011.30148 and earlier, 2017.011.30148 and earlier, 2015.006.30503 and earlier, and 2015.006.30503 and earlier have an use after free vulnerability. Successful exploitation could lead to arbitrary code execution .
CVE-2019-8218
PUBLISHED: 2019-10-17
Adobe Acrobat and Reader versions , 2019.012.20040 and earlier, 2017.011.30148 and earlier, 2017.011.30148 and earlier, 2015.006.30503 and earlier, and 2015.006.30503 and earlier have an out-of-bounds read vulnerability. Successful exploitation could lead to information disclosure .
CVE-2019-8219
PUBLISHED: 2019-10-17
Adobe Acrobat and Reader versions , 2019.012.20040 and earlier, 2017.011.30148 and earlier, 2017.011.30148 and earlier, 2015.006.30503 and earlier, and 2015.006.30503 and earlier have an use after free vulnerability. Successful exploitation could lead to arbitrary code execution .