Risk
6/25/2013
11:47 PM
Connect Directly
RSS
E-Mail
50%
50%

Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find

A bug's Common Vulnerability Scoring System (CVSS) score doesn't necessarily correlate with whether the vulnerability is being used in attacks

Relying on the measure of vulnerability severity to prioritize what to patch and what to put off for another day is a waste of effort on software flaws that pose no danger, while missing others that are being exploited, according to two researchers that plan to reveal their findings at the Black Hat Security Briefings later this year.

Click here for more of Dark Reading's Black Hat articles.

The research analyzed the severity of vulnerabilities as ranked on the popular Common Vulnerability Scoring System (CVSS) with the existence of exploits and whether those exploits were being used in the wild to attack systems. The researchers -- Luca Allodi, a Ph.D. student in the field of security economics at the University of Trento in Italy, and Fabio Massacci, professor of information systems and security at UT -- found that the CVSS score did not correlate strongly with the attribute that arguably matters most to companies: whether the vulnerability is being used to attack systems.

"The CVSS could be high, but you may have a low risk of being exploited, while you can get a low CVSS score and still be attacked," Massacci says. "There is not much correlation between the CVSS only and the chance of being attacked."

The Common Vulnerability Scoring System uses a number of qualitative characteristics of a software flaw to determine the severity of the vulnerability on a 10-point scale. The CVSS combines a number of metrics -- such as the complexity of the attack and whether it impacts a system's confidentiality, integrity, and availability -- to come up with the score.

The researchers compared CVSS scores from the National Vulnerability Database (NVD) with information from the Exploit Database on the subset of vulnerabilities for which exploits had been created and with information from Symantec on the vulnerabilities that were actually being targeted by attackers in the wild.

Vulnerabilities targeted by exploits for sale in the underground should be patched immediately, as there was a strong correlation between the sale of an exploit for a particular vulnerability and the danger of that vulnerability being attacked. However, there was less correlation between the existence of a proof-of-concept attack in the Exploit Database and the risk of attack.

The complexity of the attack -- one of the metrics used to make up the CVSS score -- also appears to have a stronger correlation to the chance of a vulnerability being targeted by attackers than the overall score itself, the researchers say.

"If your vulnerability is in an exploit kit, then patch," Allodi says. "And if it is easy to exploit, then patch. But if it is difficult -- more complex -- to exploit, then it depends on the importance of the software with a vulnerability."

[With flaw tallies varying by up to 75 percent, vulnerability data needs to be taken with a grain of salt, yet reports based on the data fail to include caveats, Black Hat presenters say. See Don't Take Vulnerability Counts At Face Value.]

Many of the criticisms echo those of researcher Dan Guido, co-founder and CEO of security startup Trail of Bits, who argued that companies should focus on which vulnerabilities are being attacked and find simple defenses that defeat the attacks. In a 2011 study of vulnerabilities targeted by popular exploit kits, for example, Guido found two mitigations that could block 90 percent of the attacks.

Doing that sort of analysis with CVSS scores is impossible, he says. The scores do not provide enough information to the information security managers, especially because two aspects of an attack are only known by the potential victim.

"The vendor has no idea what the company's network looks like and what the attacker might be after," Guido says. "And without those two critical pieces of information, it's hard to make the CVSS score relevant."

While the research highlights that CVSS has weaknesses, the scoring system is a good standard by which companies can express a single severity for software flaws, says Wolfgang Kandek, chief technology officer of vulnerability management firm Qualys. While Qualys does not use CVSS as the measure of severity for software flaws in its own service, the framework is good for the majority of companies, he says.

"It depends on the level of sophistication," Kandek says. "Our customers are good with our severity, and I know that some very sophisticated customers can pull apart CVSS values to make their own decision, but for most companies the straight score is a good measure."

Yet for companies who are trying to find the best use of their resources, focusing on CVSS scores to prioritize patching will waste effort, argues UT's Massacci. In many ways, prioritizing patching based on CVSS scores is like triaging patients in an emergency room by just their temperature, he says.

"A single number is not a good idea -- CVSS is like measuring you for a temperature and then sending you to the operating room if it's high," he says. "What you should do, like in the medical domain, is first measure if you have a fever, and then you do a blood test, and then you do an X-ray."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message. Robert Lemos is a veteran technology journalist of more than 16 years and a former research engineer, writing articles that have appeared in Business Week, CIO Magazine, CNET News.com, Computing Japan, CSO Magazine, Dark Reading, eWEEK, InfoWorld, MIT's Technology Review, ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
CraigSchiller
50%
50%
CraigSchiller,
User Rank: Apprentice
6/28/2013 | 2:06:18 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
This research is a good example of fallacious reasoning. Of course exploit databases derived from exploit kits show stronger correlation to attacks in the wild, especially when compared to a vulnerability database that is trying to document all vulnerabilities. Similarly, if you had chosen as your criteria the degree of coverage of known vulnerabilities the exploit databases would perform poorly. It's a reasonable expectation that a list of exploits in exploit kits would perform better than a list of known exploits because the exploits in exploit kits have a distribution system and individual exploits may or may not. If you would have concluded that it would be useful to include a good exploitability score along with things like CVSS scores and asset criticality ratings then I might have agreed. It's like Black or White photography <smile>, "and" is much better.</smile>
amanion
50%
50%
amanion,
User Rank: Apprentice
6/26/2013 | 10:48:46 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
CVSS does support temporal and environmental vectors and scores, however
few sites publishing CVSS scores go beyond the base (as intended, the
user is supposed to provide temporal and environmental scores). It would be interesting to perform the same analysis on complete (base+temporal+environmental) CVSS scores. Also, I don't expect the Exploitability subscore to predict attacks, I expect it to measure the relative ease of attack. Widespread attacks usually depend on large target populations. In CVSS Target Distribution is an environmental metric.
anon7395245893
50%
50%
anon7395245893,
User Rank: Apprentice
6/26/2013 | 8:45:29 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
Scoring systems will continue to evolve and should continue to evolve just as we have seen metrics other domains like fitness & diet change over the years. The good news is that CVSS clearly states which facets are being measured and represented - use it if it is useful, don't use it if it is not. The research above is accurate and insightful but going in anyone can see that CVSS is missing the sufficient modeling of the threat environment and without it, the cost to the adversary cannot be faithfully represented. In my opinion, over the next 3 to 5 years, many models will need to be developed and if done right, they will be modular and interoperable
anon7395245893
50%
50%
anon7395245893,
User Rank: Apprentice
6/26/2013 | 8:31:09 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
Correction: CVSS is a 100 point scale from 0.0 to 10.0
cmdrfrog
50%
50%
cmdrfrog,
User Rank: Apprentice
6/26/2013 | 6:45:57 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
A good opening point on the value of professional Risk Assessment. CVSS is only a severity scoring and does not equal risk because it has not been considered in the context of threats. Compliance driven organizations tend to just go with CVSS and not invest in a full risk or threat assessment becuase they are under regulatory or statutory requirement to "get compliant" regardless, so under that constraint its the best prioritization scheme in a bad situation.
Register for Dark Reading Newsletters
White Papers
Cartoon
Current Issue
Dark Reading Must Reads - September 25, 2014
Dark Reading's new Must Reads is a compendium of our best recent coverage of identity and access management. Learn about access control in the age of HTML5, how to improve authentication, why Active Directory is dead, and more.
Flash Poll
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2014-6278
Published: 2014-09-30
GNU Bash through 4.3 bash43-026 does not properly parse function definitions in the values of environment variables, which allows remote attackers to execute arbitrary commands via a crafted environment, as demonstrated by vectors involving the ForceCommand feature in OpenSSH sshd, the mod_cgi and m...

CVE-2014-6805
Published: 2014-09-30
The weibo (aka magic.weibo) application 1.2 for Android does not verify X.509 certificates from SSL servers, which allows man-in-the-middle attackers to spoof servers and obtain sensitive information via a crafted certificate.

CVE-2014-6806
Published: 2014-09-30
The Thanodi - Setswana Translator (aka com.thanodi.thanodi) application 1.0.0 for Android does not verify X.509 certificates from SSL servers, which allows man-in-the-middle attackers to spoof servers and obtain sensitive information via a crafted certificate.

CVE-2014-6807
Published: 2014-09-30
The OLA School (aka com.conduit.app_00f9890a4f0145f2aae9d714e20b273a.app) application 1.2.7.132 for Android does not verify X.509 certificates from SSL servers, which allows man-in-the-middle attackers to spoof servers and obtain sensitive information via a crafted certificate.

CVE-2014-6808
Published: 2014-09-30
The Active 24 (aka com.zentity.app.active24) application 1.0.1 for Android does not verify X.509 certificates from SSL servers, which allows man-in-the-middle attackers to spoof servers and obtain sensitive information via a crafted certificate.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
In our next Dark Reading Radio broadcast, we’ll take a close look at some of the latest research and practices in application security.