Risk
6/25/2013
11:47 PM
Connect Directly
RSS
E-Mail
50%
50%

Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find

A bug's Common Vulnerability Scoring System (CVSS) score doesn't necessarily correlate with whether the vulnerability is being used in attacks

Relying on the measure of vulnerability severity to prioritize what to patch and what to put off for another day is a waste of effort on software flaws that pose no danger, while missing others that are being exploited, according to two researchers that plan to reveal their findings at the Black Hat Security Briefings later this year.

Click here for more of Dark Reading's Black Hat articles.

The research analyzed the severity of vulnerabilities as ranked on the popular Common Vulnerability Scoring System (CVSS) with the existence of exploits and whether those exploits were being used in the wild to attack systems. The researchers -- Luca Allodi, a Ph.D. student in the field of security economics at the University of Trento in Italy, and Fabio Massacci, professor of information systems and security at UT -- found that the CVSS score did not correlate strongly with the attribute that arguably matters most to companies: whether the vulnerability is being used to attack systems.

"The CVSS could be high, but you may have a low risk of being exploited, while you can get a low CVSS score and still be attacked," Massacci says. "There is not much correlation between the CVSS only and the chance of being attacked."

The Common Vulnerability Scoring System uses a number of qualitative characteristics of a software flaw to determine the severity of the vulnerability on a 10-point scale. The CVSS combines a number of metrics -- such as the complexity of the attack and whether it impacts a system's confidentiality, integrity, and availability -- to come up with the score.

The researchers compared CVSS scores from the National Vulnerability Database (NVD) with information from the Exploit Database on the subset of vulnerabilities for which exploits had been created and with information from Symantec on the vulnerabilities that were actually being targeted by attackers in the wild.

Vulnerabilities targeted by exploits for sale in the underground should be patched immediately, as there was a strong correlation between the sale of an exploit for a particular vulnerability and the danger of that vulnerability being attacked. However, there was less correlation between the existence of a proof-of-concept attack in the Exploit Database and the risk of attack.

The complexity of the attack -- one of the metrics used to make up the CVSS score -- also appears to have a stronger correlation to the chance of a vulnerability being targeted by attackers than the overall score itself, the researchers say.

"If your vulnerability is in an exploit kit, then patch," Allodi says. "And if it is easy to exploit, then patch. But if it is difficult -- more complex -- to exploit, then it depends on the importance of the software with a vulnerability."

[With flaw tallies varying by up to 75 percent, vulnerability data needs to be taken with a grain of salt, yet reports based on the data fail to include caveats, Black Hat presenters say. See Don't Take Vulnerability Counts At Face Value.]

Many of the criticisms echo those of researcher Dan Guido, co-founder and CEO of security startup Trail of Bits, who argued that companies should focus on which vulnerabilities are being attacked and find simple defenses that defeat the attacks. In a 2011 study of vulnerabilities targeted by popular exploit kits, for example, Guido found two mitigations that could block 90 percent of the attacks.

Doing that sort of analysis with CVSS scores is impossible, he says. The scores do not provide enough information to the information security managers, especially because two aspects of an attack are only known by the potential victim.

"The vendor has no idea what the company's network looks like and what the attacker might be after," Guido says. "And without those two critical pieces of information, it's hard to make the CVSS score relevant."

While the research highlights that CVSS has weaknesses, the scoring system is a good standard by which companies can express a single severity for software flaws, says Wolfgang Kandek, chief technology officer of vulnerability management firm Qualys. While Qualys does not use CVSS as the measure of severity for software flaws in its own service, the framework is good for the majority of companies, he says.

"It depends on the level of sophistication," Kandek says. "Our customers are good with our severity, and I know that some very sophisticated customers can pull apart CVSS values to make their own decision, but for most companies the straight score is a good measure."

Yet for companies who are trying to find the best use of their resources, focusing on CVSS scores to prioritize patching will waste effort, argues UT's Massacci. In many ways, prioritizing patching based on CVSS scores is like triaging patients in an emergency room by just their temperature, he says.

"A single number is not a good idea -- CVSS is like measuring you for a temperature and then sending you to the operating room if it's high," he says. "What you should do, like in the medical domain, is first measure if you have a fever, and then you do a blood test, and then you do an X-ray."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message. Robert Lemos is a veteran technology journalist of more than 16 years and a former research engineer, writing articles that have appeared in Business Week, CIO Magazine, CNET News.com, Computing Japan, CSO Magazine, Dark Reading, eWEEK, InfoWorld, MIT's Technology Review, ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
CraigSchiller
50%
50%
CraigSchiller,
User Rank: Apprentice
6/28/2013 | 2:06:18 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
This research is a good example of fallacious reasoning. Of course exploit databases derived from exploit kits show stronger correlation to attacks in the wild, especially when compared to a vulnerability database that is trying to document all vulnerabilities. Similarly, if you had chosen as your criteria the degree of coverage of known vulnerabilities the exploit databases would perform poorly. It's a reasonable expectation that a list of exploits in exploit kits would perform better than a list of known exploits because the exploits in exploit kits have a distribution system and individual exploits may or may not. If you would have concluded that it would be useful to include a good exploitability score along with things like CVSS scores and asset criticality ratings then I might have agreed. It's like Black or White photography <smile>, "and" is much better.</smile>
amanion
50%
50%
amanion,
User Rank: Apprentice
6/26/2013 | 10:48:46 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
CVSS does support temporal and environmental vectors and scores, however
few sites publishing CVSS scores go beyond the base (as intended, the
user is supposed to provide temporal and environmental scores). It would be interesting to perform the same analysis on complete (base+temporal+environmental) CVSS scores. Also, I don't expect the Exploitability subscore to predict attacks, I expect it to measure the relative ease of attack. Widespread attacks usually depend on large target populations. In CVSS Target Distribution is an environmental metric.
anon7395245893
50%
50%
anon7395245893,
User Rank: Apprentice
6/26/2013 | 8:45:29 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
Scoring systems will continue to evolve and should continue to evolve just as we have seen metrics other domains like fitness & diet change over the years. The good news is that CVSS clearly states which facets are being measured and represented - use it if it is useful, don't use it if it is not. The research above is accurate and insightful but going in anyone can see that CVSS is missing the sufficient modeling of the threat environment and without it, the cost to the adversary cannot be faithfully represented. In my opinion, over the next 3 to 5 years, many models will need to be developed and if done right, they will be modular and interoperable
anon7395245893
50%
50%
anon7395245893,
User Rank: Apprentice
6/26/2013 | 8:31:09 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
Correction: CVSS is a 100 point scale from 0.0 to 10.0
cmdrfrog
50%
50%
cmdrfrog,
User Rank: Apprentice
6/26/2013 | 6:45:57 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
A good opening point on the value of professional Risk Assessment. CVSS is only a severity scoring and does not equal risk because it has not been considered in the context of threats. Compliance driven organizations tend to just go with CVSS and not invest in a full risk or threat assessment becuase they are under regulatory or statutory requirement to "get compliant" regardless, so under that constraint its the best prioritization scheme in a bad situation.
Register for Dark Reading Newsletters
Partner Perspectives
What's This?
In a digital world inundated with advanced security threats, Intel Security seeks to transform how we live and work to keep our information secure. Through hardware and software development, Intel Security delivers robust solutions that integrate security into every layer of every digital device. In combining the security expertise of McAfee with the innovation, performance, and trust of Intel, this vision becomes a reality.

As we rely on technology to enhance our everyday and business life, we must too consider the security of the intellectual property and confidential data that is housed on these devices. As we increase the number of devices we use, we increase the number of gateways and opportunity for security threats. Intel Security takes the “security connected” approach to ensure that every device is secure, and that all security solutions are seamlessly integrated.
Featured Writers
White Papers
Cartoon
Current Issue
Dark Reading's October Tech Digest
Fast data analysis can stymie attacks and strengthen enterprise security. Does your team have the data smarts?
Flash Poll
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2013-7407
Published: 2014-10-22
Cross-site request forgery (CSRF) vulnerability in the MRBS module for Drupal allows remote attackers to hijack the authentication of unspecified victims via unknown vectors.

CVE-2014-3675
Published: 2014-10-22
Shim allows remote attackers to cause a denial of service (out-of-bounds read) via a crafted DHCPv6 packet.

CVE-2014-3676
Published: 2014-10-22
Heap-based buffer overflow in Shim allows remote attackers to execute arbitrary code via a crafted IPv6 address, related to the "tftp:// DHCPv6 boot option."

CVE-2014-3677
Published: 2014-10-22
Unspecified vulnerability in Shim might allow attackers to execute arbitrary code via a crafted MOK list, which triggers memory corruption.

CVE-2014-4448
Published: 2014-10-22
House Arrest in Apple iOS before 8.1 relies on the hardware UID for its encryption key, which makes it easier for physically proximate attackers to obtain sensitive information from a Documents directory by obtaining this UID.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
Follow Dark Reading editors into the field as they talk with noted experts from the security world.