Only 3% of Open Source Software Bugs Are Actually Attackable, Researchers Say

With vulnerability-management workloads ballooning in the era of heightened software supply chain security risks, a study out today suggests that only about 3% of today’s flaws are actually reachable by attackers. The data implies that if application security (appsec) pros and developers work to focus on fixing and mitigating what's truly attackable, they could drastically reduce the strain on their teams.

The new study by ShiftLeft, the 2022 AppSec Progress Report, suggests that appsec and development teams can more effectively sift through vulnerabilities by focusing on the "attackable" ones. Data from the report shows that developers saw a 97% reduction in false-positive library upgrade tickets once they considered attackability when examining packages in use with critically rated vulnerabilities.

If true, this would be a welcome relief to many. Vulnerability management was already hard enough as is, but the added complication of third-party flaws — especially the scale of impact of these vulnerabilities rippling across numerous pieces of software — creates an even more daunting workload that can only be managed through effective prioritization. Security and developers can only get to so many vulnerabilities in so many applications within any given time period. They need to make sure the ones they fix or mitigate with compensating controls are the ones that count.

What Does 'Attackability' Mean for Security Vulnerabilities?

Making the determination of what's attackable comes by looking beyond the presence of open source dependencies with known vulnerabilities and examining how they're actually being used, says Manish Gupta, CEO of ShiftLeft.

"There are many tools out there that can easily find and report on these vulnerabilities. However, there is a lot of noise in these findings," Gupta says. "For example, they do not consider how the dependency is used in the application; they don't even consider whether the app even uses the dependency."

The idea of analyzing for attackability also involves assessing additional factors like whether the package that contains the CVE is loaded by the application, whether it is in use by the application, whether the package is in an attacker-controlled path, and whether it is reachable via data flows. In essence, it means taking a simplified threat modeling approach to open source vulnerabilities, with the goal of drastically cutting down on the fire drills.

CISOs have already become all too familiar with these drills. When a new high-profile supply chain vulnerability like Log4Shell or Spring4Shell hits the industry back channels, then blows up into the media headlines, their teams are called to pull long days and nights figuring out where these flaws impact their application portfolios, and even longer hours in applying fixes and mitigations to minimize risk exposures.

To that point: The report noted that 96% of vulnerable Log4J dependencies were not attackable.

Software Dependencies to the Fore

Reliance on open source dependencies — both first-hand and through third-party dependencies — is growing in modern development stacks.

"For any major application that uses a large number of dependencies, it is common to have new CVEs multiple times a month," says Gupta. "Multiply that by all the apps in the organization, one can imagine it is no easy task to keep up with all the upgrades."

While updating a package might be easy, he says the associated development work surrounding such a change can often be significant. Often a single library upgrade can precipitate a battery of new tests not just for security but for functionality and quality, and it potentially could require refactoring of code.

"Any organization serious about product quality won't ship a product without thorough testing," he explains. "Also library upgrades are not always fail-safe; there's no guarantee that new versions of open source libraries will be fully backward compatible. So, sometimes teams are also required to change how their app works before they upgrade a library."

Is Attackability Determination Feasible?

According to Mark Curphey, founder of OWASP and a longtime appsec advocate, seeking a prioritization model like this is nothing new. However, he says that picking the dimensions of analysis to determine what's risky or attackable can be more complicated in today's application environment than what ShiftLeft proposes.

"It's true and fair to say that the vast majority of vulnerable methods in open-source libraries can't be reached and therefore are not exploitable, but we are now in a world where open-source libraries are like elaborate shop fronts offering all sorts of goodies for developers to consume," Curphey tells Dark Reading. "As an industry, we recently learned from the Log4J saga that when an issue is something like a JNDI interface that few people actually used, there were still paths to exploitation, and so we all had to face the issue."

He's currently on a listening tour for his latest appsec startup, Crash Override, to ask what CSOs' biggest appsec problems are today, and almost all of them say prioritization is their No. 1 problem. He believes it may well be appsec's next big problem to solve.

"So the fundamental premise of the report makes total sense, but what we have also learned from interviews is that answering that question is very hard and more complex than 'am I using a particular bit of code,'" Curphey says. "It's things like business criticality, which includes how many users the system has, how much money flows through it, its public profile, what type of data it is processing, where it is physically located, and therefore what laws are applicable. It's technical things like how it is connected to other systems and what controls, monitoring, and alerts are in place, and the list goes on."

The other problematic thing about using "attackability" or "reachability" as a prioritization filter is understanding the underlying technical data that's being used to determine what is reachable by an attacker, says Stephen Magill, vice president of product innovation for Sonatype.

"Attackability and 'reachability' can be helpful ways of prioritizing vulnerabilities when the underlying vulnerability data is good. What is not helpful is relying on attackability or reachability as a means of compensating for bad vulnerability data," Magill says. "All too often, this is what we see the industry doing: Using inaccurate methods of identifying dependencies, coupling this with noisy data on which versions of which dependencies are vulnerable, and then using reachability-based prioritization to filter the long list of vulnerabilities that results down to something manageable."

In other words, an attackability prioritization is as only as good as the vulnerability data feeding into it, so it is caveat emptor for security teams to truly look into the hood as to how they source their vulnerability data.

"Does it only come from public feeds, or is it the result of in-depth research by a dedicated security team? Also investigate how dependencies are tracked," Magill says. "Are they just the declared dependencies in manifest files or does the tool support analysis of binary artifacts, archives, JARs, etc. These questions will help you determine the quality of the findings being prioritized. As they say 'garbage in, garbage out.'"

Finally, Magill says security leaders need to remember that many threats exist to software supply chains beyond the normal churn of bugs that are found incidentally within open source projects.

"The biggest threats to our software supply chains are malicious, purposeful attacks on open source," he says. "That is a much larger problem that we should be focused on, and completely unrelated to attackability."