Three months after the Apache Foundation disclosed the infamous Lo4j vulnerability [CVE-2021-44228] and issued a fix for it, more than 4 in 10 downloads of the logging tool from the Maven Central Java package repository continue to be known vulnerable versions.

A dashboard that Maven Central administrator Sonatype launched soon after news of the so-called Log4Shell flaw first surfaced shows that 41% of Log4j packages downloaded between Feb. 4 and March 10, 2022, are versions prior to Log4j 2.15.0. That's the patched version of the logging tool that the Apache Foundation released on Dec. 10, 2021, when the Log4Shell flaw was first disclosed. After that, the Foundation released two other updates to address two subsequent — and relatively less severe — vulnerabilities that were uncovered in the logging tool just days after the Log4Shell disclosure.

Sonatype's dashboard showed there have been more than 31.4 million downloads of Log4j in total since Dec. 10, 2021. It's unclear how many of those are vulnerable versions, but going by the latest download statistics the number could well be near, or in excess, of 10 million.

Log4j and Layers
So why are organizations and developers downloading known vulnerable versions of Log4j packages, and why are those versions available for downloads in the first place — especially given the prevalence of the flaw and the relative ease with which it can be exploited?

Travis Smith, vice president of malware threat research at Qualys, points to a couple of reasons for the continued downloads. "The main culprit is likely automated build systems, which are configured to download a specific version build of their dependencies," he says. Lesser-maintained projects especially may automatically download a specific version to avoid conflicts with updated software. "If the maintainer of that software hasn’t been paying attention to the news surrounding Log4j, their application is left open to the risk of exploitation."

The fact that Log4j is an integral part of many Java applications — and is often buried several layers deep with them — has made it extremely difficult for many organizations to detect and remediate the issue. "The inference could be made that many of the current downloads of the vulnerable version are for projects that cannot justify the time taken to upgrade," Smith says.

Another explanation for the high percentage of vulnerable Log4j downloads, according to Smith, could be that researchers are doing it to test defenses, and adversaries are doing it to test their exploits.

Ilkka Turunen, field CTO at Sonatype, says another issue is the lack of fundamental software supply chain management at many organizations. Without adequate software composition analysis tooling and a software bill of materials, organizations can have a hard time figuring out what components have gone out to which releases.

"The hard part we observed with many organizations about the Log4j fire drill was a complete lack of awareness and visibility of which third party components were used in their production," Turunen says. Often, organizations don’t understand what is in their applications and are therefore not able to quickly target affected applications and make upgrades. "In short, many companies are still trying to build their software inventory before they start to react."

Recent data that Qualys pulled from its cloud security platform in fact suggests that some 30% of the Log4j instances on the Internet still remain vulnerable for exploit, according to the company. "Apache Foundation’s Log4J is used in a myriad of places and is bundled with hundreds of packages," says Mike Parkin, senior technical engineer at Vulcan Cyber. "That broad use was part of what made it such a major vulnerability in the first place."

The fact that not every developer implements Log4J into their packages the same way is another reason why patching it has become a major issue, he says.

Why Are Vulnerable Log4J Versions Still Available?
Meanwhile, the reason why vulnerable Log4j packages still remain available for download via Maven Central is because of software dependencies, Smith and others said. Many pieces of software are dependent on the vulnerable versions of Log4j, and removing them suddenly could cause systems to break. An analysis by Google researchers a week after the Log4Shell flaw was disclosed showed that some 17,000 Java packages on Maven Central contained the vulnerability. At that time, Google discovered fixed versions were available for 25% of the impacted packages. Since then, it is likely that many more have been patched.

Yet removing vulnerable versions from the Maven repository is risky. "When we first became stewards of Maven Central, one of the key commandments we put in place was 'thou shalt not break a build,'" says Turunen. "While it can be tempting to assert our own judgment, which might be removing the vulnerabilities, what’s actually best for the community is for everyone to make their own judgments."

Brian Fox, co-founder and CTO at Sonatype, says Maven Central is more like a natural gas pipeline than a gas station where a user gets to choose their octane level each time. "If we took these downloads down from the repo, every build worldwide that comes looking for it would suddenly fail," he says.

He points to a 2016 incident where a programmer removed a package that he had developed called left-pad from the npm JavaScript registry over a dispute. Though the package consisted of just 11 lines of code, its removal broke thousands of dependent projects and caused substantial disruption across the Internet. Doing the same thing with Log4j would cause disruption where it is likely not justified and could cause more harm than good, Fox says.