Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Vulnerabilities / Threats

03:40 PM
Derek Weeks
Derek Weeks
Connect Directly
E-Mail vvv

Java Deserialization: Running Faster Than a Bear

Software components that were once good can sour instantly when new vulnerabilities are discovered within them. When that happens, the bears are coming, and you have to respond quickly.

Two men are walking through a forest. Suddenly, they see a bear off in the distance, running toward them. Adrenaline pumping, they start running away. But then one of them stops, takes some running shoes from his bag and starts putting them on.

“Frank, what are you doing?” says the other man. “Do you think you will run faster than the bear with those?”

“I don’t need to run faster than the bear,” Frank replies. “I just have to run faster than you.”

This scenario repeats itself every time a new security vulnerability is discovered in a widely used open source component. Imagine the bear as your adversary. Rushing to attack when easy prey is present. Your response time is critical.

Sneakers on. Go!

Racing to Answer Four Questions

Late last year, security researchers from FoxGlove Security announced a critical vulnerability in applications using Apache Commons Collection. The component is so widely used in Java development, countless organizations were impacted. The FoxGlove report revealed vulnerable instances of the component in open source applications like WebLogic, WebSphere, JBoss and Jenkins. Simultaneously, Java development and IT operations teams around the globe raced to discover four things:

  1. Did we use the Commons Collection component?
  2. If yes, which applications are using it?
  3. Are we vulnerable?
  4. If yes, how soon can we fix the flawed component?

As soon as the vulnerabilities were announced, the bears were charging to their doorstep. Response times were vital.

Lessons learned at PayPal

Earlier this year, PayPal shared its own experience with the deserialization vulnerability. Their engineering team blogged, “we began forking a few work-streams to assess the impact to our application infrastructure … As it turned out, this bug was found in one of our apps that was not on our core Java frameworks. Once we validated the bug, the product development and security teams quickly moved to patch the application in a couple of days.”

Knowing they were not alone in the development community racing to fix a vulnerable application, PayPal shared several lessons learned, like:

  • Where do you start?
  • Let real-world risk drive your priorities.
  • You need to make sure that the critical risk assets get the first attention.

When it comes to eliminating known vulnerabilities in critical applications, we can see from PayPal that speed, focus and community engagement all play a critical role in staying ahead of the bear.

Houston, we have a problem

When I first read about the vulnerability deserialization issue, I reached out to our Sonatype security team. They were already on the case to assess any impact to our own products (luckily, we were not impacted). My next step was reaching out to my friends at CloudBees (creators of Jenkins). Thankfully, their development team was already on top of it and released a newer version within just a few days of the alert.

But, the story does not end there. Researchers at Sonatype have identified deserialization issues in over 30,000 unique components stored in our company’s Central Repository.

The issue of vulnerabilities is far reaching. When we started looking at the download patterns of open source components from the Central Repository, we noticed the download consumption patterns of the known vulnerable versions of popular components were undistinguished from the good ones. People are using known vulnerable components for years and years after safer versions of those same components have been announced and released. In many cases, the open source projects themselves are not aware of vulnerabilities they are harboring.

 Why is this? It is not because people intend to build flawed software. It’s most likely, because they were not aware of the problem.  This is because:

  1. Security, quality and speed are critical to modern software (business) success.
  2. The world’s best software starts with the world’s best components.

Why components age like milk

We also realize that components sometimes age like milk. Components that were once good can sour instantly when new vulnerabilities are discovered within them. When that happens, the race is on. The bears are coming, and you have to respond.

[Read more about vulnerability management from Derek Weeks in 5 Steps to Improve Your Software Supply Chain Security.]

For some, the pace of their response resembles walking from the attacking bear. They have no idea what components have been consumed over the years across their software development practices. No records are kept. When a new vulnerability is announced (assuming they hear about it), the hunt reveals a manual, step-by-step analysis.

In one specific instance, a global banking entity told me that it took four people two months to detect if they were using vulnerable versions of Commons Collection across their application portfolio. (They had no running shoes.)

For others, it is a sprint at world-record pace. Organizations like Capital One, Intuit and FedEx have full visibility to the software supply chain that feeds components into their development organization. They know what components have been used and where. When new vulnerabilities are announced, they are automatically alerted to the use and location of that component.  

But responding at record pace is not an anomaly. You have to work at it. Global manufacturing giants like Toyota, Gilead Sciences, Apple and Lockheed have figured out how to run production lines at a ludicrous velocity while maintaining extremely low defect rates by continuously managing their supply chains. Knowing the bears won’t relent, it is time and it is possible for those managing software supply chains to do the same.

Gain insight into the latest threats and emerging best practices for managing them. Attend the Security Track at Interop Las Vegas, May 2-6. Register now!

Derek Weeks is passionate about applying proven supply chain management principles to improve efficiencies, reduce security risks, and sustain long-lasting competitive advantages. He is a distinguished international speaker and lectures regularly on modern software ... View Full Bio

Recommended Reading:

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
4/15/2016 | 12:41:17 PM
All you wanted to know about deserialization and more
There is another great post on SlideShare describing deserialization issues, including how to find them, what to look for, tricks, how to exploit them, and the history of deserialization vulnerabilities: http://www.slideshare.net/codewhitesec/java-deserialization-vulnerabilities-the-forgotten-bug-class

User Rank: Author
4/15/2016 | 12:38:10 PM
Identifying components subject to this vulnerability
This blog post by Brian Fox helps describe more about the issues with open source components that are subject to deserialization vulnerabilities.  Well worth reading: http://www.sonatype.org/nexus/2015/11/13/did-you-wake-up-to-an-alert-about-the-java-deserialization-vulnerability/
User Rank: Author
4/14/2016 | 8:57:31 PM
Bad strategy?
The fundamental problem here is deserializing untrusted data, not sour components. There are probably an infinite number of gadgets (like the Commons Collection library mentioned) that can be used, so the component strategy is busted. Better solutions use instrumentation like Contrast-rO0 and other free tools.
COVID-19: Latest Security News & Commentary
Dark Reading Staff 9/21/2020
Hacking Yourself: Marie Moe and Pacemaker Security
Gary McGraw Ph.D., Co-founder Berryville Institute of Machine Learning,  9/21/2020
Startup Aims to Map and Track All the IT and Security Things
Kelly Jackson Higgins, Executive Editor at Dark Reading,  9/22/2020
Register for Dark Reading Newsletters
White Papers
Current Issue
Special Report: Computing's New Normal
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
How IT Security Organizations are Attacking the Cybersecurity Problem
How IT Security Organizations are Attacking the Cybersecurity Problem
The COVID-19 pandemic turned the world -- and enterprise computing -- on end. Here's a look at how cybersecurity teams are retrenching their defense strategies, rebuilding their teams, and selecting new technologies to stop the oncoming rise of online attacks.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2020-09-23
An issue was discovered in Xen through 4.14.x. x86 PV guest kernels can experience denial of service via SYSENTER. The SYSENTER instruction leaves various state sanitization activities to software. One of Xen's sanitization paths injects a #GP fault, and incorrectly delivers it twice to the guest. T...
PUBLISHED: 2020-09-23
An issue was discovered in Xen through 4.14.x. There is mishandling of the constraint that once-valid event channels may not turn invalid. Logic in the handling of event channel operations in Xen assumes that an event channel, once valid, will not become invalid over the life time of a guest. Howeve...
PUBLISHED: 2020-09-23
An issue was discovered in Xen 4.14.x. There is a missing unlock in the XENMEM_acquire_resource error path. The RCU (Read, Copy, Update) mechanism is a synchronisation primitive. A buggy error path in the XENMEM_acquire_resource exits without releasing an RCU reference, which is conceptually similar...
PUBLISHED: 2020-09-23
An issue was discovered in Xen through 4.14.x. There are evtchn_reset() race conditions. Uses of EVTCHNOP_reset (potentially by a guest on itself) or XEN_DOMCTL_soft_reset (by itself covered by XSA-77) can lead to the violation of various internal assumptions. This may lead to out of bounds memory a...
PUBLISHED: 2020-09-23
An issue was discovered in Xen through 4.14.x. Out of bounds event channels are available to 32-bit x86 domains. The so called 2-level event channel model imposes different limits on the number of usable event channels for 32-bit x86 domains vs 64-bit or Arm (either bitness) ones. 32-bit x86 domains...