Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Edge Articles

02:20 PM
Sara Peters
Sara Peters
Edge Features
Connect Directly

Realistic Patch Management Tips, Post-SolarWinds

Patch management and testing are different, exactly the same, and completely out of hand. Here are tips from the experts on how to wrangle patches in a time of malicious software updates.

(image by Barbara Helgason, via Adobe Stock)
(image by Barbara Helgason, via Adobe Stock)

"If you didn't already know that patching introduces risk, well...now you know," says Brad Causey, CEO of security consulting and penetration testing firm Zero Day Consulting.

Causey is referring of course to the recent attack on SolarWinds that shook the industry. Software updates for SolarWinds' Orion network management software were used to distribute the Sunburst/Solarigate backdoor Trojan to some 18,000 organizations worldwide. (Note: SolarWinds is, itself, also a provider of third-party patch management services. However, those services do not appear to have been affected by the recent attacks.) 

"We're introducing risk by trying to reduce risk," Causey says.

Related Content:

How SolarWinds Busted Up Our Assumptions About Code Signing

Special Report: How IT Security Organizations Are Attacking the Cybersecurity Problem

What Should I Do About Vulnerabilities Without Fixes?

This isn't a new thing though, he says, and testing patches before deployment is standard best practice. Yet, patch testing is generally done to avoid operational snafus, not security threats. It's meant to spot a code library change that prevents three other applications from running; not to spot a backdoor Trojan.  

With the Sunburst/Solarigate attacks fresh in mind, though, is it time to revamp patch testing procedures? How can enterprise infosec teams tackle patch management securely? Here's advice from security experts on what to do now.

Be Realistic.

Causey and others say that a supply chain attack on the scale and sophistication of SolarWinds is harrowing, but it doesn't mean that enterprises need to completely reinvent patch management. Rather, IT teams just need to do some of the best practices they should have been doing all along. After all, the National Institution of Standards and Technology (NIST) lays out highly detailed guidelines on patch management in SP 800-40. 

The trouble is, SP 800-40 was last updated eight years ago, and by NIST's own reckoning, the number of vulnerabilities per year has tripled since then

"We patch all the time. We're always patching," says John Pironti, president of cybersecurity and risk consultancy IP Architects LLC.

Security hygiene, including patching, is an essential part of defense, says Pironti. Nevertheless, he says, "We're fooling ourselves if we think we can defend ourselves against a nation-state attack [like the SolarWinds incident] while continuing to release code at the speed we do."

Curtis Franklin, senior analyst of enterprise security management at Omdia, says companies must have patch management technology to help automate the process now, "because it's gotten really beyond human-scale at this point." 

Keep Trusting Patches. But... 

Despite the recent high-profile example of a malicious software update, Pironti says companies should not shy away from deploying updates.

"I think we would be doing ourselves a disservice if we started distrusting patches," he says. "I'd rather trust my vendors than question them when there's an exploit in the wild."

He does, however, say it's fair to ask for better security hygiene in the software development lifecycle. 

...Ask Software to Be More Trustworthy

"We've been trained as a society to accept flawed code," says Pironti. While regulations mandate that some industries' products meet certain safety and quality standards, enterprise software is largely unregulated. Pironti thinks that at some point this may change. "You can't let [software companies] be the barometers of what's acceptable and unacceptable risk," he says.

In the meantime, he suggests companies ask software vendors and service providers one question before any purchase: What are you doing to ensure the integrity of your third-party code?

Create a Testing Environment That's a Reasonable Representation of Reality.

In an ideal world, your testing environment would be a perfect mirror image of your production environment. It would represent every device, running every version of every operating system version, and every application, in every complex configuration that might be running in your environment at the time.

"And you'd have to invest in all that equipment nobody's using, and pay someone to maintain it, right?" points out Causey.

More realistic and affordable, though, he says, is to create a testing environment that accurately represents the systems that are the most critical -- those that are used by the widest number of users, or most critical to daily operations, or that touch the most sensitive data.  

Omdia's Franklin says that many companies succeed in creating a test environment that represents the lion's share of their endpoints. "The trouble is with their edge cases," he says.

Those edge cases might not be a problem. Until they are.

Franklin lays out an example:

It might be the system that prints out the bills of lading for the trucks leaving your manufacturing facility. And it runs a dot-matrix printer that has been cranking along since 1997. And Charlie at the freight yard knows how to hit the buttons on his Windows 98 computer to make it print all these bills of lading to keep things flowing out. And you've decided that it's simply impossible to retrain Charlie down in shipping. So you're not going to. 

And it was fine when Charlie was getting hand-written notes and typing them in. But sometime a few years ago your SAP rep said 'ya know we can put a connector that goes from SAP to Charlie's desktop.' So now Charlie's Windows 98 desktop has a link back -- probably through the Internet -- to your SAP instance.

Now, all of a sudden, Charlie's Windows 98 machine is a vulnerability. ... My guess is you don't have a Windows 98 machine in your [testing] lab. So even if [Microsoft] released an out-of-band patch for Windows 98, you couldn't test it.

You're going to have some cases like that. And they get far more numerous and bizarre in healthcare.

Franklin says most companies widely use sandboxing. "But if they're honest with themselves, they know that they can't sandbox everything. If they're doing it right, though they know what they can't sandbox." 

Use the Right Tool for the Job.

Identifying those fringe cases requires help. There are many types of technologies that will allow enterprises to locate and organize those IT assets and a wide variety of tools that help make patch management smoother. For example:

{Continued on Next Page}

Sara Peters is Senior Editor at Dark Reading and formerly the editor-in-chief of Enterprise Efficiency. Prior that she was senior editor for the Computer Security Institute, writing and speaking about virtualization, identity management, cybersecurity law, and a myriad ... View Full Bio
1 of 2

Recommended Reading:

Comment  | 
Print  | 
More Insights
Threaded  |  Newest First  |  Oldest First
User Rank: Apprentice
3/8/2021 | 1:42:31 AM
So what do we do?
In my opinion, this article is more about patch management when what we should learn from the SolarWinds incident. We already have patch management and patch systems in stages and following a priority list, thank you very much.

How is a company supposed to protect itself against malicious software updates and patches from vendors we all trusted in the first place? Every month, there is a lot of new software coming into the enterprise. Who will tell me if the binary code is clean or not? I am still looking for an answer myself.

Best regards

Jan Klingel

User Rank: Apprentice
4/6/2021 | 4:38:08 PM
Virtual patching

I am surprised the article is not mentioning the virtual patching solutions. It permits to avoid the risks of running a new software without careful (and therefore long) testing when the security issues are high or critical.

Anyway, we need to consider the concept of always running the latest version. What's the point?
Do we really need to get the latest features? Are we affected by the bugs fixed? Is the version running still supported? Is any serious security vulnerability?
The benefits of an update must be carefully measured. It could appear better to not run the last version, as long there is no security issues or bugs affecting your business.
Cartoon Caption Winner: Magic May
Flash Poll