09:44 AM
Connect Directly

Hacking The Router Patching Conundrum

Now that recent research proves that exploiting Cisco routers isn't as hard as once thought, the pressure is on for enterprises that don't regularly patch to change their ways -- without upsetting the network infrastructure

The dirty little secret about patching routers is that many enterprises don't bother for fear of the fallout any changes to their Cisco router software could have on the rest of the infrastructure. But the recent discovery of a way to easily hack the devices has turned upside down conventional wisdom that patching routers is more of a risk than an actual attack on these devices.

Researcher Felix "FX" Lindner's research earlier this year demonstrated that multiple versions of routers can be attacked -- specifically, Cisco's PowerPC routers -- shooting down the assumption that hacking routers requires separate exploits for each type of router. Enterprises traditionally have been content to avoid patching their Cisco routers because the chances of a major breach was less likely than the possibility of an unintentional outage from a router update.

"The underlying problem is that you cannot patch IOS -- you always need to update the entire image. And with this comes all kinds of compatibility issues with your configuration, hardware, and setup," says Lindner, a researcher with Recurity Labs.

Lindner demonstrated with his research that all an attacker needs is basic knowledge about the targeted device, rather than specifics of the IOS configuration. His exploit method applies to stack-buffer overflows, and he was able to execute memory writes and to disable CPU caches on Cisco routers running on the PowerPC CPU.

Router updates aren't typically a top priority, and few organizations have policies and procedures in place for patching their routers. "They're not thinking about all of the routers out there," says Dan Kaminsky, director of penetration testing for IOActive. "They're resource-constrained and overloaded: I get that. They need a good reason if they are going to deploy their limited resources to monitor yet another problem. And [Lindner] has provided a damn good reason.

"The idea that the variability of router platforms would defend you from an attacker is false. All versions have something in common [in this research], and this is not just in theory, but FX demonstrated it and used it to exploit all [PowerPC IOS] versions."

Even so, Lindner's groundbreaking research has yet to change the status quo. "For all enterprises and carriers that I know of and spoke to, nobody updates IOS when a new security vulnerability is found. The risk associated with upgrading IOS is, in fact, higher than the risk of getting 'pwned,'" Lindner says. "Most sensible network operations groups will try to filter the new issue on the border, if they still have something like a border, [and] the most advanced groups will have core-dump writing configured on their routers to catch exploitation attempts."

Cisco Systems says some of its customers patch, while others do not. "Our customers are all over the place -- some do patch diligently, and some have very strict policies," says Russ Smoak, director of technical services for Cisco. "We have the other extreme: those that have aged infrastructures as well."

Smoak says the threats to routers, in general, haven't changed much, though the reasons behind them have. Distributed denial-of-service (DDoS) attacks are typically more economically motivated now than they were before, he says. "Attacks are more subtle and more targeted," he says. "It's the same stuff, but different motivations behind it."

Cisco views its product as a target, Smoak adds. "We try to take a very paranoid view, and we do a lot of things to harden our products," he says.

Security experts agree that patching routers isn't easy, but some steps can help prevent taking down the network in the process. Recurity's Lindner says patching is, indeed, likely to cause something to break. "Many of your configurations don't work anymore, your line cards are not supported with the new release, or something else breaks," he says.

But if you keep your IOS minor version up to date, you can use the patched-IOS image, he says. Cisco could also provide some additional patch information: "What Cisco could do is test transitions from one image to the other on many platforms," Lindner says. It could publish "safe-to-replace lists," which would note that if you replace version 12.2(13) with 12.3(14), for example, certain features are not affected, he says.

For large organizations, patching Cisco routers should really be an extension of their redundancy policies, says Fred Avolio, senior professional staff with The Johns Hopkins University Applied Physics Laboratory. "You probably have redundant [routers] in place already. If you don't, you're going to have to. And you should do half of [the patches] at first and see if [they] take, and then do the rest later," Avolio says.

Assessing your risk is also important, Avolio says. "If this [vulnerability] only works on a particular configuration or version of IOS, that's part of your risk equation. But because it's more likely now that malware will be written for routers on various versions of [IOS]...the security policy you have in place needs to be adjusted because of this change in threat."

Not surprisingly, Cisco is one of those organizations that patches its own routers regularly. So how does Cisco handle the delicate router-patching process? Craig Huegen, director of IT network and data center services architecture for Cisco, says upgrading the network infrastructure involves several steps.

"First, there is a review stage, where an assessment is made of the software update. What are the known caveats, if any, to the new software? Are there any known bugs that the administrator should watch for? Have any system resource requirements changed, such as the memory or system image storage? Have any features changed?" he says.

Then the new software is downloaded and the image validated to ensure it's complete and unmodified. "In many cases, to ensure quality, images are loaded [and] activated on lab devices to ensure functionality and certify them for use," Huegen says. "Third, the new software is staged onto the production devices and prepared for use. Finally, during a scheduled maintenance window, the new software is activated [in a rolling fashion]."

Cisco gives critical security updates high priority because they could immediately affect the security or operation of the network, he says, and regular, noncritical ones are done within a regular deployment schedule. Any new feature updates to the routers are usually project-driven, he says.

"It would be a rare case that I would suggest you not need to update. For a very specific, very static application with very minimal access, it may be acceptable once long-term stability is proven through burn-in time," Huegen says.

IOActive's Kaminsky recommends that enterprises run Cisco's router-monitoring software, ensuring that branch offices and new acquisition or merger offices are also monitored. "And worry about hardware that's too old to be patched. You might have to buy new hardware," he says.

More food for thought: Consider automatic patching on the infrastructure. "We know that if we want widespread deployment of a patch, we need as little pain as possible," Kaminsky says. "You should front-load the pain into the testing process -- that's where the work is. That's the hardest part."

While there's no perfect way to patch without problems, it has become increasingly important to make router updates part of your patching routine. "It's not that you should panic. But it means that when a [router] vendor tells you to patch, then, yes, patch," Kaminsky says.

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message Kelly Jackson Higgins is Executive Editor at She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Register for Dark Reading Newsletters
Partner Perspectives
What's This?
In a digital world inundated with advanced security threats, Intel Security seeks to transform how we live and work to keep our information secure. Through hardware and software development, Intel Security delivers robust solutions that integrate security into every layer of every digital device. In combining the security expertise of McAfee with the innovation, performance, and trust of Intel, this vision becomes a reality.

As we rely on technology to enhance our everyday and business life, we must too consider the security of the intellectual property and confidential data that is housed on these devices. As we increase the number of devices we use, we increase the number of gateways and opportunity for security threats. Intel Security takes the “security connected” approach to ensure that every device is secure, and that all security solutions are seamlessly integrated.
Featured Writers
White Papers
Current Issue
Dark Reading's October Tech Digest
Fast data analysis can stymie attacks and strengthen enterprise security. Does your team have the data smarts?
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
Published: 2014-10-24
adsetgroups in Centrify Server Suite 2008 through 2014.1 and Centrify DirectControl 3.x through 4.2.0 on Linux and UNIX allows local users to read arbitrary files with root privileges by leveraging improperly protected setuid functionality.

Published: 2014-10-24
The Remote Controls feature on Samsung mobile devices does not validate the source of lock-code data received over a network, which makes it easier for remote attackers to cause a denial of service (screen locking with an arbitrary code) by triggering unexpected Find My Mobile network traffic.

Published: 2014-10-23
Untrusted search path vulnerability in Hamster Free ZIP Archiver allows local users to execute arbitrary code and conduct DLL hijacking attacks via a Trojan horse dwmapi.dll that is located in the current working directory.

Published: 2014-10-23
Open redirect vulnerability in the header function in adclick.php in OpenX 2.8.10 and earlier allows remote attackers to redirect users to arbitrary web sites and conduct phishing attacks via a URL in the (1) dest parameter to adclick.php or (2) _maxdest parameter to ck.php.

Published: 2014-10-23
Cross-site request forgery (CSRF) vulnerability in Shenzhen Tenda Technology Tenda A32 Router with firmware 5.07.53_CN allows remote attackers to hijack the authentication of administrators for requests that reboot the device via a request to goform/SysToolReboot.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
Follow Dark Reading editors into the field as they talk with noted experts from the security world.