02:46 PM
Connect Directly

'Cloudborne': Bare-Metal Cloud Servers Vulnerable to Attack

Firmware vulnerabilities provide direct access to server hardware, enabling attackers to install malware that can pass from customer to customer.

Firmware vulnerabilities in so-called bare-metal cloud servers let attackers install malware and backdoors, which remain active and grant access as servers are reassigned to new customers.

Researchers at Eclypsium are today releasing a report on firmware security issues they believe represent "a fundamental gap" in cloud infrastructure security. Their findings show baseboard management controllers (BMC) built into cloud servers could put customers at risk. While their study is based on IBM SoftLayer technology, they emphasize other providers may be exposed.

"This is a huge industry issue," says Yuriy Bulygin, Eclypsium founder and CEO, who formerly led the advanced threat research team at Intel Security.

With most infrastructure-as-a-service (IaaS) offerings, customers share resources on a physical server. Some organizations, however, have high performance requirements for certain applications or sensitive information they don't want on a machine shared with other firms.

In these cases, providers offer bare-metal cloud services. Customers buy full access to a dedicated physical server they can use however they want, without worrying it will interfere with others' data or buying and supporting additional hardware. When they're done using a bare-metal server, it's reclaimed by the provider, wiped, and repurposed for future customers.

Bare-metal cloud provides certain advantages; for example, performance improvement and the ability for businesses to install their own software stack. It also introduces new security risks as attackers have direct hardware access. This isn't the first time Eclypsium has published findings on firmware flaws: last June, they published findings on vulnerabilities in Supermicro systems.

What is Cloudborne?

Now, researchers say, bare-metal servers may not be fully erased before future use. The vulnerability, which they dubbed Cloudborne, is in the BMC – a privileged component used to manage the server. Using the Intelligence Platform Management Interface (IPMI), admins can send commands to the server or modify/reinstall an OS without physical access to the machine.

Vulnerabilities in the BMC could allow any customer to leave a backdoor on the server. "It's a fundamental gap in the cloud infrastructure, and it's exaggerated in bare-metal cloud infrastructure," says Bulygin. "The problem is that a customer – potentially a malicious customer – of a cloud service provider can have access to bare-metal instances," on which they can modify firmware and infect future users of the same machine with data theft, ransomware, and other threats.

Eclypsium conducted an experiment using IBM's SoftLayer cloud server platform, which offers bare-metal instances in most of its 35 global data centers. The team initially chose SoftLayer because of its simplified logistics and hardware access, as they explain in a blog post. But researchers also noticed Softlayer used Supermicro hardware, which based on earlier research they knew as vulnerable.

Researchers bought access to a bare-metal server, verified it was running the latest BMC firmware, and noted the product chassis and serial numbers for future identification. They made a minor change – a single bitflip inside a text comment they had prepared – and created an additional IPMI user, which they gave administrative access to the BMC channels.

They returned the server to IBM, which conducted the reclamation process, and were later able to reacquire the same server. While the new IPMI account was gone, their change to the BMC firmware remained. Researchers say this shows the BMC firmware wasn't re-flashed during reclamation, which they say makes it possible to implant malicious code into the firmware and steal data from future users.

Researchers also noticed the BMC logs were retained across provisioning, as was the root password. Since the logs were not deleted, future customers could view the actions of previous server owners and attackers could use the root password for future access.

"Most people aren't doing any verification," says John Loucaides, vice president of engineering at Eclypsium, of the reclamation process. "Most people ignore the whole firmware layer altogether." Given IBM is a large player and was affected by this issue, he anticipates other companies in the industry are affected as well.

BMC Bugs Have Been Found Before

This isn't the first time security experts found evidence of Supermicro BMC issues affecting bare-metal cloud servers. It has been a few years since researchers at Rapid7 found security issues in the Supermicro IPMI firmware, used in the BMC of Supermicro motherboards. At the time, HD Moore, then its chief research officer, analyzed the issue related to bare-metal cloud servers. Rapid7's results were similar to Eclypsium's, he says, but at the time the team felt publicly disclosing an insecure process from a specific provider wouldn't benefit the public.

"That equation has shifted a bit with consolidation among providers and the much broader adoption of cloud services," Moore says. Now, he says, Eclypsium's research is "an important problem" and "something both customers and providers should be aware of."

A compromised Supermicro BMC can be used to attack the host operating system in several says, he continues. The most straightforward is via the built-in kernel-based virtual machine (KVM) and remote media boot functionality. An attacker who installs a backdoor into a cloud server can use their access to assume control of the operating system and read the affected customer's hard drive data.

However, mitigating the problem is tough. An attacker with server access can bypass authentication when using IPMI over keyboard controller style (KCS), and create administrative accounts or flash a malicious image to the BMC, as Eclypsium did. Reflashing is handled by BMC firmware, so attackers have access even if the provider restores to a factory version.

IBM's Response

Eclypsium notified IBM of their findings; in response, IBM published a blog post indicating it has addressed the issue, and there is no evidence it has been exploited for malicious purposes.

IBM reports it is forcing all BMCs, including those reporting up-to-date firmware, to be reflashed with factory firmware before they are re-provisioned for future customers. It erases all logs in BMC firmware and regenerates all passwords for the firmware, officials report.

"IBM's approach to sanitizing servers before redeploying them is a good start, but not a complete resolution," says Moore. The firmware update process can be compromised with malicious firmware; an attacker that flashes a custom firmware can prevent providers from possibly detecting the backdoored image. He also notes that public tools exist to create custom firmware images for Supermicro components; attackers can use these to achieve access.

Researchers take issue with the fact that IBM categorized this issue as "low severity." Using the CVSS 3.0, they classified the problem as 9.3, or critical severity. "It's not a low-severity issue by any means," Loucaides says.

Related Content:



Join Dark Reading LIVE for two cybersecurity summits at Interop 2019. Learn from the industry's most knowledgeable IT security experts. Check out the Interop agenda here.

Kelly Sheridan is the Staff Editor at Dark Reading, where she focuses on cybersecurity news and analysis. She is a business technology journalist who previously reported for InformationWeek, where she covered Microsoft, and Insurance & Technology, where she covered financial ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
2/28/2019 | 3:40:20 AM
Rise in cybercrime
Tony Granims a cybersecurity expert with Critical Strategies Group has urged any organisations susceptible to cyber attacks to adequately and proactively deploy solutions that will mitigate such incidences. His predictions for an enormous increase in cyber attacks on U.S. Government agencies and companies in 2019 may just be valid.
User Rank: Strategist
2/26/2019 | 4:29:52 PM
Pre-Owned Cloud Servers
I think it's bizarre that IBM thinks that this type of vulnerability is low severity. A vulnerability that results in the reprovisioned hardware being pre-owned (pun fully intended) is critical. It seems to me that the point of Bare Metal Cloud is to avoid the performance and security issues that come from multiple clients being on the same hardware. It'd be hard to sell if people doubted that they were getting a clean system.
Crowdsourced vs. Traditional Pen Testing
Alex Haynes, Chief Information Security Officer, CDL,  3/19/2019
New Mirai Version Targets Business IoT Devices
Dark Reading Staff 3/19/2019
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: Looks like Jayne is having sushi for lunch again.
Current Issue
5 Emerging Cyber Threats to Watch for in 2019
Online attackers are constantly developing new, innovative ways to break into the enterprise. This Dark Reading Tech Digest gives an in-depth look at five emerging attack trends and exploits your security team should look out for, along with helpful recommendations on how you can prevent your organization from falling victim.
Flash Poll
The State of Cyber Security Incident Response
The State of Cyber Security Incident Response
Organizations are responding to new threats with new processes for detecting and mitigating them. Here's a look at how the discipline of incident response is evolving.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2019-03-18
An unquoted search path vulnerability was identified in Lenovo Dynamic Power Reduction Utility prior to version that could allow a malicious user with local access to execute code with administrative privileges.
PUBLISHED: 2019-03-18
Five9 Agent Desktop Plus 10.0.70 has Incorrect Access Control (issue 2 of 2).
PUBLISHED: 2019-03-17
Phamm (aka PHP LDAP Virtual Hosting Manager) 0.6.8 allows XSS via the login page (the /public/main.php action parameter).
PUBLISHED: 2019-03-15
CircuitWerkes Sicon-8, a hardware device used for managing electrical devices, ships with a web-based front-end controller and implements an authentication mechanism in JavaScript that is run in the context of a user's web browser.
PUBLISHED: 2019-03-15
An Integer overflow vulnerability exists in the batchTransfer function of a smart contract implementation for CryptoBotsBattle (CBTB), an Ethereum token. This vulnerability could be used by an attacker to create an arbitrary amount of tokens for any user.