Security researcher Federico Maggi had been collecting data – some of it sensitive in nature – from hundreds of thousands of Message Queuing Telemetry Transport (MQTT) servers he found sitting wide open on the public Internet via Shodan. "I would probe them and listen for 10 seconds or so, and just collect data from them," he says.
He found data on sensors and other devices sitting in manufacturing and automotive networks, for instance, as well as typical consumer Internet of Things (IoT) gadgets.
The majority of data, Maggi says, came from consumer devices and sensors or was data he couldn’t identify. "There was a good amount of data from factories, and I was able to find data coming from pretty expensive industrial machines, including a robot," he says.
At first, Maggi, a senior threat researcher at Trend Micro, wasn't regularly studying the millions of MQTT and other message data he was gathering via scripts he wrote that automatically extracted data from the searches; he was just occasionally checking on his findings. "I was not really paying attention to it," he says.
But all that changed earlier this year. After a chance conversation during a lunch meeting with fellow researcher Davide Quarta, Maggi decided to double down on his research after Quarta, a post-doctoral researcher for EURECOM, mentioned he was studying the MQTT protocol and had "lots of data," Maggi recalls.
So the two teamed up and dug deeper into how both MQTT and its companion, the Constrained Application Protocol, or CoAP, could be abused by attackers. They found that the widely used device-to-device communications protocols contained inherent security weaknesses, especially in the way they are implemented in IoT devices – exposing flaws that could allow attackers to execute denial-of-service (DoS) attacks on devices or gain remote control of industrial IoT or consumer IoT devices for cyber espionage or worse.
MQTT and CoAP basically serve as the backbone of IoT and industrial IoT communications. As Maggi and Quarta discovered, the protocols often are deployed in devices insecurely, leaking sensitive information such as device details, user credentials, and network configuration information. The pair of researchers will present details and data from their findings in December at Black Hat Europe in London.
Attackers could exploit these weaknesses in the protocols and their implementations to target victim organizations, as well as use an IoT device as a stepping stone further into the victim network.
One core problem is that the MQTT protocol standard itself has been evolving over the past few years; once devices get updated with new versions of the spec, that can leave older implementations at risk. It also has, at times, introduced changes that break compatibility among IoT and industrial IoT devices, for example. "Yes, this technology is very efficient and effective for what it needs to do, but it's also a changing technology," Maggi says. And that can open up security holes.
In some cases, updates that fix one issue in the code have inadvertently caused or introduced another security issue. Maggi says it took about five years for fixes to the MQTT protocol to result in a clean, bug-free specification. Meanwhile, developers aren't always able or willing to update each product with the new spec, leaving devices vulnerable. Even more concerning, the latest release of MQTT (version 5) has changes that "break" parts of the previous protocol version. "So guess how many people will adopt it now?" Maggi quips.
Quarta traced the DoS attack risk to a logic flaw in the standard itself. "Some parts of the standard were changing over time ... and there were errors in a subsequent version," he says. In a nutshell, a few sentences in the standard were left open to interpretation, he says, making it easy for developers to inadvertently implement it incorrectly. There was a code error in the protocol, as well as some missing features.
The MQTT Technical Committee in February put out a notice about the issue when Quarta reported it, with information on how to detect the issue in devices and fix it.
Quarta found that an attacker could wage a DoS attack on a device by taking advantage of MQTT's "retain" feature, which ensures all IoT clients get a message that's sent. The issue has to do with how multibyte strings – UTF-8 characters – and other strings are parsed by the protocol software. The standard includes the feature that mandates servers retain messages and that clients request acknowledgement of the delivery of each message they send.
A DoS attack could occur like this: A compromised IoT device would send an invalid UTF-8 message that enables the retain function and sabotages another parameter, for instance. The MQTT server would, in turn, comply and retain the message. A legitimate IoT device would then receive that message, but thanks to the malicious settings by the attacker, that device (the client) would close the connection. The server, expecting a response, bombards it over and over with messages, leading to a DoS of the IoT device.
"The best way to protect against a logic flaw DoS attack such as this is by keeping both client and broker software aligned," Maggi explains. "It doesn't matter whether they follow the specs or not, but they have to do the same thing."
At the network level, an IDS/IPS system could monitor and filter packets for malformed content, he says.
At Black Hat Europe, Maggi and Quarta plan to show recordings of the exploit, as well as provide a demo that shows how such a DoS would affect an industrial network. They'll show a simulated Mars Rover transporting a load while being directed by MQTT messages.
"You can imagine automation controls sending messages to the Rover to go back, load up, and load down, with each command an MQTT message," Maggi explains. "We pull off one of the exploits while messages are being delivered."
The server then is unable to deliver messages on time, so it could cause the Rover to physically drop the load, for instance. "The impact of a tiny software bug" could have safety implications, he says.
Malicious Software Updates
MQTT can also be used for software and firmware updates in IoT and industrial IoT devices. "So draw your own conclusions" about risks, Maggi says. "We found ... firmware passing through MQTT as part of an update."
Even so, the researchers say they didn't discover any attacks in the wild using MQTT. But there are some possible scenarios of other types of abuse by attackers. "I've looked into if there's any malware that uses MQTT as a command-and-control technology. That would be a convenient way to make bots communicate; it's easy to blend in because of MQTT's popularity now," Quarta says. "But I haven't found any malware using it."
While the researchers were able to easily identify MQTT servers via Shodan, what was alarming was how that led them to sensitive data. The ability to actually connect directly to the discovered devices was "more scary" than finding a device via Shodan, Maggi says. "And with zero effort, you get inundated with messages sent to the client."
But Maggi and Quarta aren't the first to point out MQTT security risks, a fact they note up front. Lucas Lundgren, a security consultant at IOActive, for instance, found in his own research tens of thousands of MQTT servers leaking sensitive information during his study between 2016 and 2017.
Lundgren was able to send commands to an exposed MQTT server sitting inside a major technology vendor's network. "It allowed me to send raw commands into a server," he told Dark Reading in a 2017 interview.
Martin Horn, a security researcher at Avast, this year discovered via his own Shodan search 49,000 MQTT servers exposed on the Net – 32,000 of them had no password protection. Horn called out the flawed implementations of MQTT as well as a lack of security for the server, which can become a single point of security failure for all of the connected IoT devices.
- 7 Steps to Start Searching with Shodan
- The Better Way: Threat Analysis & IIoT Security
- 100,000-Plus Home Routers Hijacked in Campaign to Steal Banking Credentials
- Industrial Safety Systems in the Bullseye
Black Hat Europe returns to London Dec 3-6 2018 with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions and service providers in the Business Hall. Click for information on the conference and to register.