Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

News & Commentary

End of Bibblio RCM includes -->

Cloudflare Bleeds Bad News – & Good

A slip of the fingers brings privacy concerns for millions and lessons in how to handle a security incident.

"For the want of a nail the shoe was lost, For the want of a shoe the horse was lost, For the want of a horse the rider was lost, For the want of a rider the battle was lost, For the want of a battle the kingdom was lost, And all for the want of a horseshoe-nail."
— Benjamin Franklin

Substitute the ">" symbol for an old-fashioned nail, and you have a synopsis of the latest privacy catastrophe, dubbed "Cloudbleed" by security researchers. How bad is Cloudbleed? The advice to users echoed around the Internet gives a hint of the breach's scope:

Change your passwords. All of them.

What happened to cause the problem and how it has been addressed each carry instructions for enterprise security and IT professionals. What happened is simple, and the sort of coding error that can be frustratingly difficult to catch absent the most rigorous testing protocols. In a single line of code, the intended ">=" operator was typed "==". As a result, certain operations were able to fill a buffer and keep right on writing, planting data across the system.

The flaw
According to Tavis Ormandy, a researcher with Google Project Zero, that planted data can include cookies, private messages, IP addresses, and passwords. The worst part? In a blog post about the event (made public only after the vulnerability had been repaired and remediated), Ormandy wrote, "PII [personally identifiable information] was actively being downloaded by crawlers and users during normal usage, they just didn't understand what they were seeing."

Cloudflare is a service provider with a client list that includes Uber, OKCupid, Fitbit, and various financial institutions. In addition to basic network performance services that include DNS and load balancing, Cloudflare provides security capabilities such as SSL and DDoS mitigation. Because Cloudflare provides the service to businesses rather than consumers, it's nearly impossible for an individual to know whether their personal information was exposed. And it's absolutely impossible to know whether any exposed information was actually grabbed by criminals -- until, of course, some form of identity theft takes place. Hence, the advice to change all your passwords. Now.

The problem was compounded by the way that Cloudflare operates. Cloudflare's data can be crawled and cached by services such as Google. This means that personal data was not just exposed, it was exposed in a persistent manner. That's a lot of bad news wrapped up in a single wayward character. Fortunately, there is some good news in the story to balance a bit of the bad.

Rapid response
When Ormandy discovered the issue, he immediately contacted Cloudflare using Twitter (in, as one commenter noted, the sort of message that should be absolutely horrifying to anyone on a corporate security team). In very short order, Cloudflare security personnel had reproduced the issue and shut down the services making use of the affected code. According to the company, they then put teams in the US and UK on 12-hour shifts, handing code off between the teams to keep efforts to find and remediate the problem going 24 hours a day.

Cloudflare reports that all services that included the problem were disabled globally within roughly three hours of notification. Figuring out precisely where in the code the problem lay, and how it could be repaired, took a bit longer. The company's blog post on the issue (referenced above) goes into great detail about where the problem was, what its effects were, and why it had not previously appeared. The level of detail in the disclosure is part of the good news surrounding the vulnerability; in speaking with several security researchers and IT professionals, all said that this disclosure is a model of what they would like to see in future industry events. All felt that this level of disclosure is likely to assuage customer fears and reassure regulators that all proper steps have been taken.

The final step in remediation was Cloudflare working with search engines to clear caches of released information. As the company wrote in its disclosure blog post, "With the help of Google, Yahoo, Bing and others, we found 770 unique URIs that had been cached and which contained leaked memory. Those 770 unique URIs covered 161 unique domains. The leaked memory has been purged with the help of the search engines."

Six days after the initial contact from Ormandy, Cloudflare disclosed the incident to the public. The vulnerability had existed for approximately four months and the number of customer records revealed to criminals is unknown. What is known is that this will likely be used as an example of the white-hat research system working as intended; independent discovery of an issue, verification by the affected company, rapid repair and remediation, with speedy, transparent disclosure to the public.

— Curtis Franklin, Security Editor, Light Reading

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
I Smell a RAT! New Cybersecurity Threats for the Crypto Industry
David Trepp, Partner, IT Assurance with accounting and advisory firm BPM LLP,  7/9/2021
Attacks on Kaseya Servers Led to Ransomware in Less Than 2 Hours
Robert Lemos, Contributing Writer,  7/7/2021
It's in the Game (but It Shouldn't Be)
Tal Memran, Cybersecurity Expert, CYE,  7/9/2021
Register for Dark Reading Newsletters
White Papers
Current Issue
Improving Enterprise Cybersecurity With XDR
Enterprises are looking at eXtended Detection and Response technologies to improve their abilities to detect, and respond to, threats. While endpoint detection and response is not new to enterprise security, organizations have to improve network visibility, expand data collection and expand threat hunting capabilites if they want their XDR deployments to succeed. This issue of Tech Insights also includes: a market overview for XDR from Omdia, questions to ask before deploying XDR, and an XDR primer.
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2022-06-25
In the RSS extension for MediaWiki through 1.38.1, when the $wgRSSAllowLinkTag config variable was set to true, and a new RSS feed was created with certain XSS payloads within its description tags and added to the $wgRSSUrlWhitelist config variable, stored XSS could occur via MediaWiki's template sy...
PUBLISHED: 2022-06-25
Raytion 7.2.0 allows reflected Cross-site Scripting (XSS).
PUBLISHED: 2022-06-25
Zulip is an open-source team collaboration tool. Versions 2.1.0 through and including 5.2 are vulnerable to a logic error. A stream configured as private with protected history, where new subscribers should not be allowed to see messages sent before they were subscribed, when edited causes the serve...
PUBLISHED: 2022-06-25
Argo CD is a declarative continuous deployment for Kubernetes. Argo CD versions v0.7.0 and later are vulnerable to an uncontrolled memory consumption bug, allowing an authorized malicious user to crash the repo-server service, resulting in a Denial of Service. The attacker must be an authenticated A...
PUBLISHED: 2022-06-25
ESP-IDF is the official development framework for Espressif SoCs. In Espressif’s Bluetooth Mesh SDK (`ESP-BLE-MESH`), a memory corruption vulnerability can be triggered during provisioning, because there is no check for the `SegN` field of the Transaction Start PDU. This can resul...