Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Vulnerabilities / Threats

03:55 PM
Connect Directly

Did A Faulty Memory Feature Lead To Heartbleed?

Debate arises over an older memory allocation feature in OpenSSL, and the OpenBSD community starts to tear down and revise the crypto software for its own use.

As the dust begins to settle on the Heartbleed bug, developers in the open source community are digging deeper into what really went wrong in OpenSSL to cause the encryption software to be exposed to leaking information and digital keys. One theory: A memory management feature created for OpenSSL left the software open to leaking sensitive data.

Heartbleed and its resulting heartache for the Internet could have been avoided altogether if the OpenSSL software had employed more modern memory management techniques, according to OpenBSD founder Theo de Raadt. An older memory management method used in OpenSSL for optimizing performance called ""freelist" basically keeps the contents of memory "around forever" and exacerbates problems such as Heartbleed.

"Heartbleed would not have worked at all if the [memory] allocator" had fully sanitized the memory, as other software programs do, de Raadt maintains. And OpenSSL's patch fixes only the programming error that led to Heartbleed; there still could be hundreds of security bugs in OpenSSL that have yet to be discovered.

OpenSSL's Ben Laurie says he plans to disable the memory feature that de Raadt highlights, but he argues that it's still not clear whether the feature caused Heartbleed. OpenSSL, which patched the read-overrun bug in its implementation of the Transport Layer Security protocol's Heartbeat extension, has attributed the bug to an implementation flaw in the software.

"Whilst I agree with Theo that the feature should not be enabled by default, as far as I can tell it made no difference whatsoever in the Heartbleed case," Laurie said in an email interview last night about the memory allocation method. Laurie plans to disable or remove the feature altogether. It is used "quite rarely" and was not used in Heartbleed.

However, disabling this feature could trigger another bug, according to Laurie, who is investigating that. "It does appear that under some rather specific circumstances, there is a bug when it is disabled, which will also have to be fixed."

There are at least two memory allocators in OpenSSL, he says, and the "malloc" feature favored by de Raadt and others in the open source community is almost always used by OpenSSL for memory allocation.

Laurie says he's not convinced that the use of OpenSSL's own memory allocator actually led to Heartbleed, but he has not yet been able to conduct a full investigation. "I am fairly sure... that the use of this allocator, whilst perhaps inadvisable, did not reveal any interesting information in the case of Heartbleed," he said in the email interview. "As far as I can see, all that can be leaked by the allocator is data that is sent over the network -- that is, for the most part, encrypted data. That is, data that could be sniffed off the wire -- and hence is supposed not to be useful to an attacker."

Meanwhile, de Raadt's OpenBSD group is busily hacking away at fixes for OpenSSL, all in the name of improving its security in the wake of Heartbleed. OpenBSD plans to remove old legacy code and "risky code practices" without affecting applications that use OpenSSL, de Raadt says.

He maintains that OpenSSL's memory allocator is vulnerable to attack. "Some private information must remain in memory because it will be reused by the software. But other temporary objects which should be discarded remain. Furthermore, the objects are in very predictable places. A number of modern attack methods can use this."

The Heartbleed bug has multiple components, he says. "The primary problem is the ability to access the wrong memory. The secondary problem is that the wrong memory happened to contain security sensitive information. In a more perfect world, that sensitive information would be out of reach, hiding somewhere better."

Chris Eng, vice president of research at Veracode, says hindsight is 20/20. "The decision [about memory allocation in OpenSSL] was probably made for a good reason at the time," he says. "It was probably for performance, not security reasons."

"This may have implications if they find more bugs of this type... the fact that [the] use of this memory allocation might make the impact of the bug greater," he says. "But it's unfair to say you shouldn't have made that decision" years ago.

The key is to keep memory-leakage bugs in mind in software, he says. Take Akamai's failed patch for OpenSSL. "Akamai tried to... set up a separate memory area for secret" data. "The general idea of a separate area was good. They didn't get the implementation completely correct. That shows how difficult" this is.

[Heartbleed also affects internal servers, clients, and VPN networks. See Heartbleed's Intranet & VPN Connection].

Barrett Lyon, founder and CEO of Defense.net, who has been following the online discussions about the OpenSSL memory allocation issue by OpenBSD, says that, if indeed the memory management feature sacrifices security for performance, that's a problem.

"OpenSSL as a project has been very good," Lyon says. But "it was a pretty fatal design mistake" to employ a memory allocation method that risks security.

Kelly Jackson Higgins is the Executive Editor of Dark Reading. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Ninja
4/20/2014 | 6:55:13 AM
don't blame the tools
you don't blame the tools for sloppy work. heartbleed was caused by a crass violation of an old and well known programming standard: you don't let user input data control your program.  you control the input.  in this case the attacker submits a wrong length record -- stating he has a 64k payload and providing 1 byte -- and gets a slice of server memory as a prize. the stupid award goes to the programmer.

readers should note SQL injection is a similar problem --  letting a remote user send sql programming is a sure recipie for trouble -- and sql injection has been a favorite vector -- for years
User Rank: Apprentice
4/17/2014 | 8:00:44 PM
Re: Memory management not the core issue
Aha - but imagine - if software developers and vendors had really accepted the wisdom of Multics and then the segmented memory architecture in the Intel 286 chip and beyond, memory bounds and memory typing would be handled by HARDWARE!!!!  BUT - once again, laziness - uniform addressable memory wanted for convenience sake - security and software quality - well, who pays for that, who really cares out there in the marketplace anyway - the atitude of the 1980s and 1990s now "coming home to roost".

I wonder - has anyone really gone back and looked closely at the reasons for those important computer architecture decisions based on solid computer science from the early 1980s?  (After all, Multics dates form the 1960s!) Add to that the ideas behind "madatory access control (MAC)" whereby unknown sofware of unknown provenance could NOT inherit ALL the privileges of the user, for example, downloading it from the Internet connected computers. 

AND then imagine any reputable software system vendor incorporating open-source software sub-systems, of critical security significance, into a product WITHOUT full "destructive" testing - I can, and they do!  The only choice, a Richard A Clarke has often put it is government to at last realise that the national information infrastructure is of national security importance and, like most other industries, solid and determined regulatory environments are needed before the industry will wake up and take note.  

Image if GM or Ford or Toyota or Volvo or - all car manufacturers - never crash tested their cars!  Imagine if there were no Federal Aviation Administration regulations covering aircraft and services!  Imagine no regaultions over pharmaceuticals and food!  Well - that is where we are with IT.
Kelly Jackson Higgins
Kelly Jackson Higgins,
User Rank: Strategist
4/17/2014 | 11:29:27 AM
Re: Memory management not the core issue
Thanks for the feedback, @EdMoyle. OpenBSD is doing an overhaul of OpenSSL in what they consider a security cleanup of the code. I'm not sure where all of this will go--there is a major cultural gap between the two groups of developers, it appears. My hunch is that there will be more security problems found with the code and it won't just be memory allocation issues as you note. The good news is that there is more scrutiny over encryption software, so in the end, it should get better. #optimist
Ed Moyle
Ed Moyle,
User Rank: Apprentice
4/17/2014 | 11:22:57 AM
Memory management not the core issue
First, great reporting Kelly as always.  

Second, I'm sure I'll get flames for this, but it strikes me that the mechanics of memory allocation aren't the issue here.  The lack of bounds checking (i.e. the bogus length sent to memcpy) woud still have been there no matter how the buffers in question were allocated.  

Yes, it's possible the read would have caused the call to segfault instead under a different set of circumstances (maybe, depending on architecture and where/how the buffer was allocated.)  It's also possible that certain values (e.g. keys) might have been cleaned up sooner under a different set of circumstances (again maybe, possibly, could be.)  But the core issue seems to me to still be the bogus length supplied to memcpy which isn't fixed no matter how buffers get allocated.    
User Rank: Ninja
4/16/2014 | 4:41:19 PM
Performance over Vulnerability
I agree with Chris Eng that hindsight is 20/20 and I doubt very much that presented with the very simple choice of performance over, in this case vulnerability, would have been chosen by the OpenSSL project. I think the rest of the argument was short-sighted. Performance over security is plausible to some extent if not advisable. Are you going to have systems and methodology so locked down that availability becomes an issue, to the extent where it hinders business? It goes against all principles of the CIA triad. The statement that the OpenSSL chose this flawed memory allocation method knowing that dormant within was this extreme vulnerability is ridiculous. And if I am mistaken and this was known all along, then the people behind the OpenSSL project are maniacal and there is more pressing issues at hand.
Former CISA Director Chris Krebs Discusses Risk Management & Threat Intel
Kelly Sheridan, Staff Editor, Dark Reading,  2/23/2021
Security + Fraud Protection: Your One-Two Punch Against Cyberattacks
Joshua Goldfarb, Director of Product Management at F5,  2/23/2021
Cybercrime Groups More Prolific, Focus on Healthcare in 2020
Robert Lemos, Contributing Writer,  2/22/2021
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Write a Caption, Win an Amazon Gift Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
Building the SOC of the Future
Building the SOC of the Future
Digital transformation, cloud-focused attacks, and a worldwide pandemic. The past year has changed the way business works and the way security teams operate. There is no going back.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2021-02-27
SerComm AG Combo VD625 AGSOT_2.1.0 devices allow CRLF injection (for HTTP header injection) in the download function via the Content-Disposition header.
PUBLISHED: 2021-02-27
An issue was discovered in through SaltStack Salt before 3002.5. salt.modules.cmdmod can log credentials to the info or error log level.
PUBLISHED: 2021-02-27
In SaltStack Salt before 3002.5, eauth tokens can be used once after expiration. (They might be used to run command against the salt master or minions.)
PUBLISHED: 2021-02-27
An issue was discovered in SaltStack Salt before 3002.5. Sending crafted web requests to the Salt API can result in salt.utils.thin.gen_thin() command injection because of different handling of single versus double quotes. This is related to salt/utils/thin.py.
PUBLISHED: 2021-02-27
i-doit before 1.16.0 is affected by Stored Cross-Site Scripting (XSS) issues that could allow remote authenticated attackers to inject arbitrary web script or HTML via C__MONITORING__CONFIG__TITLE, SM2__C__MONITORING__CONFIG__TITLE, C__MONITORING__CONFIG__PATH, SM2__C__MONITORING__CONFIG__PATH, C__M...