Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Vulnerabilities / Threats

03:55 PM
Connect Directly

Did A Faulty Memory Feature Lead To Heartbleed?

Debate arises over an older memory allocation feature in OpenSSL, and the OpenBSD community starts to tear down and revise the crypto software for its own use.

As the dust begins to settle on the Heartbleed bug, developers in the open source community are digging deeper into what really went wrong in OpenSSL to cause the encryption software to be exposed to leaking information and digital keys. One theory: A memory management feature created for OpenSSL left the software open to leaking sensitive data.

Heartbleed and its resulting heartache for the Internet could have been avoided altogether if the OpenSSL software had employed more modern memory management techniques, according to OpenBSD founder Theo de Raadt. An older memory management method used in OpenSSL for optimizing performance called ""freelist" basically keeps the contents of memory "around forever" and exacerbates problems such as Heartbleed.

"Heartbleed would not have worked at all if the [memory] allocator" had fully sanitized the memory, as other software programs do, de Raadt maintains. And OpenSSL's patch fixes only the programming error that led to Heartbleed; there still could be hundreds of security bugs in OpenSSL that have yet to be discovered.

OpenSSL's Ben Laurie says he plans to disable the memory feature that de Raadt highlights, but he argues that it's still not clear whether the feature caused Heartbleed. OpenSSL, which patched the read-overrun bug in its implementation of the Transport Layer Security protocol's Heartbeat extension, has attributed the bug to an implementation flaw in the software.

"Whilst I agree with Theo that the feature should not be enabled by default, as far as I can tell it made no difference whatsoever in the Heartbleed case," Laurie said in an email interview last night about the memory allocation method. Laurie plans to disable or remove the feature altogether. It is used "quite rarely" and was not used in Heartbleed.

However, disabling this feature could trigger another bug, according to Laurie, who is investigating that. "It does appear that under some rather specific circumstances, there is a bug when it is disabled, which will also have to be fixed."

There are at least two memory allocators in OpenSSL, he says, and the "malloc" feature favored by de Raadt and others in the open source community is almost always used by OpenSSL for memory allocation.

Laurie says he's not convinced that the use of OpenSSL's own memory allocator actually led to Heartbleed, but he has not yet been able to conduct a full investigation. "I am fairly sure... that the use of this allocator, whilst perhaps inadvisable, did not reveal any interesting information in the case of Heartbleed," he said in the email interview. "As far as I can see, all that can be leaked by the allocator is data that is sent over the network -- that is, for the most part, encrypted data. That is, data that could be sniffed off the wire -- and hence is supposed not to be useful to an attacker."

Meanwhile, de Raadt's OpenBSD group is busily hacking away at fixes for OpenSSL, all in the name of improving its security in the wake of Heartbleed. OpenBSD plans to remove old legacy code and "risky code practices" without affecting applications that use OpenSSL, de Raadt says.

He maintains that OpenSSL's memory allocator is vulnerable to attack. "Some private information must remain in memory because it will be reused by the software. But other temporary objects which should be discarded remain. Furthermore, the objects are in very predictable places. A number of modern attack methods can use this."

The Heartbleed bug has multiple components, he says. "The primary problem is the ability to access the wrong memory. The secondary problem is that the wrong memory happened to contain security sensitive information. In a more perfect world, that sensitive information would be out of reach, hiding somewhere better."

Chris Eng, vice president of research at Veracode, says hindsight is 20/20. "The decision [about memory allocation in OpenSSL] was probably made for a good reason at the time," he says. "It was probably for performance, not security reasons."

"This may have implications if they find more bugs of this type... the fact that [the] use of this memory allocation might make the impact of the bug greater," he says. "But it's unfair to say you shouldn't have made that decision" years ago.

The key is to keep memory-leakage bugs in mind in software, he says. Take Akamai's failed patch for OpenSSL. "Akamai tried to... set up a separate memory area for secret" data. "The general idea of a separate area was good. They didn't get the implementation completely correct. That shows how difficult" this is.

[Heartbleed also affects internal servers, clients, and VPN networks. See Heartbleed's Intranet & VPN Connection].

Barrett Lyon, founder and CEO of Defense.net, who has been following the online discussions about the OpenSSL memory allocation issue by OpenBSD, says that, if indeed the memory management feature sacrifices security for performance, that's a problem.

"OpenSSL as a project has been very good," Lyon says. But "it was a pretty fatal design mistake" to employ a memory allocation method that risks security.

Kelly Jackson Higgins is Executive Editor at DarkReading.com. She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Ninja
4/20/2014 | 6:55:13 AM
don't blame the tools
you don't blame the tools for sloppy work. heartbleed was caused by a crass violation of an old and well known programming standard: you don't let user input data control your program.  you control the input.  in this case the attacker submits a wrong length record -- stating he has a 64k payload and providing 1 byte -- and gets a slice of server memory as a prize. the stupid award goes to the programmer.

readers should note SQL injection is a similar problem --  letting a remote user send sql programming is a sure recipie for trouble -- and sql injection has been a favorite vector -- for years
User Rank: Apprentice
4/17/2014 | 8:00:44 PM
Re: Memory management not the core issue
Aha - but imagine - if software developers and vendors had really accepted the wisdom of Multics and then the segmented memory architecture in the Intel 286 chip and beyond, memory bounds and memory typing would be handled by HARDWARE!!!!  BUT - once again, laziness - uniform addressable memory wanted for convenience sake - security and software quality - well, who pays for that, who really cares out there in the marketplace anyway - the atitude of the 1980s and 1990s now "coming home to roost".

I wonder - has anyone really gone back and looked closely at the reasons for those important computer architecture decisions based on solid computer science from the early 1980s?  (After all, Multics dates form the 1960s!) Add to that the ideas behind "madatory access control (MAC)" whereby unknown sofware of unknown provenance could NOT inherit ALL the privileges of the user, for example, downloading it from the Internet connected computers. 

AND then imagine any reputable software system vendor incorporating open-source software sub-systems, of critical security significance, into a product WITHOUT full "destructive" testing - I can, and they do!  The only choice, a Richard A Clarke has often put it is government to at last realise that the national information infrastructure is of national security importance and, like most other industries, solid and determined regulatory environments are needed before the industry will wake up and take note.  

Image if GM or Ford or Toyota or Volvo or - all car manufacturers - never crash tested their cars!  Imagine if there were no Federal Aviation Administration regulations covering aircraft and services!  Imagine no regaultions over pharmaceuticals and food!  Well - that is where we are with IT.
Kelly Jackson Higgins
Kelly Jackson Higgins,
User Rank: Strategist
4/17/2014 | 11:29:27 AM
Re: Memory management not the core issue
Thanks for the feedback, @EdMoyle. OpenBSD is doing an overhaul of OpenSSL in what they consider a security cleanup of the code. I'm not sure where all of this will go--there is a major cultural gap between the two groups of developers, it appears. My hunch is that there will be more security problems found with the code and it won't just be memory allocation issues as you note. The good news is that there is more scrutiny over encryption software, so in the end, it should get better. #optimist
Ed Moyle
Ed Moyle,
User Rank: Apprentice
4/17/2014 | 11:22:57 AM
Memory management not the core issue
First, great reporting Kelly as always.  

Second, I'm sure I'll get flames for this, but it strikes me that the mechanics of memory allocation aren't the issue here.  The lack of bounds checking (i.e. the bogus length sent to memcpy) woud still have been there no matter how the buffers in question were allocated.  

Yes, it's possible the read would have caused the call to segfault instead under a different set of circumstances (maybe, depending on architecture and where/how the buffer was allocated.)  It's also possible that certain values (e.g. keys) might have been cleaned up sooner under a different set of circumstances (again maybe, possibly, could be.)  But the core issue seems to me to still be the bogus length supplied to memcpy which isn't fixed no matter how buffers get allocated.    
User Rank: Ninja
4/16/2014 | 4:41:19 PM
Performance over Vulnerability
I agree with Chris Eng that hindsight is 20/20 and I doubt very much that presented with the very simple choice of performance over, in this case vulnerability, would have been chosen by the OpenSSL project. I think the rest of the argument was short-sighted. Performance over security is plausible to some extent if not advisable. Are you going to have systems and methodology so locked down that availability becomes an issue, to the extent where it hinders business? It goes against all principles of the CIA triad. The statement that the OpenSSL chose this flawed memory allocation method knowing that dormant within was this extreme vulnerability is ridiculous. And if I am mistaken and this was known all along, then the people behind the OpenSSL project are maniacal and there is more pressing issues at hand.
10 Ways to Keep a Rogue RasPi From Wrecking Your Network
Curtis Franklin Jr., Senior Editor at Dark Reading,  7/10/2019
The Security of Cloud Applications
Hillel Solow, CTO and Co-founder, Protego,  7/11/2019
Where Businesses Waste Endpoint Security Budgets
Kelly Sheridan, Staff Editor, Dark Reading,  7/15/2019
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: "Jim, stop pretending you're drowning in tickets."
Current Issue
Building and Managing an IT Security Operations Program
As cyber threats grow, many organizations are building security operations centers (SOCs) to improve their defenses. In this Tech Digest you will learn tips on how to get the most out of a SOC in your organization - and what to do if you can't afford to build one.
Flash Poll
The State of IT Operations and Cybersecurity Operations
The State of IT Operations and Cybersecurity Operations
Your enterprise's cyber risk may depend upon the relationship between the IT team and the security team. Heres some insight on what's working and what isn't in the data center.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2019-07-16
Information disclosure in PAN-OS 7.1.23 and earlier, PAN-OS 8.0.18 and earlier, PAN-OS 8.1.8-h4 and earlier, and PAN-OS 9.0.2 and earlier may allow for an authenticated user with read-only privileges to extract the API key of the device and/or the username/password from the XML API (in PAN-OS) and p...
PUBLISHED: 2019-07-16
Command injection in PAN-0S 9.0.2 and earlier may allow an authenticated attacker to gain access to a remote shell in PAN-OS, and potentially run with the escalated user?s permissions.
PUBLISHED: 2019-07-16
A Denial of Service vulnerability in the ImageNow Server service in Hyland Perceptive Content Server before 7.1.5 allows an attacker to crash the service via a TCP connection.
PUBLISHED: 2019-07-16
Quake3e < 5ed740d is affected by: Buffer Overflow. The impact is: Possible code execution and denial of service. The component is: Argument string creation.
PUBLISHED: 2019-07-16
UPX 3.95 is affected by: Integer Overflow. The impact is: attacker can cause a denial of service. The component is: src/p_lx_elf.cpp PackLinuxElf32::PackLinuxElf32help1() Line 262. The attack vector is: the victim must open a specially crafted ELF file.