Dark Reading

Finding rootkits planted in a machine can be like searching for a needle in a haystack, and it's about to get even harder: Researcher Joanna Rutkowska will demonstrate a proof-of-concept at Black Hat DC this week that lets an attacker taint the forensics investigator's best tool for detecting wily rootkits -- hardware-based memory access.

Rutkowksa will show how an attacker could prevent forensics investigators from getting a real image of the memory where the malware resides. "Even if they somehow find out that the system is compromised, they will be unable to get the real image of memory containing the malware, and consequently, they will be unable to analyze it," says Rutkowska, senior security researcher for COSEINC.

Researchers and forensics investigators today rely more on reading hardware-based memory to get an accurate picture of the OS to help detect malware, mainly because it's difficult to find rootkits in today's complex operating systems.

"All rootkit detectors on the market today can be seen as more or less random 'hacks' that check only some limited number of well-known places in the OS," Rutkowska says.

Plus if the system has already been compromised, you can't trust any programs executing on it -- not even the rootkit detector program itself, she says. So hardware-based memory access has emerged as the best way to get a real look at what's going on.

Rutkowska wouldn't reveal details on just how she "cheats" the so-called hardware Direct Memory Access (DMA) forensics method -- she says to tune into her presentation on Wednesday. But she says her POC will disprove the conventional wisdom that the DMA approach is secure.

"I believe that this is going to be the first public presentation of how malware can cheat hardware-based memory acquisition," she says.

There are only a few hardware memory-acquisition cards out there for forensics investigators, including ones from BBN and Komuku, as well as Grand Idea Studio's Tribble, but some investigators instead use a FireWire connection to get RAM images, Rutkowska says.

Rutkowska used FireWire and AMD64-based systems in her research but says it's likely the attack would also work on an Intel-based machine. "The attack itself is not based on any implementation flaw -- it uses only documented features of AMD-based systems, and I anticipate that similar features are present on Intel systems as well."

— Kelly Jackson Higgins, Senior Editor, Dark Reading