Researchers: Bugs Can Turn Security Tools Against Their Users

European researchers have discovered flaws in antivirus and host-based IDS/IPS engines that could cause these server-based security products to "turn" on their users.

They have pinpointed hundreds of instances of two types of bugs in the parser engines of security scanners: AV evasion bugs that let attackers sneak malware past these tools and onto the corporate network; and code execution bugs, which can do anything from read and send email from a victim's email server to open a backdoor into the network.

Thierry Zoller, security engineer for German security firm n.runs, and Sergio Alvarez, head of research for n.runs, have been investigating these bugs for some time now, and Zoller says they have found an unprecedented number of bugs in these products. The pair of researchers plan to present their findings and demonstrate proof-of-concept at the HackLu2007 conference in Luxembourg in October.

Many of the bugs have now been patched by security vendors or are in the process of being patched, but Zoller says he and Alvarez are also currently reporting more of them to vendors. Even more worrisome is the pervasiveness of these holes -- security vendors often use the same parser engine code for multiple products, so the bug would not only be in their AV engines, but across their IDS/IPSes as well, Zoller says.

"These [security products] are inside the DMZ and can be used against you," he warns. "When an email gets inside a corporate network, [for example], it's mostly trusted."

AV evasion attacks can be used to sneak past the first AV scanner and get inside the corporate network. Code execution bugs affect any files that are parsed, including ZIP, RAR, ACE, and PDF. "The engines try to parse the format of the files, thus exposing flaws within the parsing engine," Zoller says, noting that when an AV engine tries to scan it, it will inadvertently execute exploit code that he and Alvarez wrote.

When AV evasion is mixed with code-execution bugs in these products, it becomes a deadly cocktail, basically using the AV or IDS/IPS parser engine as an unsuspecting accomplice in an attack. An attacker could send an email with a malicious ZIP file to the victim, and "as soon as the AV engine goes to look inside [the file], the [malicious] code will execute," unbeknownst to the victim.

An attacker could launch denial-of-service and remote-code execution exploits, Zoller adds.

But Randy Abrams, director of technical education for AV vendor ESET , says the threat from these bugs isn't likely to affect the average user and is "largely overblown." Abrams says for the malware to attack, it must actually execute, which is no small feat. "When it's inside of this malformed archive file, or even [a legitimate] one, it cannot execute."

It is possible in theory, however, for a security scanner to inadvertently execute the malicious code, he notes. "To execute, it has to get loaded into memory." And Abrams points out that this type of flaw isn't just in parser engines, but also in a file decompression engine as well.

Meanwhile, Zoller says it's actually fairly simple to fix the AV bypass/evasion flaws in AV, IDS/IPS, and Web gateway engines: "If the scanner can't open it, don't send the file" through.

So why haven't these bugs been widely addressed? "A lot of these bugs would have been found if [the vendors] had done source-code analysis," he says, although he notes that finding these bugs quickly requires investing a lot of time to develop the proper tools.

Have a comment on this story? Please click "Discuss" at the top of this page. If you'd like to contact Dark Reading's editors directly, send us a message.