A little-known class of vulnerabilities can be used to hack the trust between browsers and their plug-ins, as well as other applications, according to new research on tap at Black Hat USA next week in Las Vegas.

Researchers Ryan Smith, a vulnerability researcher at iDefense; Mark Dowd, X-Force research engineer for IBM ISS; and David Dewey, a researcher for IBM ISS; will demonstrate attacks that exploit these bugs and let an attacker hack the communication between different application components. So-called "type-confusion" vulnerabilities have typically been associated with Java applications, but the researchers found they also affect C and C++ code.

Smith says the flaws let an attacker manipulate the communications channels between a browser and its components. "These bugs result from different components in an application having to communicate in a language-agnostic way," he says.

One particularly dangerous attack the researchers will demonstrate during their presentation involves bypassing the security features in Internet Explorer. The attack exploits a so-far undisclosed vulnerability in IE's security architecture in order to circumvent one of IE's major security features -- basically rendering the browser and user exposed.

Smith, who wouldn't reveal any additional details of the hack, says the vulnerabilities he and his colleagues found in IE span all versions of the browser, including IE8, and are "systemic to the overall security architecture of IE."

"Deep down, it's an architectural problem, and to remediate it will take a lot of planning," Smith says.

In this type of attack, the bad guy would be able to remotely install malware on the victim's computer, and more. "The attack surface can be opened up wider when that bug is used in conjunction with other bugs, letting the attacker communicate with more areas of the victim's machine -- areas that the IE security architecture normally puts off-limits to attackers," Smith says. "We're going to show it has holes in it."

The researchers also will reveal other type-confusion bugs that affect other browsers, including Firefox and Safari.

"Most of these vulnerabilities [found in the past] have been one-off vulnerabilities where a researcher has said, 'This is a very interesting type of [bug].' We're trying to formulate a cohesive way to look at these vulnerabilities so authors can find them more frequently," Smith says.

An attacker would initially have to either infect a popular Web page with an exploit, or dupe a victim into visiting his malicious site. Then he could run his malicious code natively inside the browser and ultimately take control over the victim's browser and machine.

Why haven't these potentially deadly vulnerabilities been studied and abused before now? "It has been easier to find other bugs. But now that bugs are drying up [in some applications], you'll see these more frequently," Smith says.

The researchers have no plans to release their tools for these attacks, Smith says.

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.