Another Hurdle For IPsec

With Secure Sockets Layer (SSL) continuously under siege and certificate authorities in the spotlight for data breaches, is it time for rival protocol IPsec to take center stage?

IPsec has some key features, like support for various algorithms and apps, including voice-over-IP, and it provides data packet-checking for authentication. But it has never really caught fire as a VPN protocol because of concerns about its client element and potential higher-maintenance management than SSL, which is widely available commercially. Even so, some organizations are taking a second look at IPsec and running it along with SSL for more security.

"We are seeing people returning to IPsec. They are seeing the benefits, and the pros and cons," says Rainer Enders, CTO at NCP Engineering Americas, which sells both SSL and IPsec products. "IPsec is a very robust set of protocols ... For really high security apps, IPsec makes a lot more sense. It offers more security through its design as a security protocol -- not [by] the type of encryption, but the implementation."

Enders says IPsec is actually more secure than SSL, in part because many SSL implementations still run older versions of its TLS protocol. The key exchange protocol in IPsec is better built and secured, he says. "IPsec is really more secure from a security technology perspective," he says.

But not so fast, security experts say. IPsec relies on the same web of trust that SSL does, and that model has collapsed, they say.

"I think IPsec is a perfect protocol to use, but I think we would be avoiding the real problem," says security expert Taher Elgamal, who led SSL development efforts at Netscape. "If we use IPsec, we still [need] to use some PKI. So if we do not address the real issue, which is how does a client trust a server certificate, then IPsec will have the exact same issues we have today. We can still have rogue CAs, and we would still have the MD5 problems."

Security expert Dan Kaminsky concurs. "As long as SSL is broken, IPsec isn't going to fix it," Kaminsky says. "After we fix SSL, [then] there are reasons to revisit IPsec."

The underlying problem, he says, goes beyond protocols and crypto. It's key management, which dogs both SSL and IPsec: "IPsec has the exact same key management issues as SSL, with two more problems. First, you have to get the protocol to work through NATs [network address translation] and firewalls. Second, the thing doing the encryption, the kernel, is far away from the thing that is communicating with the user, the app," Kaminsky says.

So the kernel doesn't know who to validate trust for, and the app struggles to show trust to the user, he says. "The advantage of IPsec is not that it's a better protocol. It's that without application integration, you still get crypto. To do that, though, you need amazingly robust key management so the kernel itself knows who to encrypt to," Kaminsky says.

Authenticity and confidentiality are key elements of any cryptographic system, notes Will Irace, director of research and services for Fidelis Security Systems. "What's broken is the web of trust to authenticate who you are talking to is who they say they are," Irace says. And IPsec and SSL rely on that same flawed web of trust, he says.

[ Many security experts believe the Internet's trust model is broken. Figuring out how to fix it will take time and collaboration. See The Future Of Web Authentication. ]

Fixing the certificate authority (CA) infrastructure is no small feat, however. Moxie Marlinspike has developed a Web authentication model called Convergence that uses an additional layer of oversight, known as notaries. Anyone could elect to act as a notary, but security companies and Web authentication experts would likely fill the bill there, creating histories of a website's SSL certificates and rating them for consistency. And users would decide whether to trust a website rather than rely on the browser to do it for them.

So will IPsec ever take off? "The advantage of IPsec is that it's built into IPv6," says Ben Greenberg, senior security threat researcher at Fidelis. "When enterprises go IPv6, they will already have it [built in] ... and can use it to secure internal communications."

And DNSSEC also could breathe life into IPsec. "Ultimately DNSSEC is going to be the answer for a lot of key management woes, and it will revitalize IPSec as much as it will fix SSL," Kaminsky says.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.