Salvaging Digital Certificates

Download Dark Reading's November special issue on securing Web data.

One year ago, Gmail users in Iran woke up to a chilling prospect: Their sensitive and supposedly secure communications on Google's email program may have been tapped by unknown parties. A phony digital certificate in Google's name was used to impersonate the site and let the culprits mount a "man in the middle" attack to intercept and decrypt email sent to Google's servers before passing the messages along to the intended recipients.

In the search for the source of the phony certificate, all eyes turned to DigiNotar, a certificate authority in Beverwijk, Netherlands. DigiNotar admitted that it had been the victim of a cyberattack a month earlier whereby the attackers generated hundreds of bogus certificates in the names of some of the Internet's most trusted brands, including Google, Yahoo, Skype and the anonymity network Tor. Use of the fraudulent certificates was concentrated among Iranian users, leading to speculation that the attack was linked to that country's intelligence services, which have been cracking down on political dissidents.

The DigiNotar attack was the worst security compromise at a certificate authority to date. But it was hardly the only such attack. It came just months after an attack on a business affiliate of Comodo, a New Jersey CA. In that incident, the attackers generated phony certificates also in the names of prominent online brands.

These successful attacks were earth shaking because digital certificates and the encryption keys they represent are the bedrock of Internet communications. They secure everything from VPN connections to protocols such as TLS (Transport Layer Security) and SSL (Secure Sockets Layer) that protect billions of Web sessions and online transactions daily. At the heart of this system is a global public key infrastructure network of some 300 CAs entrusted with issuing certificates to individuals and organizations.

Certificate authorities are gatekeepers. They verify the identities of individuals and organizations before issuing digital certificates that contain the public and private encryption keys used to secure online exchange of information.

At least that's how it's supposed to work. As the DigiNotar attack revealed, CAs aren't Fort Knox-style identity vaults but rather are businesses subject to their own security mishaps. In the last year, CAs have come under attack from sophisticated and possibly nation-backed hacking crews, exposing security lapses and poor internal controls.

The current system leaves security-conscious businesses in a pinch. More than ever, they need secure and reliable identity services to back up their growing online presences, but the system in place is more vulnerable than ever.

What's to be done? Many experts say certificate technology still has a future and that security-conscious organizations can protect themselves by understanding the gaps in the system and taking common-sense steps to avoid them.

A Disjointed Ecosystem

DigiNotar's compromise and the resulting collapse of that certificate authority exposed deep cracks in the global PKI system. Chief among them is the sheer number of organizations with a license to issue certificates and vouch for online identities.

An offshoot of the development of SSL by Internet pioneer Netscape, CAs were envisioned as highly secure and single-purpose facilities, akin to passport offices, says Ryan Hurst, a cryptography expert and CTO at GlobalSign, a CA.

As use of the Web and e-commerce exploded, however, the job of issuing certificates turned into a business -- and a profitable one. Dozens of companies entered the market, led by firms such as VeriSign (now owned by Symantec). National governments, large corporations and commercial certificate vendors set up their own root CAs, letting them issue certificates that are accepted worldwide. Microsoft alone recognizes more than 300 CAs linked to more than 80 organizations, says Hurst, who managed that company's root server and other cryptographic initiatives for close to a decade.

Most Internet users have only a passing familiarity with this global system of online trust. Most major online service providers offer their users the option of communicating over encrypted connections; many even require it. The major browsers offer Web users clear visual cues to let them know when a particular session is protected using SSL or some other encryption method. These cues include the well-known "padlock" icon in the URL bar. Browsers keep track of reputable CAs and warn users when they visit a site that offers a suspicious or expired certificate. If you need a certificate for a website or other purposes, companies such as VeriSign and Go Daddy make buying one easy and affordable.

Convenience aside, all of those private and commercial CAs make for a rickety identity infrastructure, says Craig Spiezle, executive director and founder of the Online Trust Alliance, a nonprofit that promotes privacy, security and identity best practices. "So much of the Internet relies on SSL and the whole chain of trust," Spiezle says, "but it's a confusing, convoluted and disjointed ecosystem."

First, there's no single list of supported root CAs. Instead, each browser platform has its own policies for selecting the root CAs it recognizes. That means each browser supports a slightly different mix of CAs, with lots of overlap, says Serge Egelman, a security researcher at the University of California, Berkeley. Second, the sheer number of competing CAs has created a race to the bottom, he says, putting financial pressure on companies to cut corners.

Further down the chain of trust, browser makers such as Microsoft, Google and the Mozilla Foundation worry that they'll drive fickle users to other platforms if they issue too many security alerts and warnings, so they bundle into their software a liberal list of CAs whose certificates they accept. This approach lessens the likelihood users will get an irritating error or warning message when connecting to a website, but it also increases the chances that certificates from a compromised or disreputable CA will be trusted, Egelman says.

Having so many companies with a hand in issuing certificates also extends the risk. That was evident in the case of fraudulent certificates issued by Comodo, in which attackers compromised an administrative account at a Comodo reseller, which was acting as a registration authority, a sort of subordinate CA. The attackers then used that account to generate nine fraudulent certificates, signed by Comodo, for seven domains, including google.com and yahoo.com.

In an age of nation-backed attacks, certificate CAs with ties to authoritarian regimes also bear scrutiny. Felix Lindner, the hacker known as FX, says travelers connecting to online services within China should treat even secure Web sessions there with skepticism. "We've seen [fraudulent] certificates that checked out as valid on iPhones and other Apple devices for sites like mac.com," he says. "Your little lock icon is not good in China."

Poor SSL Implementation

Compromised CAs are just the tip of the iceberg. Poorly implemented SSL and certificate infrastructure within organizations is also a big problem.

SSL Pulse, a real-time online dashboard that surveys the security of close to 200,000 SSL-enabled websites, documents the overall health of the SSL ecosystem based on several measures, such as proper configuration and the strength of the encryption keys used to sign certificates. Close to 40% of the sites that are monitored support weak or insecure cipher suites of 128 bits or less, while about a third still support the 17-year-old SSL v2.0 protocol, which is known to be insecure, according to SSL Pulse.

In response, Microsoft and other vendors are putting pressure on software publishers and downstream websites to clean up their acts. Microsoft recently raised the mini- mum certificate key length requirements allowed for use with Windows applica- tions, blocking any application that uses keys of a length shorter than 1,024 bits, considered a baseline for security.

That's a start. However, most SSL deployments aren't visible to outside services such as SSL Pulse. Large companies, and some small ones, also can be their own certificate authorities, creating signed certificates for use by internally developed or deployed applications.

A typical global company might own hundreds of certificates issued by third-party CAs to secure communications on public-facing systems, says Paul Turner, VP of products and strategy at digital key management firm Venafi. And internally, that company might manage tens of thousands of application-specific certificates signed by an internal CA. So it's no surprise that security and management practices are all over the map.

"This is infrastructure that has grown organically within organizations," Turner says. "We have customers who, eight years ago, had hundreds of certificates that they managed. Four years ago, it was thousands. Today, it's tens of thousands or even hundreds of thousands of server certificates."

Ideally, a company keeps an inventory of those certificates and tracks their creation date, expiration date and who, internally, is responsible for them. But that's rare, Turner says, and the volume of certificates makes managing them difficult.

At one customer, Venafi staff found that an IT administrator had set up an ad hoc CA with the idea of saving his employer the cost of obtaining hundreds of application-specific certificates from an outside vendor. Those certificates, used for mission-critical internal applications, were given 10- to 20-year lifespans, but the CA itself only had a five-year lifespan. Had the problem not been identified, Turner says, all the issued certificates would have expired simultaneously, crippling the company's operations.

Stories like that aren't unusual, according to Turner. "The folks who work within these organizations are not encryption experts," he says. "Often, they follow the path of least resistance."

Loose security practices are common in other parts of the trust chain, Turner says. Web servers, database servers and other critical systems each have their own cryptographic keystores, in silos, spread across the organization. And 95% of all cryptographic keys are stored in software, not tougher-to-break hardware security modules. Instead, each platform stores them in a different location, with different levels of protection, Turner says.

Worse, administrators delegate cryptographic keystore management to IT staff, any one of whom could export the private keys for their own use. And the credentials for accessing these keystores are frequently hard-coded into internal applications that retrieve encryption keys to secure application-to-application transactions and communications. Hundreds of applications within an organization might all have the same keystore password written into their code, creating an enormous disincentive to update the passwords, Turner says.

The results are predictable: Banks and other financial services organizations that might force users to update their laptop password every 90 days may rely on a years-old and insecure password, known to current and former employees, to protect their certificate keystore, he says.

Apps And Developers In The Crosshairs

Even when the underlying certificate infrastructure is rock solid, enterprises can end up exposed because of loose coding prac-tices and improper design weaken cryptographic protections in common software such as Web browsers and mobile applications.

In the latest example of this problem, German researchers discovered that more than a thousand of the most popular free mobile applications on Google's Play marketplace fail to properly implement SSL and TLS. In a paper presented at the Conference on Computer and Communications Security (CCS 2012), the researchers, from two leading German universities, presented the results of an audit performed with an automated tool, dubbed MalloDroid. They found that 1,074 of the 13,500 popular free applications tested on Play has SSL and TLS implementations that left the applications open to man-in-the-middle attacks.

In many cases, application developers relied on dodgy SSL trust managers or hostname verifiers. Many applications didn't bother to perform authentication. Rather, they simply allowed the application to trust any certificate, regardless of who signed it, and any host that offered a certificate, regardless of whether the certificate offered was for the host offering it.

Application developers don't have a good understanding of cryptography or how PKI implementations work, GlobalSign's Hurst says. One reason is that the tools they use -- such as cryptographic APIs -- assume a high level of understanding of those topics. The result is shoddy SSL implementations.

"The problem with Android isn't so much irresponsible developers, but APIs that are designed for folks like me, not for your typical developer," Hurst says.

Five Steps To SSL Success

So what's a security-conscious company to do? Despite the many technical and organizational challenges involved with deploying a secure, reliable SSL infrastructure, organizations can take five simple steps to implement digital certificates securely and reliably.

1. Pick your CA carefully. As we noted, the market for certificates is competitive, and sometimes you get what you pay for. When picking a CA, do your homework.

"You're picking a business partner, not buying a product," Hurst says. "This is a relationship that goes beyond the delivery of a shrink-wrapped product. You have dependency on them long after they've issued you certificates." Look for CAs that are prepared to help you address issues such as key management, key revocation and phishing.

2. Prepare for the worst. If the past year has taught us anything, it's that the once boring and staid world of issuing SSL certificates isn't so boring anymore. Many organizations that relied on the DigiNotar CA -- the Dutch government among them -- found themselves in a quandary when that CA failed. They were unsure of how many certificates they needed to cancel and reissue, and they lacked a quick back-up plan. It can take days or weeks to get up and running with a new CA and replace all of your certificates, Venafi's Turner warns. And, if a CA has been compromised, you probably won't be the only organization scrambling for a safe harbor. Turner recommends having a back-up CA, with a list of domains you need authorized and the ability to churn out replacement certificates on short order.

3. Know your environment. A first step to SSL sanity is knowing which certificates are floating around in your organization's name. That list should include certificates issued by public and internal CAs. Do an inventory, reviewing all of your applications and servers, noting any certificates they pass and any they accept from "relying parties" -- the individuals and systems that electronically interact with the certificate holder. Next, identify and document who the system and application owners are and how to contact them. Document which CAs issued certificates, their expiration dates, the algorithms used and key lengths.

Finally, document the trust anchor for each CA -- the CA that would be used to validate the certificate (typically, the root CA) and make sure you have the contact information for that organization and know the procedures necessary to revoke and replace any compromised certificates.

4. Have a key management strategy. One of the biggest exposures large companies have to SSL risk is through their critical, legacy internal applications. Many of these companies use SSL or SSH (Secure Shell) encryption to secure communications with users and with other systems with which they interact. However, most organizations have only a cursory understanding of how those keys are implemented.

Companies rarely have a way to change the many old, easy-to-break embedded keys and insecure protocols they have in use, and they're wary of incurring downtime to do so. Address the problem head-on by making an inventory of the encryption keys used within your application infrastructure, assessing their strength and security, and devising a plan to replace weak keys and strengthen protections for all your keys, such as with a hardware security module.

5. Clean up SSL deployment and implementation. Misconfiguration of SSL servers is endemic. As Qualys's SSL Pulse tracker has shown, many companies continue to rely on or support outdated protocols and vulnerable ciphers. Our experts recommend performing an audit of your public and private SSL deployments, ensuring that they meet industry standards for secure implementation. Make sure that your SSL implementation uses secure protocols and unbreakable ciphers, and that you're using SSL to encrypt all of the traffic from your website. For internal or special-purpose applications, consider limiting the list of supported CAs to limit your exposure to forged certificates.

Platform vendors such as Apple, Google and Microsoft would do well to create APIs and other development tools that enforce best practices, make it easy to understand configuration changes that can increase security, and limit the options available to average developers.

Sidebar: Beyond SSL: The Convergence Project

The compromise and eventual collapse of Dutch certificate authority DigiNotar was a disaster for the tens of thousands of private and public sector organizations that relied on the CA to vouch for their online identities. Furthermore, many governments -- including Iran and China -- maintain their own CAs, which are recognized by software publishers and browsers but can become problematic in the age of state-sponsored attacks.

Enter Convergence, the brainchild of SSL security expert Moxie Marlinspike. Based on research done for the Perspectives Project at Carnegie Mellon University, Convergence aims to replace the static list of trusted CAs with a dynamic, global network of "notaries" who vouch for the identity of a given website.

Convergence provides what Marlinspike calls "trust agility." Users choose which notaries they trust and for how long -- granting and revoking trust themselves, rather than deferring trust decisions to browser makers, governments and other authorities.

For now, Convergence is in the beta stage, with two notaries (Marlinspike's Thoughtcrime Labs and security firm Qualys) and a free Firefox plug-in. However, the plan is to get more notaries to sign up, expand Convergence to work with more browser platforms and begin offering Convergence as an alternative to certificate authorities.