Data, meet spies: The unfinished state of Web crypto

Revelations about the National Security Agency's surveillance abilities have highlighted shortcomings in many Internet companies' security practices that can expose users' confidential communications to government eavesdroppers.

Secret government files leaked by Edward Snowden outline a U.S. and U.K. surveillance apparatus that's able to vacuum up domestic and international data flows by the exabyte. One classified document describes "collection of communications on fiber cables and infrastructure as data flows past," and another refers to the NSA's network-based surveillance of Microsoft's Hotmail servers.

Most Internet companies, however, do not use an privacy-protective encryption technique that has existed for over 20 years -- it's called forward secrecy -- that cleverly encodes Web browsing and Web e-mail in a way that frustrates fiber taps by national governments.

Lack of adoption by Apple, Twitter, Microsoft, Yahoo, AOL and others is probably due to "performance concerns and not valuing forward secrecy enough," says Ivan Ristic, director of engineering at the cloud security firm Qualys. Google, by contrast, adopted it two years ago.

Traditionally, "https" Web links have used a single master encryption key to encode hundreds of millions of user connections. That creates an obvious vulnerability: an eavesdropper who obtains that master key can decrypt and peruse millions of supposedly private connections and conversations.

That vulnerability vanishes through forward secrecy's use of temporary individual keys, a different one for each encrypted Web session, instead of relying on a single master key. Through a bit of adroit mathematics that Whitfield Diffie and other cryptographers outlined in 1992, the Web e-mail or browsing session is believed to become impenetrable even to a government eavesdropper such as the NSA that can passively tap into fiber links.

Forward secrecy is an "important technique" that all Web companies should adopt, says Dan Auerbach, a staff technologist at the Electronic Frontier Foundation in San Francisco. It means, he says, an "attacker cannot use the same key to decode all past messages ever sent through those channels."

A survey of major Web companies shows that only Google has configured its Web servers to support forward secrecy by default.

Forward secrecy means an organization with the means to tap into Tier 1 Internet providers "can't decrypt previously recorded traffic," says Adam Langley, a software engineer at Google. "Forward security means you can't go back in time."

Langley announced Google's adoption of forward secrecy, sometimes called perfect forward secrecy, in a 2011 blog post that said an eavesdropper able to break a master key "will no longer be able to decrypt months' worth of connections." The company also published the source code its engineers created using a so-called elliptic curve algorithm in hopes that other companies would adopt it too.

Facebook is currently working on implementing forward secrecy and is planning to enable it for users soon, a person familiar with the company's plans said.

The social network is already experimenting with forward secrecy on its public Web servers. Facebook has enabled some encryption techniques that use forward secrecy, but has not made them the default.

"What that means that these suites will probably almost never be used, and are there only for the rare case that there are some clients that do not support any other suites," says Ristic, Qualys' engineering director, referring to Facebook. (You can check if a Web site uses forward secrecy through Qualys' SSL Server Test or the GnuTLS utility.)

A LinkedIn spokesman provided CNET with a statement saying: "At this point, like many other large platforms, LinkedIn had not enabled [forward secrecy], although we are aware of it and keeping our eye on it. It's still early days for [forward secrecy], and there are site performance implications. So for the time being, our security efforts are focused elsewhere."

A Microsoft spokesman declined to comment. Apple, Yahoo, AOL, and Twitter representatives did not respond to queries.

Disclosures that Snowden, the former NSA contractor now staying in the transit area of Moscow's Sheremetyevo Airpot, made over the last few weeks have shed additional light on the ability of the NSA and other intelligence agencies to tap into fiber links without the knowledge or participation of the Internet companies.

A leaked NSA slide on "upstream" data collection from "fiber cables and infrastructure as data flows past" suggests the spy agency is tapping into Internet backbone links operated by companies such as AT&T, CenturyLink, XO Communications, Verizon, and Level 3 Communications -- and using that passive access to vacuum up communications.

Documents that came to light in 2006 in a lawsuit brought by the Electronic Frontier Foundation offer insight into the spy agency's relationship with Tier 1 providers. Mark Klein, who worked as an AT&T technician for over 22 years, disclosed (PDF) that he witnessed domestic voice and Internet traffic being surreptitiously "diverted" through a "splitter cabinet" to secure room 641A in one of the company's San Francisco facilities. The room was accessible only to NSA-cleared technicians.

A classified directive released last week signed by Attorney General Eric Holder and published by the Guardian indicates the NSA can keep encrypted data it intercepts forever -- giving its supercomputers plenty of time in the future to attempt a brute force attack on master encryption keys it's unable to penetrate today. Holder secretly authorized the NSA to retain encrypted data "for a period sufficient to allow thorough exploitation."

Other intelligence agencies are no less interested. A U.S. security researcher disclosed last month that he was contacted by a telecommunications company in Saudi Arabia for help with "monitoring encrypted data." In 2011, Gmail users in Iran were targeted by a concerted effort to bypass browser encryption. Gamma International, which sells interception gear to governments, boasts in its marketing literature (PDF) that its FinFisher targets Web encryption.

Taking stock of forward secrecy
Without forward secrecy, https-encrypted data a spy agency intercepts could be decrypted if the agency can obtain a Web company's master keys through a court order, through cryptanalysis, through bribing or subverting an employee, or through extralegal means. With forward secrecy enabled, however, an intelligence agency would have to mount what's known as an active or man-in-the-middle attack, which is far more difficult to perform and could be detected by modern browsers.

One reason Web companies have been reluctant to embrace forward secrecy is the cost: an estimate from 2011 said the additional cost of encrypting a connection was at least 15 percent higher, which can be a significant increase for firms that handle millions of users a day and billions of connections a year. Other estimates are even higher.

Another obstacle is that both the Web browser and Web server both must be able to speak in what amounts to the same encryption dialect. Unless both are able to mutually agree to switch to the same forward secret cipher, the connection will continue in a less secure manner with the weaker protection provided by a single master key.

A recent survey by Netcraft found that browser support for forward secrecy "varied significantly." Microsoft's Internet Explorer, the survey found, "does particularly poorly" and is generally unable to make a fully secure connection when connecting to Web sites that use more mainstream ciphers for forward secrecy.

Netcraft said that while Apple's Safari browser supports many ciphers used for forward secrecy, sometimes it will default to a less secure channel. "Web servers respecting the browser's preferences will end up selecting a non-[forward secret] cipher suite," even if the Web server itself would prefer otherwise, Netcraft said. Firefox, Opera, and Google's Chrome browser performed better.

The recent revelations about government surveillance should prompt companies to move more quickly toward stronger encryption, says Auerbach, the EFF technologist. An analogy, he said, is "if you break into my house, you'll be able to see not only what's in there right now, but also everything in the past: all the furniture that used to be there, all the people and conversations that used to take place in the house."

With forward secrecy, Auerbach says, even if you break into a house, "you still won't know what was happening before you got there."

Last updated at 1:00 p.m. PT