Surveillance State

Editors' note: Correction, March 3, 12:46 p.m. PST: This post, which originally carried the headline "White House ditches YouTube after privacy complaints," significantly misconstrued the White House's policy on and use of YouTube. In the interests of disclosure and transparency, we are leaving the contents as originally posted, with two subsequent update notes and with the exception of the headline change. See also our follow-up story, "No, the White House hasn't ditched YouTube."

* * * * * * * * * * * * * Original story follows * * * * * * * * * * * * *

Updated at 5:50 p.m. PST March 2: The New York Times is reporting that the White House has denied any change in online video policy. While the White House spokesperson admitted to using an in-house flash based solution for the latest of the president's weekly video messages, he said the White House is just "experimenting" with different solutions.

Updated at 2:59 a.m. PST March 3: Late Monday, Google posted on its Public Policy Blog a rebuttal to this report: "White House videos on YouTube."

Responding to complaints by privacy activists, the White House has quietly abandoned YouTube as the provider of the embedded videos on the president's official home page.

With the release of the latest weekly video address, the White House has shifted to a Flash-based video solution using Akamai's content delivery network.

The White House's decision to move away from the Google-owned video-sharing site will likely be met with praise by privacy activists and could mark the beginning of a real backlash in response to Google's insatiable thirst for detailed data on the browsing habits of Web surfers.

Ironically, the decision by the White House comes days after YouTube began to roll out new policies to better protect the privacy of visitors who view videos embedded into federal government Web sites. The move by YouTube may prove to be too little, too late.

This is the new embedded video tool used by the White House.

(Credit: Whitehouse.gov)

The White House's decision to embed YouTube videos in the president's official home page drew instant criticism from privacy activists. In addition to several critical posts on my blog, by the Electronic Frontier Foundation (here and here), the Center for Democracy and Technology and the Center for Digital Democracy blasted the choice of video providers.

The focus of the criticism was on the use of long-term tracking cookies by the Google-owned video-sharing site. When the new White House site first went live in January, every visitor to the president's blog would be issued a tracking cookie, even those who did not click the "play" button to watch the video.

The White House acted quickly, and soon deployed a technical fix to the cookie issue, which protected Web surfers who did not click the play button. However, the tens of thousand of people who clicked play were still issued a cookie, and thus tracked by YouTube.

In an unannounced change over the weekend, the White House appears to have solved the remaining cookie privacy issue for those Web site visitors who wish to watch the president's weekly video message.

Out with YouTube, in with Akamai
As of Saturday, the White House seems to have ditched YouTube as its video provider. Visitors to the White House blog can now click play to view a Flash-based video that loads directly from the White House's own Web servers. This solution, which appears to use Akamai's content delivery network, does not make use of tracking cookies.

The president's tech team seems to have finally hit on an optimal solution--one which protects the privacy of the visitors to the White House site, while still permitting the president to spread his message.

The White House is still posting copies of the videos to its official YouTube channel. However, the president no longer provides free advertising to YouTube by embedding those videos on a taxpayer-funded site.

Furthermore, the White House has copied one of the coolest of YouTube's social features: the ability for users to easily share and embed videos on their own sites. Each of the White House-hosted videos includes an "embed" link under it that can be copied and pasted onto any other Web site or blog.

It is unclear whether this switch away from YouTube marks a permanent shift in policy for the White House, or whether the Oval Office geek squad is merely testing an alternate video provider. While the latest video is served using Akamai's servers, the older videos remain as embedded YouTube files.

YouTube's new cookie rules
The timing of the White House's decision to switch to Akamai is rather strange, given the recent moves by YouTube to offer a more privacy-preserving solution for videos used on federal government sites.

Within the last couple weeks, YouTube has silently rolled out its own updates in response to the cookie-related criticism. People wishing to embed a YouTube video can now select a delayed cookies option when copying the embed URL.

This is the new delayed cookies option for YouTube embeds.

(Credit: Screenshot of YouTube)

That choice will cause the embedded videos to be served from an alternate domain, www.youtube-nocookie.com, which registrar records reveal was first registered on January 23 2009, just one day after this blog first mentioned the White House/YouTube cookie issue.

New documentation on the YouTube site reveals:

Enabling delayed cookies means that the YouTube video player will not set any non-session cookies on the computer of a visitor (viewing the page on which the YouTube video is embedded). The YouTube video player may set non-session cookies on the visitor's computer once the visitor clicks on the YouTube video player.

This option is rather similar (yet still inferior) to the technical fix that was previously used (and since disabled) by the White House, as well as the open source MyTube tool developed by the Electronic Frontier Foundation.

A prominent privacy policy
In another new move by YouTube, the site now appears to be directly embedding a link to its privacy policy in all videos that are played from government sites.

This is the new privacy policy link in .gov-hosted YouTube videos.

(Credit: Whitehouse.gov)

When those same videos are viewed at YouTube.com, or when embedded in a blog or other non-.gov site, the clickable link to the privacy policy is gone.

Webmasters for various state agencies seemed to notice the new policy last week and initially complained to YouTube, thinking that the new youtube-nocookie.com was a phishing site.

A representative from YouTube told the Webmasters:

The privacy policy link you see on your embed player is in response to federal regulations regarding privacy on embed players. We're working to remove it from state and local .gov sites as soon as possible.

Still not perfect
While the decision by the White House to ditch YouTube is a good one, unresolved issues remain.

First, as previously noted by the Electronic Frontier Foundation, the White House Web site makes use of an "invisible pixel" style Web bug/tracker on every page on the site, hosted by WebTrends.com.

Ideally, the White House should take its Web analytics technology in-house and abandon the use of this third party tracking technology. Otherwise, at the very least, the White House privacy policy should be updated to note the tracking cookies used by WebTrends.

Second, the White House still has not published the waivers it issued to YouTube (and potentially other third parties), which permitted the sites to use long-term tracking cookies. The Electronic Frontier Foundation has repeatedly asked for these documents-- requests that the White House has ignored.

Given the president's much-publicized commitment to transparency, it is time that the White House publishes these documents.

Third, in its recent move to include privacy policy links in videos embedded at .gov Web sites, YouTube has clearly demonstrated that it has the ability to modify the services it provides depending on the referrer information associated with incoming requests. YouTube should build on this and adopt a policy of not logging any data associated with .gov-referred requests.

That is, the site would be free to keep logs on the videos viewed by visitors to its own site as well as those embedded on blogs, but it would opt to immediately forget all identifying information associated with requests from government sites.

While the White House seems to understand the cookie privacy issue, it is unlikely that members of the House and Senate are equally as tech savvy. After all, some of them can barely figure out Twitter.

YouTube videos are heavily used on the Web sites of those in the House and Senate. YouTube should adopt sane logging policies for visitors who view these videos, so that we don't have to wait for the House and Senate to fix the problem themselves.

YouTube did not return a request for comment, while a representative for the White House Web team declined to speak on the record.

(Credit: Recovery.gov)

Update: As of 8 a.m. PST, within three hours of this story first going live, it appears that President Obama's Web team has (silently) pulled the robots.txt file from the Recovery.gov Web site. The site is now open to Web crawlers of all kinds.

The Obama administration has apparently opted to forbid Google and other search engines from indexing any content on the newly launched Recovery.gov.

Is this even more evidence that the administration's much-publicized commitment to transparency is simply hype?

Recovery.gov, which went live Tuesday, is set to act as a central clearinghouse for information related to the newly signed American Recovery and Reinvestment Act. The legislation is designed to stimulate the flagging U.S. economy.

In a video message, available on YouTube and embedded into the new site, President Obama states that the "size and scale of (the stimulus) plan demands unprecedented efforts to root out waste, inefficiency, and unnecessary spending. Recovery.gov will be the online portal for these efforts." He adds that the new site will be used to publish information on how the stimulus funds will be spent in a "timely, targeted, and transparent manner."

Although the site is advertised as proof of the president's commitment to transparency, its technical design seems to betray that spirit. Most importantly, the site currently blocks all requests by search engines, which would ordinarily download and index each page to make the information more accessible to the Web-searching public.

The site's robots.txt file has just a few lines of text:

# Deny all search bots, web spiders
User-agent: *
Disallow: /

Although the White House Web team did not immediately respond to a request for comment, the single-line comment at the top of the file indicates that the blocking of search engines is no accident but rather a statement of policy.

Many sites use a robots.txt file to communicate, in machine-readable terms, the Web pages that they do and don't wish to be indexed by search engines. While the files don't carry much, if any, legal weight, most search engines act as good Internet citizens and honor the requests.

Luckily for the millions of Americans who might wish to find out how their money is going to be spent, it seems that Google has opted to ignore the administration's restrictive robots.txt on the stimulus-related site. It is unclear if this is due to an error or a manual override by someone at Google, but a quick search turns up more than 60 Web pages on Recovery.gov that have been indexed by the search engine's Web crawlers in just the past three days.

Also, the stimulus bill requires that the site be run by the new Recovery Accountability and Transparency Board, but it seems to currently be under the control of the White House Web team--the same folks who revamped Whitehouse.gov and whose use of the robots.txt search engine-blocking code was expanded after the site initially was praised by bloggers for its openness.

It is this blogger's hope that with a bit of gentle prodding by members of the pro-transparency community, Recovery.gov's administrators will correct the "unintentional oversight" that was made in launching the site with such an restrictive robots.txt file.

When the mainstream media first announced Barack Obama's "victory" in keeping his BlackBerry, the focus was on the security of the device, and keeping the U.S. president's e-mail communications private from spies and hackers.

The news coverage and analysis by armchair security experts thus far has failed to focus on the real threat: attacks against President Obama's location privacy, and the potential physical security risks that come with someone knowing the president's real-time physical location.

President Obama and his BlackBerry at the White House in late January.

(Credit: UPI Photo/Ron Sachs/Pool)

Serial numbers
Before we dive in, let's take a moment to note that each mobile phone has a unique serial number, known as an IMEI, or MEID. This unique number is transmitted in clear text, every time the phone communicates with a nearby cell tower. Thus, while the contents of a phone call or the data session (for e-mail) are usually encrypted, anyone with the right equipment can home in on a particular IMEI and identify the location of the source of that signal.

The most common device used to locate a phone by its IMEI is a "Triggerfish", a piece of equipment that is routinely used by law enforcement and intelligence agencies. This kind of device tricks nearby cell phones into transmitting their serial numbers and other information by impersonating a cell tower.

The devices, which are actually fairly low-tech, were used to hunt down famed hacker Kevin Mitnick back in the 1990s. Most interesting of all, according to Department of Justice documents, Triggerfish can be used to reveal a suspect's location "without the user knowing about it and without involving the cell phone provider."

The expensive brand-name Triggerfish devices, made by the Harris Corp., are sold only to government agencies. However, it is almost certain that foreign governments have similar technology. Furthermore, someone with a low budget could likely use the open-source GNU Radio platform, which can already decipher GSM signals, to roll their own phone sniffer.

Finding Obama
We know that the president has been given a White House-issued BlackBerry phone. As a result, Obama's smartphone is broadcasting its IMEI serial number for anyone with the right equipment to detect.

Of course, the president is never alone, and so it is likely that anyone sniffing the wireless spectrum near the president would pick up hundreds of different BlackBerrys in the area.

However, Obama's aides do have to go home at some point, whereas Obama sleeps at the White House. This means that over the course of several days or weeks, it should be possible for a patient adversary to determine which IMEI belongs to the president's phone, and which IMEIs are associated with the phones of aides, simply by following the president (at a distance) and monitoring the spectrum at all hours.

As staffers go home for the evening, and Secret Service agents rotate out of duty, an adversary can strike their IMEI numbers off of the list. Within days, that initial list of 100 BlackBerrys can be reduced down to a single IMEI identifying the president's phone

Were someone to learn the president's IMEI, they could use it to gain valuable (and dangerous) information. For example, by pointing an antenna at the White House, it'd be possible to instantly determine if the president was inside. With a sophisticated-enough antenna, it might even be possible to determine which vehicle the president is sitting in while traveling in a motorcade, or to determine if the Secret Service is driving an empty limousine along a high-profile route to draw attention, while the president travels to a venue in an unmarked vehicle. The digital trail left by the president's BlackBerry would soon announce his presence to those keeping an eye out for his IMEI.

I am sure that others could come up with even more nefarious uses for real-time access to the president's physical location. I will leave that task to the blogosphere.

Burners
The simple solution to this problem, of course, is for the President to regularly change his IMEI serial number by getting a new phone. However, this presents another problem: that of the odd man out.

Imagine that foreign spies point a directional antenna at the White House and are thus able to capture the IMEI numbers of Obama and his team, as they leave and return to the White House from various events.

If a new IMEI number were to suddenly appear, be used for one week, disappear, and then be replaced by a new IMEI, which was also used for a week, before also disappearing, it would soon be obvious that a single person was changing phones. This pattern would be even more obvious, if everyone else in the president's entourage kept using their own phone--and thus broadcast the same IMEI, week after week.

Simply put, the only way that President Obama can gain some level of anonymity with regard to his IMEI number is if everyone in his team also changes their IMEI numbers with the same regularity.

Fans of the HBO TV show The Wire (a group that includes Obama) will no doubt remember the use of cheap prepaid "burner" phones by the fictional drug dealers. In order to avoid being wiretapped by the police, the entire criminal gang would dispose of their phones at once and switch to brand-new devices.

Essentially, the White House needs to start using burners.

Cost-effective protection
It would be extremely expensive (and wasteful) for the president and his staff to get a new BlackBerry each week. Luckily, there are two options available to the White House tech staff that allow them to protect the president's location privacy in a cost-effective (and environmentally friendly) way:

First, the White House geek team can simply shuffle the BlackBerrys used by the President's staff. That is, take away everyone's phone, mix them up, restore the software to the factory default, then issue a "new" phone to each staffer.

Within minutes, the phones would synchronize with the White House e-mail servers, and thus the "new" devices would have instant access to the e-mails and information that had been on the previous device.

The inconvenience factor of such a solution could also be significantly reduced by having twice as many phones as employees--that way, staff would not have to go without their phone for more than a minute or two, as they were swapped each week.

As long as this shuffling of phones were done randomly, the IMEI numbers would be sufficiently anonymized. Sure, a potential attacker would know that the device belonged to a member of the White House staff, but they would not know whether if belonged to a lowly intern, the press secretary, or the president.

A slightly more laborious method would be to hack the software running on the BlackBerrys and flash the devices with a new serial number. While this is quite possibly a violation of the Digital Millennium Copyright Act (which prohibits most forms of phone hacking), it is unlikely that Research In Motion (which makes the BlackBerry) would sue the White House for engaging in such reverse engineering.

Of course, the downside of giving each phone a new serial number is that these phones would then need to be re-registered with the wireless communication company, which would otherwise refuse to provide the devices with service. However, this additional burden for the White House techies would yield significant security benefits, as each phone would be given a clean IMEI number not associated with the White House.

Insiders
In this article, I've focused solely on the scenario of a bad guy with an antenna. There is also the very real (and significant) risk of an insider working for the phone company.

Insiders are a notoriously difficult security problem to fix, something Obama has likely already learned, after his passport file was read by a contractor working for the State Department.

Even if every person working for the White House's telecommunications carrier were honest, it could also be possible to social-engineer the information out of a customer service representative (otherwise known as "pretexting").

Alternatively, an adversary could simply hack into the computer systems used by the phone company in order to get information on Obama's phone. Is was this latter approach that was followed by an unknown attacker who was able to spy on the phone calls of more than 100 Greek government officials during the 2004 Olympics.

Foreign trips
President Obama is likely to go on many foreign trips during his four (or more) years in office. In addition to burdening taxpayers with the obscene international roaming rates associated with his foreign BlackBerry usage, there are new and more serious security concerns to consider.

The federal government can most likely trust AT&T and the other wireless carriers. After all, they did join forces with the National Security Agency to spy on millions of American's phone calls without a warrant. The telecommunication companies in foreign countries are far less likely to be pro-United States, and in some cases, they are likely to be working closely with foreign intelligence agencies.

Thus, as long as President Obama keeps his BlackBerry turned on while he is in China, it is likely that the Chinese government will be closely monitoring his location, as reported by the president's phone to the Chinese government-owned phone company. The same sort of security issues will likely arise in many other countries.

Due to these security concerns, this blogger would be extremely surprised if the Secret Service permitted the President to use his BlackBerry when on foreign trips.

As you can see, the use of a BlackBerry by the president creates a number of very real security headaches that are no doubt keeping several people at the Secret Service awake at night. While the initial focus of the press was on the e-mail and smartphone technology in the president's phone, the real threats and risks are actually associated with more boring functions of the device.

Further reading: M. Jakobsson and S. Wetzel. "Security Weaknesses in Bluetooth" (PDF) describes some very similar location privacy attacks against mobile phones using Bluetooth-based sniffers.

The White House has silently tripled the number of Web pages that it forbids Google and other search engines from accessing. Is this a bad omen or much ado about nothing?

Within hours of Barack Obama being sworn in as president, bloggers and tech journalists began to closely examine the new White House Web site for hidden indicators as to how he would shape future tech policy.

While I focused my efforts on the White House privacy policy, others looked to the new administration's robots.txt file, which lays out boundaries that search engines like Google should follow when scraping the site.

When the new Obama geek team posted its sparse robots.txt to the Web, tech pundits soon hailed it as a sign of the President's commitment to openness, transparency, and proof that someone tech-savvy was finally running the show.

Blogger Jason Kottke hailed the move, writing that it was "a small and nerdy measure of the huge change in the executive branch of the U.S. government today." Another blogger, Ben Orenstein, compared the new Obama robots.txt file to the 2,400-line file used by the Bush White House, "I think you've got a lovely little microcosm; one that points to a hopeful and open future."

The big fuss?

These digerati were excited by the fact that the new White House robots.txt file contained just two lines:

User-agent: *
Disallow: /includes/

Fast-forward one week, and the White House has silently started to expand its use of the robots.txt search engine-blocking mechanism. As of Friday morning, the file now contains the following text:

User-agent: *
Disallow: /includes/
Disallow: /search/
Disallow: /omb/search/

While it would be accurate to state that the White House has in one day tripled the number of sites it excludes from Google crawling, it is also important to note that this is not a big deal--in fact, it doesn't matter at all.

For the most part, the Bush White House's use of robots.txt was totally legitimate, something that Kevin Fox, an engineer at Friendfeed told the folks at Google Blogoscoped:

This is a bit silly. The old robots.txt excludes internal search result pages and redundant text versions of HTML pages. This is exactly what robots.txt is for. Google's Webmaster Guidelines state "Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don't add much value for users coming from search engines."

It's understandable that the robots.txt of an 8-year-old site is longer than that of a 1-day-old site, and it's not as if '/secrets/top' or '/katrina/response/' were put in the robots file.

Fun as it may be, this is a nonstory.

Those bloggers drunk on hope who desperately wanted to see proof of Obama's commitment to his campaign promises of transparency and Google Government now find themselves with a difficult choice: they can either accept and acknowledge that robots.txt files are not a set of digital tea leaves through which you can read the new administration, or, if robots.txt does carry weight, they can try to come up with a way of explaining a 200 percent increase in the number of directories blocked by Obama's Web team as anything but Cheney-esque secrecy.

Simply put, the robots.txt file was created and managed by engineers, not lawyers or policy makers. It is not the place to judge the president on tech policy issues.

The president's tech policy should instead be judged on real issues: how many former RIAA and MPAA lawyers will be given positions of power in the administration, who ends up working at the FTC and FCC, and who will be named the new cybersecurity czar.

As for the president's commitment to transparency, he has already violated his pledge to post all nonemergency bills on the Whitehouse.gov Web site for five days before signing them. The text of the Lilly Ledbetter Fair Pay Act of 2009, which was signed into law yesterday, was certainly not posted to Whitehouse.gov for anywhere near five days.

Obama's broken commitment to transparency remains advertised on the White House blog:

One significant addition to WhiteHouse.gov reflects a campaign promise from the president: we will publish all nonemergency legislation to the Web site for five days, and allow the public to review and comment before the president signs it.

It is by looking to these kinds of concrete issues by which we can judge the president, not robots.txt

As President Obama's $825+ billion financial stimulus package works its way through Congress, a number of groups have started to call for increased transparency in the way that data on the proposed spending will be shared with citizens.

Most noteworthy are demands from public-interest groups and academics that the the data be provided in a format conducive to user-generated mashups and remixes.

The American Recovery and Reinvestment Act of 2009 passed through the House Appropriations Committee a couple weeks ago, and it is expected to come up for a full House vote in the coming weeks.

In addition to authorizing the spending of an obscene amount of money, the act also mandates the creation of a Web site to "foster greater accountability and transparency" in the use of those funds.

While the bill does a great job in mandating the kinds of information that will be put online (contracts, audits, inspector general reports, etc.), it is rather vague with regard to details on how the information will be provided.

The only hints include language mandating that the information be "easy to understand" and "regularly updated," and include a "database of findings from audits," "printable reports," and "user-friendly visual presentations to enhance public awareness of the use of funds."

Such statements bring to mind the possibility of yet another boring and difficult-to-navigate federal government Web site, perhaps similar to the Federal Communications Commission's antiquated and ineffective home page, or the Federal Elections Commission's slothlike campaign donation search engine.

Faced with the possibility of another Web 1.0 Web site designed by the federal bureaucracy, a number of pro-transparency activists and tech policy academics have started to weigh in on the issue, all of them demanding the same thing: full, easy, and free access to the complete data set powering the Recovery.gov Web site.

For example, while the FEC's donation search engine was often slow and unresponsive during last year's presidential campaign, a number of third parties were able to create fantastic mashups of the campaign donation data--the most notable of these being the Hufington Post's FundRace tool, which provides users with a Google map view of each donation to the presidential campaigns.

The numerous independent sites allowing for the easy navigation of campaign donation data was possible because of the legal requirement that all FEC data be made available in full to the public. As a result, public-interest groups and media organizations were able to create their own innovative mashups and remixes of the data, providing faster and more responsive Web interfaces than the FEC's overwhelmed servers, as well as creating innovative visualization methods for navigating the data set.

John Wonderlich, program director at the nonpartisan Sunlight Foundation, outlined the general problem:

We'd like the site to serve not just the amateur information consumer, but also the programmers that can skillfully remix the information. The citizen observer's role seems well-addressed by the legislation that mandated the site (with requirements for "printable reports," feedback, and to be "easy to understand"), while the needs of the programmer are largely unaddressed. The data should be available in formats that facilitate more advanced use by programmers and analysts alike.

Certainly, the data should be made available following the 8 Principles of Open Data: (1) complete, (2) primary (as it is collected at the source), (3) timely, (4) accessible, (5) machine-processable, (6) nondiscriminatory, (7) nonproprietary, and (8) and license-free. XML and CSV are a minimum.

Search is great, if you are looking to find information about any one thing. But original analysis and visualization require access to data in bulk. If the goal of putting the data online is to increase accountability and transparency, then it is necessary (to) provide bulk data access.

Echoing this last point, David Robinson, the associate director of the Center for Information Technology Policy at Princeton University, told me that "(no) one person or organization could possibly anticipate all the ways that Americans will want to analyze, reuse, or cross-reference the information that Recovery.gov will offer. And no one person or organization needs to do so, as long as the data itself is readily available."

In 2008, Robinson and his colleagues at Princeton published a paper calling for the government to provide open access to the raw data used by all federal Web sites. The highly influential paper has been widely circulated among technology policy circles in recent months.

Jim Harper, the director of information policy studies at the Cato Institute, feels that the entire back-end database should be made available.

"This is a little tricky, because people have to settle on a format, and then require submissions in that format from contractors and state and local entities, etc.," Harper told me. "But if the administration wants to be transparent, a little forcing will go a long way. States and contractors will learn how to deal with standardized data quickly, if it makes the difference on getting federal dollars."

A month ago, Harper moderated a one-day forum at Cato, in which a number of policy experts called for open access to government data. A video and podcast of that event can be found here.

Given that this bill has largely been written and shaped behind closed doors, it remains unclear how much of an impact these pro-transparency activists will have on the legislation that will create the Recovery.gov Web site. As of press time, calls for comment left with the House and Senate Appropriations Committees had yet to be returned.

Someone at the White House appears to be listening to those of us in the privacy community.

For the third time in just six days, the Obama administration has modified the White House Web site privacy policy in response to criticism from the blogosphere.

When the site launched on January 20, it exempted YouTube from federal anticookie tracking rules that would have otherwise cast a legal shadow over the use of embedded videos on the White House blog.

Reacting to criticism from the blogosphere, the White House first modified its Web site on Friday to limit the cookie exposure to only those users who clicked on videos. Then, on Sunday, the White House again tinkered with its privacy policy to scrub YouTube's name from the cookie exemption.

The original YouTube-specific exemption stated:

For videos that are visible on WhiteHouse.gov, a "persistent cookie" is set by third-party providers when you click to play the video.

This persistent cookie is used by YouTube to help maintain the integrity of video statistics. A waiver has been issued by the White House Counsel's office to allow for the use of this persistent cookie.

However, by Sunday evening, the exemption had been edited to remove all mention of YouTube:

For videos that are visible on WhiteHouse.gov, a "persistent cookie" is set by third-party providers when you click to play the video.

This persistent cookie is used by some third-party providers to help maintain the integrity of video statistics. A waiver has been issued by the White House Counsel's office to allow for the use of this persistent cookie.

The decision by the White House to revisit the cookie exemption does not come as a complete shock. The YouTube rule had in just a few short days generated both bad press and direct criticism from several public-interest groups.

It should be noted that this change is, for the most part, cosmetic. YouTube continues to be the only company whose video content is embedded within the White House Web site. Furthermore, the Google-owned video-sharing site is the only one that has received both official legal clearance from the White House Counsel and direct assistance by the White House tech staff (who embed the YouTube content) in planting tracking cookies within the Web browsers of millions of Americans.

Google CEO Eric Schmidt, who has advised President Obama and who personally donated $25,000 to the president's inauguration celebration (out of a total of $150,000 by six Google executives) must be rather pleased.

Still no transparency
In spite of Obama's much-publicized commitment to transparency, the White House has yet to actually provide a copy of the waiver (something this blogger has requested from White House officials informally, as well as via the Freedom of Information Act).

The text of the original privacy policy implied that a specific waiver had been issued for the cookies forced upon end users who intentionally viewed YouTube videos embedded within the White House Web site. The text now implies a far broader waiver for multiple video-sharing Web sites. However, it remains unclear if a new waiver has been issued, or if the old waiver was broad enough to cover multiple sites.

When I first wrote about the privacy policy text last week, I criticized the White House for providing YouTube with a specific exemption. At the time, I noted that no other company had received such special treatment.

The motivation of my criticism was to try to shame the White House staff into doing away with the exemption--as cookies are in no way required in order to serve online video. Instead of recognizing the need to protect consumer privacy, White House officials reacted by expanding the exemption to other companies.

In many ways, the current policy is actually worse than before: non-tech-savvy consumers now have no idea how many companies might be forcing their Web browser to accept tracking cookies. At least up until last week, visitors could take some comfort in the knowledge that only one company might be invading their privacy when they visited the White House Web site (and then only by a firm that had pledged to "do no evil"). Now, at least according to the White House's wide exemption, there could be many.

Last week, I said we should be reasonable and give the White House Web team a bit of time--after all, it is in a brand-new office, managing a new computer network, and scrambling to meet the demands of a very busy boss. However, if the team has had enough time to tinker with the privacy policy at least three times in the past six days, then it has more than enough time to post a copy of the waiver.

Just 12 hours after this blog highlighted the privacy problems associated with the White House's use of embedded YouTube videos, the Obama team rushed to deploy a technical fix that significantly protects the privacy of many (but not all) of the site's visitors.

Since its launch three days ago, President Obama's White House Web site has included several embedded YouTube videos. While this certainly demonstrates that the 44th president is Web 2.0 savvy, the decision to embed YouTube videos has also enabled the Google-owned video-sharing site to sneakily collect data on the millions of people who visit Whitehouse.gov--even those users who never click the "play" button to actually watch one of the videos.

Change.gov, the Web site for the Obama/Biden transition team, also made extensive use of YouTube videos. This practice was something that I sharply criticized back in November, citing the cookie-related privacy risks as well as the decade-old rules prohibiting the use of long-term tracking cookies on federal agency Web sites.

Unfortunately, when the new White House Web site launched, rather than fix the privacy issues that had plagued the transition team's Web site, Obama's legal team instead opted to provide YouTube with an exemption to those pesky federal regulations, letting it use long-term cookies to track visitors to the White House Web site. No other company was singled out and granted such a waiver.

It seems that someone in the White House read my blog post yesterday--as within 12 hours of the story going live, Obama's Web team rolled out a technical fix that severely limits YouTube's ability to track most visitors to the White House Web site.

By late Thursday evening, each embedded YouTube video had been replaced with an image of a video player, which a user must click on before the real YouTube player will be loaded. The result of this change is that YouTube is now only able to use cookies to track users who click on the "play" button on an embedded YouTube video--the majority of people who scroll through a page without clicking play will not be tracked.

This is clearly a step in the right direction--and it is particularly interesting to see that the White House has essentially rolled their own version of the Electronic Frontier Foundation's MyTube privacy tool.

While this is great news (especially after just a few hours), it is by no means a comprehensive solution, but a Band-Aid. Those users who do click the "play" button will be secretly tracked as they navigate the White House Web site--and if those users have visited YouTube or any other Google-run Web site in the past, the fact that they watched an Obama video will be added to the existing massive pile of data the company has compiled on each of them.

Simply put, there is no good reason for Google to be able to data mine a citizen's interaction with the president--especially when watching a video that was produced and uploaded by the White House at the taxpayers' expense.

The White House is already making use of Akamai's commercial edge caching services, and the transition team made full use of Amazon's Simple Storage Service for the download-friendly version of Obama's weekly address. Rather than using YouTube, the State Department has for some time opted to pay for a commercial, flash-based video streaming solution provided by Brightcove for its propaganda information site America.gov.

If the Obama team is willing to pay for some of its Web 2.0 technology, why can't they also follow the State Department's lead and cough up a few bucks for a streaming video service that doesn't cross-subsidize its offerings by tracking the Web habits of users.

Finally, if the White House lawyers are going to waive long-standing federal privacy rules for YouTube, merely mentioning the existence of that waiver is not enough. Given Obama's much publicized commitment to transparency, I think it's quite reasonable to ask that the team post the text of each and every waiver to the federal cookie policy to its Web site. Members of the public have a right to know the reasons that were used to justify exempting YouTube's cookies from these otherwise strict rules. If the YouTube waiver cannot withstand the analysis of legal experts and the ridicule of tech bloggers, it probably shouldn't have been authorized.

The White House Web site has been live for just three days, and in just the past day, Obama's administration has given us some reason to believe that it takes Web privacy seriously. Over the next few weeks, it'll have a chance to prove it.

Update: 12 hours after posting this story, the White House (partially) reversed itself. The rather dubious YouTube-only waiver from federal Web privacy rules has been maintained, but the White House Web site has been updated to limit the exposure of visitors to YouTube's tracking efforts to only those people who actually click the "play" button on a YouTube video. For more details on the new changes, read this blog post.

The new Web site for Obama's White House is already drawing attention from privacy activists and tech bloggers. While the initial focus has been on the site's policies relating to search engine robots, a far more interesting tidbit has so far escaped the public eye: the White House has quietly exempted YouTube from strict rules relating to the use of cookies on federal agency Web sites.

The new White House Web site privacy policy promises that the site will not use long-term tracking cookies, complying with a decade-old rule prohibiting such user tracking by federal agencies. However, the privacy policy then reveals that Obama's legal team has exempted YouTube from this rule (YouTube videos are embedded at various places around the White House Web site).

While the White House might not be tracking visitors, the Google-owned video sharing site is free to use persistent cookies to track the browsing behavior of millions of visitors to Obama's home in cyberspace.

No other company has been singled out and rewarded with such a waiver.

In a blog post back in November, I criticized the Obama transition team's Change.gov Web site for its use of embedded YouTube videos. At the time, I stated that the practice might violate long-standing federal rules that forbid federal agencies from using persistent tracking cookies on their Web sites. It turns out that I was wrong: the transition team was technically not a federal agency and thus not bound by the anti-cookie rules.

Now that Obama is president, his official Web site is required to abide by the cookie regulations. Furthermore, as of Wednesday afternoon, several YouTube videos have been embedded on the White House blog. As soon as a visitor surfs to one of the blog pages that contain a YouTube video, a long-term tracking cookie is automatically set in the user's browser--even for those users who do not click the "play" button.

Someone on the Obama legal team seems to have read my previous blog post, as they've modified the White House privacy policy to specifically exclude YouTube's tracking cookies from federal rules that would otherwise prohibit their use:

"For videos that are visible on WhiteHouse.gov, a 'persistent cookie' is set by third party providers when you click to play the video.

This persistent cookie is used by YouTube to help maintain the integrity of video statistics. A waiver has been issued by the White House Counsel's office to allow for the use of this persistent cookie."

YouTube and cookies
Each time a new user visits YouTube, a unique permanent tracking cookie is issued by the Web site to the user's browser, which it stores. Whenever the user later revisits YouTube, that cookie is transmitted to the video-sharing site, allowing it to identify users and monitor their video viewing habits.

YouTube is also able to set and access a user's tracking cookie when she visits a third-party Web page that has embedded a video stored on the YouTube site (such as a blog or other Web site), even if the user never clicks the play button.

The moment that the flash file containing the video player is downloaded from YouTube's servers and displayed in the user's browser as part of another Web page, the cookie is transmitted to YouTube's servers. Considering how widespread the practice of embedding YouTube videos has become, this gives Google an amazing amount of data on the Web-browsing activities of hundreds of millions of Internet users--many of whom may not realize that such tracking data is being collected.

The White House policy is not being followed
The YouTube-related text in the new White House privacy policy implies that not all users will be tracked by YouTube. The policy notes that:

"If you would like to view a video without the use of persistent cookies, a link to download the video file is typically provided just below the video."

As of Thursday morning, this statement is false.

In multiple tests by this blogger with both Internet Explorer and Firefox, merely visiting pages on the White House blog causes YouTube to set a long-term tracking cookie in the browser--even if the user does not press the play button to start the video. After eight months, this cookie will be automatically deleted by the user's browser--unless, of course, the user visits another Web page somewhere else on the Internet containing a YouTube-embedded video, in which case, the eight-month cookie clock is reset. Given how widespread YouTube video embeds have become, this cookie essentially lasts forever.

While it is obvious that I am rather critical of this entire affair, I am willing to give the Obama Web team the benefit of the doubt in one area: the fact that their current Web infrastructure does not deliver on the promises made by their privacy policy.

The Obama White House Web site is only two days old, and so it is certainly possible that the team simply hasn't gotten around to deploying a more privacy-preserving system for YouTube video embeds. Protecting users who do not click "play" from automatically receiving a cookie is certainly possible; the Electronic Frontier Foundation in 2008 released a wrapper script for YouTube videos that provided this very feature. Let us hope that the Obama team deploys such a technology in due course.

Can YouTube be justified as a "compelling need"?
For the past 10 years, federal agencies have been prohibited from using tracking cookies on their Web sites, except in a few special cases. The Office of Management and Budget rule M-03-22 states that:

"Agencies are prohibited from using persistent cookies or any other means (e.g., web beacons) to track visitors' activity on the Internet except .... [when there is] a compelling need."

The question we must now focus on is this: Is the need for Obama to use embedded videos hosted by YouTube (and not, say, another company's video-streaming platform that does not force cookies upon its users) a use that can be reasonably described as compelling?

Presumably, this has been justified on the basis that YouTube forces cookies on the visitors of any Web site that embeds one of its videos. However, while Joe or Jane blogger has no bargaining power with YouTube/Google, the federal government certainly does.

In just the past couple weeks, YouTube has launched dedicated pages for both the House and Senate to show off their own videos, and the site also recently started allowing users to directly download copies of some videos. This latter feature has not yet been widely deployed across the site, and is seems to be limited to videos posted by Obama's team.

Given the famously close connections between Obama and Google, you'd think his tech team could negotiate for a cookie-less way to embed videos. At a technical level, this would be an easy enough change, even if it would deny Google the ability to collect even more information on millions of Americans.

Cookies and other federal agencies
Finally, the new White House YouTube rule may have a far broader impact on the way that federal agencies use Web 2.0 content. Simply put, if another federal agency embeds a YouTube video in its Web site without first having the agency's legal team issue a waiver, have federal rules been violated?

Up until this week, federal agencies have been free to embed Web 2.0 content in their own sites without any real need to consider the privacy risks posed to end users. The fact that the White House Counsel has felt it necessary to issue such a waiver for YouTube videos appearing on the White House Web page could be reasonably interpreted to mean that such a waiver is now required for all embedded Web 2.0 content that might force cookies upon end users. This is certainly new legal ground.

Consider, for example, the Transportation Security Administration, which has posted YouTube videos to its blog numerous times over the past year. Its privacy policy makes no mention of YouTube cookies. Could this lead to issues for the TSA Web team, or perhaps even congressional investigations? Given my own history with TSA, I certainly hope so.

It's that time of year again: predictions for the next 12 months, most of which are likely to be wrong, and a few that, if right, will further cement Surveillance State's status as a top tier tech blog...maybe.

• President Obama will break the heart of Net neutrality activists by picking pro-telecom industry people for the FCC. On the other hand, Obama will pick someone great for the position of privacy czar, and then castrate him/her by not giving the position any power.
• Comcast, AT&T and other ISPs will begin the mass deployment of monthly download caps. However, they will strike profit sharing deals with Google/YouTube and Apple to exempt such traffic from customers' monthly bandwidth limits. Customers who go over the cap will have to pay extra--thus also conveniently killing off much of the P2P market (since no one will pay for BitTorrent), without having to resort to Deep Packet Inspection.
• Google and Yahoo will win the war to define the terms of the privacy vs. data logging debate: The search engines will settle on storing search log data for three to six months, but Microsoft will (unfortunately) fail to define the debate on how the data is anonymized, rather than after how many months. Google and Yahoo will continue to engage in privacy theater by not effectively anonymizing their logs.
• We will not see the passage of any comprehensive overhaul of privacy law in 2009. Efforts to restore privacy to searches of laptops at the border will fail. If legislation does pass, it'll be toothless.
• Bruce Schneier will be the next cybersecurity czar for the federal government.
• The Transportation Security Administration will reverse the liquid ban, but will continue to engage in pointless security theater. The replacement for head honcho Kip Hawley will not shake things up.
• The RIAA will suffer its first major loss in the courts, and will be forced to pay more than $100,000 in damages (in addition to legal costs). Likewise, attempts by the RIAA and MPAA to institute "three strikes" rules in the U.S. will fail.
• The copyright office will reject most of the applications for new DMCA exemptions. It will likely extend the Sony rootkit exemption (although expanding it to downloads/DVDs), and will also likely approve the exemption expansion request for academics to use DVD ripping software for classroom use. All of the other requests will be turned down.
• The transition to digital TV will be a giant trainwreck. Politicians from all sides will rush to point the finger and blame the FCC, and in particular, (by then) former Commissioner Kevin Martin.
• Senator Herb Kohl's investigation into text message pricing will go nowhere, the carriers will not drop prices, and the class action lawsuits will be thrown out of court.