Google warns entire Internet is malware

Updates at 9:10 a.m., 9:45 a.m., 10:30 a.m., 10:40 a.m., 11:25 a.m., and 12:15 p.m. PST: Google's and StopBadware.org's numerous responses added. Rewrites have been made throughout to sum up the issue.

For about an hour on Saturday morning, Google listed every site on the Internet as malware.

After the initial problem was fixed, it took a couple of hours to iron out who actually was to blame--Google or a nonprofit known as StopBadware.org.

TechCrunch and CNET reported around 7 a.m. PST that every site found via Google search was flagged with this message: "This site may harm your computer." As part of Google's malware protection, clicking on a flagged site's link would pull up an additional warning. Although a link could simply be cut and paste, Google's warning was unnerving enough to keep some people from pushing their luck.

Twitter was awash in the news, with thousands of people posting about their kindred experiences on Google search.

In a blog posting just after 9 a.m. PST, Marissa Mayer, Google vice president of search products & user experience, attributed the problem to "human error" and to a URL list provided by StopBadware.org. But about 30 minutes later, a blog posting on StopBadware.org disputed her explanation. An hour after that, Mayer posted Google's mea culpa.

Below is Mayer's 9:02 a.m. PST posting, with her 10:29 a.m. PST update folded in. Her update acknowledges that StopBadware.org did not provide the wrong information and that it was solely Google's fault. In her update, Mayer wrote: "This post was revised as more precise information has become available."

Note: The sentences that Mayer removed in her update are noted with strike-outs and brackets. The sentences she added in her update are in bold. Here is Mayer's explanation:

If you did a Google search between 6:30 a.m. PST and 7:25 a.m. PST this morning, you likely saw that the message "This site may harm your computer" accompanied each and every search result. This was clearly an error, and we are very sorry for the inconvenience caused to our users.

What happened? Very simply, human error. Google flags search results with the message "This site may harm your computer" if the site is known to install malicious software in the background or otherwise surreptitiously. We do this to protect our users against visiting sites that could harm their computers.

[We work with a non-profit called StopBadware.org to get our list of URLs. StopBadware carefully researches each consumer complaint to decide fairly whether that URL belongs on the list. Since each case needs to be individually researched, this list is maintained by humans, not algorithms.]

We maintain a list of such sites through both manual and automated methods. We work with a non-profit called StopBadware.org to come up with criteria for maintaining this list, and to provide simple processes for webmasters to remove their site from the list.

[We periodically receive updates to that list and received one such update to release on the site this morning.]

We periodically update that list and released one such update to the site this morning.

Unfortunately (and here's the human error), the URL of '/' was mistakenly checked in as a value to the file and '/' expands to all URLs. Fortunately, our on-call site reliability team found the problem quickly and reverted the file. Since we push these updates in a staggered and rolling fashion, the errors began appearing between 6:27 a.m. and 6:40 a.m. and began disappearing between 7:10 and 7:25 a.m., so the duration of the problem for any particular user was approximately 40 minutes.

Thanks to our team for their quick work in finding this. And again, our apologies to any of you who were inconvenienced this morning, and to site owners whose pages were incorrectly labelled. We will carefully investigate this incident and put more robust file checks in place to prevent it from happening again.

Mayer's update followed several blog postings from StopBadware.org manager Maxim Weinstein. StopBadware.org, which is coordinated through Harvard's Berkman Center for Internet & Society, doesn't partner only with Google. Its other partners include PayPal, VeriSign, Trend Micro, and Consumer Reports WebWatch.

After Weinstein read Mayer's initial explanation, he asserted that her posting was wrong. At 9:31 a.m. PST, he wrote on the nonprofit's blog:

Google has posted an update on their official blog that erroneously states that Google gets its list of URLs from us. This is not accurate. Google generates its own list of badware URLs, and no data that we generate is supposed to affect the warnings in Google's search listings. We are attempting to work with Google to clarify their statement.

About 10 minutes later, Weinstein updated the post with this:

Google is working on an updated statement. Meanwhile, to clarify some false press reports, it does not appear to be the case that Google has taken down the warnings for legitimately bad sites. We have spot checked a couple known bad sites, and Google is still flagging those sites as bad. i.e., the problem appears to be corrected on their end.

In an e-mail to CNET News at 10:08 a.m. PST, Weinstein reiterated that "Mayer's explanation was inaccurate. She has informed me that Google is working on an updated statement to clarify the facts."

In StopBadware.org's defense, Weinstein added:

Google scans websites to identify sites that may be dangerous to users. When it finds such sites, Google issues warnings in the search results. This morning, they inadvertently added these warnings to nearly all websites, causing user confusion.

StopBadware.org does not provide the data for these warnings. Our role is to use the data provided to us by Google for research/analysis, to support/assist webmasters in cleaning up sites and navigating the review process when their sites are clean, and to provide a third-party review when users hit roadblocks with Google's own process.

Our site was taken down this morning as a result of extremely heavy traffic due to the Google glitch, which led many users to seek additional information. As StopBadware.org is mentioned on the Google warning page that users see when they click on a search result that Google has flagged as bad, many people associated the warnings with us.

In a follow-up e-mail at 11:14 a.m. PST, Weinstein wrote that he was satisfied with Google's corrected response.

"I believe Google's updated statement accurately clarifies that Google does not receive the URL data from us and that we were not involved in this morning's glitch," he wrote.

What a way to start the weekend...

Note: I am in no way related to Maxim Weinstein.