All the news that robots pick

The Web search giant was hit with a lawsuit from French news agency Agence France Presse, forcing it to start to pull thousands of photos and news stories from its service. Then critics lashed out over its decision to include reports from National Vanguard, a publication that espouses white supremacy. In response, Google said it will remove the publication from its index.

Both are black eyes to Google's theory that computers virtually unassisted by human editors can pick the top stories of the day and beat traditional media at its own craft.

Google's own description of the service, which is still in beta after three years, defies the two instances that cropped up this week: "Google News is a highly unusual news service in that our results are compiled solely by computer algorithms, without human intervention."

The tensions hit on the growing pains of changing news consumption and distribution. On the one hand, readers are eagerly using aggregation services like Google News to save time and find news they're interested in from one location. But the digital melting pot of news also has raised questions about the need for standards that go beyond what technology can provide.

"It's a searchable newsstand, and it's a wonderful source," said Janice E. Castro, director of Graduate Journalism Programs at Northwestern University's Medill School of Journalism and former editor of Time.com. "But you're used to being able to say, 'There's the good newspaper; there's the poor stuff.' In search, it's all the same color and all the same size, and it's not ranked by quality."

"The best is mixed up with things that are far from the best," Castro said.

Google's feet are being held to the fire because it uses its technology to mine the depths of the Web to compile news. Yahoo News, in contrast, searches for news but also forms partnerships with content providers to populate its service. Google declined to comment on whether it has licensing deals with content owners.

In addition, Google News and similar news aggregation sites have become considerably powerful, forcing news organizations like the AFP to rethink their purpose and news distribution strategies. An increasing number of people turn to search as a way to access news, and many publishers have failed to answer

John Battelle, a Web search pundit and former publisher of The Industry Standard, said that Google is an object of concern for publishers because it has yet to form a business model for its aggregation service, as opposed to Yahoo.

"That creates fear, uncertainty and doubt around their true intentions with the product," Battelle wrote in an e-mail, though he does not believe those intentions are "evil."

Visitors to Google News have nearly doubled to 5.9 million visitors since February 2004, according to ComScore Media Metrix. Yet Google News is not as popular as New York Times Digital, CNN, AOL News or Yahoo News, the leading news destination online.

Google uses algorithms to find popular news of the day and to cluster different sources on a given story, with links and photos from various publishers. But behind Google's technology, the company has pre-selected roughly 4,500 sources of information, and it continually reviews new sources to include in its searchable collection.

The question on many critics' minds is, what standards does Google use to select a news source?

"We're demanding transparency of mainstream news. Well, it's high time we get transparency from Google News," Jeff Jarvis, a blogger and president of Advance.net, wrote on Buzz Machine.

Jarvis added: "Google: Release a complete list of your news sources now. And institute a means for questioning those choices and for suggesting other choices now."

The call for transparency was in response to revelations that the National Vanguard was included in Google News' index. And according to the blog HonestReporting.com, Google News has previously included Jihad Unspun, a Web site that publishes anti-Semitic content.

Google spokesman Steve Langdon said the company does not allow hate content into its news service. "If we are made aware of articles that include hate content, we will remove them," he said.

The company has several guidelines for choosing news sources, including ensuring that the publication is edited. But it does not detail those guidelines on its site, except to say that "news sources are selected without regard to political viewpoint or ideology, enabling you to see how different news organizations are reporting the same story."

Aggregators vs. publishers
Google is also facing dissent from at least one of its news sources. Last week, AFP sued Google for allegedly using its news articles and photos without authorization. The French company is suing for $17.5 million in damages and seeks to permanently bar Google from using its materials.

Despite Google's policy to remove content at a publisher's request, AFP sued the company for past damages. Most publishers, however, want to be included in Google News because they believe it is a benefit to them and their readers, Langdon said.

AFP's complaint charges that Google infringes on its copyright by reusing its story "leads" as well as the headlines and photos.

Fred von Lohmann, an attorney at the Electronic Frontier Foundation, said a legal precedence has been established that allows Web publishers to link to thumbnail images, however. He also said the use of headlines and excerpts from the lead of a news story is fair use, and believes that Google is in the right.

"It would be a different World Wide Web if you had to ask for permission before you linked to something, and the same thing applies to news," Lohmann said.

Still, Google could face more of these lawsuits and pressure to engineer a more transparent news service.

"There's this weird tension," said Eric Goldman, assistant professor at the University of Marquette. "On the one hand, they need to tighten up who's included in their index, but then on the other hand, if they're too tight, someone is going to zip by them with hotter fresher news."