Google protects its search results

Emoungu is not alone. This month, about 100 Comcast subscribers were temporarily shut out of Google when the search company charged the high-speed Internet access provider with hosting some accounts that had abused its terms of service by performing "automated queries." The crackdown cut a wide swath, taking out a block of IP addresses, shutting down the guilty and innocent alike.

"We are not accusing you personally of having violated our Terms of Service," said Google's notice to Emoungu, a computer programmer who has been a subscriber to Comcast's high-speed Internet service for two years. "You are most likely an innocent victim of someone else's bad behavior. We're really sorry to have had to take this action."

Lockouts are hardly unprecedented on Google. The company has long banned computer-generated search requests, which can sop up substantial system resources and help unscrupulous marketers manipulate its search rankings. But the latest clashes hint at an escalation in a war that may see many more Web surfers caught in the crossfire.

At stake in the battle is the purity of Google's search results, which are widely viewed as among the most relevant for any given keyword on the Web. That reputation has made Google an Internet darling, drawing large crowds of surfers to its site and accolades for its technology.

A survey this week from Web researcher OneStat.com showed that Google drew the largest global search audience on the Web, with nearly 46 percent of surfers using its site. Web portal Yahoo came in second with 20.6 percent usage, and Microsoft's MSN followed with 7.8 percent reach. America Online's search service was seventh on the list, attracting 1.8 percent of Web users worldwide.

Success has made Google a target. It faces competition from a raft of search newcomers offering twists on its successful formula such as Ask Jeeves' Teoma and LookSmart's WiseNut. In addition, marketers and others are eager to game the most popular and influential Web search tool around.

Tricks of the trade
Much of the conflict to date has raged over search engine optimization services and software that explicitly aims to push certain Web sites up the rankings for specific key words. Such tools may use a variety of techniques, such as "link spamming" to improve rank placements and automated queries to judge the success of such campaigns.

The tactics play on Google's ranking methods, which use links as a kind of vote to vouch for the authority of a given Web site. Since Google's search method automatically indexes billions of Web pages, the ranking system is crucial to ferreting out the most valuable sites for specific keywords.

FirstPlace Software's WebPosition Gold, one of the most popular search optimization tools, has served as a flashpoint in the debate. It has attracted a range of customers interested in inflating their site rankings.

The software helps create "doorway pages" that can be used to stack Web pages with popular keywords as a way to drive surfers to other pages that have little or no relation to those search terms. The software also reports Web site ranking and tracks the number of visitors to a site--a feature that can run users afoul of Google's ban on automated queries. Among the search engines that WebPosition Gold analyzes include Yahoo, Google, Overture, WebCrawler, Excite and AltaVista.

FirstPlace Software President Brent Winters said that although Google does not like the automated queries, businesses have "an inherent right" to know what their rankings are to measure their business results. He said the Missouri-based company tries to educate its customers as to what the search engines consider spam, but it does not have control over what a customer does with the software.

"You can buy a hammer to build a house or you can buy a hammer and go smash your neighbor's window," Winters said. "So we can't control how people are going to use the tool. We just try to educate people to use it properly and to not abuse the search engines and to keep queries to a minimum."

Winters said his company has attempted to work out negotiations with Google on multiple occasions but that the search company has "not been all that cooperative." He said Google has not responded to WebPosition Gold's proposals, such as working together to reduce the number of queries by the average user or to shift more queries to off-peak periods.

When asked about Google's negotiations with software companies such as WebPosition Gold, Google declined to comment, saying only that automated queries slow its system.

"It can place a demand on our servers and can slow down Google as a service for the millions of millions who use it everyday," said Google spokesman Nathan Tyler.

Juicing results
The Comcast automated query crackdown comes as Google faces increasing attacks on the objectivity of its search results from link spammers, whose ranks now include legions of casual self-publishers who have turned the practice of driving sites to the top of keyword rankings into a competitive Web sport.

While these attacks are not always associated with automated queries, they are part of the larger challenge that Google is confronting as it seeks to keep its results clean.

Danny Sullivan, an editor with SearchEngineWatch.com, said people have used links for years to help promote Web sites. But as time progressed, Google and other search companies found that people were creating artificial links that existed solely to fool search engines. Sullivan said an ambitious link spammer can create hundreds of Web sites and interlink them for the sole purpose of pushing up its ranking on Google.

As a result, Sullivan said he wasn't surprised that Google shut down some Comcast customers because of the automated queries. He said it's been a "huge issue" for the search engines because it takes up a lot of time and resources to keep "junk" out of their listings.

"There are lots of people who still successfully run automated queries against Google," Sullivan said. "While it violates their terms and conditions and while they absolutely do not like it, it's more trouble than it's worth to try to shut them down."

Automated queries and link spammers are not the only thorn in Google's side.

Other tactics that people use to manipulate their Web site's rankings include the use of cloaking software, which lets people create customized pages meant to please a search engine's algorithm. People may also create multiple Web sites that are similar to each other, hoping that one will stick with a search engine.

Another fast-growing tactic used to boost a Web site's rank is called "Google-bombing."

Adam Mathes, a 22-year-old Stanford University student studying computer science, discovered what's considered to be the first Google bombing campaign last April. He said the concept of Google bombing was that "it was concerted effort for lots of people to link a specific page with specific terms" in an attempt to make that page show up among Google's top-ranked Web sites. He found that Google not only shows you pages that have the exact keywords, but also it will show pages that are linked to that page with those words.

Mathes said he discovered this when he was playing a joke on a friend. He said when he searched the words "Internet Rockstar," his friend's Web site, BenBrown.com, would be the first hits on Google. Mathes said at the time, his friend's site did not have those terms on it, but a lot of other sites linked to him with that term did. So he launched the so-called Google bombing campaign by linking his other friend, Andy Pressman, with the term "talentless hack." Mathes said the campaign was effective because in the first couple of months, Pressman was the first hit for the words "talentless hack" on Google.

Mathes said he choose Google because it was "the premier search engine among the geeks" a year ago. He said he still has respect for Google, but "in the end, it's just technology and all these search engines can be manipulated somehow."

The clearest losers
Regardless of intent, such attacks may have raised the stakes for Google, whose reputation for fending off manipulators of all kinds is under more stress than ever.

In this battle, ordinary Web surfers will be the clearest losers, caught between a possible decline in the quality of search results on one side, and aggressive spam policing on the other.

Google's enforcement tactics include denying service to blocks of IP addresses when it cannot track down a specific abuser. The company's notice this month to Comcast users, for example, said it had shut off access to its services because "some person or people" had violated its terms of service agreement.

Google declined to comment about the service denials, saying that the company does not discuss details it undertakes to protect it from spam or unauthorized abuse.

"We're very clear in our terms of service about key areas that we define as inappropriate use," said Google's Tyler.

Comcast also declined to discuss any details but said it was "an isolated incident" that happened April 5, affecting less than a hundred people. A company representative said the incident was "resolved very quickly," lasting no more than a few hours.

Vince Bernardo, a Comcast customer caught in the crackdown, said he immediately contacted friends and co-workers who used Comcast in South New Jersey and they confirmed having the same problem.

"I've set Google as my home page, and it's a very critical part of my Internet life," Bernardo said in an e-mail to News.com. "It's an invaluable research tool for leisure, general curiosity and work."