Search sites work to clean up their act

The commercial practices of search engines are once again in the spotlight after a recent warning shot from federal regulators over inadequate disclosure of paid links.

The Federal Trade Commission sent a letter late last month urging several Web sites to provide conspicuous labels for commercial search listings or face possible action, and minor changes are already rippling through search destinations, including Terra Lycos, Ask Jeeves, LookSmart, America Online and Microsoft's MSN.

But the reforms are unlikely to go beyond relabeling, at least for now, leaving untouched the myriad arrangements that have transformed many search engines from Web library catalogs to the online equivalent of the Yellow Pages.

"Everything is for sale," said Dana Todd, founder of interactive agency SiteLab. "As a consumer, I'm appalled; as an advertiser, I'm delighted."

The tug-of-war over search engine marketing comes as advertisers increasingly turn to the medium as one of the most effective ways to reach their target audiences on the Internet--a trend that has been eagerly embraced by Web sites that have seen other forms of ad sales wither.

The trend has raised concerns that the public might be misled about the editorial independence of search listings, which have frequently been promoted as unbiased research tools.

At the center of the controversy are two types of for-fee programs common among the search providers: paid placement and paid inclusion.

In a nutshell, paid placement allows an advertiser to pay for higher ranking or prominence on a results page, relative to a keyword search.

Overture Services, the pied piper of commercial search, lets marketers bid for placement in search results, paying a set price for each click. It licenses results to portals, including Yahoo and MSN--which split sales with Overture--and the listings typically appear at the top or bottom of results pages.

The handsome profits culled from the system have drawn many followers. Portals such as MSN and Yahoo sell their own placements to beef up revenue. Search site Google added an auction-style ad program to complement its natural results, which set it into rivalry with Overture. The company has won accounts with AOL, EarthLink and Ask Jeeves, previously held by Overture.

But the rampant practice drew barbs last year from Portland, Ore.-based Commercial Alert, which said the sites misled consumers by promoting paid links as "recommended" or "featured" sites. The FTC quickly chimed in. Companies named in the complaint were AltaVista, AOL Time Warner, Direct Hit Technologies (owned by Ask Jeeves), iWon, LookSmart, Microsoft and Terra Lycos. A copy of the recent FTC letter was also sent to Overture, Yahoo, InfoSpace, About.com, Google and Disney.

Now, many sites, including Terra Lycos, LookSmart, Altavista and Ask Jeeves, are conforming by choosing a more explicit term for commercial results appearing throughout their pages.

"Response to the FTC letter has been positive," said Danny Sullivan, editor of industry newsletter and Web site SearchEngineWatch.com. "Language on most of the search provider sites has all shifted to 'sponsored.'"

Others have also complied or are in accordance with the FTC's guidelines. AOL updated its Netscape search results with the word "sponsored." Yahoo had already used the term for its commercial links.

In contrast, MSN promotes "featured" sites, which include links to advertisers or partners of the online portal, as well as sponsored sites.

In an e-mail, MSN Product Manager Parul Shah said the company has prominently placed links to an explanation of its results that clearly disclose sponsored and non-sponsored results. She added that MSN plans to update its search pages in September to add additional explanatory links.

Paid inclusion under fire
The other major form of search engine advertising under the FTC review is so-called paid inclusion. Although this practice is less well known than paid placement among search engine users, it is no less popular among marketers, and may raise more troubling questions when it comes to sorting out its effects on actual search results.

"Paid inclusion gets more complicated because the programs vary a lot," said Dean Forbes, an attorney with the FTC's division of advertising practices.

"If all of the sites in the search engine's results were paid inclusion then that should be disclosed because that makes it an ad medium," he said.

Paid inclusion largely pertains to "organic" search engines such as Inktomi, AltaVista and Fast Search and Transfer's AlltheWeb, which provide technology that scours the Web and uses mathematical algorithms to compile relevant results. Under financial pressure, many such sites developed programs to guarantee companies that they would "crawl" or search a Web address more often, for a price.

Such programs have been growing in popularity in the past year.

Inktomi, for example, last year started Index Connect, which lets small and medium-sized sites pay a flat, annual fee to have their Web addresses regularly indexed. Larger sites, such as Amazon.com and eBay, that want assurance that more than 1,000 addresses are indexed pay on a sliding scale, depending on the category. These sites can pay anywhere from 15 cents to $1 each time a Web surfer clicks on a hosted page--which is akin to the Overture model.

Others to introduce similar programs include AltaVista, Ask Jeeves' Teoma and Fast's AlltheWeb, which plans to begin a paid inclusion service in September. InfoSpace, which operates Excite.com, just this week introduced a new paid inclusion program that lets marketers have their Web site content refreshed in its index every 48 hours.

Google delivers unbiased search results from a vast index of Web sites; the company does not accept fees for cataloging sites.

Full disclosure?
Companies that use paid inclusion are taking steps to better disclose the practice in the wake of the FTC's letter last month.

Fast's AlltheWeb publicly announced support for the FTC's request and it added a link, called "what's included," to describe its paid inclusion program. AltaVista now includes a tiny link near its results pages called "about," which leads to a disclaimer that describes how companies can pay to have their sites visited more frequently.

Teoma also will include an "about" link in coming weeks that is close to search results. LookSmart this week plans to add a pop-up link that outlines how commercial listings can appear in a directory of site reviews.

Gary Ruskin, executive director of Commercial Alert, called the changes a boon to consumers that will aid in the editorial integrity of search services.

"Many of these search engines have near zero editorial integrity," Ruskin said. "If (listings) are for sale, the public ought to know; then they are not search engines but ad delivery mechanisms."

In the future, analysts say, paid inclusion could become the subject of further inspection.

"The issues are very complicated, and the FTC has brought us closer to some clarity," said SearchEngineWatch's Sullivan. "But it may very well be a first step."

One area of concern for Web site owners is that the search providers could artificially keep their indices stale to promote the for-fee program. The question some ask is what's the incentive to buy into a search index if the technology is already visiting all of its pages every week or two? But if search providers let themselves grow outdated, they face rivals at every turn.

Inktomi, for example, recently increased the frequency and depth of its search engine to "crawl" about 2 billion documents on the Web every two weeks--a benchmark that rivals Google and Fast. Just a week ago, it searched only about 500 million documents about every month, according to Vish Makhijani, vice president of Web search at Inktomi. Fast in the last week announced that it is searching 2.4 billion documents.

All of the companies say that the paid indexing does not affect the ordering or relevance of search results served to consumers and they comprise, in general, less than 5 percent of the total results.

"At the end of the day we're an editorial (service); we have to strike a natural balance with the business aspect," Makhijani said. "We lose distribution if we are compromising the results for commercial benefit."