Search engines try to find their sound

To be Googled, or not to be Googled?

That was the question facing National Public Radio's online director Maria Thomas earlier this year. The answer would seem obvious for anyone doing business on the Web, where being included in search results can mean the difference between success and oblivion.

But in Thomas' case it wasn't that simple: Most "spiders" that crawl and index the Web are effectively blind to audio and video content, making NPR's highly regarded radio programming all but invisible to mainstream search engines.

Indexing files by looking at their audio features is still a work in progress for big search engines, including Google. So NPR eventually hit on a plan to instantly turn audio broadcasts into text files that can be recognized and picked up by search engine spiders.

"Our site is primarily full of rich audio, and we want people to find it when it's relevant," Thomas said. "The big search engines' technologies don't have the ability to get inside the audio or video. With the little bit of text we have on NPR, it's not always good enough to find our content, and reference the page."

Consumers armed with broadband connections at home are driving new demand for multimedia content and setting off a new wave of technology development among search engine companies eager to extend their empires from the static world of text to the dynamic realm of video and audio.

The stakes are enormous, not just for the search engines, but for content owners hoping to harness the Internet, stand out in the online information glut and attract new audiences. The winning search companies could become the gatekeepers in a new era of media increasingly defined by consumers' ability to seek out programming on their own terms and consume when and how they want.

NPR is not the first company to bend over backward for recognition on the search engines, which drive the bulk of traffic to hundreds of thousands of Web sites. But it may be the first broadcaster to adopt a guerrillalike strategy for insinuating its audio in the search indices. And the company's strategy is working so far: In recent weeks, NPR audio has begun regularly appearing on the index pages of Google News and Yahoo News, and clips also crop up when people search for news-related keywords, such as "Abu Ghraib," the name of the notorious prison in Iraq.

Thomas did not divulge specific traffic figures, but she said that since stories started showing up in results churned out by major search engines, the NPR site has seen record spikes in visitors for high-interest news stories, such as the murder of American hostage Nick Berg in Iraq.

Sound and fury signifying as nothing
NPR's move points to the limitations of Google and Yahoo at a time when broadband Internet connections are becoming more popular among consumers, fostering new demand for multimedia content. Publishers are increasingly adding exclusive audio and video content for online access; educators are streaming courses online; broadcasters are bringing vast archives online in digital form; and small-time publishers are finding it cheaper and cheaper to create, produce and host multimedia on the Web. Yet you wouldn't find any of it on the primary search engines.

That's created an opportunity for specialty search engines focused on filling the gap. Already, technology companies including Singingfish, StreamSage, Hewlett-Packard, Virage, Nexidia and others have emerged to address some of the challenges. Yahoo and AOL are players, too. Yahoo owns AltaVista, which has one of the Web's oldest audio and video search engines, but so far, Yahoo has not sought to feature the technology. America Online, another dark horse in the search race, bought Singingfish earlier this year.

"There's a tremendous upsurge in the amount of streams available on the Internet," Singingfish General Manager Karen Howe said. "Because of broadband adoption, which is so strong in the enterprise and in the home, accessing high-quality content now is not such a pain for the user." Singingfish fields about 6 million searches a day, up from 3 million in January, and records about 80,000 new streams a day.

Yahoo's and Google's drawbacks could underscore the need for even more specialty search engines or prompt advancements from the status quo. Unleashing new features almost daily, the major search engines are in a race to win the hearts and clicks of Web surfers with their search tools, largely because it means more advertising revenue in their pockets. And with Google's upcoming $2.7 billion IPO, the competition could get even more heated, launching the two companies into new realms of rivalry.

The technologies that Yahoo and Google rely on today are focused on mining text on the Internet for content relative to keywords. Among other techniques, they analyze the interconnectedness between Web pages and examine the headers and anchor text on a page so that they can return appropriate Web sites for any keyword, or set of keywords a surfer does a search on. Even images--an increasingly popular source for Web searches on Google and Yahoo--are tied to text that defines the pictures.

Similarly, some search engines now analyze text or keywords that describe a multimedia file's content, or what's called metadata. Singingfish, for example, relies on 70 fields of descriptive material about a file--such as author, bit rate, file size--to catalog it, but the company regularly runs into shortcomings in such information.

Others transcribe portions of the audio or video and then analyze the language for meaning, topics of conversation, and relevance to a search term.

Most ambitiously of all, a handful are bent on searching inside the files to extract meaning and relevance by examining audio and video features directly.

Weeding the garble
StreamSage is starting to make waves with its audio and video search technology, introduced late last year. The Washington, D.C.-based company developed software after roughly three years of research that uses speech recognition technology to transcribe audio and video. It then uses contextual analysis to understand the language and parse the themes of the content. As a result, it can generate a kind of table of contents for the topics discussed in the files.

The downfalls of this method are that it can be extremely difficult to be 100 percent accurate. In fact, experts say the language-detection technology is typically only 80 percent accurate. Language hurdles such as accents, jargon and dialect can trip up the technology, for example.

StreamSage introduced a Web site called CampaignSearch.com last week to showcase its technology. The site lets people search audio and video files on the Web for clips from the presidential candidates, including files on sites such as Whitehouse.org, CSPAN Voice of America and others.

"It's a timely demonstration of StreamSage's technology," said Seth Murray, the company's president.

For example, the technology can dissect an hour-long speech from Democratic presidential nominee John Kerry to find a segment in which he talks about health care, and earmark that 4 minutes of the broadcast for access.

StreamSage has flown under the radar during its last four years of operation while it has invested heavily in research and development. Its chief scientist, Tim Sibley, is known for his work in computational linguistics. StreamSage has received funding from research grants, including the National Institute for Standards and Technology's Advanced Technology Program. Harvard University uses StreamSage's technology to allow medical school students to search past lectures on related subjects. AOL is using the technology to provide closed captions for streaming video and audio on AOL Broadband.

NPR is using StreamSage to transcribe its audio programs as they're broadcast, thereby helping them to get listed faster. NPR does commission transcripts for many of its programs, but the traditionally manual process of transcription would be too slow for a search related to timely news. Using speech recognition technology, StreamSage can create text from audio much more quickly, and then feed those transcripts to Google and Yahoo.

NPR's Thomas said her outfit eventually replaces the transcripts from StreamSage with those from humans because the human-rendered records much more accurately reflect the audio and video content. StreamSage's results can be garbled.

NPR also licenses technology from Singingfish to meticulously label its audio files with relevant information, or metadata.

In its own first step toward offering multimedia search, Mountain View, Calif.-based Google registers NPR audio files on Google News, its specialty news aggregation service. A search for a headline topic that is discussed on audio-only NPR programs, such as Talk of the Nation, will uncover a link to the audio program and the specific segment covered, for example.

A Google representative confirmed a relationship with NPR but declined to comment further on the technology. Up until now, Google has not listed multimedia files because the company has sought to avoid the legal uncertainties of indexing and linking to copyrighted works that owners may want protected, company executives have said in the past. Beyond those reasons, audio and video file searching can be a much more difficult technical task to solve than cataloging the Web.

StreamSage's Murray said he's not worried about potential copyright issues because his company is not housing the information. Rather Streamsage points people to audio and video around the Web, just like Google or Yahoo does.

Exactly how far search engines can go in linking to multimedia files has yet to be worked out definitively in the courts. The recording industry last year quietly settled a long running dispute with MP3Board.com over alleged illegal links to music files without any money changing hands, said MP3Board's attorney, Ira Rothken.

Yahoo also announced a relationship with NPR, in February, when it outlined its "content acquisition" program, a systematic effort to include more hard-to-get information in its searchable database.

An NPR affiliate in Boston, WBUR.org, is using similar technology from Hewlett-Packard, called Speechbot. Robin Lubbock, director of new media for WBUR, said the broadcaster is using Speechbot to translate audio into text so employees and visitors can search for content on its own Web site.

Virage, which is now owned by Autonomy, has technology that analyzes in-stream audio and video and lets people zip to the part of the stream they want. Yet it can be an expensive enterprise solution.

Jay Webster, chief technology officer of interactive agency Fathom Online, said that for most audio or video broadcasters to get ranked in search engine results, they would have to employ some manual indexing of their own first.

"Where it gets cool is if you could search on any keyword and find it within audio and that audio would come up in search results," Webster said. "But I don't think we're there yet."