Yahoo BOSS + Twitter + Google App Engine = fresh news

A Yahoo engineer has combined a Yahoo search service with Twitter to offer a new way to present news that's both fresh and pertinent.

Yahoo BOSS engineer Vik Singh
Yahoo BOSS engineer Vik Singh Stephen Shankland/CNET News

Here's Web 2.0 at its finest: A Yahoo programmer has combined his own project, Yahoo BOSS (Build Your Own Search Service) , with Twitter and Google App Engine to create a new way to determine what news is both new and important.

The service, called TweetNews, presents Yahoo news search results in a different way, using results from the same search on Twitter to determine what should get high placement, according to a blog posting about it by BOSS engineer Vik Singh.

BOSS supplies Yahoo search results in a form that can be repackaged, processed, and published for free, though Yahoo asks for revenue sharing for popular services.

TweetNews combines human interest, as judged by Twitter users, with a measure of authority, as judged by publications that make the cut for Yahoo News search. The application also includes an expandable "related tweets" button that supplies links to people's Twitter references to the various news stories.

"Twitter as a ranking signal for search freshness may prove to be very useful if constructed properly," Singh said in the blog.

Here's a screenshot of the search in action, using the terms "hudson plane" to illustrate the news items Twitter users find most pertinent.

Vik Singh's TweetNews application shows how Twitter can be used to find the most pertinent breaking news.
Vik Singh's TweetNews application shows how Twitter can be used to find the most pertinent breaking news. Vik Singh/TweetNews/CNET News

The application is publicly available as a service running on Google App Engine--not the first time Singh has demonstrated BOSS ideas on his main competitor's application hosting system . Google hosts applications on App Engine for free, but only within various limits, and Singh's However, the application exceeded its quota within a few hours of his posting.

The application isn't just a novel demo, though. It's an attempt to solve a challenging problem in determining what breaking news is most pertinent to people. Here's how Singh describes the challenge:

Freshness (especially in the context of search) is a challenging problem. Traditional PageRank style algorithms don't really work here as it takes time for a fresh URL to garner enough links to beat an older high ranking URL. One approach is to use cluster sizes as a feature for measuring the popularity of a story (i.e. Google News). Although quite effective IMO this may not be fast enough all the time. For the cluster size to grow requires other sources to write about the same story. Traditional media can be slow however, especially on local topics.

I remember when I saw breaking Twitter messages describing the California Wildfires. When I searched Google/Yahoo/Microsoft right at that moment I barely got anything (

What I found most interesting in both of these cases was that news articles did exist on these topics, but just weren't valued highly enough yet or not focusing on the right stories (as the majority of tweets were). So why not just do that? Order these fresh news articles (which mostly provide authority and in-depth coverage) based on the number of related fresh tweets as well as show the tweets under each. That's this service.

Tags:
Software
About the author

Stephen Shankland has been a reporter at CNET since 1998 and covers browsers, Web development, digital photography and new technology. In the past he has been CNET's beat reporter for Google, Yahoo, Linux, open-source software, servers and supercomputers. He has a soft spot in his heart for standards groups and I/O interfaces.

 

Join the discussion

Conversation powered by Livefyre

Don't Miss
Hot Products
Trending on CNET

HOT ON CNET

Is your phone battery always at 4 percent?

These battery packs will give your device the extra juice to power through all of those texts and phone calls.