Google News turns 8 amid news industry in flux
A boon and a bane to news sites and bloggers, the heavily trafficked news aggregator is drawing readers but needs to adapt with a quickly changing industry.
MOUNTAIN VIEW, Calif.--Krishna Bharat, founder and engineering head of Google News, was stuck in New Orleans at a conference in the days after September 11, 2001, and like so many others desperately searching for news about the attacks on New York and Washington, D.C. Not only was it hard to find the most authoritative information online, it was hard to find information from differing points of view.
"(There was) a lot of time to think about current events and news and trying to get out of there," Bharat said in a recent interview with CNET. "I came back here and said, 'I know how to extract content from news sites, how can I make this process of understanding much more improved?'"
It was that frustration that led to the eventual creation of Google News on September 22, 2002. It aggregated online news content from around the world and quickly became popular and controversial, as some news publishers complained about what they perceived as an attempt to draw traffic away from their own Web sites and others pondered the ranking criteria.
The rancor of that debate has quieted, but Google News is facing new challenges. The nature of what we consider to be "news" is rapidly changing as a new generation of journalists and media companies born entirely of the Web flood it with written, spoken, and visual content both in traditional story form and through microblogging sites. Meanwhile, readers are demanding more control over the sources of their news, eroding that original goal of diversity.
Bharat is convinced that Google can engineer its way through these challenges. "The future of Google News is better personalization and better social input," he said, implying a future (some might argue it's already here) where the most relevant and authoritative content is that recommended by friends and tied to one's preferences.
Leading the way
In developing the first edition of Google News, Bharat built an algorithm by mid-October 2001 that organized news stories based on freshness and the number of news outlets that were covering a particular story. It turned out that most news companies were presenting their content in a similar way, so Bharat was able to use a lot of the work from his previous gig indexing retailers' sites for product search, which he said was a much more complicated problem.
The news process, however, had to be approached differently from regular Google search because of the nature of the content: at the time, fresh news stories posted minutes ago could take a long time to show up in regular search results since few if any sites were linking to those Web pages. Google's overall search algorithms now surface fresh content in regular search results very quickly. But in late 2001, Google started experimenting with a standalone news "onebox"--Google's term for a section of a search results page dedicated to one type of result--powered by Bharat's ranking science.
Google launched the aggregation page the following September. Google declined to share numbers regarding traffic growth for this story, but did say that the site is currently driving 1 billion page views to news publishers' Web sites per month and Google currently operates 72 separate editions of Google News in 30 different languages.
Bharat and his team made several tweaks over the years to add new features and make the shift from a static site published once every 10 minutes to a site that is updated almost instantaneously with the influx of news. The most recent change, a user-interface redesign, was controversial among die-hard Google News users but core to how Bharat and Google see the news business evolving.
"You're balancing the need to inform with the need to engage," Bharat said, which is actually a line that could have emerged from any veteran newspaper editor: in other words, ideally you'd like to serve some news vegetables along with dessert. But Google, raised in the church of data, thinks that the best way to provide that balance is by automatically tracking which stories are most popular among news editors by observing how many different organizations report on a topic, and blend a mix of which stories are being clicked on the most by readers.
"We can't sit there and editorially decide what's good and what's bad," Bharat said. "The only scalable solution is where we tap the intelligence of others."
Printing the news that fits
There are two trends in the evolution of the news business that will keep Bharat busy over the next eight years. One is the rise in "news spam," SEO-bait articles often written by low-paid freelancers that are designed mostly to surface within Google, rather than inform, educate, or entertain readers with coherent writing.
So-called "content mills" like Associated Content and Demand Media are churning out short news-related pieces of content by the thousands in hopes of driving traffic to their sites. Google appears to be of two minds about this issue: on one hand, Google loves free and openly-published content that it can index and surface within search (and therefore serve ads against) and feels that news quality can be a very subjective thing. However, it also wants to provide a quality product in Google News that will keep people coming back to the site, for much the same reason.
Bharat implied that Google is working on a way to refine the signals it uses to rank news stories in a way that filters out the most egregious examples of news spam without branding certain companies as offenders because of certain stories. "What we are very sensitive to is user experience, but we don't want to be anecdote driven, we want to be sensitive to statistically relevant feedback," he said.
There's also the challenge of living up to the original intent of Google News: providing news from different points of view. Bharat, a native of India, has been surprised at the lack of international news that most Americans receive and created the product in hopes of changing that. But the media world is an increasingly partisan place, which has led Google toward giving individuals a way to manage their own home page.
While Google feels it has to provide those controls in order to best serve its users, it's also working with the news industry on things like Fast Flip and Living Stories to present important and engaging content in a unique way in hopes of exposing those users to ideas and concepts they may not have found by limiting their sources of incoming news. It has also embarked on those projects in hopes of convincing members of the news industry that Google has come in peace, and not to destroy their businesses, as some have alleged.
Bharat, a self-described "news junkie" has tried with Google News to create "a mirror" of the news ecosystem, which means it by definition surfaces all the good and the bad that emerges from the daily news cycle. That reflection has changed dramatically over the past eight years and seems likely to change ever further as the news media shifts from the offline world to the online one.
Google's ability to harness that world will be tested by such a shift. Google search is driven by the company's desire to provide the right answer to a given query. Selecting the "right" news story is a far more difficult and potentially dangerous process.