Twitter search is broken

In the Webware 100 this year, both Twitter and Twitter Search are finalists, in separate categories. (Vote here and here.) I added the search feature of Twitter as a product because it's a research tool quite separate from the social/publishing network of Twitter itself. In some ways I think it's even more important.

But Twitter Search has been failing me. It's still good when the result set is small. Last winter, when I was looking for tweets on road conditions over the Siskiyou Pass, it was useful. But try using it to see what's happening on a popular topic (like "iPhone"). What a mess!

The problem with Twitter Search is that it's purely time-based. Even though you can create fairly complex and specific search queries with the Advanced Search feature, the results you get are not sorted except by time.

What we need is for Twitter Search to also take relevance into account when displaying results. What is relevance? Timeliness is a part of it--a big part. But let the sort order be influenced by some other data and you might get something incredible.

At the very least, Twitter Search could be more relevant if it (optionally) displayed only results from the people a user was following, or if it ranked more highly those tweets that had been retweeted or linked to from elsewhere on the Web. Or better yet, make the highest-ranked results those from users who are typically retweeted or linked to the most.

You can't discount the importance of timeliness, since Twitter Search is unparalleled at taking the pulse of the Web moment-to-moment. But as a research tool it needs more finesse. And while I realize that adding features like these (as options, please) would undermine the brutal simplicity of Twitter Search, they'd also make it more useful.

Google, of course, figured this out long ago. Search is about relevance. You have to determine what real humans are paying attention to if you want to give them information they can use. There's no universal solution to this, so Google constantly tweaks its PageRank algorithm. That shows how important ranking technology is.

I've been using Twitter Search a lot at work, and it's been by turns fantastic and fantastically frustrating. It's one of the reasons I'd love to see Google get hold of Twitter's data stream, either through a partnership or acquisition. Until then, for a few tips and links for more effective Twitter searching, turn the page...

Twitter Search tips

If searching for a person who's on Twitter, search for the username, e.g., @rafe. You can also search for tweets directly to or from people with to:@username and from:@username.

Use Advanced Search. There are very good options to refine a query, especially by date, which can be a huge help. You can also type in advanced queries directly. See the Search Operators page for help.

See also How to search Twitter smarter from CIO.

To find Web pages linked to from Twitter users, check out the beta Twitter search feature in OneRiot.

I use TweetDeck to monitor multiple searches at a time--very useful for tracking real-time news. Other multicolumn Twitter clients, like Nambu, Seesmic Desktop, and PeopleBrowsr, also allow you set up and monitor multiple searches. Sideline, a Yahoo experiment, is also worth a look.

And here's one that's not yet out but looks promising: Tweetprobe, which provides search analytics from the Twittersphere. It should be useful for marketers and brand managers. There will be free and pro versions, and I hope to get a demo version to try in a week or so.