Why open source search engines don't have a chance

If you've been following the tale behind Wikia, you would know that Wikipedia co-founder Jimmy Wales has something up his sleeve that he believes will revolutionize the way we all search the web.

If you've been following the tale behind Wikia, you would know that Wikipedia co-founder Jimmy Wales has something up his sleeve that he believes will revolutionize the way we all search the web.

According to the company, Wikia has four main components: the indexing of the Web, developing a search engine application, an algorithm, and using people to help filter sites and rank results.

And while the first three are pretty standard, it's the fourth component that has me shaking my head. One of the highest costs associated with starting a search engine is buying enough servers and creating enough software that will allow a service to continually crawl the Web for just about anything it can (or is allowed to) find. But in order to solve this cost issue, Wikia is hoping people from around the world will log onto Grub and install Web crawling software that will do the necessary crawling work during the computer's idle time. So far, about one thousand people have signed on; Wikia is hoping for 100,000.

For some reason, a host of companies are creating these "user-generated" search experiences where companies are giving you the opportunity to filter search results and make the search engine more appealing. Maybe I'm a cynic and a skeptic, but this idea is pure rubbish. And to make matters worse, some of these companies actually believe an all-out destruction of Google is possible. I think they're kidding themselves.

Google has successfully created the best search engine by spending money and knowing what is best. And while you may not always find exactly what you're looking for when you search Google, do you think it would be any better with your peers filtering the results for you?

The only place I can see this user-generated, human-filtered service working is in a niche market. For example, Wikia, in an attempt to bring users in on the searching, will be releasing its own algorithm in the hope that people will improve it. Does the company honestly believe it will help on a general search query like 'computer'?

There are too many factors involved when getting you the best search results. But when you search for something that's detailed and only a select few actually care about it, that's when the new version of search engines will shine. The reason for this is simple: niche enthusiasts will modify the results in such a way that only the best results for that topic will come up -- it's easy, there's not much on that topic. And after all, who would know better about the Wabash Star Trek convention than those people putting it on and attending it?

Another issue I see with human-generated results is the issue of popularity. Who's to say that website A offers worse material or insight just because it doesn't command the same kind of popularity as website B? Sure, I can hear it now: well the people will know that and they will promote that site. Don't be so naive -- popular sites will take the day because more people like that site. It's as simple as that.

And just for the sake of argument, how would the computer illiterate come into play? Would they even know about Wikia? Would they know that they can download Web crawling software to help make the results more effective and they also have the ability to work with the general population to make search results better? My guess is they couldn't care less. There are still people using dial-up even though broadband is available in their area. It's not because they like slow speeds, it's because they have something that works well-enough for them and there's no reason to change. The same will hold true for Wikia, Mahalo and the rest of these new searching experiences.

I applaud Wikia and the rest for making a valiant effort at dethroning Google as the leader in Web searching. But if these companies really want to make a splash, the employed tactics need to be thrown out in favor of better algorithms and a unique experience. That said, these search engines could work in niche markets and there's no reason why each of these companies shouldn't go after those -- there's still a good amount of money to make in those areas.

Google is here to stay and whether we want to admit or not, tailored search results and edited content will not unseat Google as the best search engine on the Internet.

Tags:
Smart Home
About the author

Don Reisinger is a technology columnist who has covered everything from HDTVs to computers to Flowbee Haircut Systems. Besides his work with CNET, Don's work has been featured in a variety of other publications including PC World and a host of Ziff-Davis publications.

 

ARTICLE DISCUSSION

Conversation powered by Livefyre

Don't Miss
Hot Products
Trending on CNET

Hot on CNET

The Next Big Thing

Consoles go wide and far beyond gaming with power and realism.