Open-source Lucene threatens Microsoft, Google enterprise search

As the open-source search project Lucene grows in credibility and commercialization, it poses a clear and present danger to the proprietary search businesses of Microsoft, Google, Autonomy, and more.

It must be depressing to be Microsoft these days.

You spend $1.2 billion to acquire enterprise search leader FAST in January 2008 and then another $100 million on semantic search vendor Powerset in July 2008, only to have the excellent Apache's Lucene, an open-source search project, and Solr, an enterprise search server based on Lucene, offer better performance at a 100 percent discount.

Not very sporting of the open-source community, now is it?

Granted, Lucene and Solr still lack some of the spit and polish that Microsoft FAST, Autonomy, Google's enterprise search appliance, and other proprietary competitors offer, but this isn't slowing its adoption. As CMS Watch's Kas Thomas notes, interest in Lucene and Solr is skyrocketing, as measured by job postings (among other data):

Indeed.com job postings suggest high interest in Solr and Lucene

This was OK when Lucene stood alone, relatively rough and unadorned by Solr. But Solr makes Lucene much more palatable to enterprises that worry about getting mired in Microsoft (or Autonomy or Google or...), particularly with the uncertainty and unrest prior to its acquisition that may have led to an employee brain drain at FAST, and interest in Solr is improved further by the arrival of Lucid Imagination, a company started in 2007 to commercialize Lucene and Solr.

I talked Wednesday with Marc Krellenstein, co-founder and CTO at Lucid Imagination, and learned that while Lucene and Solr have been doing exceptionally well on their own, Lucid Imagination is in pole position to help advance the development of these open-source projects by offering dedicated development resources and to make a solid business for itself in the process.

Just having a company associated with Lucene and Solr may already be enough to get enterprises off the fence and behind the search project.

According to Krellenstein, Solr delivers significant performance improvements over proprietary alternatives. The goal is to continue to improve its functionality, which currently has roughly 80 percent of the total functionality of rival search products while also advancing innovation to surpass these rivals. The company and project have been making steady progress in these areas.

Did the Lucene/Solr community just upend billions of dollars in Microsoft, Google, Autonomy, and others' investments in proprietary search? Time will tell, but $1.2 billion for FAST is looking mighty expensive compared with the $0.00 Microsoft could have paid for Lucene.

Unfortunately, that's what its customers may be thinking, too. Like the U.S. intelligence community, for starters, which is now standardizing on Solr/Lucene. Microsoft and its peers must be hoping this will remain confidential.

Not a chance.

Disclosure: I am an adviser to Lucid Imagination.


Follow me on Twitter @mjasay.

Tags:
Tech Culture
About the author

    Matt Asay is chief operating officer at Canonical, the company behind the Ubuntu Linux operating system. Prior to Canonical, Matt was general manager of the Americas division and vice president of business development at Alfresco, an open-source applications company. Matt brings a decade of in-the-trenches open-source business and legal experience to The Open Road, with an emphasis on emerging open-source business strategies and opportunities. He is a member of the CNET Blog Network and is not an employee of CNET. You can follow Matt on Twitter @mjasay.

     

    ARTICLE DISCUSSION

    Conversation powered by Livefyre

    Don't Miss
    Hot Products
    Trending on CNET

    Hot on CNET

    CNET's giving away a 3D printer

    Enter for a chance to win* the MakerBot Replicator 3D Printer and all the supplies you need to get started.