Cloud platforms of the future: Hadoop and Eucalyptus

Open source is driving development of the cloud. My picks for the most interesting software of 2008 are Hadoop and Eucalyptus.

Dave Rosenberg Co-founder, MuleSource
Dave Rosenberg has more than 15 years of technology and marketing experience that spans from Bell Labs to startup IPOs to open-source and cloud software companies. He is CEO and founder of Nodeable, co-founder of MuleSoft, and managing director for Hardy Way. He is an adviser to DataStax, IT Database, and Puppet Labs.
Dave Rosenberg
2 min read

Without a doubt, the cloud and all its forms and meanings were big news in 2008. Besides the huge growth of Amazon EC2 and Google App Engine, we saw Salesforce launch Force.com, a true platform-as-a-service.

My picks for the most interesting software of 2008 are Hadoop and Eucalyptus.

Hadoop is an Apache project, the "open source implementation of MapReduce, a powerful tool designed for the detailed analysis and transformation of very large data sets," which basically means you can process a ton of data on commodity hardware.

Hadoop is going commercial through Cloudera and while details are not publicly available, let's just say there are some very important and interesting foundations being laid for the way that people deal with computing and processing power.

Eucalyptus is an "open-source software infrastructure for implementing 'cloud computing' on clusters. The current interface to Eucalyptus is compatible with Amazon's EC2 interface, but the infrastructure is designed to support multiple client-side interfaces. Eucalyptus is implemented using commonly available Linux tools and basic Web-service technologies making it easy to install and maintain."

In layman's terms, Eucalyptus makes a group of Linux boxes act very similar to Amazon EC2, which means someday soon you will be able to run an internal cloud.

The fact that both of these platforms are open source speaks to two aspects of the evolution of the cloud.

  • There is an appetite for software that is more bleeding-edge or like an "invention" to be open source as enterprises (and developers) want to be able to tweak it to their needs.
  • Fewer and fewer companies are willing to plunk down huge dollar amounts for something that may/may not suit their needs. It will be difficult to justify spending anything in 2009, let alone for software that you haven't proven already.

The cloud will continue to evolve, but these two projects and the respective commercial efforts behind them will be interesting to watch.

Disclosure: I am an advisor to Eucalyptus.