Cloud platforms of the future: Hadoop and Eucalyptus

Without a doubt, the cloud and all its forms and meanings were big news in 2008. Besides the huge growth of Amazon EC2 and Google App Engine, we saw Salesforce launch Force.com, a true platform-as-a-service.

My picks for the most interesting software of 2008 are Hadoop and Eucalyptus.

Hadoop is an Apache project, the "open source implementation of MapReduce, a powerful tool designed for the detailed analysis and transformation of very large data sets," which basically means you can process a ton of data on commodity hardware.

Hadoop is going commercial through Cloudera and while details are not publicly available, let's just say there are some very important and interesting foundations being laid for the way that people deal with computing and processing power.

Eucalyptus is an "open-source software infrastructure for implementing 'cloud computing' on clusters. The current interface to Eucalyptus is compatible with Amazon's EC2 interface, but the infrastructure is designed to support multiple client-side interfaces. Eucalyptus is implemented using commonly available Linux tools and basic Web-service technologies making it easy to install and maintain."

In layman's terms, Eucalyptus makes a group of Linux boxes act very similar to Amazon EC2, which means someday soon you will be able to run an internal cloud.

The fact that both of these platforms are open source speaks to two aspects of the evolution of the cloud.

• There is an appetite for software that is more bleeding-edge or like an "invention" to be open source as enterprises (and developers) want to be able to tweak it to their needs.
• Fewer and fewer companies are willing to plunk down huge dollar amounts for something that may/may not suit their needs. It will be difficult to justify spending anything in 2009, let alone for software that you haven't proven already.

The cloud will continue to evolve, but these two projects and the respective commercial efforts behind them will be interesting to watch.

Disclosure: I am an advisor to Eucalyptus.