Hadoop: For those who actually want to get work done

Sometimes people open source code simply to be cool. Other times they do it to be useful.

I loved this post in The Register about Doug Cutting and his Hadoop open-source project. I've written about Hadoop before and the vision it shows on Yahoo!'s part, but El Reg is having none of that, castigating companies and projects like Twitter that release parts of their software as open source for media, not business, effect:

Twitter, which is widely accepted as the drum major of the Web 2.0 failure parade, released an open source project called Starling in January of this year. Starling is the Ruby-based messaging system that runs Twitter's backend. Yes, Twitter, the nonprofit web service known widely for its downtime, dropped its disaster-producing xxxxpile on the world. Why? Maybe they thought more competent developers would fix their problems. The more likely scenario is that they wanted to get a [a boost] from the fake tech media to make themselves look more important. I am guessing this is why no code has been released for Starling since it was open sourced. Oops.

Twitter decided they would be cute and trendy. They wrote their code in Ruby: the official state language of the hipster-developer nation. Doug Cutting, on the other hand, decided he would get xxxx done, and wrote Hadoop in Java. Starling was hidden away in some corner and forgotten (it's hosted at RubyForge...). Hadoop lives prominently at the Apache Software Foundation. Starling is a re-hash of an existing Java Enterprise API called JMS that has several open source implementations. Hadoop is an implementation of Google's MapReduce, a system that publicly only existed on paper. Hadoop has the added benefit of actually working.

Winner? Hadoop. Sometimes open source is cool because it's cool. Other times it's cool because it's useful (but hard). I like the latter kind.

Please head over to The Register and read the full article. One of the best I've read in a long while, the language notwithstanding. It's a great reminder to those of us in the open-source world prone to hubris, and it didn't even come from Savio. :-)

Featured Video