Scaling Twitter redux--the ESB should be your best friend

Scaling a message-based system is a common enterprise problem. Web 2.0 needs to get a little enterprise-like to make things work better.

As we Twitt-iots sit around bemoaning the fact that we can't send each other useless junk on a flaky service, I thought I would take this chance to address the notion that this message-scaling problem is new.

It's not. It's very common, and it can be solved.

Scaling a messaging platform is why IBM sells a boatload of MQ series, why the AMQP protocol was developed, and why JMS is nearly ubiquitous. Pretty much every large enterprise has similar scale issues related to messaging, especially in financial services. But they don't have downtime, and if they did as frequently as Twitter, the people behind them would all get fired.

This is a topic I actually know something about. (Disclosure: my company develops an open-source enterprise service bus, or ESB, called Mule.) All of our use cases involve some kind of complicated messaging architecture, whether it be Web-service based, publish and subscribe, one to many, direct connection, etc. And most deal with data transformations and a vast array of protocols.

In my view, what Twitter needs is to adopt a bus-type of architecture that separates the transport from the application and uses a middleman to process the transactions. This is a very common enterprise scenario that needs to be applied. This is what an ESB does.

One example in the Web 2.0 world is OpSource, which is delivering billions of transactions a day as an SaaS provider using the ESB at the crux of the transactions. This is much larger than Twitter and has actual revenue impact. Maybe OpSource can host Twitter?

About the author

Dave Rosenberg has more than 15 years of technology and marketing experience that spans from Bell Labs to startup IPOs to open-source and cloud software companies. He is CEO and founder of Nodeable, co-founder of MuleSoft, and managing director for Hardy Way. He is an adviser to DataStax, IT Database, and Puppet Labs.


Discuss Scaling Twitter redux--the ESB should be...

Conversation powered by Livefyre

Show Comments Hide Comments
Latest Articles from CNET
How your next 'flagship' smartphone may cost less than $300