Amazon APIs as cloud standards? Not so fast
Some people consider Amazon's APIs to be the de facto cloud computing API standard. I don't agree for several reasons--not the least of which is that it's too early for cloud standards in the first place.
commentary Last month, as a part of the Structure conference in San Francisco, I had the privilege of moderating a panel on the subject of hybrid cloud computing. The panelists were some of the early pioneers of cloud computing, and included the likes of Marten Mickos, CEO of Eucalyptus; Michael Crandell, CEO of Rightscale; and Joe Weinman, vice president of Strategy and Business Development at AT&T.
At one point during the conversation, Mickos made a statement to the effect that Eucalyptus sees the Amazon Web Service APIs as a "candidate for a standard much like the PC standard of the '80s," the latter having enabled innovation of operating systems and applications thanks to a single hardware standard shared by a wide variety of computer manufacturers.
Ellen Rubin, vice president of products and founder of cloud gateway appliance vendor CloudSwitch, wrote a post in response to Mickos' claim that critically examined whether or not it is possible for the Amazon APIs to be a standard today:
If there were an industry standard, Amazon certainly has a strong claim for it. They're the clear leader, with technology second to none. They've made huge contributions to advance cloud computing. Their API is highly proven and widely used, their cloud is highly scalable, and they have by far the biggest traction of any cloud. So full credit to Amazon for leading the way in bringing cloud computing into the mainstream. But it's a big leap from there to saying that Amazon should be the basis for an industry standard.
I've heard the decree that the Amazon APIs are a "defacto standard" for about a year now, first from Simon Wardley (who was at Linux vendor Canonical at the time) and later from others including cloud blogger and Enomaly CTO Reuven Cohen.
While I agree that the AWS EC2 and S3 APIs are today the market-leading infrastructure-as-a-service APIs if measured by number of users, I agree with Ruben that that position doesn't necessarily equate to a long-term standard. There are three reasons for this:
The Amazon APIs define only one set of cloud features: Amazon's. Yes, Amazon is by far the leader in cloud server and storage services, and yes, they have by far the strongest ecosystem supporting their capabilities. However, EC2 is actually a strictly defined server and network architecture that leaves little room for innovation in distributed application architectures and infrastructure configuration.
Now if the development world figures out how to overcome all performance, availability, and security issues using this architecture, that's not such a big deal. However, there is a reason that enterprise infrastructure architectures evolved the way they did, and there are many who believe that the shortcomings of Amazon's architecture will limit its effectiveness for a variety of applications.
In fact, the one area where Amazon EC2 has caused headaches for some people is in the realm of I/O perfomance. Both network and storage access has proven to have unpredictable latencies. One lead developer of a well known open-source database told me he has looked to take some clients to other cloud environments because of this issue.
Similarly, with storage, you get a basic set of features that Amazon has enabled for you. You can obviously deploy any database you'd like on EC2, but that means you won't be using the "standard" Amazon API. I'm not sure that's the whole idea here.
Do four vendors a standard make? In reality, while many vendors and developers consume the Amazon EC2 and S3 APIs, only four vendors actually deliver them: Amazon, Eucalyptus, Cloud.com, and newcomer Nimbula. While it is impressive that these products (two of which are open source) would see the EC2 APIs as the mark to shoot for, it is important to keep my previous point in mind. What these projects have done is define themselves as feature-compatible with AWS, which again makes them right for some workloads, but not so much for others.
Any standard has to start somewhere, though, so one could look at this as the beginning of what will become a "must have" API set, at least for flat topology, high scale cloud environments. There is a problem, though. Amazon hasn't done anything to ensure the API is in the public domain and legally available for commercial resale. All three of these companies are in some sense playing with fire.
The good news is that most (and maybe all?) can replace the AWS API with some other API option with relative ease. However, from a marketing perspective, they'd have to start over again with promoting the new API as the new standard.
It's just too early for almost anything to be a cloud standard. The truth is, nobody in this industry can accurately predict today what the next 10 years of computing will look like. What architectures will "win"? What edge cases will be critical to the function of commerce? What new technologies will change the core requirements of the cloud computing operations model?
In a recent blog post, Enomaly's Cohen revised his position on the AWS API, and points out another key factor hindering its declaration as a standard: today, does anybody other than product vendors really care about portable cloud APIs? Most users of the cloud are, in fact, focusing their efforts on one cloud vendor or another. They have yet to feel the pain of converting key operations code due to a change in cloud vendor. It just isn't a high-demand aspect of cloud computing right now.
This may be further supported by the lack of buzz around the apparent breakdown of the Open Cloud Computing Interface standard effort in the Object Grid Forum. After a disagreement regarding licensing, it appears that a key member is taking critical intellectual property and parting ways. There really hasn't been much outrage expressed by the general cloud user community, which tells me the standard wasn't that important to them at this point.
There are plenty of reasons to desire a consistent, standard cloud computing API. However, there are also many reasons why it is just premature to declare a single winner--or any winner, for that matter. As good at the AWS EC2 and S3 APIs are for their respective contexts, they are just popular APIs, and not yet anything that can be declared a de facto standard for the entire cloud community.