Devops to grow with cloud, data services (Q&A)

Devops is a relatively new concept that centers around the interdependence of development and operations and has been on the rise in the Web 2.0 world of virtualization and the cloud. The characteristics of devops include concepts like "architect, developer, tester, product manager, project manager--all in one" and "ability to write code beyond simple scripts" all working toward an ideal of managing infrastructure in an automated fashion.

One of the players in this market is Luke Kanies, founder and CEO of the Puppet Labs, which provides support and service to users of the Puppet open-source server automation tool, and is hosting its Puppet Camp 2010 next month in San Francisco. (Disclaimer: I serve as an adviser to Puppet Labs.)

I asked Kanies some questions about devops, automation, systems management, and the cloud.

Q: Give us a brief overview of the rise of devops and why it matters?
Kanies: Devops is essentially a cultural movement toward more development-like operations. First and foremost this means acknowledging and impressing the fact that your infrastructure is code, so you should be using developer tools and practice to maintain and interact with it. It also means that you should have the same requirements of your infrastructure as you do of your applications--you need an API, high quality data, version control, access control, auditing, and more.

The reason it matters is that the problems of IT have outstripped its ability to deal with them--our tools and practices largely aren't built for a world where you can turn up 36,000 machines in a week or deploy 1,000 machines an hour, nor where your boss can expect full deployment of an application across thousands of nodes in seconds or minutes rather than weeks or months. Devops attempts to fix these problems with a culture and practice that adopts and adapts development tools in the infrastructure and builds a culture of delivery and agility.

There is a fairly small universe of products addressing the specific devops needs--is this just an early market or hard to figure out?
Kanies: I think it's mostly because devops is a cultural movement more than a technical shift. In the same way that there aren't many Agile Development tools, there won't be many devops tools. In that same way, however, you can expect a bevy of consultants and books on the topic.

With the explosion of the cloud, there has been a lot of talk about cloud management. With your sysadmin background, how does cloud management differ from traditional on-premise IT systems management?
Kanies: In my experience, it's mostly about expectations and rate-limiting steps. The technology isn't fundamentally different from the virtualization technology we were using years ago, but now we expect to be able to turn up hundreds of machines a day, and we expect no portion of the process to take more than a few minutes. We've got customers who demand the ability to turn up 500 machines in 15 minutes, and this just wasn't a reasonable demand 5 years ago.

I think the biggest difference most sysadmins will experience--and by "most" here I definitely exclude the leading-edge, devops, ruby-wielding, git-using ninjas, but rather the typical corporate sysadmin--is that they can no longer just be faster than Dell or Red Hat, they have to be faster than VMware, even if it's not reasonable. This means a lot of assumptions and excuses have to be let go, and people have to be willing to see their department as a service provider, just like the cloud providers.

That last bit, service, is a very important point to me--IT has come to be at the heart of most critical corporate initiatives, but the cloud can allow companies to push IT into a service provider role, rather than a central role. This can empowering, but it can also be threatening, and it's up to the technologist to pick a side.

Where do you see data center automation going in next 12 to 18 months?
Kanies: It's clear that the automation tide has turned--you just can't do a cloud or cloud-like deployment without it. The big question now is how much value the organization can get from it. It's sufficient for an automation solution to just make IT more agile and effective, but it leaves a lot of value on the table. If you can go further and provide self-service interfaces to the application owners--such as for application deployment and upgrades--along with vast amounts of data in a useful presentation layer to executives, then you've gone from being a cost center to a competitive advantage.