The emergence of applications-managed storage

Yesterday, I heard a story about a CIO of one of the big NYC-based financial services firms explaining to his audience what was delaying further virtualization of the firm's application environment. The reason wasn't related to hardware or software.

The problem was staffing. Given the current state of virtual server management practices, the number of virtual machines was outrunning the number of people needed to effectively manage them.

IT has traditionally been about automating things. But automated IT management, and particularly automated management of the storage environment, has traditionally been approached cautiously at best by IT administrators. Nevertheless, a number of technologies are now emerging that will facilitate automated data and storage management in a way that is linked directly to the needs of the application environment.

Storage administrators, if they so desire, will soon have the ability to automate storage and data management process that are tied to a real-time awareness of the applications they serve. Think of this as applications-managed storage. Think of applications-managed storage as one way to manage more virtualization with proportionately fewer people.

The storage array controller, in addition to interpreting I/O commands and RAID stacks, has long been used as a platform for data and storage management services. However, with the advent of virtual server operating systems like Microsoft Hyper-V and VMware vSphere, these and other data management services are now being offered by the OS layer. The OS layer can now serve as a platform for storage management services. A growing number of different types of vendors--storage, server virtualization, and systems vendors--are beginning to deliver products that manage storage from the vantage point of the OS layer. Why? A virtualized OS can "see" up into the application layer and down into the storage layer. It has a simultaneous view of both worlds.

But the virtual OS isn't the only link between applications and storage available. Specialized network traffic processing chips are being developed that can, among other things, render the needed application awareness. Fiber optic sensors inserted between the server and storage layers could also be used to develop application-to-storage awareness.

As such, application-aware storage management services that are guided by both historical and unfolding events taking place at the application layer can be used to automate a response from the storage layer based on those events.

There is also a growing list of management APIs provided by applications and server virtualization vendors. Oracle now offers APIs for its Oracle Enterprise Manager. VMware has delivered a list of APIs that includes those for vCenter, vMotion, and Site Recovery Manager (SRM). Microsoft offers APIs for Hyper-V and Systems Operations Manager. On the storage side, storage vendors have long used device specific APIs to allow for management of disk arrays by storage resource management applications as well as the SNIA-developed Storage Management Interface Standard (SMI-S). As a result, management tools, sometimes referred to as "plug-ins," that connect the application environment to storage management services through these APIs are appearing.

It is no longer a given that networked storage can only or will only be managed by storage architects and administrators. Those in the IT administrative group with a more generalized IT background will also have an increasing ability to manage storage, including (dare I say it?) database administrators.

Emerging tools, through applications-managed automation, will allow existing staff to accomplish more data and storage management, and as a result, be more productive in ways that matter the most to applications users.