Storage software takes center stage

Specifically, the Ohio-based financial services company was looking for the right program to ensure that data stored on mainframes was being effectively duplicated, so it would be backed up in case of disaster.

IBM happened to have a product, called Extended Remote Copy, that would do just what the bank needed. That pushed IBM to the top of the list and led to a deal in which KeyBank eventually purchased some 31 terabytes of storage while Big Blue agreed to manage the bank's backup facility.

KeyBank's experience is becoming increasingly common as more corporate buyers worry less about how much storage they need and more about how to handle what they've got. As a result, storage companies are beginning to change their focus from just increasing the size and performance of their storage systems to offering software that can make the gear more adaptable and easier to manage.

Major hardware vendors such as EMC, Hitachi Data Systems, IBM and Hewlett-Packard are pouring investment into management software. And in most cases, it's going into developing applications that can control competitors' storage arrays. The sector also is becoming something of a hotbed for start-ups, two more of which announced their formation on Monday.

HP, for instance, now spends more research dollars on storage software than on hardware, said Mark Sorenson, vice president of HP's storage software division. EMC says that it spends three-quarters of its research dollars on software and that the bulk of its recent acquisitions have been in that area.

A time for change
The impetus for the change comes, in part, from a narrowing of opportunities in hardware. Storage gear has improved enough that all of the leaders offer products good enough to handle most tasks--hampering the companies' ability to charge more for new technology.

"As the storage industry has matured, the storage arrays across the industry (have become) pretty darn good," Sorenson said. "They are all pretty fast, and they are all pretty reliable."

Another reason for the shift is that storage providers see a move to software as a way to maintain their profitability as market pressure intensifies for them to lower prices for disks.

That's not to say that some of their customers aren't still taking the traditional route of buying the latest and greatest hardware and then finding compatible software.

"I don't think the software is driving the hardware just yet," said Nancy Marrone, a senior analyst at Enterprise Storage Group, a Milford, Mass.-based storage consultancy. But Marrone said experiences like that of KeyBank foreshadow a day when software preferences begin to dictate which hardware a storage customer purchases.

Customers are starting to make their preferences heard in at least one area--storage management software.

Although disks are cheap, the work force needed to manage the storage arrays is not. It can be a strain for companies to add more capacity, as most of the tasks associated with storage are still done manually--for example, checking how much storage is out there, billing other departments for usage and adding new storage.

This makes automation attractive. There is a clear market, analysts say, for management software that can allow storage administrators to focus more on designing their storage networks and less on day-to-day maintenance.

"Storage is one of the last big unsolved problems in IT as far as I can tell," said Jeremy Burton, chief marketing officer for Veritas, which is a leader in backup and recovery software and is endeavoring to move further into storage management.

Storage utopia
The storage industry already has a grand vision of how this software should someday work. Companies such as Veritas, IBM and HP speak of a day when management programs are so advanced that a storage administrator won't have to even really know what hardware is out there. Administrators will set up policies for various types of data, and then the management software will automatically find the appropriate hardware and ensure that space is made available for the data.

"That's utopia, right, but we're pretty far from that," said Scott Genereux, a vice president at Hitachi Data Systems. "I don't know if we'll ever get there."

Customers want storage tasks to become more automated, but they need the change to be more gradual, said Chris Gahagan, senior vice president of storage infrastructure software for EMC.

"Data centers can't absorb massive amounts of process change," he said. Solve a customer's most pressing problem, then move on to the next one, he added.

Still, there is clearly a need for software that can help companies get a handle on their storage. Most companies today don't really know all the storage they have or how much of it is in use. As a result--and because storage has become comparatively cheap to acquire--many companies just keep adding more disks.

Hitachi's Genereux cited one recent customer that had 200 terabytes of storage distributed throughout its network. Although only 40 percent of that capacity was being used, the company was on the verge of buying another 100 terabytes.

The amount of corporate data that needs to be stored is growing at 80 percent a year, analysts estimate, but budgets remain flat. This means companies have to make better use of the storage they have. Although hardware makers would prefer to just sell more boxes, they are also working on improving their software products.

One of the first steps along the way is virtualization, which allows people to organize information based on where it fits logically in an organizational structure rather than where it is stored. For example, all of the finance department's files can show up as a single entity even though they might be stored on any number of different disk arrays.

A number of companies are already doing this, and more are following suit. IBM, for example, is bringing out a Linux-based storage visualization engine this year. TotalStorage SAN Volume Controller, formerly known as Lodestone, is designed to give managers a complete view of storage arrays, allowing them to see where they have free capacity and what type of storage it is.

Raising a standard
Also key to offering more powerful software is making sure that programs can talk to lots of different types of hardware.

The storage industry is developing a standard way for hardware to identify itself and to list its capabilities. The standard, which has been years in the making and is just now entering the final stages of ratification is known by a number of names--such as SNIA Storage Management Initiative Specification (SMI-S)--as well as by its former code-name Bluefin.

IBM, Hitachi, Sun Microsystems and Veritas have been particularly strong backers of the standard, although EMC has also pledged to support it throughout its product line.

Next month, the proposed 1.0 version of the standard will be available for public review, according to Sheila Childs, the Legato executive who heads the board of directors of the Storage Networking Industry Association (SNIA), a trade group backing the standard. The release of the finalized standard is scheduled for July, with some products supporting the standard beginning to show up later this year.

The biggest proponents say standards are needed to allow the networked storage industry to really grow. Sun's James Staten notes that it wasn't until the TCP/IP standard gained broad acceptance that the networking industry really took off.

Staten, who is director of software marketing for Sun's storage unit, notes that companies like Cisco Systems and 3Com that aggressively adopted the standard won out in the market, compared with companies like Novell that pushed their own architecture.

That lack of standards, Staten said, is one reason that so many companies buy storage that attaches directly to a server rather than gear that can be part of a storage network.

The lack has also held back innovation on the software side, says Veritas' Burton. Instead of helping automate tasks, many earlier software efforts have centered on trying to talk to all of the incompatible types of storage hardware out there.

However, the industry likely won't feel the impact of having the standard--a unified way for storage hardware to broadcast its capabilities--until at least next year, said Enterprise Storage Group's Marrone.

There is concern in the industry that the standard will offer only basic information to management software. A customer may still need to buy a storage gear maker's proprietary software to take advantage of all the features in its hardware.

Competitive code-share
During the wait for the standard to take hold, hardware vendors have done their own side deals to achieve some piecemeal interoperability. In these cases, the companies swap code, so that rivals' software can access some of their gear, and vice-versa.

Despite the industry's shift toward separating out hardware and software, some customers still prefer to buy a complete storage system from one vendor that's more likely to work well, even if it means higher initial purchase costs and a lack of flexibility down the road.

That's the choice Phoenix-based America West Airlines made when it went with EMC for a storage-area network to manage its back-office data, flight operations and other sensitive information. The airline first went with EMC in 2000 and has continued to add the company's hardware and software as it has expanded its storage network.

"I have a very conservative approach when it comes to infrastructure issues," said Joe Beery, the airline's chief information officer. "Find something that works really well, and leave it alone. You put in what works well, you don't mess with it, and you go on and do other things you need to do to."

Indeed, Bob Sampson, vice president of sales and strategy in IBM Systems Group, said just because software is becoming more important, that doesn't mean customers won't have to continue to ratchet up the speed and capacity of their disk arrays.

"It is not air cover for not having strong products," he said.