Tried and true beating out InfiniBand

InfiniBand, which requires the industry to adopt new hardware and software, was perhaps just a bit too revolutionary for most computing companies to adopt. Companies needed a better system than the existing PCI technology for jobs such as plugging in network cards. But InfiniBand lost out to faster versions of PCI--the last nail in the coffin being PCI-X 2.0, which this week entered final review by the companies developing it.

Although some in the industry sigh with relief that a comparatively mundane technology will suffice, the episode is a cautionary tale about overinflated expectations. The lesson is especially clear for server manufacturers, which must choose technology for systems still years in the future; the long lead times are necessary to accommodate conservative customers intolerant of the disruptions imposed by frequent hardware and software changes.

"InfiniBand is not going to be a PCI replacement. The industry recognized that about nine months ago," said Tom Macdonald, general manager of Intel's Advanced Components Division. But there still will be InfiniBand products. "It's not hundreds of millions or tens of millions, but it's a significant opportunity."

InfiniBand was the product of a battle of industry giants pitting Intel, Dell Computer and Sun Microsystems against IBM, Hewlett-Packard and Compaq Computer. After months of wrangling, the companies called a truce--resolving their standards battle--and coined the flashy new name.

The technology, though yet to go from demonstration mode into being included in real-world products, will be used in data centers packed with computers and storage devices, as originally planned. But it no longer is expected to become ubiquitous, industry analysts and companies say.

"We're in the reality phase right now instead of the hype phase, and that's always a bit of a downer," said IDC analyst Vernon Turner.

If InfiniBand fans had tried to push the technology for desktop computers as well as servers, it might have had a better chance, said MDR/Instat senior analyst Cary Snyder. "That was a strategic mistake. Because of that, they're relegated to more of a niche area," he said.

Easier said than done
When InfiniBand arrived on the scene, the consortium pushing it said the specification would "lead to an input-output architecture that will eventually replace existing shared-bus I/O." In other words, it would replace PCI.

But the trade group that governs PCI has just extended its life span another few years. First came PCI-X, which doubles the transfer speed of PCI and is just now shipping in volume. And this week, the group governing PCI said another generation, PCI-X 2.0, will mean an even longer life span for the standard.

The companies developing it are now giving it a final review. PCI-X 2.0 will double and then quadruple the speed of PCI-X.

"It's a no-brainer. It just makes so much sense," said Kimball Brown, vice president of business development at ServerWorks, a company that manufactures the chipset that's key to linking CPUs to computer subsystems such as memory or the PCI data pathway. "We think (PCI-X 2.0) will be the standard interconnect coming out of servers for many years to come."

ServerWorks, whose chipsets are used in servers from all the major Intel server sellers, should have its first PCI-X 2.0 chips ready by the end of the year, Brown said.

"It's incredibly difficult for entirely new, revolutionary schemes to ever get a toehold," said Insight 64 analyst Nathan Brookwood. Designers repeatedly worry that a design has run out of steam, but "then the technologists find a way to extend it and extend it and extend it."

PCI-X, just now arriving in today's servers, can transfer 1.07 gigabytes of data per second, said Roger Tipley, president of the PCI special-interest group that governs the specification. PCI-X 2.0 pumps data faster for each tick of the clock, with the PCI-X 266 standard at 2.1 gigabytes per second and PCI-X 533 at 4.3 gigabytes per second.

PCI-X 2.0 will be good for connections to 10 gigabit-per-second Ethernet networks, high-speed hard disk controllers and forthcoming 10 gigabit-per-second Fibre Channel connections to storage networks, Tipley said.

But the longevity of PCI-X 2.0 means it's overlapping with another version of PCI called 3GIO, for third-generation input-output.

Intel and other supporters believe 3GIO will coexist with PCI-X. Intel's Macdonald expects that beginning in 2004, 3GIO will start catching on in desktop and laptop computers, which will be able to use 3GIO for plugging in data-hungry graphics cards as well as network cards.

Tipley said the 3GIO specification provides an initial transfer speed of 0.5 gigabytes per second that will increase to 1, 2, 4 and 8 gigabytes per second over the years.

ServerWorks' Brown believes server makers will stick with what works for the time being, and that means PCI-X, not 3GIO. "3GIO will come, but it still has to prove itself in the marketplace," he said.

Uses for InfiniBand
InfiniBand still has several possible uses. For example, it's good for setting up high-speed connections between servers and to storage systems--for example, joining computers into "clusters" that can share loads and keep running even if some individual machines fail.

The clustering role is becoming more important with newer versions of database software from IBM and Oracle that spreads the database across several smaller servers instead of one gigantic, expensive machine. But that method is comparatively untested, and corporate customers are conservative.

Other companies, such as IBM and InfiniBand chipmaker Mellanox, see InfiniBand as a way to make super-slim blade servers even slimmer by separating computer components into different cabinets. InfiniBand would join CPU modules with storage systems and adapters to connect to the network.

But compared with the technology's earlier glory days, those are niche applications.

"In the enterprise, it'll be there, but as far as it being everywhere like it could have been at one time, it isn't," Snyder said.