Off-the-shelf supercomputing

So he built his own.

Warren wired together 140 off-the-shelf computers with Digital processors, loaded them with the Linux operating system, connected them with ordinary Ethernet equipment and added 35.8GB of memory. Now he's got a machine that can perform 48.5 billion calculations per second.

Code-named Avalon, Warren's laboratory special could rank among the top 100 on the list of the world's 500 fastest computers. A smaller, 70-processor version of Avalon ranked number 315 as of last June.

The total price tag is $313,000--dirt cheap for a supercomputer.

In a way, Avalon is to supercomputers what Linux is to operating systems: a do-it-yourself, get-your-hands-dirty solution that turned out to be as powerful as commercially available counterparts.

Warren has been interested in building his own supercomputer since he was an undergraduate. As processors grew faster and faster, it was only a matter of time before it was possible "to put one of these things together out of off-the-shelf parts," he said.

While it's a remarkable achievement, there's more to a supercomputer than its hardware, said Jerry Sheridan, an analyst with Dataquest. "A processor doesn't make a supercomputer. There are other elements: reliability, availability, processing power."

Sheridan is impressed with the Avalon achievement, "but I would get more excited if it was a new technology and somebody was going to [offer] it commercially or offer it in a wide variety of configurations," he added.

Meanwhile, Ben Passarelli of Silicon Graphics--which bought the famed Cray Research supercomputing company--doesn't believe commercial supercomputers are going away.

Avalon's architecture works for computationally intensive tasks, but it has its limits, Passarelli said.

The worst problem: "You're limited to things that have a small I/O [input/output] requirement," he said. "That rules out anything where you're trying to analyze or pore through vast amounts of data," such as a million point-of-sale transaction records.

Unsurprisingly, Passarelli favors SGI's architecture, in which the computer uses a single shared memory. Such a system is more flexible so it can tackle a wider variety of problems, he said.

Avalon uses 140 533-MHz Alpha 21164A chips, each in its own box with 256MB of memory. Each box has a 100 MB/sec Ethernet connection to a 3Com Ethernet switch, so each processor can "talk" to any of the others. And each box runs Linux, the upstart version of Unix written by Linus Torvalds and other unpaid programmers.

The Avalon designers researchers chose Linux chiefly because the operating system's freely available source code can be customized, explained Dave Neal, another Los Alamos researcher who helps to run Avalon. "Mike's really tweaked the kernel," slicing out the features Avalon doesn't need. In addition, archived Linux discussions make it possible to ferret out the source of most problems.

"The other part of it is the damn thing is free," Neal added.

Linux also is very robust, Neal said. He's in charge of 80 workstations used for various scientific purposes, and 60 of those run Linux. "From a scientific computing point of view, it's kind of a godsend," Neal said. "Linux is a force to be reckoned with."

Warren and his colleagues have been using Avalon for specialized scientific calculations, such as performing astronomical calculations. They presently have 60 users who get to sign up to use half the machine for 24-hour time slots. The problems the researchers have tackled bear on pretty technical subjects: unstable periodic orbits, spatiotemporal chaotic systems, nonlinear Schodinger equations.

Avalon's architecture isn't for everyone, he cautioned. "If you prefer a shared memory programming model, Avalon is not going to suit you at all," he said. "Avalon isn't meant to compete with commercial supercomputers that can solve a huge variety of problems."

Nonetheless, these types of computers seem to fill a gap in the market. House-brand supercomputers will likely appeal to universities who can use it not only for getting a lot of horsepower on the cheap. Such systems can also be used as a way to teach students parallel programming techniques.

Of course, there's no saying that more adventurous types couldn't bring it to market.

"I'm waiting to see the first person who uses one of these as a monster Web server. You can make the machine look like one very big, very fast Web server."