Beowulf computing method makes business inroads

The Linux operating system has found another inroad into the business world: a low-cost, high-power, number-crunching method called Beowulf.

Beowulf is a method of ganging lots of Linux-based computers together to tackle heavy-duty calculation jobs. It's sort of a do-it-yourself, low-budget supercomputer that's proven popular at universities and government labs.

Now, the broadening appeal of the technique has led companies such as IBM and Compaq Computer to see Beowulf as a possible product to add to the computing lineup.

The move parallels the adoption of the Linux operating system in other parts of the corporate world. In the space of a few years, Linux has spread from an amateur programming project--created by Finnish student Linus Torvalds and cherished by volunteers--into an operating system popular at universities, then to a credible product that's made its way into server lines at the world's largest computing companies.

"We are very, very interested in this space and are looking at it very hard," said Tom Figgatt, who leads the e-business section for IBM's Netfinity server group. Beowulf Customers come from companies working on pharmaceuticals, oil searching, structural analysis, and visualization of complex data.

And though Compaq is still focusing its Beowulf efforts on the scientific and technical community, the company sees commercial possibilities, such as converting the vast amounts of data in company databases into useful information. For example, "you could pull all the bar-code information from every supermarket in the U.S. and find out where gum is mostly sold," said David Mitchell, a marketing manager for Compaq's Alpha servers.

Linux isn't the only way to tie Beowulf systems together, but it's a popular one, largely because its open-source nature means it can be modified to work as fast as possible.

"We have some government [Beowulf clusters] using Tru64 Unix, because it's optimized to run on Alphas and you get an extra added punch," Mitchell said. "You pay for the licensing per processor, but it's the fastest you can get."

Beowulf interest is starting to grow. Doubleclick is one site that uses gangs of Linux boxes as part of the process of delivering ads to Web sites all over the Internet. It doesn't use Beowulf systems right now, but it's considering using them to replace expensive hardware from Cisco Systems, said Greg Tagaris, director of operations at Doubleclick.

The reason for going to Beowulf? It would simplify the company's network, be less expensive, easier to expand, and cut down on "single points of failure," the equipment that brings down the entire system if it stops working.

IBM whipped together a 37-processor Beowulf demonstration for a Linux conference in March, tied a Cray supercomputer speed record for a complex graphics task, and spurred a multitude of queries with customers interested in the technology, Figgatt said. IBM now has several Beowulf systems running at customer sites, Figgatt said.

Compaq builds Beowulf clusters at its custom assembly section and has several models ready to go, Mitchell said. "Rather than having to buy all the separate components and put it together, you can order a package," he said. Compaq has several configurations based on either Intel or Alpha chips.

While good at number-crunching, Beowulf needs to be easier to use, and that's where IBM will focus its attention, Figgatt said. "Our more commercial customers say I need management tools, workload balancing, and high availability. That's what we'll focus on," he said.

Several smaller players also are involved. Giganet has just brought its high-speed networking hardware over to Linux, and The Linux Store began selling Beowulf systems in June.

Beowulf not a universal cure
While Beowulf systems are good for number-crunching, they're not suited for all server tasks, analysts and industry representatives say.

Beowulf wouldn't be good for a program that executes large numbers of transactions, such as an airline reservation system. It would, however, work for business tasks such as deciding how to design an assembly line or which mix of currencies to buy, said International Data Corporation analyst Dan Kusnetzky.

Also, setting up a Beowulf system and getting software to run on it is tricky, Kusnetzky said. "This is software that is fairly raw," he said, and to use Beowulf, a program has to be split up so pieces can run in parallel on all the nodes of the Beowulf cluster. "That requires quite a bit more expertise than most people have."

Oracle, the No. 1 database software, evaluated Beowulf systems and decided against them, said Bob Shimp, senior director of product marketing at Oracle. "We did have an engineer who looked through all that technology. The decision was that it isn't easily applied to commercial databases," he said.

But there are other ways of ganging together Linux computers that Oracle is investigating, in particular for its Oracle Parallel Server product. "We are looking at a variety of other avenues of clustering technology on Linux," Shimp said. "We'd love to find a way to put [Oracle Parallel Server] on Linux."

One thing holding Beowulf back for nonscientific uses is the difficulty of writing software, said VA Linux Systems chief executive Larry Augustin. In the scientific area, people are comfortable writing their own programs, he said.

But to enable other uses of lots and lots of Linux systems, VA Linux Systems will release software next week that makes it easier for a single administrator to manage large numbers of Linux boxes.

Origins
Beowulf systems originated in 1994 at NASA, where Thomas Sterling and Don Becker built a machine using 16 Intel computers.

Later, as the systems grew more popular, variations emerged, such as those using Macintosh computers or one at Oak Ridge National Laboratory using nothing but cast-off computers. In 1998, Red Hat began selling the software needed to set up a Beowulf cluster in a version called Extreme Linux.

Depending on the task at hand, Beowulf systems need fast processors, fast networking technology, and a fast operating system.

The systems also need a way to pass messages between the different nodes of the computer. Popular methods include PVM (parallel virtual machine) and MPI (message-passing interface), which are freely available, but Sun Microsystems has announced that it plans to make its own message-passing method available under its community source license.

Compaq sees Beowulf clusters as a way to sell its Alpha processors, and IBM has an identical philosophy with its PowerPC chips, Figgatt said. "Our brethren in the RS/6000 division are looking at this opportunity as well," he said. There may be a linkage between IBM systems the based on both Intel and PowerPC systems.