Xen getting multiprocessor support

Xen, software that lets multiple operating systems run on the same computer, will become significantly more powerful with an upcoming version that introduces multiprocessor support.

Xen is a "hypervisor"--a thin layer of software that governs how different operating systems get access to computer resources such as processors and memory. But the current version is hobbled by the fact that each operating system can use only one processor.

Version 3, which programmers hope to begin testing this month, will lift that limit, said Ian Pratt, a founder and lead programmer of the open-source project and the company XenSource that's trying to commercialize it. "We hope to get 3.0.0 out in August--sooner if we're ready," Pratt said, though he cautioned, "I'm a firm believer in having releases driven by product quality rather than dates."

The multiprocessor support is important, said Illuminata analyst Gordon Haff. "Some applications obviously need more than one processor's worth of performance," he said, and Xen also stands to benefit from not being pigeonholed as software with only limited abilities.

Running multiple operating systems on a single machine--an ability that Unix servers and mainframes have had for years--is useful for several reasons. It makes it easier for one server to share multiple tasks and therefore replace several independent machines. And personal computers can be divided into noninterfering domains for business use, personal use or systems management. Mainstream adoption of multi-OS practices likely will be hastened by features in Intel processors this year and Advanced Micro Devices chips in 2006 that make the task significantly easier.

Xen has gained active help from Intel, IBM, AMD, Hewlett-Packard, Sun Microsystems, Red Hat and Novell. In prevailing industry argot, its job is to virtualize computer resources, letting operating systems run in separate "virtual machines," often called partitions, rather than directly on the hardware itself.

But Xen also has rivals to fend off. EMC subsidiary VMware has a head start of years, already supports virtual machines that use up to two physical processors, and will receive four-processor support by the end of 2005. And Microsoft is working on its own hypervisor, a product expected to ship in 2007.

Another potentially formidable competitor could have been IBM's rHype, or Research Hypervisor, given that Big Blue has decades of experience with the technology. But in a mailing list posting, an IBM programmer said the project--or at least the version for Intel chips--"was mostly for demonstration purposes" and steered people toward Xen.

Xen uses an approach called paravirtualization that stops a step short of the full-on virtualization used by market leader VMware. That approach boosts performance, Pratt and analysts agree, but today requires that operating systems be modified to take advantage of it. The top Linux sellers, Red Hat and Novell, have released test versions of their open-source operating system products with tentative support.

Paravirtualization means that Xen doesn't face some VMware limits for

multiprocessor servers. "I've seen excellent results on eight-way systems--that's the biggest we have access to--but I know people are using Xen on some 32-way systems," Pratt said.

Other Xen 3 features
Multiprocessor support isn't the only change coming. Also on tap is support for Intel's Virtualization Technology (VT), which is expected to ease Xen's operating system compatibility, and support for the 64-bit extensions that expand memory capacity for newer x86 processors such as Intel's Xeon and Advanced Micro Devices' Opteron.

Further improvements are planned for later updates to version 3, Pratt said. One is a "shared buffer cache," which would let separate partitions share a processor's high-speed cache memory for faster data access. Another is faster networking links among different virtual machines using an approach that will work even if a virtual machine is moved from one physical computer to another.

AMD's answer to Intel's VT is a technology code-named Pacifica. Xen support for Pacifica is a "work in progress," Pratt said.

It's likely a single version of Xen will be able to support both VT and Pacifica, Pratt said.

Details on how that is likely to happen emerged in June, when AMD programmer Elsie Wahlig proposed on Xen's mailing list a common interface to the two technologies. Wahlig worked with Pratt, another lead Xen programmer named Keir Fraser, and IBM to develop the proposal.

Intel's Jun Nakajima said Intel would help modify Xen to accommodate the change. "We are evaluating this proposal and will work on enhancing the VT-x code to provide the right level of abstraction to support other VT-x like technologies," Nakajima said. VT-x refers to the version of VT that runs on Intel's Xeon processors, as opposed to VT-i for its Itanium chips.

Xen still has a ways to go, however, Haff said. "Xen remains relatively immature compared to VMware in particular," he said. "We'll be able to say it's truly mature when it gets rolled into enterprise Linux distributions and has all the ancillary command and control tools."

Another requirement for virtual machine software will be to get certification from software companies, said Gabriel Consulting Group analyst Daniel Olds. "That's a killer in the corporate environment and will keep VMware and Xen on the outside looking in" until it's addressed, he said.

The BitKeeper snafu
Xen developers are going through some of the same growing pains as the Linux operating system with which Xen is closely allied. In April, Linux programmers abruptly shifted to a new mechanism to control the project's underlying source code, and now Xen is facing the same plight.

Linux and Xen programmers had managed their source code with a proprietary tool called BitKeeper from a company called BitMover. BitMover had permitted open-source programmers to use the tool, but the company canceled the offer this year.

"The BitKeeper issue has come at a very bad time for us, as we really don't want to change (source code managers) until 3.0 ships," Pratt said.

And there aren't any simple fixes, he added. "We're strongly considering buying some BitKeeper licenses for the key developers to tide us over. We tried a bunch of different tools, and none of them come close to BitKeeper," Pratt said.

BitMover released a tool last week to ease the transition, though. The tool lets a BitKeeper source code repository be converted to a widely used alternative, Concurrent Version System, or CVS.