Optical networking: The next generation

The National LambdaRail (NLR) project is linking universities across the United States in an all-optical network consisting of thousands of miles of fiber; it's the first such network of its kind. NLR's research focus--and potential future impact on the commercial market--are leading some networking experts to make comparisons between the project and the early investments that led to the Internet itself.

Last month, NLR completed the first full East-West phase of deployment, which included links between Denver and Chicago, Atlanta and Jacksonville, and Seattle and Denver. Phase two, which is expected to be complete by May or June 2005, will cover the southern region of the United States. This part of the project will link universities from Louisiana, Texas, Oklahoma, New Mexico, Arizona, Salt Lake City and New York.

"The National LambdaRail is the next step in the natural evolution of research and education in data communications," said Tom West, chief executive of the National LambdaRail. "For the first time, researchers will actually own underlying infrastructure, something that is crucial in developing advanced science applications and network research."

Forget Internet2 and its 10-gigabit-per-second network, called Abilene. Experts say NLR is the most ambitious networking initiative since the U.S. Department of Defense commissioned the ARPAnet in 1969 and the National Science Foundation worked on NSFnet in the late 1980s--two efforts considered crucial to the development and commercialization of the Internet.

Like Abilene, NLR is backed heavily by Internet2, the university research consortium dedicated to creating next-generation networking technologies. But NRL offers something its sister project can't--a complete fiber infrastructure on which researchers can build their own Internet Protocol networks. By contrast, Abilene provides an IP connection over infrastructure rented from commercial backbone providers, an arrangement that ultimately limits research possibilities.

The problem that has faced the research community since the commercialization of the Internet is that they have become beholden to commercial carriers that own the fiber and basic infrastructure of the communications networks. They are often forced to sign multiyear contracts that exceed their research needs. And because researchers don't own the access to the fundamental building blocks of the network, they can't conduct cutting-edge experiments on the network itself.

Now, for the first time in years, experts say, researchers once again have full access to a research network, providing unmatched opportunities to push networking technology forward.

"LambdaRail is creating the ARPAnet all over again," said Scot Colburn, a network engineer at the National Center for Atmospheric Research, which plans to hook into LambdaRail next year. "People in the academic community will now be able to play with the protocols and the basic infrastructure in a way they can't do now."

Help for optical networking?
The biggest likely beneficiary of NLR is the optical networking industry.

During the boom years, carriers such as WorldCom were predicting unprecedented growth on their networks, and new optical networking seemed like just the technology to feed the need. Carriers racked up debt as they spent billions of dollars in digging trenches and laying fiber. Billions of dollars also were pumped into equipment start-ups to make devices that could efficiently use this fiber to transmit massive amounts of data at lightning speeds.

Since the telecommunications bubble burst, hundreds of these companies have gone bankrupt, and "optical" has become a dirty word in the networking world. A final accounting of the damage may not be over even yet: Corning, one of the largest makers of optical fiber, cable and photonics products, on Thursday said it will record $2.8 billion to $2.9 billion in noncash charges in its third quarter related to its telecommunications business.

Given the current climate, the advent of NLR and the research possibilities it is opening up are already being hailed as a godsend for the beleaguered sector.

"I think that NLR has definitely raised the consciousness of optical technology," Colburn said. "As a networking guy, a year ago I didn't have to know a lot about optical networking, but now I do."

Network engineers agree that it could take years before networking research conducted on the National LambdaRail infrastructure ever makes it into commercial products or services. But when it does, the entire food chain of companies in the telecommunications market stands to benefit. These companies include carriers such as Level 3 Communications and Qwest Communications International; equipment makers such as Cisco Systems and Nortel Networks; and fiber and optical component makers like Corning and JDS Uniphase.

"By nature, the research and education community will always be a few steps ahead of the commercial market," said John Verduzco, sales director for research and education at Level 3. "Our work with the researchers involved in the NLR project should theoretically pay dividends in the future. It's possible some of the research could spawn offshoots that we productize later."

A new kind of research network
Like fiber networks laid in the late 1990s, NLR relies on technology called dense wave division multiplexing (DWDM) that splits light on a fiber into hundreds of wavelengths.

While Internet2 users share a single 10gbps network, NLR users can have their own dedicated 10gbps link to themselves. Experts say Abilene provides more than enough capacity to run most next-generation applications, such as high-definition video, but that it doesn't offer enough capacity for some of the highest-performing supercomputing applications.

Because Internet2 is a shared network, researchers are constantly trying to tune the infrastructure to increase performance, measured by so-called Land Speed Record tests.

The last record was set in September, when scientists at CERN (European Organization for Nuclear Research), the California Institute of Technology, Advanced Micro Devices, Cisco, Microsoft Research, Newisys and S2IO sent 859 gigabytes of data in less than 17 minutes at a rate of 6.63gbps--a speed that equals the transfer of a full-length DVD movie in four seconds. The transfer experiment was done between Geneva, the home of CERN, and Pasadena, Calif., where Caltech is based, or a distance of approximately 15,766 kilometers.

In theory, researchers using a dedicated 10gbps wavelength, or "lambda," from NLR should be able to transmit hundreds of gigabytes of data at 10gbps without much problem. While most researchers don't yet need that kind of capacity, some are already looking forward to applications that could take advantage of a high-speed, dedicated network.

For example, at the National Center for Atmospheric Research in Colorado, researchers are developing new climate models that incorporate more complex chemical interactions, extensions into the stratosphere, and biogeochemical processes.

Verification of these processes involves a comparison with observational data, which may not be stored at NCAR. Researchers plan to use NLR to access remote computing and data resources, said Jeff Kiehl, chairman of the Community Climate System Model Scientific Steering Committee for NCAR.

The Pittsburgh Supercomputing Center, which was the first research group to connect to LambdaRail in November 2003, is using the LambdaRail infrastructure instead of a connection from a commercial provider to connect to the National Science Foundation's Teragrid facility in Chicago.

Wendy Huntoon, assistant director of networking at the Pittsburgh Supercomputing Center, said researchers at the center have already submitted proposals to study the effects of packet size on network performance.

"The main reason we went with NLR is because it provided additional flexibility that couldn't be matched from the commercial carriers," Huntoon said. "For example, if our researchers wanted to change the backbone for any reason, they could without having to renegotiate a contract."

Creating partnerships
NLR currently has 18 members consisting of universities and research groups around the country. Each member has pledged to contribute $5 million over the next five years to the project. Internet2 holds two memberships and has pledged $10 million.

In exchange for its $10 million contribution, Internet2 is using a 10gpbs wavelength to design a hybrid network that uses both IP packet switching and dynamically provisioned lambdas. The project, called HOPI, or hybrid optical and packet infrastructure, will use wide-area lambdas with IP routers and lambda switches capable of high capacity and dynamic provisioning.

To date, the NLR consortium has raised more than $80 million. West said $30 million of that money is earmarked for building out the optical infrastructure.

While NLR has leased fiber from a number of service providers, including Level 3, Qwest, AT&T and WilTel Communications, it's using equipment to build the infrastructure from only one company: Cisco.

Through its exclusive partnership, Cisco is supplying NLR with optical DWDM multiplexers, Ethernet switches and IP routers.

Cisco said its involvement in NLR goes beyond simply providing researchers with equipment. The company is a strategic participant in NLR and holds two board seats, which have been filled by prominent researchers outside of Cisco's organization. The company also plans to fund individual projects that use LambdaRail through its University Research Program.

"NLR can serve as the testbed for so many new projects involving networking," said Javad Boroumand, a senior manager in Cisco's academic research and technology group. "If you use history as a basis, the Internet and Napster didn't come from technology companies but from the research community. We want to help drive this innovation."

Moving forward
NLR provides the fiber network across the country, but universities that want to use the infrastructure still have to find a way to hook into the network. As a result, universities in the same geographic region are banding together to purchase their own local or regional fiber.

"There is still a serious last-mile problem," Colburn said. "It's great we have this nationwide infrastructure, but it can only be used if you have the fiber to connect to it."

Internet2 has established the National Research and Education Fiber Company (FiberCo) to help these groups acquire regional fiber. Specifically, FiberCo acts as the middleman between universities and carriers that own the rights to the fiber.

"In many ways, telecom carriers weren't set up to sell to higher education," said Greg Wood, a spokesman for Internet2. "FiberCo helps negotiate some of these terms to make the process much easier."