Six million America Online customers were locked out of cyberspace for at least 19 hours Wednesday in one of the worst outages in online history.
AOL crashed at 1 a.m. Pacific time while it was installing new software for its network routing system, a problem area for other Internet service providers. Company officials reported that the system finally was restored at 7:45 p.m., but some users still said they could not log on as late as 8:30 p.m., Pacific time, nearly midnight Eastern time.
"Wednesday was a bad day for me--and, I know, for many of you," began a "community update" posted online by AOL chairman Steve Case late tonight. "The AOL system was unavailable for most of the day, so if you tried to sign on and couldn't, you were not, unfortunately, alone."
As part of his profusely apologetic remarks, Case guaranteed that the company would reimburse subscribers for time lost on the service, whose business model depends on maintaining a sense of community among its members. "We regret this extraordinary delay and extend our gratitude for the patience and continued support of our members," he said in a press release.
Like telephone, electric, and cable services, Internet service providers crash from time to time, but none is as large as AOL. The last major ISP outage occurred June 19 when Netcom, a half-million-strong service provider, crashed for 13 hours. That same day, by coincidence, AOL's email system went down for about an hour.
Many AOL users were at least frustrated by today's blackout, if not furious. Several businesses rely on AOL to distribute their services, such as newspapers and magazines, and some depend on it for stock quotes, breaking news, and electronic mail.
But others saw the outage simply as an inconvenience. "Honestly, it's just a major inconvenience," said E. David Ellington, president and chief executive of NetNoir, which had planned to start a three-day redesign of its AOL site today. "It's not the end of my business. It's technology."
The main thing, Ellington said, was preserving his lines of communication. "I'm not really panicked. I'd be concerned if all my email was wiped out," he said.
That isn't going to happen, according to assurances from AOL spokeswoman Pam McGraw. She vowed that email would be delivered once AOL went back online.
McGraw said the problem occurred during software installation Wednesday morning. AOL users know that system maintenance is performed from 1 a.m. to 3 a.m. Pacific time, she said, when "there's a possibility the system is unavailable."
This time, however, it stayed unavailable a lot longer. "During the process of installing new software, we ran into some technical problems," McGraw said. She said the new software was to be used for the network routing system but would not be more specific.
Coincidentally, the Netcom crash two months ago occurred after someone also made a mistake with the routing system, said Bob Tomasi, vice president of operations for the national ISP.
Instead of closing off a routing gateway, he said, "we inadvertently opened it up." The result was rather than Internet traffic flowing from Netcom onto the Internet, traffic from all across the Net was flowing back into Netcom--virtually choking its routers and shutting them down.
Netcom had to reset its 250 routers, which took several hours.
Because he'd been through it, Tomasi was hardly gloating over AOL's travails. "I'm going to be the last person to stand up and say these guys have problems. Come over here," he said. "These problems do happen. In fact, if there was something I could do over here to help them, I'd do it."
Tomasi added that customers did not abandon Netcom. "It didn't seem to drive people away at all," he said. "We apologized and acknowledged the fact that it was caused by our mistake."
Likewise, David Locke, an analyst with Volpe, Welty doubted that AOL would suffer long-term damage from the outage. "If your phone line's down for some reason, it doesn't make you think, 'My God, I should switch from AT&T to some local small provider.'"
Realistically, he said, "the vast majority of those 6 million [customers] aren't even going to realize it was down." Those who do may understand that it's a new technology, and things like this can happen.
Bob Metcalfe, a columnist for trade publication InfoWorld who has long warned about the fragility of the Internet, said the massive outage is "a wake-up call. It's a reminder that these online services are becoming less and less an optional toy game. The Internet is becoming more and more valuable to people conducting their lives through it and when it goes out, it hurts."
Locke endorses such acknowledgement of reality. "The world is not bulletproof. It's not foolproof. The people who are online today really understand that," he said. In the end, "it's like beating up Microsoft every time my machine crashes. I just give it the finger and move on."
"I look at this whole thing as kind of a nonevent. It would be better if it didn't happen, and the stock will probably go down $2 or $3," he said. (In fact, AOL's stock closed 5/8 of a point higher today than Tuesday, to 34-7/8.)
Ellington, sounding ever the optimist, managed to find a silver lining in an otherwise disastrous day: "I picked up the phone and called some folks. I actually had to talk to them."