From the logs--electronic records of the information passed on the network--it quickly became evident that a server with gate-keeping control over different parts of the system was getting downright chatty with a foreign computer via the Internet.
"I didn't know what the information meant," Riebrandt said. "I just knew that someone was talking to (the server). And it was talking back."
After an afternoon's investigation, Riebrandt and the other administrators overseeing security concluded that the attackers had compromised the network. So they reinstalled the system, using a secure backup they'd prepared.
But the attackers added insult to injury: They came back the next day, hacking the server in exactly the same way. Riebrandt and the others still don't know how it had happened.
Luckily they'll get a chance to learn from their mistakes--without grave consequences. The attackers weren't foreign-sponsored spies orcreeping through the Pentagon's computer systems, but a Department of Defense "red team" attempting to poke holes in a mock military network run by students of the Naval Postgraduate School here.
Hardening the nation's Internet defenses against cyberattack has been a goal long discussed in policy circles, but results have been slow in coming. The Clinton administration drafted thein 1999 and released it for public comment in 2000. Included in the plan were 10 steps that the government should take to defend important national infrastructure, including communications and the Internet, against attack.
Yet onlyhave concrete steps been taken, including discussions of separate networks for intra-agency data, computer security scholarships in return for service, and budget increases.
While not part of the National Plan, the Cyber-Defense Exercise does address one of the plan's 10 steps: training more security professionals.
The four-day exercise, which ended Thursday, pitted so-called blue teams of students from six different military academies against professional military red teams. The red teams are made up of government employees from the National Security Agency and soldiers from the U.S. Air Force's 92nd Information Warfare Aggressor Squadron and the Army's Land Information Warfare Activity.
The 30 participants from the Navy Postgraduate School seemed to have done well. Aside from the primary domain controller whose security got cracked twice, the red teams were able to compromise only one other server. That was an unsecured backup system that wasn't supposed to be part of the exercise but had accidentally been left connected to the network during the 6 a.m. to 2 p.m. PDT attack window.
"I feel pretty confident that we won," said Allen Harper, a second-year NPS student and a captain with the U.S. Marine Corps. The students won't actually know the final results for two more weeks.
In last year's contest, the Navy Postgraduate School topped the score of the other two schools that took part. However, NPS couldn't take home the trophy because it's a graduate school and not an undergraduate academy.
Most of the students who join in have no previous hands-on experience in securing a network. "There were a lot of people out of their comfort zone," Harper explained. "But they stepped up to the plate and did really well."
For instance, Harper himself, as a communications officer in an infantry battalion, hadn't had any direct experience with security. And fellow team member Lynzi Ziegenhagen used to be a product manager for a wireless-software company. Now she's in the first crop of computer-securitystudents, one of 11 who took part in the Cyber-Defense Exercise.
"I really didn't know anything about security before I got here," Ziegenhagen said.
As leader of the students in the team that was responsible for securing the network's Web servers, Ziegenhagen says she's learned a lot in the last week, especially since Web servers were among the first computers attacked.
The exercise wasn't limited to just the U.S. military. Valter Monteiro, a lieutenant commander with the Brazilian Navy and a student at the NPS, was one of three students from other nations' military services to take part. He secured the network's routers--the essential hardware that directs information to the right destination.
While Monteiro had six years of experience setting up Cisco routers for networks in the Brazilian military, he had never concentrated on security as much as he did for the Cyber-Defense Exercise. He said the hands-on exercise was a refreshing change. "The approach is different. In Brazil, a masters is more theoretical."
Indeed, specific lessons aside, what Monteiro seemed to take away from the experience was a strong appetite for computer security. The exercise seems to have that effect.
"No one here is getting a grade," said Marine Capt. Harper, this year's team leader. "And yet we are all willing to skip classes to be here. In classes we learn a lot, but this is a way to measure ourselves."
Harper himself has gone from being marginally competent with computers before he started the program to becoming a security guru.
After last year's exercise, when Harper was first exposed to the hands-on side of security, he and a core group of students went to the United States' largest hacker convention, Def Con, to take part in the annual capture-the-flag tournament. The group went on the offensive for the showdown, in which teams of hackers attempt to compromise key servers on a mock network. Surprisingly, the rookies nabbed second place, losing by only a slim margin, Harper said.
This year's Cyber-Defense Exercise puts Harper and his team back on defense, however.
Early each morning, a student on the blue team had to show a white team referee that the network services were up and running. The white team, the "U.N. observers" of this particular exercise, were analysts from the Computer Emergency Response Team Coordination Center at Carnegie Mellon University (CERT). They would evaluate each side's claims of penetration and response.
After proving the network was up, the blue team students had to keep their hands off the computers during the 6 a.m. to 2 p.m. attack window. Between noon and 2 p.m., they could watch what was happening but could not react. After 2 p.m., the group would then go to work, searching the network for evidence that the red team had gotten in.
Keeping each service up on the network--e-mail or FTP file access, for instance--granted the blue team points. But the red team could steal those points away by successfully compromising the service. Discovering the attack and responding would then be the blue team's only way to get points back.
"We'd lose points throughout the day and then try to gain them back by reporting what (compromises) we found," Harper said.
On Monday, the attackers mainly settled for scanning the network for weaknesses, said Harper. The red team came in over the Internet on a secure virtual private network set up specially for the exercise. By Tuesday, the scans dropped off and attacks began. The backup server that had been left on the network quickly became a casualty.
"They owned it," said a chagrined Harper.
The attackers also sprung a couple of hoaxes, attempting to leave evidence that they had broken in, when in fact they hadn't.
On Wednesday, the attackers took over the Windows 2000 server that was acting as the domain controller, allowing users access to various network services. While they compromised the machine, they weren't able to do much, said Riebrandt, the security administrator for the NPS labs.
"This box definitely took some hits, but it stayed secure," Riebrandt said.
"It's expected that they would get in," added Harper. "We have to defend against a hundred different things, but they only have to find one mistake to use against us."
In fact, the NPS students did well. Their network was so secure that the red team asked for a gift: a password to the group's FTP server. Even with the password, however, early analysis on Thursday seemed to indicate that the network had withstood further attack.
What did you learn in school today?
The exercise had many serious lessons, said J.D. Fulp, professor of computer science at the Naval Postgraduate School and the adviser for the NPS blue team.
"This totally demystifies a discipline that most people don't get hands-on experience with," Fulp said.
The exercise gives students a fairly controlled environment in which to view an attack, improving their analysis skills and allowing them to see the potential consequences of weak security.
And there's another lesson, Fulp said. "The basic premise is that the install-and-patch approach doesn't work." Operating systems need to be designed to be secure from the get-go, without the need for constant monitoring and tweaking.
The core dozen students working part time on security at the NPS add up to a far larger staff than that in place at many Fortune 1000 companies. And those companies are dealing with networks that are far larger than the 16 computers connected together in the NPS lab.
Having to continually monitor every computer on the network and patch every system is far too much work, Fulp said.
But until better OSes arrive, the military--and private industry--can benefit from the exercise. The price tag, an initial $100,000 for equipment per participating school, is modest compared with the threat of unsecured networks, Fulp said.
"The money that is earmarked for cyberdefense...at least some needs to go to these programs," Fulp said. "This is the core of what we need to do."