Open-source team fights buffer overflows

Theo de Raadt, the project leader for the group, believes that the group's latest improvements to the Unix variant, due to be released on May 1, will make causing a buffer overflow extremely difficult, if not impossible. A buffer overflow is a memory error in software that allows an attacker to run a malicious program.

"I could say that I am killing buffer overflows, but I am in the security community, so I have to put it in quotes," he told attendees at the CanSecWest security show on Thursday.

The memory bugs have resisted extermination for almost 30 years, and de Raadt said that any claims that an open-source group has done so would need to be tested.

Some attendees are already incredulous that the changes will eliminate buffer overflows.

"It's just adding another layer" to the security, said Nicolas Fischbach, senior manager for security at Colt Telecom, a Swiss communications provider. "It won't make a huge difference because there are always bugs that are found in software."

An overflow exploit generally works when an attacker sends a program requesting too much information. The data usually includes two components: one that crashes the application and one that's either a program or a memory address that points to a program that the attacker would like to run. When the application crashes due to the first component, the operating system will execute the second.

The OpenBSD team hardened the operating system to this type of attack using three tactics.

The group randomized where in memory the "stack"--a structure that holds applications and their data--resides, so that code designed to exploit buffer overflows will have to be tailored to the system's memory layout.

"Buffer overflows take advantage of a certain memory layout," de Raadt said. "It's a tiny waste of memory, with very little overhead, but it makes things a little bit more difficult. We are trying to make the (code) crackers work a little bit more."

In addition, the group restructured how critical addresses are stored on the stack, so that it's harder to get buffer overflows to result in a running program. The team placed a small tag, called a canary, in the memory structure to detect if addresses had been modified, a common method hackers use to get a legitimate program to run malicious code.

Finally, the group found a way to hack the BSD file system and divide main memory into a writable portion and an executable portion. Pieces of programs and data, known as pages, that are stored to memory will be placed into one of the two areas.

"We want to make sure that no page is both executable or writable simultaneously," he said. "The goal is that no hackers should be able to write code and then execute it."

The problem for the OpenBSD group is that while 64-bit processors have such memory protections available, the most-popular 32-bit processors don't. So the group has had to work around the issue and break up a computer's memory into writable and executable areas.

"You can draw a line in the sand--before that line you can execute, above you can't," de Raadt said.

While the other security features will be available in the May 1 release, the protected memory page structure for 32-bit processors--such as Intel x86 chips and the PowerPC chips--won't be ready for another six months, he said.

The research was funded by a $2.3 million grant from the Defense Advanced Research Projects Agency (DARPA) to the OpenBSD Project, but the latest changes go beyond the original grant request, de Raadt said.

"This really wasn't part of the DARPA grant," he said. "But it happened because the DARPA grant happened, because when you throw a bunch of...guys into a room and get them drunk, this is what you get." De Raadt was careful to point out that the group paid for its own beer.