Google's Niels Provos battles malware on the Web

During 2000 and 2001, Ph.D. student Niels Provos would occasionally drive from the University of Michigan across the border into Canada and spend the weekend working on an open-source cryptography project that would end up becoming one of the most widely used network security technologies ever: OpenSSH. He couldn't work on it in his Ann Arbor office, or he would have run afoul of restrictive U.S. export regulations designed to keep strong crypto out of the hands of foreigners.

Several years later, Provos moved his research papers and software related to steganography, which is the science of hiding secret messages, from servers at the U.S. university to a server in the Netherlands to avoid violating Michigan law. He was concerned (and so was the Electronic Frontier Foundation) that the law--which made it illegal to develop software that conceals "the existence or place of origin of any telecommunications service"--was so vague as to allow it to apply to his research. After the legislation was later watered down, he moved his stuff back to the states.

"One of the things I love about him is he's guileless," says Dug Song, who studied with Provos at the University of Michigan, drove him on some of his cross-border code-writing runs, and later co-founded Duo Security. "He's a very honest and open person, [and] he has a lot of integrity."

Now at Google, Provos leads the Safe Browsing team, whose technology identifies sites that are dangerous and flags them in Google search results so searchers won't have their computers compromised. The Safe Browsing technology, the first search engine-based service of its kind when it launched in mid-2006, keeps hundreds of millions of Internet users safe every day from malware lurking in Web sites.

In addition to worrying about protecting Web surfers' computers, Provos also is conscientious about safeguarding consumer privacy and he pushed for Google to delete user data collected by the team after two weeks, which is a short retention period for any Internet company collecting data. "We spent a lot of time figuring out what was right," he told CNET in a recent interview. "Google never knows which site you go to," using obfuscation techniques when checking for malicious content, and this method also reduces the amount of data transmitted to the browser, he added.

"He's always thinking about the user, especially when it comes to privacy matters," says Panayiotis Mavrommatis, a senior software engineer on the Safe Browsing team. "He's not afraid to say, we cannot store this information or we should give the user the option to opt out when it comes to privacy."

21st century craftsman
Provos' background explains a lot of his current interests. He grew up in the northern German town of Lubeck (founded in the 12th century) with one brother, who is an antiques dealer; his father, a judge; and his mother, who taught English at night school. The family's house was full of swords, old weapons, and armor. Both his parents practiced the Japanese martial art Aikido, which Provos took up, along with Judo. He and his brother also played soccer and spent a lot of time in the woods nearby, building caves and tree houses.

Provos also played Dungeons and Dragons role playing games in high school with friends. His father sometimes joined in and later wrote a medieval fantasy adventure book based on the stories the group came up with, titled "Sargon Schatz," which means "Sargon's Treasure." Provos self-published his father's book in 2008, shortly before his father died.

Now 38, Provos still does Aikido, as well as Japanese Kendo sword fighting, which his father did. He also took up blacksmithing a few years ago and has made kitchen knives and a Japanese Tanto Samurai sword. He is working on a replica of a Viking chest, forging a metal sliding lock, as well as metal straps and hinges as they would historically have been made.

"I strongly believe you need to find a good balance between work and play," he says. With blacksmithing, Provos likes the "tangible results, and (the fact that) it has nothing to do with computers." In general, though, his "off" hours have proved to be valuable for his work.

"Most of the ideas I thought were worthwhile I had when I was not working," he says. "It used to happen when I was walking the dog or taking a shower, when I was doing things not related to the actual problem. Your mind becomes free and all of a sudden this insight happens."

The discipline for mastering a skill and the appreciation of creating things that are functional and beautifully architected are also reflected in Provos' work in security as a hacker and defender of people's computers and digital secrets. He hammers metal into weapons and turns bits and bytes into digital locks and keys.

"Niels is a craftsman," Song says of his former collaborator and the man who was best man at his wedding. "He brings an academic discipline to engineering--and rigor, but also a craftsman's approach, to software design and implementation."

In addition to working on OpenSSH, free open-source software used to encrypt Internet communications, Provos wrote libevent, a software library released in 2000 that allows programmers to write scalable network programs.

"He has written some of the most widely used and important system software in the world...which forms the core of many of the most important modern network daemons and programs," Song says. "Libevent was revolutionary at the time in promoting event driven programming."

Provos also volunteers time serving on the board of directors for USENIX, or the Advanced Computing Technical Association, and working on the Honeynet Project, an international nonprofit security research organization.

Going to Google
Provos' path to security and Google wasn't the most obvious one, given that he started out studying physics at the University of Hamburg. "I really enjoyed physics at first, but at some point you reach these questions about the universe that you can't answer anymore. And all the measurements you do only result in models you can't truly explain," he says. "Then I went to mathematics and everything could be explained and my world view was good again. But mathematics was kind of dry. Then I started working on cryptography and realized I liked doing computer science."

He decided to get a PhD in the United States, and the proximity to Canada where he could work on cryptography away from U.S. government scrutiny made the University of Michigan an ideal choice. When he was finishing his degree, Provos assumed he would get a job in a research lab or as a professor at a university. But then he saw a little advertisement in an Association for Computing Machinery publication seeking smart people to work at Google.

"I hadn't really thought of Google at all, but I thought, 'why not?'" he says. "I went to the interview and everyone was really smart and enthusiastic about what they worked on...They have fantastic infrastructure and there is so much opportunity to work on interesting projects." He joined the Mountain View, Calif.-based company in 2003.

In the early days, he was working on protecting sites against denial-of-service attacks. But then "I realized that malware and compromised Web sites was becoming more of a problem," he says. "The danger was that this could erode confidence in the Web and at the end of the day hurt Google too."

So he proposed that engineers leverage the information Google gleans about all the sites its system crawls, figure out which contain malware, and display warning signs before people click on those results. The system was designed so it could be automated on a large scale. Google also tells Webmasters of sites that are flagged what the problem is and offers suggestions for how to fix it.

Safe Browsing protects every Google Web search and warns users of Chrome, Firefox, and Safari when they are about to visit a site that has malware. Webmasters can use the Safe Browsing Application Programming Interface to warn site visitors away from links on their site that lead to malware-infested pages and to prevent users from posting phishing links on the site.

"What I enjoy about Google is that it's possible to do these things. When I started the project, nobody was working on this," Provos says. "There is a strong team of great engineers. The work is challenging and interesting. I'm very happy."

Provos has shown his ability to handle pressure gracefully when there is a bump in the road, even a big bump. On Saturday, January 31, 2009, for about 45 minutes, Google mistakenly flagged every Web site in its search results as having malware on it. The problem was quickly fixed, but fearing that his colleagues would be demoralized, Provos called everyone on his team to give them a pep talk.

"He was basically trying to make sure everyone was OK in terms of stress level," says team member Mavrommatis. "Google is known for not going down ever, and this was one of the major outages in the history of the company. That sort of highlights his personal relations with the team...He is very good with people and a great manager because he cares about the people."