Web traffic redirected to China in mystery mix-up

Workers at Internet network operation centers around the world are trying to figure out why traffic to sites such as YouTube, Twitter, and Facebook was redirected to servers in China this week, giving Web surfers around the globe a glimpse of what Chinese Internet users see when they try to access those blocked sites.

On Wednesday, someone at Chile's Domain Name System (DNS) registry, the Internet Protocol (IP) address lookup system, said a local Internet service provider had noticed strange behavior and asked his counterparts in other parts of the world about it on an industry e-mail list.

Specifically, one of the main DNS root servers, called the I Root Server and operated in Sweden, was directing visitors trying to go to those sites instead to servers in China. This effectively sent people behind the Great Firewall of China, a strictly controlled network of servers and routers the People's Republic of China uses to filter the Internet and block its citizens from accessing content deemed politically sensitive.

Representatives from Twitter and Facebook did not respond to calls and e-mails seeking comment on Thursday night.

A spokesman for Google, which owns YouTube, declined to comment, saying "this appears to be a specific ISP level issue." He said it was not related to Google's English-language corporate site appearing in Chinese, Danish, and other languages on Wednesday, which the company attributed to a bug.

There will no doubt be speculation that the DNS mix-up is related to tensions among Google, the United States, and China over Google's claim that its network and Gmail users who are human rights activists were targeted late last year by attacks originating in China. In a highly public action earlier this week, Google moved its Google.cn site to Hong Kong.

"For a long time, we have believed that China modifies DNS answers; no surprise there," Rodney Joffe, a senior vice president at DNS service provider Neustar, told CNET in an interview late Thursday. "They do it because they want to make sure that, for example, people inside China are subject to the censorship."

But what was a surprise, he said, was that a server inside of China was able to redirect Web traffic to servers inside that country.

As a result, Internet users around the world trying to go to those three popular U.S.-based social-networking sites, as well as to as many as 20 or 30 other sites, were either being redirected to alternative sites offered in China or saw error messages indicating that the sites they were seeking did not exist, Joffe said. He declined to name any other Web sites that were affected.

"The issue isn't so much that the system was giving bogus answers, but that it was giving bogus answers (to users) outside China," he said.

A representative from the China Internet Network Information Center denied any malfeasance. "We wanna clarify that CNNIC never did any interceptions or other things for the mirror site of I root server," Xiaodong Lee, vice president and chief technology officer at CNNIC, wrote in an e-mail on the DNS operations list. "CNNIC only provides the stable Internet connection, power and necessary hand support."

Xiaodong did not respond to an e-mail seeking comment.

Netnod/Autonomica, which operates the I Root server, also denied any responsibility. "We do not intercept, interfere, rewrite or otherwise alter either queries, responses or the content of the zone itself," Netnod CEO Kurtis Linqvist wrote in an e-mail. "We are currently investigating the issue reported, but at this time there is nothing additional" to share.

It could have been an innocent mistake or it could have been that China was caught hijacking Web traffic, Joffe said. He would only comment on what the speculation was in the DNS community and declined to give his opinion on what he thought happened.

"One could look at it from the point of view that China was getting the attention of the rest of the world as a result of the Google incident," he said. "At the next level, the most paranoid one, perhaps China is involved in attempting to hijack DNS requests" on a regular basis.

Dan Kaminsky, who publicized and helped address serious security problems with DNS and again last year, said he was inclined to think it was intentional on the part of China, although he acknowledged that he was not personally familiar with the details and had only heard about the situation from others.

"What seems to be going on right now is a diplomatic war between the U.S. and China, and in such a war there seems be a battle between an America company, Google, and whatever organization is running the DNS in China," he said. "For a long time, we haven't had strong authentication on the Internet because it has simply been too difficult to deploy. We got away with it because not too many people were seeking to interfere with Net traffic."

A representative of Arbor Networks speculated that an ISP misconfigured its BGP (Border Gateway Protocol) system, which is used to route Internet traffic. "I don't think it was done intentionally," Danny McPherson, chief security officer with Arbor Networks, told IDG News Service earlier Thursday. "This is an example of how easy it is for this information to be contaminated or corrupted or leaked out beyond the boundaries of what it was supposed to be."

Such a problem happened in February 2008 when a Pakistan DNS provider knocked YouTube offline for a few hours.

Tech blog site Ars Technica, which appears to have first reported the story, noted that "these queries were answered by a root server residing in China, and China has been applying this type of creativity to DNS queries since at least 2002."

Whatever the cause of the problem, it has now been fixed, said Brad Williams, a spokesman at VeriSign, which operates the top level domain that the affected sites use.

"In our regular network checks, we recently noticed that routes were being announced outside of China for our anycast server there," Williams said in a statement. "As this was an aberration, we notified our technical partner in China and helped them resolve the issue. Our network checks show that the issue is now resolved."

Roy Arends, a researcher at U.K. registry Nominet, said he had seen the problem happen before and wrote a study on it last year.

"I wanted to keep this internal, however, the cat is out of the bag now," Arends wrote in a post to the DNS Operations e-mail list. He did not respond to an e-mail late Thursday.

Neustar's Joffe declined to say whether U.S. authorities had been notified of the incident, but did comment on the magnitude of it.

"This was a real world example of the Net security industry's worst nightmare," he said. "And last night it happened."

Update 8:50 a.m. PDT March 26:
Another expert from the DNS operations e-mail list said he did not believe there was malicious intent behind the actions.

"This looks more like careless behavior; that they didn't care that much about problems for international users," Bert Hubert, founder of Dutch-based software provider PowerDNS.com, said in an interview Friday.

"The wider problems are that it appears that someone in China can disrupt Facebook for someone in California," he said. "It appears we can no longer see the Internet as a friendly shared resource and that strict boundaries will have to be put in place. The problem is the technology is not really there to make that happen."