Smoking out photo hoaxes with software

Did Brad Pitt really meet up with aliens at the World Economic Forum? Software for detecting photo fraud may have the answer. Photos: Pictures that lie

Dartmouth College professor Hany Farid is no fan of Josef Stalin, but he acknowledges that the photo retouching done during the Soviet era was top notch.

"That was impressive work. I've seen some of the originals," Farid said. The Soviets just didn't airbrush their victims out, he added. They painted in new backgrounds on the negatives.

Farid's interest in photo retouching isn't just historical. The professor of computer science and applied mathematics runs the university's Image Science Group, which has emerged as one of the chief research centers in the U.S. for developing software to detect manipulation in digital photographs.

Photo hoaxes

While some of the group's software is now used by the FBI and large media organizations such as Reuters, a version written in Java will come out soon that will be easier to use and thereby allow more police and media organizations to sniff out fraud. The current software is written in Matlab, a numerical computing environment.

"I hope to have a beta out in the next six months," Farid said. "Right now, you need someone who is reasonably well-trained to use it."

Photo manipulation is a lot more common than you might think, according to L. Frank Kenney, an analyst at Gartner. That Newsweek cover of Martha Stewart on her release from prison? It's Martha's head, but a model's body. Some people believe hip hop artist Tupac Shakur remains alive, in part because of the images that have cropped up since his reported death in 1996.

Although it's difficult to estimate the size of the market for fraud detection tools, the demand is substantial, according to Kenney.

"How much is the presidency of a country worth, or control of a company? People tend not to read the retractions," he said. "Once the stuff is indelibly embedded in your memory, it is tough to get out."

The Journal of Cell Biology, a premier academic journal, estimates that around 25 percent of manuscripts accepted for publication contain at least one image that has been "inappropriately manipulated" and must be resubmitted. That means it has been touched up, although in the vast majority of cases, the author is only trying to clean the background and the changes do not affect the scientific efficacy of the results. Still, around 1 percent of accepted articles contain manipulated images that do significantly affect the results, said executive editor Mike Rossner. Those papers get rejected.

"Our goal is to have an accurate interpretation of data as possible," Rossner said. "These (images) are (of) things like radioactivity detected on a piece of X-ray film."

Law enforcement officials have also had to turn to the software to prosecute child pornographers. In 2002, the Supreme Court in Ashcroft v. Free Speech Coalition overturned parts of the Child Pornography Protection Act for being overly broad, ruling that only images of actual minors, and not computer-generated simulations, are illegal.

Since that decision, a common defense has become that the images found on a hard drive are artificially created.

"The burden is now on the prosecution. These cases used to be slam dunks," Farid said.

How it works
Fraud detection software for images essentially searches for photographic anomalies that the human brain ignores or can't detect.

Humans, for instance, ignore lighting irregularities in two-dimensional images. While the direction of light can be re-adjusted in 3D images from video games, it is difficult to harmonize in 2D photographs. The light in the famous doctored photo that puts Sen. John Kerry next to actress Jane Fonda at a protest rally actually comes from two different directions.

"The lighting is off by 40 degrees," Farid said. "We are insensitive to it, but computers detect it."

Although modern researchers have in clinical studies documented humans' ability to filter out lighting incongruities, 15th-century painters were aware of the way humans process images and exploited that knowledge to create seemingly realistic lighting effects that would have been nearly impossible to replicate in real life.

"The lighting is totally bizarre in some Renaissance paintings," he said.

Featured Video