Tech lessons learned from the wisdom of crowds

Prediction markets have some smart things to say about everything from DRAM prices to when Microsoft software will be finished. Tech turns to prediction markets

SUNNYVALE, Calif.--In a 1945 paper, the great Austrian economist F.A. Hayek described how prices set by a free market are really "a mechanism for communicating information" about the probability of future events.

If a war in the Middle East is seen as likely, for instance, oil prices will probably increase. Hayek's insight showed that the results can be surprisingly accurate, as long as enough people are allowed to wager real money on the outcome.

Now, technology firms are using a modern twist on this idea, called prediction markets, as a way to save money, harness the distributed knowledge of their rank-and-file employees, and even answer questions like: When will this software ship? And what will memory prices be like in a few months?

At a micro-conference that Yahoo convened at its headquarters here on Wednesday evening, representatives from Google, Yahoo, Microsoft and Hewlett-Packard described their experiments with prediction markets. "The really valuable knowledge is in the organization as a whole," said moderator James Surowiecki, author of the book The Wisdom of Crowds.

The companies' experiences went as follows:

Google: It's no secret that Google is using prediction markets internally: Bo Cowgill, a Google project manager, described it in a blog post last year.

Cowgill suggested that companies use reputation systems that encourage employee participation by showing whether someone is scoring better or worse in the prediction realm than his colleagues. "You can have winners in each department," Cowgill said, adding that he'd like Google's employee directory to reveal each person's accuracy.

Google offers cash prizes and T-shirts to employees who score well, and the winners are calculated quarterly, Cowgill said. But he still stressed reputation systems as a better method: "In order to make it attractive in a forward sense you have to spend more and more money or the laws are going to have to change. I don't think either is likely to happen anytime soon."

HP: HP Labs has created betting software called BRAIN that's designed for employees and managers; pharmaceutical giant Pfizer will use it starting next year. The unique feature is that BRAIN assigns people a profile based on how risk-averse they are.

To test BRAIN, HP compared it to existing ways of forecasting memory prices--a key figure since memory represents up to 10 percent of the cost of a computer and HP spends billions of dollars on it a year. Instead of a 4 percent error rate for the existing method of forecasting DDR and DDR2 512MB memory prices, the prediction market had a 2.5 percent error rate, or a 37 percent improvement. A second trial to forecast HP Services operating profits yielded similar results.

"We are able to pay real cash money to people without running afoul" of the Commodity Futures Trading Commission said HP's Leslie Fine, because the betting market gives participants only rewards and does not let them lose money.

Yahoo:David Pennock, a principal research scientist at Yahoo Research, said the company has created a currency called a Yootle. It's described as a "scorekeeping system for favors owed."

Pennock offered as an example a programmer offering to write a piece of code for a few Yootles. Or, when organizing a dinner outing, one employee could use an internal SMS tool to bid 2 Yootles for Italian and 4 Yootles for Mexican. "If you don't get to go to the restaurant you want to, you get compensation" in Yootles, he said.

Related to Yootles is Yahoo Research's experiment with a fantasy prediction market for technology called the Tech Buzz Game. It's a modified version of software licensed from NewsFutures in conjunction with O'Reilly Media and features topics like Atlantic hurricanes and portable media devices. Winners are those who predict how popular a topic will be on Yahoo Search.

Microsoft: When Todd Proebsting, director of Microsoft's Center for Software Excellence, tested a prediction market internally, managers quickly gave it their blessing.

The goal: to have 25 members of a development team predict when a Microsoft product would ship (this was an internal product, not one sold externally). The prediction market was set up in August 2004, and the product that "had been in the works for a long time" was scheduled to ship in November 2004.

Featured Video