Did so many get it so wrong?

As pundits rush to explain what happened on Wall Street, fingers point to the thought leaders. We read exposés on Mary Meeker and Frank Quattrone--once dubbed geniuses--and wonder how they could have sponsored initial public offerings for the nth online grocer or women's portal. Or we ask ourselves how stock analysts could ride a stock all the way down from $150 to $3, all the while touting it as a "strong buy."

These are people who were supposed to be making smart decisions. How did they get it so wrong? The answer is, they didn't.

I submit that the major market players behaved perfectly rationally given the incentives put in place for them. And they were wildly profitable in doing so.

Investment banks certainly weren't offered incentives to temper market valuations during the good times. It's those high valuations that brought in corporate finance business, generating huge investment banking fees and trading commissions. Sure, the banks' reputations were at stake, but as long as the market rose, nobody cared.

This brings us to the sell-side analysts--those responsible for making sound "buy" and "sell" (though more often "buy" than "sell") recommendations on companies' stocks. The analysts are supposed to be the real brains of the operation--those running the numbers and conducting rigorous fundamental research.

Why didn't they put the kibosh on things?

It helps to look at how analysts get paid. Sure, part of analysts' pay depends on how popular they are with investors, as judged by Institutional Investor, a large-scale poll of institutional money managers and the bellwether analyst ranking.

But as analysts increasingly are paid on the corporate finance business they bring in to the firm, rankings become less relevant. And corporate finance clients typically choose investment banks based on the analyst's ability and willingness to "support" their stock in the market, creating a perverse incentive structure for analysts.

Banks will pay their rock-star analysts--those able to attract lucrative corporate finance clients--up to $10 million a year in salary and bonus. It's no wonder analysts have "buy" or "strong buy" ratings on more than 90 percent of the stocks they cover.

Surely institutional investors--large money managers who take significant positions in stocks--had the opportunity to pull their money out of the market when they saw valuations spiral out of control. But institutions had a lot to gain by continuing to play a dangerous game of musical chairs, so long as the music continued to play. Provided that these large investors could get allocations in hot IPOs, which they could later flip to smaller institutions and retail investors--at much higher prices--they made money hand over fist.

Although investment banks historically penalized this behavior, many looked the other way during the frothy market. Some argue that the investment banks further provided incentives to investors to flip through the long-established convention of underpricing IPOs.

Even though demand for IPO shares spiked during the 1998-1999 time frame, IPO premiums rose only slightly, creating an even greater demand imbalance than already existed. Who could resist tripling or quadrupling an investment inside of 10 seconds? Even momentum investors--investors who "pile on" to a stock trend, buying when the stock is moving up and selling (or short selling) when it is moving down--acted according to incentives. When the good momentum players occasionally did get caught on the wrong side of a stock movement, they took their medicine and waited to get back into the game.

As much as investors were willing to take a risk and buy a high-priced stock, there were always retail investors willing to pay a higher price. This was an incentive to investors to bid up stock prices indefinitely and then pull out at the first sign of a downturn.

Musical chairs
It's the old familiar story. When the music stopped playing and everyone scrambled for a chair, the retail investors were the only ones left.

But how do retail investors protect themselves from getting fleeced by the professionals next time? Here's a clue: Take a look at the incentives. Rather than trying to outsmart the market, why don't retail investors align their incentives with those of money managers by investing in the funds they manage?

If the buck stops with the money manager the next time around, you can bet he or she will invest a lot more wisely.