Perspective: Tech's big challenge: Decentralization

What's the connection between Wi-Fi wireless networks, Weblogs and Web services? They are among the few technologies thriving amid the industrywide downturn. What's more, they are examples of the trend toward decentralization.

In the coming decade, decentralization will be the critical challenge for the technology, media and telecommunications industries. Each has developed with the assumption that powerful central forces will manage development. Enterprise IT has "big iron" servers and monolithic software applications; communications has carriers investing in huge infrastructure build-outs; and media has content owners controlling distributions channels.

These approaches are under siege--and not because there's a New Economy, or because information deserves to be free, or because of any fluctuation in the stock market. Centralized systems are failing for two simple reasons: They can't scale, and they don't reflect the real world of people.

The world is becoming increasingly complex. Companies manage supply chains in real time, while hundreds of thousands of gamers gather in shared virtual worlds. Networks must carry vast and growing amounts of traffic, with no end in sight. Centralized systems eventually crumble under the strain of that complexity.

Decentralized approaches often seem impractical, but they work in practice. The Internet itself is a prime example--it works because the content, the domain name system and the routers are radically distributed.

But it's the human element that is really driving the pressure for decentralized solutions. This shouldn't be too surprising. Biological phenomena like the human body and the global biosphere have had billions of years to evolve, and they are the most complex decentralized systems we encounter.

More concretely, people are seeking ways to communicate and collaborate across the artificial boundaries of organizations and geography. They want their music, on their terms, just as they want high-speed connectivity anywhere, any time.

Decentralized approaches often seem impractical, but they work in practice.

I know what you're thinking. This sounds like a warmed-over version of the peer-to-peer (P2P) hype that people like me were spouting during the glory days of Napster.

Napster is dead, and so are most of the copycat start-ups that followed it in search of bubble-era IPO nirvana. Yet take a closer look. By using a decentralized architecture, Napster saved $600 million in storage costs and over $6 million per month in bandwidth expenses, according to Bear Stearns analysts Chris Kwak and Robert Fagin. Name one company that wouldn't want to enjoy those savings, especially in a difficult economy.

Moreover, decentralization is much broader than Napster-like file sharing.

Take Wi-Fi, for example. As cellular carriers struggle to build out expensive 3G wireless networks, people are adopting Wi-Fi in droves. Wi-Fi access points are popping up everywhere, making up for what they lack in ubiquity with radically lower costs, rapid innovation and easier deployment.

The mantra of Sun founder Bill Joy, one of the smartest people in technology, is that most smart people don't work at Sun. His statement is true of any company, and it's an insight relevant in many other areas.

No software application hosts all the critical data in a company. No media outlet produces all the good news analysis. And no network operator controls all the access points or all the users. We must build systems that take advantage of the power of the network, but that do so in ways that make business sense.

Lotus Notes creator Ray Ozzie, among the world's most celebrated software designers, is another believer in decentralization. His new company, Groove, offers a business collaboration platform that puts most of the smarts on each user's computer, rather than in central servers.

Ozzie saw how real businesspeople worked through fluid, cross-organizational teams, and how they struggled against central information technology administrators. He also saw how his own children voraciously adopted instant messaging and multiplayer online gaming, two applications that have been decentralized from the beginning.

Because decentralization inherently breaks down boundaries, it cuts across familiar industry categories. It's everywhere you look...but that makes it easy to miss. Some of the smartest software executives I know aren't aware of the revolutionary potential of open spectrum, while most telecom experts don't understand Web services.

Parallels abound. As it struggles with digital content distribution, for example, the entertainment industry could learn from the experience of computer companies such as IBM that had to reorient their business models away from centralized mainframes and proprietary systems. But it's hard for people to see the forest when they're used to scrutinizing the trees.

Microsoft faces a similar problem explaining its .Net initiative, which is basically a commitment to decentralized computing. The Redmond giant has done a poor job communicating what .Net is about, causing much head-scratching in the technology world. Part of the problem, I believe, is simply that describing decentralization is hard. Our minds always want something central and stable to latch onto.

The entertainment industry could learn from the experience of computer companies such as IBM that had to reorient their business models away from centralized mainframes and proprietary systems.

Microsoft is also trying to walk a fine line. It sees itself benefiting from decentralization because more power in the PCs and other devices at the edges of the network means more systems running Windows. Users and competitors, however, often see Windows itself as a central authority. Linux competes effectively by decentralizing the software development process Microsoft controls in-house.

As the Microsoft example shows, decentralization is neither automatic nor absolute. The most decentralized system doesn't always win. The challenge is to find the equilibrium points--the optimum group sizes, the viable models and the appropriate social compromises.

Ray Ozzie, for example, imagined Groove as a tool for groups of two to 25 people. Any more would probably result in too many voices for a coherent conversation.

Indeed, most Groove "shared spaces" fall into the expected range. Surprisingly, though, 25 percent include only a single member, and hundreds of Groove spaces have 100 or more members. To better understand and respond to decentralization, this is the kind of data we need to examine in more detail.

Although decentralization is a long-term challenge, the good news is that it's also an opportunity. Businesses that can capitalize on decentralization--as both creators and users of technology--will be best-positioned for the future.