Open source and the Long Tail: An interview with Chris Anderson
How does the Wired editor's Long Tail theory apply to open source?
Recently I was fortunate to interview Chris Anderson, editor-in-chief of Wired and the keynote speaker for the Open Source Think Tank, coming up February 7-9 in Napa Valley, Calif. Given Chris' views, I think he's an ideal person to headline an event whose theme is "The Future of Commercial Open Source." (While attendance is by invitation only, you can still apply for admittance.)
Everyone has heard about Chris Anderson's article, book, and blog, The Long Tail. If you haven't, you don't live on this planet (not that there's anything wrong with that). Anderson's theory--that there is big opportunity in lots of little markets--resonates in a world whose technology increasingly permits, encourages, and even requires that we move beyond mass market product development to cater to individual tastes.
As Chris put it in his original Wired article:
For too long we've been suffering the tyranny of lowest-common-denominator fare, subjected to brain-dead summer blockbusters and manufactured pop. Why? Economics. Many of our assumptions about popular taste are actually artifacts of poor supply-and-demand matching--a market response to inefficient distribution.
Free products (or, at least, their discovery) from the physical world, however, and the economics of consumption change. Dramatically.
I spent some time talking with Chris to see how his theory applies to open source. His ideas pushed me to re-examine my own, as my thoughts on how the Long Tail would apply to open source turned out to be a bit naive...
Q: How does open source play into the Long Tail?
Anderson: Anybody writing software for anyone to consume for any reason. That's the Long Tail applied to open source. Open source is fundamentally a marketplace in which anyone can participate. It's a meritocracy. It's a more efficient way of discovering development talent, rather than the old model of hiring out of universities and such.
This has serious repercussions for an industry that is starved for talent. The distribution of participants is more dispersed in the online world. As a result, you get more new ideas, different approaches, talents, and energies that come from surprising places. These would not have been identified in past years. (I interjected that Ross Mason, founder of the Mule project and inhabitant of Malta, is a good example of this.)
Look at Google's Summer of Code. If you map its participants you find develpoers in far-flung places like Madagascar. Things like language, academic training, credentials, and such don't matter as much with open source, widening the talent pool. This should make for better code.
Taste, talent, and demand is much more diverse than traditional markets relfect. Traditional markets reflect traditional bottlenecks with distribution, not necessarily reality. The Long Tail economy gives us a broader distribution of demand; the open-source Long Tail economy gives us a broader distribution of development talent and, by extension, development projects.
Did open source inspire the Long Tail?
No...It actually started with the entertainment world: music, books, movies, etc. When you start talking about a diversity of tastes, music is the obvious example. Wal-Mart is a classic case of constraining the market because its model is to only distribute "hits."
It wasn't until a year afterwards that I really started to look at open source as an example of the Long Tail phenomenon. Open source turns software development on its head. If open source were only available as executables and if it were only sold through physical locations, it wouldn't be the Long Tail explosion that it has become.
But there's more to come. The shift of software from the desktop to the Web will really be the making of open-source software. The Long Tail side of software will almost certainly be Web-based because the Web lowers the barriers to adoption of software. There will always be some software best delivered as packaged bits. But the big problem with packaged software--or one big problem--is the risk associated with installation. It just might not work. The Web removes that problem.
Keep in mind that sites like Google, Digg, etc. are simply open-source projects of a sort. They tend to be heavily modified instances of open source. Hence, any definition of open source that doesn't include Web sites in its definition is going to miss the mark.
Interesting...But will open source survive the Web? One of the big problems with the so-called Web 2.0 companies is that they're net consumers of open source, not net producers.
The Web is built on the 1 percent model. One percent of your users are contributors. Should open source be any different? It doesn't need everyone to be a contributor.
Regardless, the Long Tail of open source is not Linux. It's super-targeted niche services delivered over the Web. That, and the development underlying Sourceforge, is not going to change regardless of contributions from the Web companies.
Are there particular projects that you think have embodied your theory the best?
Sourceforge. This is exactly what I have in mind when I talk about the Long Tail of open source. Many projects on Sourceforge are obscure to the point of being almost impossible to believe that there is demand for them. Many are only written by one person and have only that user. What's fascinating about Sourceforge is that it's nondiscriminatory: the marketplace sorts out the hits.
However, keep in mind that I'm not just talking about open source here. Consider Salesforce.com with its AppExchange. By building an open platform, Salesforce.com is enabling a Long Tail economy around itself.
Your theory suggests that you need "both ends of the curve"--both the short tail (hits) and the long tail (nonhits). Is there a problem with this model if the "grown-up" open-source projects move off Sourceforge? This tends to be the case. Will Sourceforge become less useful? Less Long Tail, in other words?
Good point. I'm not sure. There is a potential problem if the popular open-source projects move away on Sourceforge.
Even open-source software projects tend to be measured on the "hits" basis you disparage. Can you conceive of a company that aggregates the "losers" and makes one big "hit" out of it?
What's the value add? What justifies paying for something that can be had for free? I don't see a need for this. I can't see a model where any particular vendor can support all the disparate bits. A caveat emptor model (like Sourceforge) works. I don't see how Red Hat could economically justify offering the kind of granular project support you suggest. Open source is a Long Tail phenomenon, but any given open-source company is just one player in that larger phenomenon.
Fascinating stuff, and just a glimpse into Chris' thoughts on the Long Tail of open source. Remember, you can still apply for admittance to the Open Source Think Tank. Hearing Chris explore this subject in more detail would be valuable.