Wikipedia as a Bottom-Up Process

In March 2000, an entrepreneur named Jimmy Wales hired a newly-minted philosopher named Larry Sanger to start a new kind of encyclopedia. Called Nupedia, its goal was to harness the power of the Internet to build a high-quality, volunteer-driven reference work for the masses. Anxious to maximize quality, Wales and Sanger instituted a rigorous peer-review process: Before content appeared on the site, it had to be thoroughly scrutinized by subject-matter experts to ensure it was of the highest quality.

The project went nowhere. The review process proceeded at a glacial pace, with only a handful of articles being published in the first year. In January 2001, Sanger suggested augmenting Nupedia with a side-project called Wikipedia, which anyone would be able to edit without going through the cumbersome Nupedia review process. They didn’t expect Wikipedia articles to be very good, but they hoped they might produce some raw material that could later be fashioned into publishable articles by the Nupedia editors.

Of course, you know how this story ends. Nupedia continued to languish, producing a grand total of 24 articles in its lifetime. Wikipedia took off faster than anyone—including Sanger and Wales themselves—could have imagined. It had 10,000 articles before its first birthday and 100,000 articles before its second. Wales soon realized that the Wikipedia effort didn’t need an editor-in-chief and laid Sanger off. And the site kept growing; it’s on track to reach 3 million articles by the end of 2009.

Wikipedia’s success was so counter-intuitive that even its founders didn’t anticipate how well it would work. If you’ll forgive me for citing Clay Shirky again (I’ll be doing that a lot on this blog), here’s how he described it in an interview I did with him last year:

When people first hear about wikis, they always say, “Wow, that’s really cool.” And then about two minutes later, everybody has the same reaction: “Oh, that actually won’t work.” People just don’t think that that pattern can create new value. The social problems seem too obvious and intractable. The mind-shift is… that you have to start looking at Wikipedia as a different kind of product than Britannica. Wikipedia actually is an ongoing process…

It’s like a coral reef. The action is actually in the edits and the arguing: the living part of it. The calcified deposits, the stuff that nobody’s arguing about any longer, is what most people experience as Wikipedia, in the same way that most people experience a coral reef as the residue of what all the little creatures are doing.

One of the mistakes people tend to make in evaluating bottom-up systems is that they tend to underestimate how much happened “under the hood” to produce a finished product. Each little creature toils away his whole life to add a tiny bit of material to the coral reef. Similarly, it’s not uncommon for Wikipedia editors to thousands of words arguing over one paragraph or even one sentence of an article. The Wikipedia article on Michael Jackson is about 15,000 words long. The discussion section on the Michael Jackson goes on for 27 pages, each containing about 10,000 words of discussion. All that complexity is hidden from a casual user, who only sees the tidy, finished product and not the long, messy process that produced it.

To bring things back to yesterday’s discussion of Charles Darwin, I think a big part of the reason people find evolution counterintuitive is that it's really hard to grasp the concept of a billion years. Lots of people think the human body is too complex to be the result of trial and error, but that's because it's extremely difficult to grasp just how much trial and error it took. Indeed, it likely took so many steps that we couldn’t possibly understand them all even if we tried. Therefore, the key to thinking clearly about it is the one Shirky recommends: to focus on understanding the the process on its own terms. We can make some useful observations about how natural selection and the Wikipedia editing process work, but we’re never going to be able to understand or predict how the processes will play out in every individual circumstance.