The epistemology of Wikipedia

I'm not actually going to start writing this yet. It just struck me as a great idea for a column--assimilating some issues that were raised in a recent thread in sci.physics.relativity. The column would answer such questions as:

Wikipedia doesn't require any sort of credentials for contributors. Why think it is in the least reliable? "You can't really evaluate the validity of something unless you know something about the author and his/her qualifications to speak on the topic and/or you are provided with the appropriate references to support his/her claims." That's a common enough attitude--what, do we think that's wrong? Why?
Is there anything about the Wikipedia process by itself, unaided by an approval process, that tends to the overall improvement of the reliability of the articles?
Would it be rational, at some point in the future, to trust what any random article from Wikipedia says?
It is possible that ten anonymous people, whose credentials no one knows, to work on a single article. If such an article is more reliable than an article worked on by, say, just one or two people, why is that?
Generally, how can recent insights in social epistemology be applied and/or tested by Wikipedia's experience?
If I am an expert, and I want the truth stated as accurately as possible, then why on earth should I contribute to a project wherein my words can be edited by all manner of nonexperts who don't even really understand them? Since nothing's stopping a crank from replacing my brilliant prose with nonsense, what's the point?
It might be interesting to compare the Wikipedia process with the pragmatic theory of truth--the notion that truth is whatever is considered true within the "ideal limit of inquiry." (I think this theory is false, by the way, but the comparison would be interesting.)

I think these questions have very interesting answers, and the questions and answers can be organized in a sort of systematic way that explores the methodological and epistemological underpinnings of why Wikipedia works. If you want to start discussing this stuff without me, go ahead. :-) I guess I'd use any such discussion to help build the essay.

--Larry_Sanger

Yes, I think the Wikipedia process can result in articles better than those written by single persons no matter how qualified, and for good reason. Individual humans, even--indeed especially--experts, have egos, biases, and cultural assumptions; but groups of humans will tend to average those things out. Very few prople are even capable of seeing all of their prejudices, and need others of different backgrounds to point them out. That's one reason why humans have things like conferences, so that experts can get together and check each others assumptions. But this process is even superior to that. The process here also eliminates the intimidation and control factors of more common human interactions--nobody can be shouted down, and everyone's text looks the same. Likewise, the time limits of human conferences don't apply. If you can't think of a good reply to a question in a conference in a few seconds, it doesn't get asked. Here, it may take days or even weeks, but it does get asked and usually answered. And the physical limitations of getting like-minded people together don't apply.

You can't get away with evasion here--no assertion goes unquestioned, and no questioner is asked for his "credentials", as it should be. "Authority" is nothing but a useful shortcut used in the world of humans because we haven't had the luxury of technology like this that makes expertise less relevant. What matters is the argument, not the arguers, and this technology supports--indeed enforces--that. Facts are facts, no matter who writes about them, and even non-experts in a field can tell when an expert is avoiding the facts or pushing bogus arguments. Should experts then post here? Absolutely, but only those with the stomach to face the opposition and the backbone to stand behind their case with good arguments. If they became experts because they got it right, this medium will make them shine. If they became experts through bluster and bluff, this medium will expose them. Should I trust a random article? Certainly not. But I certainly trust my own bullshit-detection skills, combined with my knowledge of how the process works here, to be able to separate the wheat from the chaff. There will be a lot of chaff, but I fully expect the wheat here to surpass the quality of traditional works for all of the above reasons. --Lee Daniel Crocker

I agree with most of this, but think that the chaff here is somewhat an effect of people starting with blank slates. Once basic articles are written, the person who has an english major cleans up the grammar, a philosopher looks at the logic, a specialist in a related field looks at the data and compares it to what he or she knows. Some of the articles I have written contained errors, but over time they have been ruthlessly weeded out by everyone who looked at them and made corrections. Once a framework is in place, it actually becomes easier to improve articles, as the small tidbits of information that I have becomes part of a mass of information from many sources. Can I trust an article here? I think so, but I also need to look at the /Talk that has gone on, and some of the past revisions to see if the article is brand new, or has been seen by enough "eyeballs". Research based on information here requires some judgement instead of just blind faith. -- mike dill

--- Slightly off-topic, but all that's needed to certify Wikipedia is to test a good statistical sample of its articles with certified experts. This would show how closely Wikipedia approaches a conventional encyclopedia. I'd think a good statistician could correlate or regress the test data with automatically-collected data, like number of revisions, number of authors, etc, to test the above hypotheses. In turn, these facts would then permit the hapless coder(s) supportin WIkipedia to produce a list of articles that probably need attention, and maybe even diagnose what type of attention. Ray Van De Walker