Research:Accuracy and quality of Wikipedia entries

Add links
From Meta, a Wikimedia project coordination wiki
Contact
Naomi Norman
Imogen Casebourne
Chris Davies
Michelle Fernandes
This page documents a completed research project.


Key Personnel[edit]

This research is funded by a grant from the Wikimedia Foundation and carried out by a team of researchers based at Epic, a UK-based e-learning company, and at the University of Oxford. Key personnel on the project include:

  • Naomi Norman, Epic (PI)
  • Chris Davies, University of Oxford (co-PI)
  • Michelle Caroline Fernandes, University of Oxford (RA)
  • Imogen Casebourne, Epic

Project Summary[edit]

The objective of this project is to assess the accuracy and quality of a small sample of Wikipedia entries across a limited range of subjects and languages. The goals are:

  1. to identify errors, omissions and other quality issues in a sample of Wikipedia articles
  2. to compare the number of errors, omissions and other quality issues in a corresponding sample of articles on the same subjects from popular online encyclopedias for each language considered
  3. to compare article quality between the two samples using both expert reviews and reader feedback.

The intention is that this pilot project will establish the best possible research approach to measure and surface the accuracy and quality of articles in Wikipedia, and to provide preliminary methodological recommendations towards a more comprehensive study of the quality of Wikipedia articles across subjects and languages.

Background Information[edit]

In December 2005, the scientific journal Nature, undertook a study to compare the accuracy of science entries on Wikipedia with those of the online version of Encyclopedia Britannica.[1] The key difference between the two encyclopedias is that while Wikipedia relies on voluntary contributors, regardless of academic background, whereas entries in the Encyclopedia Britannica are compiled by selected paid expert advisors and editors. In forty-two reviews that were submitted, on average, four inaccuracies were identified per Wikipedia article in comparison to three per Encyclopedia Britannica article. Encyclopedia Britannica refuted the study’s findings, stating lapses in the methodology and asking for Nature to issue a public retraction of the article.[2] Nature responded by rejecting Encyclopedia Britannica’s criticisms and refused to retract.[3]

In the period following 2005, Wikipedia has been working hard with its readers and editors to improve the accuracy and quality of its entries. The current project is a small-scale preliminary study, inspired by Nature’s study, but that aims to employ greater rigor, involving academics and scholars, and examining more than just English language entries and in a broader range of subjects.

Methods[edit]

Recruitment[edit]

Graduate students from all academic disciplines at the University of Oxford will be contacted to participate in the study. The students will be asked to identify five or six respected academics in their field of study.

Sampling[edit]

Eight graduate students and twenty-four expert academics will comprise the review panel. The review panel will consist of native English, Spanish and Arabic speakers. The panel will be evenly distributed across four disciplines: humanities; social sciences; mathematical, physical and life sciences and medical sciences.

Review Process[edit]

Each student and expert academic will be asked to review two pairs of articles in their relevant subject domain and native language. In each pair, one article will be a Wikipedia entry and the other will an article from a popular alternative online encyclopedia. The students and expert academics will not be aware of the source of the articles. The articles will be assessed for a number of factors relating to the quality of the article. These will include comprehensiveness, accuracy, verifiability by references and objectivity. Articles will be rated using a survey tool specially designed for the study. The participants will be asked to justify their judgments, citing relevant references, and suggest appropriate corrections.

Results[edit]

Dissemination[edit]

The research will be presented at relevant conferences and seminars (such as Wikimania, the Oxford University Department of Education). The output of this research will be published in an open access journal.

Wikimedia Policies, Ethics, and Human Subjects Protection[edit]

Our foremost priority is to conduct our research in an ethical, respectful, and non-disruptive manner. We will ensure that we conform to strict standards of informed consent and transparency in data collection methods. All participants will be informed about our affiliation, purpose and research goals. We will make all efforts to address any risks associated with participation in this study.

Benefits for the Wikimedia community[edit]

This work will help to:

  1. identify errors, omissions and other quality issues in Wikipedia articles
  2. determine comparisons in quality between Wikipedia entries and analogous entries from a popular alternative online version of an encyclopedia
  3. compare reviews from readers with those of experts.

The results of this study will inform the design of a large-scale, empirical project across a range of languages and subject domains.

Time Line[edit]

November 2011

  • Sample and recruit students and academic experts
  • Identify pairs of articles of relevance

December 2011

  • Write and build article review

January 2012

  • Complete data collection
  • Collate relevant data from Wikipedia

February 2012

  • Code data
  • Data analysis

March 2012

  • Draft report
  • Submit final report

April 2012

  • Presentation of research

Funding[edit]

This project is supported by a grant from the Wikimedia Foundation.

References[edit]

  1. ↑ Giles, J. (2005) Internet encyclopaedias go head to head, Nature, vol.438, 15 December 2005, pp. 900-901.
  2. ↑ Encyclopædia Britannica, Inc. (March 2006), Fatally flawed: refuting the recent study on encyclopaedic accuracy by the journal Nature, [Online], Available at: http://corporate.britannica.com/britannica_nature_response.pdf [Accessed 11/03/11]
  3. ↑ Nature (23 March 2006), Encyclopaedia Britannica and Nature: a response, [Online], Available at http://www.nature.com/press_releases/Britannica_response.pdf [Accessed 11/03/11].

See also[edit]

There is source content on Wikisource:

External links[edit]

Contacts[edit]

Retrieved from "https://meta.wikimedia.org/w/index.php?title=Research:Accuracy_and_quality_of_Wikipedia_entries&oldid=20103604"