Research:The role of citations in how readers evaluate Wikipedia articles
As part of an ongoing effort to assess and improve the integrity of the knowledge captured in Wikimedia projects, this project will involve research into the role that citations (references) play in helping Wikipedia readers achieve their learning goals—i.e. the ostensible reason that they have chosen to read a Wikipedia article in the first place.
Previous research suggests that citations can help readers achieve their learning goals in at least two ways:
- by providing them with evidence that the information about a particular topic on Wikipedia is credible, and
- by providing them with signposts to additional information resources about that topic
However, it is not clear whether citations are a major factor in reader's credibility judgements—their decisions to believe, and act on, the information they read in Wikipedia—or whether other content, context, or structural factors are as or more important than extensive citation of reliable sources.
Learning more about the role that citations play in readers' experience of Wikipedia can help Wikimedia contributors focus and prioritize their sourcing efforts, and help Wikimedia Foundation Product teams build features and functionality to support readers’ learning goals and their digital literacy.
- Literature review
- This study will begin with a review of existing academic literature related to reader perceptions of the trustworthiness of web content (including Wikipedia content) and the ways that readers use Wikipedia content to learn in both formal and informal learning contexts.
- Subsequent to that analysis, the researchers will develop a survey instrument, based on the the one developed for Research:Characterizing Wikipedia Reader Behavior, supplemented with additional questions focused on gaining insight into the factors that mediate readers' perceptions of a Wikipedia article's quality and credibility. A link to the surveys will be deployed to a sample of readers on English Wikipedia, when they view articles, using the QuickSurvey extension.
- Ethnographic interviews will be conducted with survey respondents who indicate their willingness to be contacted for follow-up discussions. These interviews will help the researchers gain a deeper understanding into the factors that mediate a reader's trust of Wikipedia content, including but not limited to citations (e.g. when, how, and why they use citations). These interviews will also help tease out what characteristics of a Wikipedia reader or their context of use (information needs, available technology, profession, education level, sociocultural background) are associated with likelihood to engage with citations and the role of other article features that readers draw upon when evaluating the accuracy, neutrality, or usefulness of Wikipedia content. Study information for interview participants.
- July-September 2018
- background research, study plan
- October 2018 - January 2019
- develop, deploy and analyze first survey on English Wikipedia; develop taxonomy of reasons for trusting Wikipedia content.
- February - May 2019
- deploy second-round survey on English and analyze data
- May - July 2019
- recruit for an begin conducting follow-up interviews;
- August - September 2019
- analyze interview results; report and publish project findings
First round (EN)
Survey ran from 2019/1/7 to 2019/1/9. We received 297 complete survey responses (425 responses total).
Related project pages
- Towards Modeling Citation Quality
- Characterizing Wikipedia Reader Behavior
- Characterizing Wikipedia Citation Usage
- Investigating Wikipedia's role as a gateway to medical content
- Student use of free online information resources
- Citation Practices in Wikipedia