Research:The Role of Images for Knowledge Understanding

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search
12:04, 20 March 2020 (UTC)
Katharina Reinecke
Duration:  2020-April – 2021-
Light-Bulb icon by Till Teenck.svg

This page documents a proposed research project.
Information may be incomplete and may change before the project starts.

In this project, we study the importance of images in knowledge understanding through Wikipedia. With a set of large-scale user studies, we will quantify and qualify how readers around the world rely on images when navigating knowledge in Wikipedia. The final goal of this project is to generate insights regarding how much readers from different demographics use images (intentionally or unintentionally) when reading Wikipedia, and to analyze the layouts and the types of images which are more useful for knowledge understanding in Wikipedia, across different article topics.


Wikimedia Research’s Knowledge Gaps white paper underlines the need for methods to quantify the global usage of multimedia content in Wikimedia spaces. As part of this strategic direction, we want to conduct research on the role of images in knowledge understanding and learning through Wikipedia. Understanding these aspects is crucial to enable future work in the area of machine learning for image search and classification. While both the Wikimedia Research team and the WMF organization are planning to deploy techniques to improve discoverability of images and pictorial representations of articles, we actually do not know the extent to which readers use images when navigating the site, and the role visual content plays in free knowledge diffusion around the world.

Research Questions[edit]

  • Do images help readers learn or understand better what they are reading?
  • For which kinds of tasks/articles are images more useful?
  • What type of images or visualizations [infographic, flowchart, picture, 3D images, moving images, graphs, etc] were useful in this context,i.e., the tasks/articles/field?
  • For which cultures/countries are images useful, and does learning [behaviour and values] through images change if the reader is a native speaker of the language of the text?
  • Which images are more/less useful for this task, and where do they appear in the text?
  • And are images more/less useful for different demographics?
  • What tasks or objectives do images and visualizations perform or achieve in the process of information dissemination, attention and depth of knowledge gained and total user/readers' experience?


To answer these research questions, we will design a large-scale user study.

Task. In this study, participants will be asked to read a Wikipedia article, and then answer a set multiple-choice questions about its content. Questions could also be given to participants beforehands, in a “scavenger hunt”-type of experiment. After each article, an “accuracy” score reflecting how much the participant understood the article will be calculated based on the answers provided to the multiple-choice questions.

Interventions. Articles will be artificially manipulated to add/remove/displace images, and the effect of such interventions on the accuracy score will be recorded. Articles from different topics will be used, to understand how the importance of images varies across different areas of knowledge.

Platform. We will run the study on the Lab in the Wild platform. "Lab in the wild" seems like an ideal setting to test tasks on understanding/learning with images in Wikipedia for the following reasons: (1) It has an audience of diverse people speaking many different languages and coming from ~40 countries, and allows localization and translation to multiple languages; (2) Its experiments are "gamified" by nature: contributors are not paid, what they get as a reward is a feedback regarding a skill, and how they compare to others; (3) It is open (people don't have to sign up and register).


Policy, Ethics and Human Subjects Research[edit]

An umbrella IRB approval is available for LabintheWild studies that are of similar nature, collect similar demographics, and carry minimal risk to participants. For this study, we will have to get approval for the informed consent form that participants see at the start of the study as a modification to the umbrella IRB.


Using data collected from the large-scale user study we have been able to identify unique factors that contribute to participant accuracy for specific question types. During the study participants were asked about non-visual, visual, and image aspects of the article content that they saw. This was done by separately asking participants non-visual, visual, and image questions generated from the article content manually. With each article we randomly chose for them to see the respective Wikipedia article infographic image. Using R libraries we have ran mixed-effect models for each question type.

  • For non-visual question how long participants took on viewing the article before answering questions along with their demographic information of education level and English proficiency made a significant impact on their accuracy.
  • For visual question both article time and English proficiency also had a significant impact. For this question type we also saw the time that participants took on the question having an effect as well.
  • For image questions question time and article time came out to be significant, but this was the only type of question where images had an impact.

Besides identifying what variables have an impact on participants' ability to retain certain article knowledge we were also able to contribute by creating the dataset of articles and questions used in this study.

On Phabricator[edit]

Follow this task.