Research:Knowledge Gaps Index/Taxonomy/Full paper

This paper, titled "A Taxonomy of Knowledge Gaps for Wikimedia Projects", was previously made available at https://arxiv.org/pdf/2008.12314.pdf . Licensed under CC-BY.

A Taxonomy of Knowledge Gaps for Wikimedia Projects (First Draft)

MIRIAM REDI, Wikimedia Foundation

MARTIN GERLACH, Wikimedia Foundation

ISAAC JOHNSON, Wikimedia Foundation

JONATHAN MORGAN, Wikimedia Foundation

LEILA ZIA, Wikimedia Foundation

EXECUTIVE SUMMARY

In January 2019, prompted by the Wikimedia Movement’s 2030 strategic direction, the Research team at the Wikimedia Foundation identified the need to develop a knowledge gaps index—a composite index to support the decision makers across the Wikimedia movement by providing: a framework to encourage structured and targeted brainstorming discussions; data on the state of the knowledge gaps across the Wikimedia projects that can inform decision making and assist with measuring the long term impact of large scale initiatives in the Movement.

Since July 2019 and as the first step toward building the knowledge gap index, the Research team has developed the first complete draft of a taxonomy of knowledge gaps for the Wikimedia projects. We studied more than 200 references by scholars, researchers, practitioners, community members and affiliates—exposing evidence of knowledge gaps in readership, contributorship, and content of Wikimedia projects. We elaborated the findings and compiled the taxonomy of knowledge gaps in this paper, where we describe, group and classify knowledge gaps into a structured framework. The taxonomy you will learn more about in the rest of this work will serve as a basis to operationalize and quantify knowledge equity, one of the two 2030 strategic directions, through the knowledge gaps index.

We hope you enjoy the read and join the conversation about how we can improve the taxonomy of knowledge gaps for the Wikimedia projects. If you have any suggestion or feedback, please reach out to us via the Knowledge Gaps Index Meta page.

INTRODUCTION

With almost 54 million articles written by roughly 500,000 monthly editorsdoes this mean 500,000 different editors in a month? Something else? across more than 160 actively edited languages, Wikipedia is the most important source of encyclopedic knowledge and one of the most important knowledge resources available on the internet. Every month, the project attracts users on more than 1.5 billion unique devices from across the globe, for a total of more than 15 billion monthly pageviews.

While global and massively popular resources, Wikipedia and its sister projects such as Wikimedia Commons and Wikidata, suffer from a wide range of knowledge gaps which we define as disparities in participation or coverage of a specific group of readers, contributors, or content.

A typical example of a knowledge gap is the gender gap, one of the most well-studied gaps in the Wikiverse. Researchers and practitioners have investigated the gender gap by measuring the representation of different gender groups in content, readers, and contributors, and found, for example, that less than 20% of the biographies in Wikipedia are about women, roughly 75% of readers in Wikipedia are men, and that this disparity becomes even more extreme when analyzing editors’ gender distribution.

Beyond the gender gap, only a handful of works have studied ways to quantify or address other kinds of knowledge gaps—for example by analyzing readers’ and editors’ motivations for accessing the site, studying the role of technical skills and awareness in Wikimedia contributorship and readership, designing algorithms to identify and prioritize missing content in different languages, or proposing question-and-answer facilities to satisfy readers’ information needs.

While these works made substantial progress towards measuring and identifying directions for bridging individual gaps, they did not provide a holistic and systematic framework to understand the scope of knowledge gaps in Wikimedia and the relationships between them.

To see why having such a comprehensive map of the space of knowledge gaps could be key for Wikimedia projects, consider the main directions identified as part of the Movement Strategy: by 2030, Wikimedia will become the essential infrastructure of the ecosystem of free knowledge, and achieve knowledge equity by focusing efforts on including all “knowledge and communities that have been left out by structures of power and privilege”. To support this goal, and measure the progress towards knowledge equity, the Research team at the Wikimedia Foundation has identified the need to operationalize knowledge equity into a knowledge gaps index – a composite index tracking the collective evolution of knowledge gaps in Wikimedia.

While extremely important, operationalizing knowledge equity by studying and measuring its underlying factors is not a trivial task. The first step towards this goal is to generate a systematic picture of those Wikimedia audiences, groups, and cultures that could be underrepresented in terms of participation, representation, and coverage.

Therefore, in this work we propose the first taxonomy of knowledge gaps in the context of Wikimedia projects. We identify three macro-dimensions of the Wikimedia ecosystem, namely Readers, Contributors, and Content as the root of the taxonomy. We then review studies and discussions from scholars, researchers, practitioners and community members, and compile for each of these dimensions a list of knowledge gaps as areas of the Wikiverse where we found evidence of potential inequality. Finally, we group these gaps into facets containing highly related knowledge gaps. The final 3-layer taxonomy is illustrated in Figure 1.

Fig. 1. The Knowledge Gaps Taxonomy

This taxonomy provides a theoretical framework based on which we can develop different approaches to address knowledge gaps. First, it constitutes a necessary first step for the development of robust metrics and indicators to quantify gaps, which allow to track progress towards knowledge equity. Second, it facilitates the conceptualization of the interdependence between different gaps in order to understand potential causes and barriers, and to design more effective interventions. Third, with this taxonomy, we aim at fostering conversations across the Wikimedia and academic communities around the nature and composition of the “content and communities that have been left out by structures of power and privilege”.

In the remainder of this paper, we will first present each macro branch of the taxonomy: Readers in Section 2, Contributors in Section 3 and Content in Section 4. We will then provide some suggestions on how different audiences can use and benefit from this taxonomy (see Section 5), expose interesting directions for future work in Section 6, and finally look at the methodology used to compile the taxonomy (Section 7).

READERS

The readership dimension of knowledge gaps encompasses all those gaps related to readers’ access to Wikimedia sites. We identify different areas where readers are under-represented according to literature, surveys, and community strategic directions and organize them around three main facets: sociodemographics, information needs, and accessibility. We also highlight different initiatives aiming at closing each gap. We define readers as all users who connect to the site to consume Wikimedia content. While there exists a body of research studying how content consumption happens outside of Wikimedia, e.g., voice assistants, search engines, or thirdparty apps, we scope down our definition of readership to readers who come directly to the projects to access content. The objectives for this dimension are mainly motivated by the 2020 Movement Strategy, which recommends that Wikimedia platforms should be designed to “enable everyone—irrespective of gender, culture, age, technological background or skills, or physical abilities—to enjoy a positive experience during both consumption and contribution to knowledge throughout the Wikimedia ecosystem”.

Facet	Gap	Description	Source
Sociodemographics Objective: readers with different social status, demographics, and cultural background can easily and safely accessing free knowledge	Gender	Difference between readers of different gender identities in how and how much they access the sites.	literature, surveys, strategy
	Age	Difference between readers of different age in how and how much they access the sites.	literature, surveys, strategy, community
	Locale	Differences in readership between rural areas, towns, and cities	literature, surveys, community
	Language	Differences in readership depending on readers’ ability to read one or more languages	surveys, strategy, community
	Income	Difference on how readers with different income, wealth, or employment status access Wikimedia sites	literature, surveys
	Education	Differences in readership depending on readers’ educational background	literature surveys, community
	Beliefs	Difference in how and how much people having different beliefs access content on Wikimedia sites	community
	Background	Differences in readership among people with different cultural, political and sexual preferences	community
Information Need Objective: readers with different information needs can find and consume free knowledge	Motivation	Differences in readership depending on the reason behind readers’ visit to the site	literature, surveys
	Information Depth	Differences in readership depending on the depth of information for which a reader is looking	literature, surveys
	Familiarity	Differences in readership depending on one’s prior familiarity with a topic	literature, surveys, community
Accessibility Objective: readers with different technical setup and skills can easily access Wikimedia projects	Internet connectivity	Contrasts among the ability of readers with different internet connections to access Wikimedia sites	surveys, strategy, community
	Device	Difference in accessibility to the site depending on readers’ devices	surveys, strategy, community
	Tech Skills	Differences in readership depending on readers’ general internet skill	literature, strategy, other
	Disabilities	Disparities in ability to access the knowledge within Wikipedia depedning on individual disabilities	literature, community

Sociodemographics Gaps

Sociodemographic readership gaps have been widely discussed across the Wikimedia movement, exposed by surveys organized by the Wikimedia Foundation and the chapters, and analyzed by research studying the Wikimedia ecosystem. This facet includes gaps in readership related to demographics such as gender, age, language and location, and social status such as income and education. The objective for this facet states that different socio-demographics groups should not face greater barriers to their ability to access and consume content through Wikimedia projects.

Gender

The gender gap is the difference between readers of different genders in how and how much they access the sites. The gender gap has been measured through many readership surveys organized by the Wikimedia Foundation, different chapters, independent organizations and research communities by asking respondents for their gender identity. This makes the gender gap one of the better understood constructs with respect to readership. However, almost all surveys have only included men and women as gender identities, making it difficult to draw any conclusions about readership among people with non-binary gender identities. Furthermore, conclusions about gender gaps might differ depending on the definition of readers. For example, the most recent survey in this space defined readers as people who read Wikipedia daily, and found that, in almost every country/language surveyed (with the exception of Romanian), there is a substantial gender gap, i.e. men tend to generate many more page views than women even in regions where previous surveys have shown no strong gender gaps in occasional usage. A number of factors cause some genders to be underrepresented in Wikimedia’s readership population, and across the years, different community initiatives have focused on bridging this gap.

Age

The age gap reflects how and how much readers of different age access Wikimedia sites. There exists a large volume of data on readers’ age, collected by surveys and academic literature. In cases where the agedistribution at country level is available (so the age distributions can be compared against the full population), the data indicates that readers tend to be much younger than the general populace. Among other things, the quality and presentation of content might affect this gap, and various initiatives across the movement focus on making content more available and readable for specific age ranges, such as Simple English Wikipedia, the Wikijuniors project from Wikibooks, or WikiProject Accessibility, which, among other things, aims at making content more accessible to elderly readers.

Locale

The locale gap refers to the different levels of readership between rural areas, towns, and cities. Proximity to urban areas tends to be a strong proxy for availability of services. While there are no global standardized definitions for how to define urban/rural, which makes it difficult to compare self-reported locale to official statistics, results from WMF-led and academic surveys suggest strong over-representation of urban areas among the reader population. To help bridging this gap, projects such as WikiConnect and the WMF’s “New Readers" initiative work on connecting Wikipedia with rural areas and those people who don’t have direct access to Wikipedia.

Language

The language gap reflects the different levels of readership depending on readers’ ability to read one or more languages. What languages an individual can read greatly impacts what content is available to them and can introduce greater barriers if they are forced to read content in a language that is less familiar to them. Surveys have been conducted to estimate readers’ literacy suggesting that certain languages have highly-literate readers. For example, languages that are specific to one country show high levels of literacy amongst readers. In contrast, other languages such as English or French, which are more strongly associated with colonialism, have many readers for which English / French is not their native language. In order to address this issue, in English Simple Wikipedia was introduced using a simpler grammar and a limited vocabulary. While improving readability in comparison to English Wikipedia, research has shown that its level is still not ideal for readers with limited language literacy. Other community initiatives attempting to bridge this gap aim at growing under-represented languages, for example Scribe or the GapFinder tool.

Income

The income gap is the difference in how readers with different income, wealth, or employment status access Wikimedia sites. Income, wealth, and employment status are three separate constructs, but they all relate to an individual’s means and class status. Of note, it can be difficult to disentangle the impact of these concepts from other, highly-correlated measures such as education level. Surveys and recent research consistently indicate that people with more income are also more likely to be readers of Wikipedia though the relationship between employment status and readership is less clear. The other strong, and sometimes countervailing, trend is that students also tend to be much heavier readers of Wikipedia.

Education

The education gap reflects differences in readership depending on readers’ educational background. While education systems vary across different countries, one commonly-used country-agnostic proxy for education level is the number of years of education. Surveys from researchers and from WMF and affiliates consistently demonstrate that individuals with higher levels of education are more likely to be readers of Wikipedia. Some Wikimedia projects are trying to close this gap, by specifically targeting readers with different levels of education, for example Wikiversity; relatedly, initiatives such as “Wikimedia+Education” specifically focus on creating community around the topic of making Wikipedia more integrated with education systems at different levels.

Background

The background gap includes all differences in readership among people with different sexual preferences or cultural, political and religious beliefs. Studies on readers from emerging markets found some evidence that local socio-political constructs might impact the way in which Wikipedia’s credibility is perceived. While we could not find studies of how sexuality and culture relates to readership, there have been discussions of how sexuality is represented in content that make it clear that a reader’s sexual identity could relate to how much they trust and feel safe within Wikipedia, and initiatives such as the "Wikimedia LGBT+ Portal" actively work on creating a “safer environment for LGBT+ readers of Wikimedia projects, with a special interest in improving the experiences of LGBT+ youth on Wikimedia projects”. Finally, Wikipedia by nature aims to structure its content to be welcoming to readers with different individual identities, perspectives and opinions. For example, the “Neutral Point of View” policy encourages editors to represent “fairly, proportionately, and, as far as possible, without editorial bias, all the significant views that have been published by reliable sources on a topic”. Most friendly space policies in Wikimedia also encourage inclusiveness at many levels, including the respect of people’s religious views. Moreover, surveys around community health report religion as one prominent factor creating barriers to involvement in Wikimedia communities.

Ethnicity and race bring additional context to an individual’s cultural background and often relate strongly with access to resources. Studies in the United States have shown gaps in usage along racial lines and presumably these same trends are found in other countries in that people who are disenfranchised due to their race or ethnicity also would be less likely to access Wikipedia.

Information Need Gaps

This facet deals with the motivation, information needs and prior knowledge held by a given reader, and how those affects their likelihood to read Wikipedia. Hypotheses for why readership might vary under these different information need contexts include readers’ perceptions of the utility of Wikipedia, their ability to locate information effectively, and external re-use of Wikipedia content (e.g., rich search results) that satisfy readers’ needs before they reach the Wikipedia platform. It is nearly impossible to predict a given reader’s motivation, familiarity, or information depth based upon the article they are reading and anecdotally readers experience the full range of these contexts over time or even within a given day or reading session.

The objective in this facet is to make the site and its content suitable for people with different information needs.

Motivation

The motivation gaps reflect different levels of readership depending on the reason behind readers’ visit to the site. There is quite a bit of variability between languages in readers’ predominant motivations. The 2019 Reader Demographics Surveys confirmed that these differences cannot be fully explained by individual demographics.

Information Depth

The information depth gaps reflect different levels of readership depending on what level of content a reader is looking for on the site—i.e. checking a quick fact, looking for an overview of a topic, or an in-depth read. Analogous to reader motivations, there is quite a bit of variability between languages in what readers’ predominant information depths are. The 2019 Reader Demographics Surveys confirmed that these differences cannot be explained by individual demographics. Still not fully understood is how these information needs might relate to the utility of Wikipedia for a given reader and whether readers with information needs like “fact” or even “overview” are largely consuming Wikipedia via external re-use and are therefore underrepresented in the existing data. Additional information about reader information depth was gathered in past reader surveys.

Familiarity

The familiarity gaps reflect different levels of readership depending on one’s prior familiarity with a topic. Similar to the motivation, someone’s prior knowledge about a topic might affect their perception of Wikipedia as an appropriate data source, and average readers’ prior knowledge on the topic they read varies across Wikipedia language editions. Ongoing community discussions about content “Depth vs Breadth” indirectly reason about solutions to address these gaps.

Accessibility Gaps

This facets describes the gaps in readership related to the ability to consume Wikimedia content. There are a number of factors that affect the accessibility of Wikimedia content, including the physical resources available to someone, and their individual abilities. While there is overlap to the gaps described in other facets, the accessibility facet specifically focuses on barriers that prevent individuals who would like to read Wikimedia content from accessing the knowledge, such as internet availability, tech skills, or disabilities. Evidence from Wikipedia Zero indicates how extreme and contextual these gaps can be: for example, pageviews from Angola dropped by 80% following the deactivation of Wikipedia Zero whereas no change was seen in Kuwait. The Wikimedia Foundation has done much formative research in this domain, which led to supporting offline access and awareness campaigns. The objective in this facet, inspired by the “Better User Experience” recommendation of the movement strategy, is to break down the technical barriers preventing people from accessing free knowledge.

Internet Connectivity

The internet connectivity gaps reflect contrasts among the ability of readers with different internet connections to access Wikimedia sites. Even though a given Wikipedia article is relatively small in size, slow internet connections can still make it very difficult to reach and browse Wikipedia. From surveys linking internet connectivity to readership, we see much lower readership among low-speed internet users. Evidence from early experiments after the introduction of a new data center in Singapore showed that lower latency correlates with higher longer-term reader engagement. Internet cost can also be a huge barrier to internet access even when a country has good connectivity. The data on usage after Wikipedia Zero program ended suggest that in places where data have indeed a high cost, it is a significant barrier to readership. Initiatives such as Kiwix and other projects from the Inuka team at the Wikimedia Foundation aim at bridging this gap by making Wikipedia more available offline.

Device

The device gap is the difference in accessibility to the site depending on readers’ devices. With predominantly mobile usage in countries that are only now coming online, making Wikipedia more accessible via mobile phones is a high priority. Efforts are already underway to support KaiOS (lightweight phone OS) and continued development of the mobile interface as well as official Android and iOS apps is important for welcoming this increasingly mobile population.

Tech Skills

The tech skills gap reflects different levels of readership depending on readers’ general internet skills. The latter capture one’s experience with the internet and ability to not only find the content one is looking for but also the ability to verify it. Academic research has found that high internet skills are associated with an increase in participation and readership on Wikipedia. External initiatives such as Mozilla’s Digital Skills Observatory, a research project studying the impact of digital skills training on confidence and agency of low-income first-time smartphone users, provide useful insight in how to bridge this gap.

Disabilities

The disabilities gap reflects how individual disabilities might affect one’s ability to access the knowledge within Wikipedia. While individuals who are blind might be the most salient example, disabilities fall into many categories: cognitive, developmental, intellectual, mental, physical, or sensory disabilities. There is scant literature on the degree to which individuals who read Wikipedia have various disabilities beyond anecdotal evidence—e.g., an interview with Graham Pierce, an individual who is blind and a prolific editor, tips for improving accessibility, a survey from 2008 that indicated that Wikipedia was a popular site for individuals who use screen readers. A number of projects focus on improving the accessibility of Wikimedia sites though, such as WikiProject Accessibility, WikiProject Usability, WikiBlind User Group, and Para-Wikimedians User Group.

While there is little data on readers with disabilities, the barriers facing these individuals are not new so initial metrics might focus on measuring the accessibility of a given project—e.g., proportion of images with captions—as opposed to levels of readership in communities of people with disabilities. In general, providing a variety of ways to access content—e.g., text, images, video, audio—while reducing barriers to understanding—e.g., good color contrast, high readability— is a good approach to improving the accessibility for readers of all abilities. For example, Wikispeech provides text-to-speech for Wikipedia articles and VideoWiki provides a tool for collaboratively editing videos from images and wikitext.

CONTRIBUTORS

The contributor dimension of knowledge gaps covers all gaps related to categories of people contributing to Wikimedia sites. We define contributors as all individuals who edit or otherwise maintain Wikimedia content. For the purpose of this taxonomy, this definition does not include technical contributors—i.e. the individuals who build the MediaWiki software on which Wikimedia sites run—though the software and choices made in its design certainly are highly impactful on what types of contributors feel supported and what content is created. Contributors gaps are organized into three main facets: sociodemographics, motivations, and accessibility. Similar to the Readership dimension, the objectives in the Contributors facets are informed by the 2020 Movement Strategy, which invite Wikimedia communities to be safe and inclusive and projects to be designed so that everyone is welcome to contribute.

Facet	Gap	Description	Source
Sociodemographics Objective: contributors with different social status, demographics, and cultural background can easily and safely access and contribute to free knowledge	Gender	Differences between contributors of different gender identities in how and how much they contribute to the sites.	literature, surveys, strategy, community
	Age	Differences between contributors of different ages in how and how much they contribute to the sites.	literature, surveys, strategy, community
	Locale	Differences between contributors of different locales (urban, rural) in how and how much they contribute to the sites. literature, surveys, community
	Language	Differences between contributors of different reading abilities in a language in how and how much they contribute to the sites.	surveys, strategy, community
	Income	Differences between contributors with different income, wealth, or employment status in how and how much they contribute to the sites.	literature, surveys, community
	Education	Differences between contributors of different educational backgrounds in how and how much they contribute to the sites.	literature surveys, community
Contextual Objective: contributors with different motivations and roles can access and contribute to free knowledge	Motivation	Differences in contribution depending on one’s reason for contributing to the site.	literature, surveys
	Role	Differences in contribution depending on the type of editing that one chooses to do.	literature, surveys, community
Accessibility Objective: contributors with different technical resources and abilities can easily access and contribute to Wikimedia projects	Internet connectivity	Disparities in ability to contribute to the knowledge within Wikipedia depending on one’s access to high-speed internet	surveys, strategy, community
	Device	Disparities in ability to contribute to the knowledge within Wikipedia depending on one’s device.	surveys, strategy, community
	Tech Skills	Disparities in ability to contribute to the knowledge within Wikipedia depending on one’s general internet skills	literature, strategy
	Disabilities	Disparities in ability to contribute to the knowledge within Wikipedia depending on individual disabilities	literature, community

Sociodemographics Gaps

Similar to their Readers counterpart, contributors’ Sociodemographic gaps, i.e. demographics gaps such as gender, age, language and location, and social status gaps such as income and education, have been widely discussed by different sources. The objective for this facet states that groups with different socio-demographic characteristics should not face greater barriers to their ability to access and contribute content through Wikimedia projects.

Gender

The gender gap is the difference between individuals of different gender identities in how likely they are to contribute to Wikimedia sites. The gender gap has been measured through many surveys organized by the Wikimedia Foundation, different chapters, independent organizations and research communities by asking respondents for their gender identity, though generally limiting the possible answers to the binary man/woman choice. Conclusions about gender gaps might differ depending on the definition of contributors. For example, recent surveys in this space stratified editors by how many edits were associated with their account. For all three languages surveyed—Arabic, English, and Norwegian—the gender gap is less extreme for editors with fewer edits. While researchers are still working on investigating the causes behind some gender identities being under-represented in Wikimedia’s contributor population, different community initiatives are focusing on bridging this gap.

Age

The age gap is the difference between individuals of different age in how likely they are to contribute to Wikimedia sites. Readers’ age data has been collected by surveys and academic literature. These studies found that the median age of contributors varies substantially, but the median age tracks slightly lower than country-by-country median age. This is best captured by the 2018 Community Insights survey. Median age ranges from around 40 in Western Europe to the 20s in Eastern Europe and “Global South”. Globally, the median age has increased overtime from the low 30s to high 30s. Throughout the years, several initiatives have been developed to bridge this gap and involve senior citizens as contributors for Wikimedia projects.

Locale

The locale gap is the difference between where an individual lives—e.g., rural areas, towns, cities—and how likely they are to contribute to Wikimedia sites. Results from WMF-led and academic research suggest strong over-representation of urban areas among the contributor population. Some projects have started to help grow the pool of contributors from rural areas, for example activities for Wikimedia Spain or the “Wiki Loves Villages” initiatives.

Language

The language gap is the difference between an individual’s fluency in a language and how likely they are to contribute to Wikimedia sites. Surveys have been conducted to estimate contributors’ literacy or language skills and the Babel system is widespread on user talk pages and offers an alternative to understanding the fluency of contributors. Though it may feel intuitive that fluency would be required to contribute, lowering the barrier to contribution by lower-fluency individuals can be important for effective patrolling in small wikis, increase the diversity of contributors, and allow for the cross-pollination of content that might otherwise remain locked up in other languages. Many editors are multilingual and contribute to Wikipedia in a variety of languages, with small wikis heavily depending on multilingual editors and English the most common second-language outside of one’s native language.

Income

The income gap is the difference between individuals of different income, wealth, or employment and how likely they are to contribute to Wikimedia sites. Research on contributor income and employment does not show consistent patterns in contribution by income but higher contribution rates among individuals who are employed. This is presumably convoluted by the high number of student contributors to Wikipedia.

Education

The education gap is how an individual’s level of education affects the likelihood that they contribute to Wikimedia sites. Surveys consistently demonstrate that individuals with higher levels of education are more likely to be contributors to Wikipedia. Despite this skew, many academics, students, librarians, and other scholars do not contribute to Wikipedia and organizations like Wiki Education and the Wikipedia Library work to change this.

Additional Characteristics

It is not possible to document all the ways in which gaps might be identified in the sociodemographics of the contributor population. We describe here several characteristics, however, that either are largely unstudied or too contextual to be easily studied in a global manner.

While studies of how sexuality relates to contributors are limited, there have been surveys and discussions of how sexuality is represented in content that make it clear that a contributor’s sexual identity could relate to how much they trust and feel safe within Wikipedia. For more information, see the LGBT+ Portal.

Ethnicity and race are very contextual as to what it means about an individual’s status and access to resources. Studies in the United States have shown gaps in contribution along racial lines and presumably these same trends are found in other countries where groups of people who are disenfranchised due to their race/ethnicity are also less likely to contribute to Wikipedia. See also and initiatives such as AfroCrowd, Black Lunch Table, and Whose Knowledge?.

No comprehensive surveys were found that included measures of individual beliefs related to politics, religion, or culture such as those asked by the European Values Survey. Research has attempted to assign political leanings to editors, however, based upon their contribution history, showing interest in how individual beliefs of contributors affect the process of collaboration, the degree to which they contribute, and what articles they edit. One of the main recommendation within the 2030 movement strategy encourage communities to bridge these gaps by making the community more inclusive and safe. One of the main mechanisms the community uses to ensure respect for minorities and for anyone disregarding of religious belief or sexual orientation is the Friendly Space Policy, which is in force at every community event.

Contextual Gaps

Individuals contribute to Wikipedia for many different reasons and in many different ways. Our tools need to support this diversity of motivations and types of work or we risk exacerbating gaps by only supporting certain types of contributors. Of note, while research that seeks to reduce barriers to contributing (e.g., socialization via Wikipedia Teahouse, personalized edit recommendations from SuggestBot or GapFinder) has been demonstrated to increase contributions, research that has focused on playing directly to editors motivations is a more cautionary tale and can instead supersede existing motivations (see Gamified Onboarding), or be perceived as manipulative.

Motivation

Motivation gaps reflect different levels of contributions depending on the reason behind an individual’s desire to contribute to the site. Contributor motivations include both the intrinsic (e.g., support of free knowledge, for fun, curiosity, personal satisfaction) and extrinsic (e.g., fixing errors, promoting a topic, professional or school-related reasons, learning a new skill, money). This wide range of motivations affect contributors’ levels of activity and roles that they take on. Contributors’ initial motivations can also differ greatly from the reasons they stay. See “why do people edit?” , Balestra et al., various surveys, or the detailed results from the 2012 Editor Survey for additional details.

Role

Roles reflect different levels of contributions depending on what type of work an individual does to support the wikis. There are explicit roles—i.e. user access levels—on Wikipedia through which users ascend. However, within these access levels, individuals can do very different types of work. Yang et al. identified seven editor roles that users of all access levels undertake. These can be expanded to more specific types of work but offer a high-level view and can be modeled from edit activity

• Social Networker: communicate on user pages and communication namespaces
• Fact Checker: removal / verification of content
• Substantive Expert: content producers – adding substantive content to articles
• Copy Editor: grammar, paraphrase, relocation
• Wiki Gnomes: clean up content and wikitext markup issues
• Vandal Fighter: reverting vandalism, warning editors, and other patrolling work
• Fact Updater: updating templated content or Wikidata
• Wikipedian: working behind the scenes (non-article mainspace) to keep things organized etc.

Surveys generally ask contributors what types of work they do on-wiki, but these tasks can be matched to the above roles. Initiatives such as the Wikipedia Teahouse or the Newcomer Page help newer Wikipedians learn how to familiarize themselves with the platform and progressively become more expert editors.

Accessibility Gaps

This facets describes the gaps in contribution related to the ability to access and edit Wikimedia sites. These gaps specifically focus on barriers that prevent individuals who would like to contribute to Wikimedia from sharing their knowledge, such as internet availability, tech skills, or disabilities. These gaps can be extreme and far-reaching. Zhang and Zhu found that blocks to Chinese Wikipedia in Mainland China in October 2005, beyond preventing contributions from individuals located in Mainland China, also led to a 42.8% drop in contributions from non-blocked individuals. The objectives in this facet, inspired by the “Improve User Experience” recommendation of the movement strategy, encourage the breaking down of the technical barriers preventing people with accessibility limitations to access free knowledge.

Internet Connectivity

The internet connectivity gap relates to how internet access affects one’s ability to contribute. This covers both internet speed and internet cost. While Wikipedia content is relatively lightweight, cost and speed have been identified as a major barriers to contributors. Graham et al. found that removing broadband access barriers is a necessary but insufficient condition for content generation. Initiatives like GLOW work on addressing this gap by subsidizing internet costs for contributors.

Device

The device gap refers to different levels of contributions depending on what device someone has available. Editing classically requires a good laptop or desktop computer in order to easily use the available editing interfaces while potentially also gathering research in other windows. There has been a shift to mobile that has been accompanied with huge improvements in mobile editing interfaces (e.g., Suggested Edits). Additionally, efforts have been made to provide laptops (e.g., GLOW) to new editors in order to reduce this technological burden. Various surveys have asked contributors about their devices.

Tech Skills

The tech skills gap reflects different levels of contributions depending on an individual’s general internet skills. While extensive effort has been put into simplifying the process of contributing to Wikipedia, editing articles, uploading images, and identifying sources can still pose substantial technical challenges for individuals. Academic research has found that high internet skills are associated with an increase in awareness that Wikipedia can be edited and having edited Wikipedia. Edit-a-thons in particular can help to bridge this gap and the 2030 strategy calls for further investment in skills and training.

Disabilities

The disabilities gap reflects how individual disabilities might affect one’s ability to access and contribute to the knowledge within Wikipedia. While individuals who are blind might be the most salient example, disabilities fall into many categories: cognitive, developmental, intellectual, mental, physical, or sensory disabilities. There is scant evidence beyond the anecdotal of the degree to which individuals with various disabilities contribute to Wikipedia (see Userboxes in WikiProject Accessibility, an interview with Graham Pierce, and evaluation from 2008 of editing with a screenreader). Various groups have also been established for individuals with disabilities such as the WikiBlind User Group and Para-Wikimedians User Group.

CONTENT

Wikipedia is incomplete by design. The opportunity to share new information with the world is a major motivating factor among both new and established Wikipedia contributors. However, when important information about a topic is absent, incomplete, biased, or otherwise inaccessible to readers, these content gaps can undermine Wikipedia’s ability to serve the needs of its global audience.

The goal of this Section is to characterize gaps in content coverage. In the widest sense, “content” is information about a topic, i.e. a piece of knowledge that could be the focus of one or more Wikipedia articles. “Coverage” refers to how well Wikimedia project content addresses a particular topic. In turn, a content gap refers to differences in coverage of one or more topics.

Facet	Gap	Description	Source
Policy Objective: content is consistent with core content policies	Verifiability	Differences in the use of reliable sources in order to verify content.	literature, community
	Neutrality	Biases in the content across Wikipedia articles .	literature. community
Accessibility Objective: content is accessible to different audiences	Multimedia	Differences in coverage with respect to the type of media used to share the content	literature, strategy, community
	Structured Data	Differences in the use of information which is indexed and machine-readable	literature, strategy, community
	Readability	Barriers for accessing or consuming information originating from content	literature, strategy, community
Diversity Objective: content covers knowledge that is underrepresented, marginalized, and locally relevant	Gender	Differences in content coverage depending on the gender identity of subjects	literature, strategy, community
	Geography	Differences in coverage of topics related to geographic regions or population distribution	literature, strategy, community
	Impactful topics	Differences in coverage of topics that are of common interest	literature, strategy, community
	Cultural context topics	Differences in coverage of topics related to the history, heritage, and characteristics of a current or former cultural group	literature, strategy, community

Policy Gaps

Wikipedia content is governed by three principal core content policies – Neutral point of view, Verifiability, and No original research – which define the scope and the material that should exist in the online encyclopedia. These policies shape the way in which content is added to Wikipedia, and, to some extent, to its sister projects. Gaps in this facet relate to the two main policies of neutrality and verifiability reflecting biases and lack of reliability in Wikipedia articles, respectively. Thus, the objectives in this facet are mainly inspired by the guidelines within the core content policies, which mandate that content in Wikipedia should be verifiable and neutral.

Verifiability Gap

The verifiability gap reflects differences in the use of reliable sources in order to verify content in Wikipedia. In fact, Wikipedia’s Verifiability core content policy requires every piece of information which has been challenged – or is likely to be challenged – to be backed by a reliable source. Thus, adding high-quality citations is one of the key mechanisms that the communities have to bridge the Verifiability gaps. Previous research has deepened our understanding of citation usage by highlighting gaps in terms of the types of sources used or their accessibility as well as in developing methods for how to automatically detect the citation span of individual sources or classify what content is missing citations. There exist different initiatives to help editors monitor citation quality at scale such as the Citation Detective tool, which automatically produces dumps of sentences requiring citations in English Wikipedia, or Wikicite, whose goals include the “improvement of citations in Wikimedia projects and an open, collaborative repository of bibliographic data for innovative applications”. Furthermore, the Wikipedia Library portal constitutes a fundamental tool to help bridging the verifiability gap, as it provides reliable sources that editors can use to improve the articles, as well as promoting community initiatives such as the “1Lib1Ref” campaign, which encourages librarians to add references missing in Wikipedia. While citations and references are a key tool to monitor the quality of knowledge coming from written sources, how to incorporate oral knowledge within the verifiability framework, remains an open point for discussion.

Neutrality Gap

The neutrality gap reflects biases in the content across Wikipedia articles. In fact, the Neutral Point of View (NPOV) policy demands that all encyclopedic content should represent all significant views fairly, proportionately, and without bias. To help understanding and bridging this gap, researchers in various fields have tried to characterize the dynamics of NPOV disputes, as well as quantify the NPOV gap from explicit (statements supporting a certain POV) or implicit (omission of certain aspects) bias. Examples include the use of NPOV-templates to automatically detect biased language in the content of Wikipedia, or documenting biases in specific topics such as politics or culture among others. The Neutral point of view Noticeboard is one of the main sub-communities dedicated to the discussion of neutrality of content in English Wikipedia.

Accessibility Gaps

Content in Wikimedia projects is multimodal by nature, and can take very different forms, such as images, text, structured data, etc., and the form of presentation clearly affects how accessible content is to different audiences. The gaps in this facet cover the different nature of the content types available in Wikimedia spaces through the lenses of their ability to break down barriers preventing people from accessing free knowledge. The objectives in this facet mainly focus on bridging accessibility gaps from a content perspective, aiming at having more images, structured data, and readable text across Wikimedia projects.

Multimedia Gap

The multimedia gap reflects differences in coverage with respect to the type of media used to share the content. Acknowledging the potential of different forms of media beyond text (image, audio, video, geospatial, etc.) to convey content to different audiences, the Movement strategy recommends building the necessary technology to make free knowledge content accessible in various formats and support more diverse modes of consumption and contribution to Wikimedia projects. For example, with the bulk of a typical article made up of text, the use of images is encouraged in order to increase readers’ understanding of the subject matter (see, for example, English Wikipedia’s policy on image use). However, on average, half of Wikipedia articles are missing images, and around 95% of Wikidata items do not have a value for the image-property (P18). This is exacerbated by the fact that with few exceptions (e.g., the role of visual and multimedia aspects in Wikipedia and other projects has largely been ignored by researchers. To help bridging this gap, several recent research initiatives aim at designing smart tools for image analysis and adding structure to the Wikimedia Commons repository. Tools built by community members also support editors in discovering and adding images in Wikimedia projects, including the WDFIST and the Wikishootme tools. Finally, several community initiatives are organized around adding pictorial representations of under-represented people and topics, such as Wiki Loves Monuments, Visible Wiki Women, and Wikipedia Pages Wanting Photos.

Structured Data

The structured data gap reflects the differences in the use of structured data across Wikimedia projects, namely the amount of information which is organized and indexed in a machine-readable fashion. Structured data offers the potential for managing data for Wikimedia projects on a global scale, allowing easier indexing and offering tools for multilingual data creation and description. One of the most prominent examples of structured data is Wikidata, a structured data knowledge base which was designed with the goal of increasing knowledge diversity. Information from Wikidata is being re-used in Wikipedia articles in different ways, most notably in infoboxes but perhaps most commonly in metadata templates. The Wikidata Concepts Monitor provides quantitative insight into the degree of Wikidata usage, albeit its definition of usage in terms of aspects makes it non-trivial to interpret the statistics with respect to individual templates or infoboxes. In addition, it must be noted that, beyond some established cases, the degree to which Wikidata should be used in Wikimedia projects is part of ongoing discussions within the respective communities (see for example the case in English Wikipedia). Given the importance and potential of structured data, the WMF’s Medium Term Plan 2019-2020 (Platform Evolution) aims at increasing its usage across Wikimedia projects. Product initiatives such as the Structured Data on Commons program, as well as new proposals for an Abstract Wikipedia aim at closing this gap and enable communities to consume and contribute structured data across languages.

Readability

The readability gap refers to how difficult it is to read the content of an article or a piece of information in Wikimedia projects in comparison to the reading abilities of their readers. Using statistical tools such as the Gunning-Fog index, researchers have tried to quantify this gap by computing language complexity of texts in Wikipedia articles. Different studies have shown that the difficulty is considerably above the reading ability of average adults even for projects such as Simple English Wikipedia, which are explicitly aimed at closing this gap.

An indirect approach to bridge the gap is to make content available in different languages given the high proportion of non-native readers in some languages. For this, researchers developed machine-learning models to discover and prioritize articles missing in a given Wikipedia language edition which, in turn, are recommended for creation to editors. Platforms such as the Content Translation Tool help in translating articles to any language available in Wikipedia.

Additionally, content might be structured in such a way that it is inaccessible to readers with various cognitive, developmental, intellectual, mental, physical, or sensory disabilities—e.g., blindness, dyslexia. For English Wikipedia, the Accessibility component of the Manual of Style was written to address these issues and the Accessibility Dos and Don’ts provides a quick overview of some of the more prominent accessibility gaps in content. In particular, the following barriers are identified (to which we add how they might be better tracked):

• Do use high contrast and color-blind friendly color schemes: {{Template:Overcolored}}, {{Template:Cleanup colors}}, and {{Template:Overcoloured}}, all of which add articles to [[Category:Wikipedia articles with colour accessibility problems]].
• Do provide alt text and a caption for most images: {{Template:Alternative text missing}} adds articles to [[Category:Unclassified articles missing image alternative text]], but this should also be detectable from wikitext dumps.
• Do provide a text description of any charts or diagrams: for tables, this should be detectable via wikitext dumps.
• Do nest section headings sequentially: pseudoheadings would be harder to detect but finding articles where section headers are out of order should just require going through parsed wikitext (or parsing the wikitext dumps with mwparserfromhell or similar libraries).
• Do create correctly structured tables: it should be possible to detect lack of headers / scope for tables but other issues such as tables for layout or incorrectly structured tables would be much harder to detect.
• Do encase non-English words or phrases in {{lang}}: {{Template:Cleanup lang}} tracks instances where the {{lang}} template should be used and adds them to [[Category:Pages with nonEnglish text lacking appropriate markup]]. There are also machine-learning models that do language identification but they are unlikely to work well for identifying the short stretches of other-language text for which these templates tend to be used.

Content diversity

The content diversity facets includes all gaps related to topic coverage in Wikimedia projects. Developing a comprehensive, hierarchically-structured, and canonical representation of Wikipedia content is a non-trivial if not impossible task. Wikipedia’s own category structure for content is not hierarchical, and the language and culture-specific nature of topic relevance make the creation of an exhaustive list of topic gaps infeasible. Therefore, our aim here is to provide a list of example topic gaps that have received widespread attention in public media, research studies, and include topics highly curated by community projects and discussed across different parts of the movement, such as gender, geography, and culture gaps. Objectives in this facet are inspired by both the Wikimedia Foundation’s Medium-term plan, which supports diverse content creation “by actively foster[ing] the inclusion of underrepresented and marginalized knowledge, and ensur[ing] content is locally relevant to communities”, as well as the the movement strategy directions which encourages “improving coverage of collectively-identified priority topics that impact our world and improve people’s lives”. Community initiatives such as the Wikiproject Countering Systemic Bias, the Wikipedia Diversity Observatory, or WhoseKnowledge? aim to address imbalances in the coverage of subjects and topics.

Gender Gaps

Topic gender gaps refer to the differences in content coverage depending on the gender identity of subjects. The gender gap – the fact that the majority of Wikimedia content about people focuses on male subjects – is one of the most-studied examples of content gaps and well-documented in the research literature and interactive tools such as the Wikidata Human Gender Inidicators, Denezeleh, or the Wikidata Concepts Monitor. A number of Community initiatives such as the Gender Gap Portal, the Wikiproject Women, as well as organizations such as Whose Knowledge?, are focusing their efforts and working hard to address the gender content gap across Wikimedia projects.

Geographical Gaps

The geographical gap captures coverage differences in topics related to geographic regions or population distribution. Looking at geo-tagged information on Wikipedia content, research has shown that the geographic coverage differs substantially across language editions and that geographic coverage is extremely uneven and clustered with a strong bias towards content related to the United States and Western Europe. To address this gap, several initiatives across Wikimedia projects aim at increasing content coverage of underrepresented areas, for example the Africa Portal in Wikipedia or the Wiki Loves Africa Contest in Wikimedia Commons.

Impactful Topics Gaps

The topic gap captures coverage differences among common interest and impactful topics implicitly considered equally within the scope of an encyclopedia. The abundance of topics varies substantially across different Wikipedia language editions, and it is of crucial interest to assess differences in coverage of specific topics; for example the coverage of medical knowledge on Wikipedia since “[its] health content is the most frequently visited resource for health information on the internet” (see also the WikiProject Medicine in English Wikipedia). While the Movement Strategy indicates that the community is still missing tools to identify which topics are most impactful in the world, initiatives such as WikiProject Vital Articles or All Human Knowledge help in addressing this gap by gathering lists of important and impactful topics which should be present in all Wikipedia editions.

Cultural Context Topics Gaps

The cultural context gap captures coverage differences related to the history, heritage, and characteristics of a current or former cultural group. Cultural identity has been shown to be a crucial part for the motivation of editors to contribute, and the project Wikipedia Diversity Observatory aims to define and quantify the extent of articles which can be considered cultural context content. Initiatives specific to individual projects, such as Wikiproject French Caribbean Culture in Wikidata gather efforts to address culture-specific content gaps.

HOW TO USE THIS TAXONOMY

The taxonomy of knowledge gaps developed in this research and the corresponding literature review can be utilized in a variety of ways by different actors in the free knowledge ecosystem and the Wikimedia projects. This includes Wikimedia community organizers, affiliates, Wikimedia Foundation staff, contributors, researchers and partners. Below we describe the different aspect of the taxonomy and how they can be utilized

The taxonomy as a framework for conversations. As evidenced through the work in developing the taxonomy of knowledge gaps, Wikimedia projects face many different knowledge gaps. However, prior to the development of this taxonomy, the bulk of the attention of large actors within the Movement such as the Wikimedia Foundation has been heavily focused on specific gap types. We hope that the more comprehensive taxonomy of gaps can provide a framework for the decision makers to learn about the different gap types, brainstorm about their possible relationships, and devise ways to address them.

The references that support the gaps. As part of the process of developing the taxonomy of knowledge gaps, we conducted a major literature review. This review of past work which is presented as sources in the tables throughout this paper as well as in the reference section can be a valuable resource for those who are eager to expand their knowledge of the particular gaps and learn about their possible causes.

A first step towards measuring knowledge gaps. The definitions of the different gap types provided through the taxonomy is a necessary step for developing metrics to measure the different gap types. The measurement of the gap types and understanding the relationships between them can further help to develop the knowledge gap index, a composite index that can operationalize knowledge equity in the Wikimedia projects and help support more data-informed decision-making processes.

FUTURE WORK

This taxonomy represents a first step towards understanding the underlying mechanisms that prevent us from reaching knowledge equity, and designing solutions to remove those barriers preventing people from accessing free knowledge. In this Section, we identify the next steps to reach this goal by building on top of the taxonomy presented in this manuscript. We ordered such areas for future work by increasing complexity.

Metrics

In order to track progress towards knowledge equity, it is necessary to quantify each of the knowledge gaps described in this taxonomy. For this we need to i) carefully define metrics reflecting the gap extent across different categories ii) identify relevant reference data sources to apply the metric and measure the gap amplitude. The main challenges in this endeavour are:

• Operationalizing a gap: The definition of each gap must be translated into a set of measurable quantities. For example, for the content gender gap in Wikipedia one often uses the property P21 (sex or gender) of the corresponding item in Wikidata. Operationalizing other gaps might be less trivial. The case of topical content gaps serves as an illustrative example. Identifying content related to, e.g., medicine, is possible through manual annotation in Wikiprojects though it is time-intensive and requires constant updating which might be unfeasible for the many small and medium-sized projects. Automatic methods such as the ORES topic classification alleviate some of these issues, however, they are currently limited in their general applicability as they only support a small set of languages and a fixed set of 64 topics.
• Availability of data: Some data for specific gaps is just not available or hard to obtain for privacy or other reasons. For example, obtaining information readers’ age in order to measure the age socio-demographic gap requires careful legal frameworks and succinctly capturing complex and contextual social phenomena like race is largely not possible at a global scale. Furthermore, some data from, e.g. surveys, might come in irregular intervals or is not consistent across time, as for the case of metrics related to contributorship gap.
• One gap, many metrics: For every individual gap, there exist multiple, equally relevant, ways in which to measure it. Thus, it is important to note that there many metrics which capture different aspects of the same gap. The content gender gap provides an illustrative example, as it constitutes one of the most well-studied gaps. Public tools documenting the gap (such as Wikidata Human Gender Indicators or Denelezh) document the number of articles on men and women, respectively, thus capturing only the selection of content. Research has shown that it is equally important to go other aspects such as extent (e.g. comparing the quality of articles on men and women) or the framing (comparing the language in the articles of men and women). Building on initial insights, future research needs to understand the role of different metrics of the same gap and how the well-explored content gender gap can inform on metrics in less-explored gaps.
• Goal: The definition of a metric for a gap allows one to make a statement about whether the gap is small or large. Such definition thus leads to conclusions about whether a gap is closed or not. What this goal should be might not be as obvious as it seems. For example, in the context of the content gender gap, recent discussions in the community revolve around what are suitable baselines for the comparison of the number of biographies on men and women. Future work involves curation of external datasets such as sociodemographic data on country-level and consultation and discussions with communities on how to define the metrics.

Knowledge gap index

One of the end goals of this taxonomy is to generate a knowledge gaps index, namely a single indicator combining metrics from different gaps, measuring the overall knowledge equity of Wikimedia projects. Socio-economic indices have been adopted by many organizations, advocacy groups and policy makers such as the Global Innovation Index, the Human Development Index from the United Nations, or the Gender Equality Index from the European Union. Such indices are a useful tool for analysis of policies and communication with the public because they can give a coarse-grained view on complex multi-dimensional realities. The index requires the projection and aggregation of different metrics into a composite index and is developed over the course of 10 steps, the first being the construction of a theoretical framework. The main challenges to reach this goal are:

• Availability of metrics: An index requires the availability of validated metrics (see the discussion in the previous subsection).
• Robustness: Even if all metrics are available, the robust construction of a composite index is non-trivial. According to recommended procedures, this taxonomy indeed represents only the first, theoretical step towards a comprehensive, robust and high-quality index. Additional steps would include, for example, uncertainty and sensitivity analysis.

Interactions between gaps

The taxonomy presented here reflects a simplified version of the complex landscape of knowledge gaps by depicting readership, contributorship, and content as three independent dimensions.

However, these dimensions are interrelated and feed into each other, as pointed out by the community, research, and Wikimedia Foundation in their medium-term plan. Apart from few examples (e.g. on the misalignment between content and readership), our understanding of how different gaps interact with each other is still very limited, and part of our future research work includes shedding light on these interactions

Causes and Interventions

In order to bridge knowledge gaps, we not only need metrics to monitor the efficacy of individual interventions, but also require an understanding of the underlying causes to design effective interventions. However, causal evidence on causes is scarce. Recent research has identified several candidate hypothesis for individual gaps. An example comes from previous work analyzing knowledge gaps in readership and contributorship: the authors found that the gender gap is higher in contributors than in readers, and proposed interventions including continued efforts in female editors recruitment, as well as awareness campaigns to increase awareness that Wikipedia is editable among women readers.

METHODOLOGY

The Taxonomy of Knowledge Gaps emerges as the result of hours of literature review, survey analysis, in-depth movement strategy reading, research brainstorming and Wikimedia Foundationwide discussions. To help understand the methodology we used to compile this taxonomy, in this Section we explain its main guiding principles and structural characteristics.

Guiding Principles

Our task is to order elements of Wikimedia spaces lacking knowledge equity in a taxonomy of knowledge gaps. There are endless ways in which we could identify an order through this large pool of unstructured pieces of information. To help narrowing down the scope while delivering a consistent final product, we let different themes, principles and values guide the selection of the taxonomy classes and levels.

Driven by communities.

The taxonomy is largely inspired by values and principles of the Wikimedia communities and the latest recommendations of the Movement Strategy. At the same time, the taxonomy is grounded in the literature produced by the broader academic and scientific communities who study Wikimedia space from a computational and sociological perspective (for an overview, see reviews or the Wikimedia Research Newsletter).

Neutrality.

Similar to Bloom’s taxonomy of educational objectives, one of the foundational principles of this taxonomy is its neutrality. Our mission is to define a classification of knowledge gaps, thus helping to define some of the barriers preventing people from accessing free knowledge. While doing so, we aim at being as impartial and inclusive as possible, without expressing value judgements on the importance of one gap over another.

Flexibility.

Like any project in Wikimedia spaces, we want this taxonomy to be “editable”. This product is a starting point for an open, structured discussion on knowledge gaps, and we will make the taxonomy available on a Wiki to foster in-depth conversations and research studies. Also, its inherent structure allows the taxonomy to evolve and be expanded and completed by experts and community members who want to contribute to the improvement of its content.

Measurability

One of the end goals of this taxonomy is to help Wikimedia communities measure the impact of their initiatives and content creation. With this in mind, when possible, we explicitly formulated gaps and objectives to incorporate elements that are quantifiable via surveys, large-scale data analysis or other computational methods.

Sources

To understand relevant components of the taxonomy of knowledge gaps, and make decisions about which aspects of Wikimedia ecosystems to include in the taxonomy, we gathered information from different sources describing or discussing inequalities among readers, contributors, and content in Wikimedia spaces. We explain here our main data sources, and the rationale behind choosing each of them.

Academic Literature.

We tap into ideas from the bulk of academic literature which studies knowledge gaps in Wikimedia. The majority of related research works belongs to the broad field of “computational social science” and tries to characterise and quantify different aspects of Wikimedia communities using a computational approach. For example, researchers have studied gender gaps across different dimensions, quantified the usage of visual content across different languages or estimated the geographical bias of Wikipedia content. Research works aiming at bridging knowledge gaps using tools from recommender systems and natural language processing also largely helped the creation of this taxonomy: these include methods to grow Wikipedia languages via recommendations, as well as machine learning models to score readability of Wikipedia articles, or discover sentences needing citations.

Community Surveys

Throughout the years, the Wikimedia Foundation, Wikimedia Affiliates, and independent organizations have run surveys to characterize Wikimedia readers and editors. We tap into the questions asked in these surveys to define some of the gaps and facets in our taxonomy, for example sociodemographics gaps or accessibility gaps. Most of these surveys are referenced throughout this manuscript, and a superset of them can be found by checking the appropriate categories on meta.wikimedia.org.

Movement Strategy

The third source of content for this taxonomy is a set of guidelines from the Wikimedia Movement Strategy. Such guidelines include broad strategic directions, namely Knowledge Equity and Knowledge as a Service, as well as more specific recommendations to implement such strategic directions. We also borrow from Wikimedia Foundation’s medium term plan priorities. Collectively, these guidelines helped us identify gaps, barriers, and common objectives related to free knowledge.

Community Initiatives

Throughout the years, the Wikimedia community have worked on many initiatives and discussions aiming to address knowledge gap. This not only includes research (e.g. trying to conceptualize or measure the gap) but also covers a wide variety of initiatives such as WikiProjects (e.g. WikiProject Women).

Taxonomy Structure

Beyond the identification of potential gaps within the Wikimedia ecosystem, one of the main challenges in building a taxonomy of knowledge gaps is to provide a structure for how to analyze and classify the different gaps. In this section we give an overview over the overall structure, and define and motivate the different levels in the taxonomy.

Gap.

A gap corresponds to an individual aspect of the Wikimedia ecosystem (for example readers’ gender, or content topics) for which we found signals of unbalanced coverage across its inner categories (for example, proportion of people who identify as men, women or non-binary across readers, in the case of the reader gender gap). For each gap, we include the following characterizing fields:

• Description: This field is the definition of the gap, and describes which type of disparity is covered by the gap. For example, the gender gap is defined as “the difference between readers of different genders in how and how much they access the sites”.
• Sources: The decision to include a gap in the taxonomy is heavily driven by existing sources. The source property of a gap includes all the references we examined when characterising and describing a given gap, grouped by source type (literature, survey, strategic guidelines and/or community).

Facet.

We grouped semantically-related gaps into facets, namely macro-categories describing the general semantics of a gap. For example, gaps such as “gender” or “age” both belong to the “sociodemgraphics” facet. Gaps in the same facet are also characterized by the same Objective. The objective represents the ideal goal to be reached in order to close or address the knowledge gaps in a facet. To make sure we follow the neutrality principle, the objectives we propose are largely inspired by movement strategy recommendations and priorities, thus echoing the results of months of community discussions around the future of Wikimedia projects.

Dimension.

We identify three foundational dimensions as root nodes in our taxonomy grouping different facets: Wikimedia readers, Wikimedia contributors, and Wikimedia content. We will expand the rationale behind the choice of the dimensions in the next Subsection.

Root Dimensions: Readers, Contributors, and Content

This taxonomy is developed along three dimensions representing macro-elements of the Wikimedia ecosystem: Readers, Contributors, and Content. The choice of these dimensions is broadly inspired by two models summarizing the inner mechanisms of Wikimedia communities.

Figure 2a

A model proposed by Wikimedia Foundation as part of their medium-term plan sketches a diagram in the form of a spinning wheel describing the relationship between awareness, consumption, contributors, content, and advocacy, as shown in Figure 2a.

The model shows how people initially engage with the site (i.e. they become Wikimedia readers) through spontaneous search or awareness campaigns. Some readers then might become more involved in the values or the mission of the Wikimedia movement, thus potentially converting into contributors or donors. When more people engage with the site, the content production becomes more diverse, thus making the site more inclusive for new readers, and the cycle begins again.

Figure 2b

A model by Shaw and Hargittai (“The Pipeline of Online Participation Inequalities: The Case of Wikipedia Editing”) proposes a framework in the form of a pipeline to describe how gaps form at different levels of engagement with Wikipedia (see Figure 2b). Specifically, they propose a series of steps through which a user must go to become a contributor: being an internet user, having heard of the site (awareness of Wikipedia), having visited the site (Wikipedia reader), knowing it’s possible to contribute to the site (awareness they can edit), and then finally having contributed (is a contributor). At each of these steps, different groups of people might be more or less likely to drop off in participation.

For each dimension of this Knowledge Gaps Taxonomy – Readership (Section 2), Contributorship (Section 3) and, Content (Section 4) – we presented a short overview of its meaning and the corresponding taxonomy in a tabular. For each of the dimension’s facet, we provided a detailed explanation for the rationale behind each of the facets