Community Insights/Community Insights 2021 Report/Collaboration, Diversity & Inclusion (2021)

2021 Community Insights Community Climates Supplement

• Executive summary
• Methodology
• Statistical endnotes
• Data appendices

The experienced social environment: Diversity, Inclusion, & Collaboration

Wikimedians do not contribute to the movement in a vacuum—an environment which enables collaborative relationships helps individuals achieve shared goals. Drawing from existing research on collaborative environments in the education sector and business sector, the Global Data & Insights team has identified eight indicators of a collaborative and engaged environment, as well as five indicators of an inclusive environment, in the Wikimedia context. Understanding how different Wikimedians experience different aspects of collaboration and inclusion can help identify critical levers for improving on-wiki experiences and movement success (See Appendix:Methodology).

Key findings

• Across the eight “Collaborative Engagement” factors used in this analysis, Wikimedians felt best about Engagement itself (taking pride in, recommending, and feeling inspired by Wikimedia projects); Feelings of Belonging (feeling that they belong in and are respected in the movement); Fairness (feeling that their contributions to projects aren’t unfairly removed); and Collaborative Intention (feeling that people in the movement collaborate, support, build cooperative relationships and consult with each other). The “Awareness of Self & Others” factor — whether respondents felt movement members are aware of their own biases, and whether they value feedback from or motivate others around them — continues to demonstrate unfavorable experiences.
• As in 2019, most Diversity & Inclusion factors were experienced more favorably than Collaborative Engagement factors overall. Across the five Diversity and Inclusion factors, the strongest were Non-Discrimination (the extent to which respondents witnessed or personally experienced discrimination or harassment on Wikimedia projects) and Individual Commitment to Diversity (for example, personal comfort working with people from different backgrounds, and viewing multicultural collaboration as valuable).
• Many Collaborative Engagement factors continue to be experienced most favorably by those who take on organizing responsibilities. Developers, Editors, and On-wiki Admins experienced these factors less favorably.
  • Movement Organizers reported somewhat more favorably about contributors’ Self-Awareness & Awareness of Others. Active Editors and Developers scored lower than Admins and Organizers in terms of Engagement and perceptions of Fairness.
  • Active Editors without organizing or admin roles also were less likely to report strong Feelings of Belonging or Individual Commitment to Diversity. However, they were also the least likely to report personally experiencing or witnessing discrimination.
  • In terms of Movement Strategy, Editors, Developers and especially On-wiki Admins scored less favorably than Movement Organizers, consistent with the broader observations that not all contributor groups seek to engage with the strategy process. In the 2017 movement strategy process, the phase which targeted online contributors most, online Editor groups engaged at a ratio of only 1 to 5 compared to affiliate and movement organizer groups. However, awareness of the strategy process has also shifted as strategy aims pivot away from building awareness and toward implementation.
  • On-wiki admin roles disconnected from movement organizing correlated with less favorable perceptions of Movement Leadership. Movement Organizers, and even Organizing Admins, were more likely to give favorable ratings than their non-organizing On-wiki Admin counterparts. This indicates room for improvement in supporting the global movement and for the Foundation and other organizations to communicate well about their projects and initiatives among this movement leadership audience.
• In 2020, we saw increased favorable perceptions of Fairness, Movement Leadership and Feelings of Belonging. This is especially notable, as each factor had decreased in favorability between 2018 and 2019.

About the data

Figure 1: Editors, Developers, Organizing Admins and On-wiki Admins were 4 to 11% more likely to identify as men than the overall average of contributors in 2020. Movement Organizers were the only group which had fewer respondents identifying as men than the overall average.

Data were collected within the Community Insights survey, which was conducted during September-October 2020. Participants were sampled from across Wikimedia community spaces and data weighted according to the Community Insights survey sampling and weighting strategy (See also: Methodological Appendix, Community Insights Report 2021).

The Gender Gap across contributors groups

Before diving in, it is important to recognize there is a much larger gender gap for Editing and On-wiki Admin audiences than for Movement Organizers. Admins are nearly half as likely to identify as women than contributors overall, while Movement Organizers were more than twice as likely to do so.¹

Geographic differences across contributor groups

Figure 2: In Africa, we heard from disproportionately more organizers and organizing admins than editors.

A second significant difference in our audience distribution should also be noted. In Africa we heard from disproportionately more organizers and organizing admins compared to editors who did not hold these roles.²

Factors of Collaborative Engagement

Our 2020 set of Collaborative Engagement constructs include:

Awareness of Self and Others (5 questions)

To what extent are people aware of others and their own motivations?

Collaborative Intention (4)

How much do contributors feel that others are interested in building successful cooperative relationships?

Engagement (5)

How much do community members identify with, are inspired by, and promote being a part of the Wikimedia movement and its projects?

Fairness (1)

To what extent do community members experience their contributions being challenged and/or removed unfairly or inappropriately?

Feelings of Belonging (2)

How respected do people feel as a part of the Wikimedia movement, its organizations, and decision-making processes?

Movement Leadership (4)

How confident are community members in the Foundation's ability to support the global movement and for the Foundation and other organizations to communicate well about their projects and initiatives?

Movement Strategy Leadership (4)

How confident are communities in the movement strategy process?

Problem Solving & Negotiating (2)

How much do community members feel that others seek fair solutions and are willing to talk through competing personal interests?

Figure 3: The strongest factors of Collaborative Engagement were Engagement itself, Feelings of Belonging, Fairness, and Collaborative Intention. Awareness of Self & Others continues to demonstrate unfavorable experiences.

Overall the strongest factors of Collaborative Engagement were Engagement itself, followed by Feelings of Belonging, Fairness, and Collaborative Intention.

Awareness of Self & Others, as in previous years, was the least favorably rated factor. Contributors were four and a half times as likely to rate Engagement favorably than Awareness of Self & Others. Other factors mentioned were three to four times more likely to receive favorable ratings than Awareness of Self & Others.

Many Collaborative Engagement factors continue to be experienced most favorably by those who take on administrative and organizing responsibilities. While Movement Organizers and Organizing Admins consistently scored above average across Collaborative Engagement factors, Developers, Editors, and On-wiki Admins had more variation across categories [see the Figures below]. For example, Developers scored lowest among the five audience groups on Engagement (84% favorable) and Fairness (57% favorable), and were significantly less likely to rate these factors favorably compared to Organizing Admins; however, Developers ranked above the overall averages regarding Feelings of Belonging, Collaborative Intention, and Movement Leadership. Similarly, while On-wiki Admins were most likely to rate Fairness favorably (84%) among audience groups, they were less likely to rate Engagement favorably and least likely to rate Collaborative Intention, and Movement Strategy and Leadership favorably compared to other contributors.³ Such variations in audience experience reflect the specific roles, levels of engagement, and responsibilities of contributor groups, which can help to guide targeted efforts to improve community resilience.

In terms of Movement Strategy, similar to 2019, Editors, Developers and On-wiki Admins scored significantly lower than Movement Organizers. On-wiki Admins were least likely to report favorable perceptions of Movement Strategy (26% favorable) and Movement Leadership (36% favorable), indicating substantial room for improvement in supporting the global movement and for the Foundation and other organizations to communicate well about their projects and initiatives among this movement leadership audience. (Note: Superscript notations clarify significant differences: * p < 0.05; ** p < 0.01; *** p < 0.001).³

• 
  Figure 4. Means of Awareness of Self & Others scores by audience group, based on a 5-point scale (1 = least favorable, 5 = most favorable).
• 
  Figure 5. Means of Engagement scores by audience group.
• 
  Figure 6. Means of Fairness scores by audience group.
• 
  Figure 7. Means of Feelings of Belonging scores by audience group.
• 
  Figure 8. Means of Collaborative Intention scores by audience group.
• 
  Figure 9. Means of Movement Leadership scores by audience group.
• 
  Figure 10. Means of Movement Strategy scores by audience group.

From 2019 to 2020, four Collaborative Engagement factors demonstrated statistically significant improvement. Three factors — Fairness, Feelings of Belonging, and Movement Leadership — are especially noteworthy as their favorability was in decline between 2018 and 2019, as outlined in the table below. Compared to 2019, survey participants gave 6% more favorable ratings to both Fairness and Engagement in 2020, and 5% more favorable ratings to Feelings of Belonging and Movement Leadership.⁴ All year-over-year comparisons are available in the Appendix (See Appendix: Changes from 2019 to 2020).

Collaborative Engagement: Changes from 2019 to 2020	Change 2018-2019	Change 2019-2020
Fairness***	🔻	▲
Feelings of Belonging***	🔻	▲
Movement Leadership***	🔻	▲
Engagement***	▽	▲

Diversity & Inclusion

The 2020 survey constructs include:

Non-Discrimination (2)

To what extent do community members experience unsafe environments or unfair treatment due to their identity?

Inclusive Culture (3)

Do community members feel that Wikimedia communities are making progress with diversity initiatives?

Inclusive Interactions (4)

Do community members experience an environment that supports the free and open expression of ideas among contributors of different backgrounds?

Individual Commitment to Diversity (5)

How much do community members value working with contributors of different backgrounds?

Leadership Commitment to Diversity (4)

Do community members think that movement leaders encourage and support diverse participation and content creation?

Figure 11: As in 2019, most Diversity & Inclusion factors were experienced more favorably than Collaborative Engagement factors, especially Individual Commitment to Diversity and Non-Discrimination.

Across all five Diversity & Inclusion factors, the most favorably rated factor was Individual Commitment to Diversity followed by Non-Discrimination, indicating positive attitudes toward interacting with people of different backgrounds and infrequent experience of discrimination. The least favorably rated factors continue to be Leadership Commitment to Diversity and Inclusive Culture. Leadership Commitment to Diversity and Inclusive Culture factors were a third less likely to receive a favorable rating than Individual Commitment to Diversity and Non-Discrimination.

Again, there were some differences between audiences among Diversity & Inclusion factors. Active Editors without organizing or admin roles were less likely to report both personally experiencing and witnessing discrimination than other contributor audiences. However, they were also less likely to report positively regarding Individual or Leadership Commitments to Diversity compared to Movement Organizers and Organizing Admins, and regarding norms of Inclusive Culture compared to Movement Organizers. Essentially, less exposure to discrimination correlated to those who experienced less individual motivation, community policy, and normative supports for diversity and inclusion. More specifically, movement organizing and on-wiki administration tended to correlate with higher prevalence of witnessing someone treated unfairly because they are part of a specific cultural or social group (for example, their race, gender, sexual orientation, religion, disability, etc.) or personally feeling unsafe or uncomfortable contributing to Wikimedia projects online in the last 12 months. Movement Organizers were about 2 to 2.5 times more likely to personally witness someone be treated unfairly because they are part of a specific cultural or social group than active Editors, and Organizing Admins were more than 1.7 times more likely to have experienced than On-wiki Admins who do not organize. In addition, Movement Organizers and especially On-wiki Admins, whether they organized or not, were 30 to 42 percent more likely, respectively, to have felt unsafe or uncomfortable contributing to Wikimedia projects online in the last 12 months. While Movement Organizers and Organizing Admins were more likely to report witnessing or experiencing discrimination, they reported 12 to 13 percent more favorably regarding Inclusive Culture compared to On-wiki Admins. Compared to both Editors and On-wiki Admins, Movement Organizers and Organizing Admins were also more likely to report favorably regarding Individual and Leadership Commitment to Diversity. (Note: Superscript notations clarify significant differences: * p < 0.05;  ** p < 0.01; *** p < 0.001).⁵

• 
  Figure 12. Means of Non-Discrimination scores by audience group.
• 
  Figure 13. Means of Non-Discrimination scores by audience group.
• 
  Figure 14. Means of Non-Discrimination scores by audience group.
• 
  Figure 15. Means of Non-Discrimination scores by audience group.

As with the Collaborative Engagement constructs, there were some signs of a reversal of downward trends, one of which was significant. Per the table below, Non-Discrimination demonstrated a 4% increase in favorable participant ratings from 2019 to 2020. Other factors, which had shown a statistically significant increase or decline in 2019 compared to 2018, began to trend upward by one to two percent comparatively. However, these differences, as well as similar slight upward trends in the other two Diversity & Inclusion factors, did not reach statistical significance except in the case of Non-Discrimination.⁶ (See also Appendix: Changes from 2019 to 2020).

Diversity & Inclusion : Changes from 2018 to 2019	Change 2018-2019	Change 2019-2020
Non-Discrimination*	🔻	▲
Inclusive Interactions	▲	△
Inclusive Culture	🔻	△

Considerations & conclusion

In 2018, the Wikimedia movement set out to identify specific strategic recommendations to target the identified shared direction for the Wikimedia Movement:

• Knowledge as a Service, to build tools for collaborative partnerships to exchange free knowledge
• Knowledge Equity, to break down social barriers to bring along those who have been left out by structures of power and privilege.

In 2021, these recommendations need to be reviewed and discussed across movement stakeholders before they are finalized and implemented fully in planning. The data in this supplement is relevant to several movement strategy recommendations. These metrics are mapped along those relevant key strategy recommendations, along with their summary data on the strength of each factor overall and year-over-year trends in Appendix: Metrics to movement goals mapping.

There are many ways to use these data: (1) Look to weaknesses (2) Look to strengths (3) Look for the gaps.

Look to weaknesses to identify areas for intervention.

With a problem-orientation, one might use the data to identify weak points and either work to further investigate and understand the weakness, or to change what they think may be causing the weakness.

For instance, focusing on perceptions of Problem Solving and Negotiating, one may seek to understand the training gaps and opportunities for contributors to develop these key collaborative skills and work to improve training tools and practices. Alternatively, one may recognize that training gap and potential intervention more immediately and be ready to pilot an intervention.

Look to the strengths to identify reliable community skills and resiliency traits.

With a strengths-based approach, one might use the data to identify strong points and either work to further investigate and understand those strengths to try to maintain them, or work to expand those strengths to other spaces.

For instance, focusing on Engagement, one may consider that this factor rates especially high for Organizing Admins and Movement Organizers and identify potential engagement strategies used with these groups which might be introduced to other contributor audiences. Alternatively, one may simply look to these high scores as benchmarks or maintenance goals.

Look for the gaps between contributor or demographic groups to identify unequal advantages or barriers.

Focusing on gaps between different audiences and demographic groups one may better identify barriers or potential intervention areas that could change the situation for those audiences being left behind.  

Gaps in the weaknesses to identify those who are advantaged.

Focusing on gaps between different audiences and demographic groups, one may better identify barriers or potential intervention areas that could change the situation for those audiences being left behind.

Gaps in the weaknesses to identify those who are advantaged. Similar to above, one might look to the weakest factors to identify gaps between audiences and demographics which stand out as scoring favorably.

For instance, with regard to Awareness of Self & Others, one might look to the different experiences of dominant versus non-dominant group contributors to identify potential interventions to mirror those more positive experiences such as developing easier pathways to make personal connections between contributors without risking their privacy; or developing collaborative projects and initiatives to connect contributors to work on a shared goal which crosses project and/or contributor spaces.

OR

Focusing on Feelings of Belonging, one might look to potential interventions through recognition programs or implement process design changes to prevent community practices which create a climate for high-exposure social criticism.

Gaps in the strengths to identify those being left behind.

Alternatively, one may also look to the strengths and dig in to explore where certain contributing groups seem to fall behind.  

For instance, focusing on perceptions of Fairness, one may seek to understand the level of training and awareness of editing policies and/or may choose to work to improve perceptions of fairness surrounding online contributions by improving training tools and practices.

OR  

Focusing on Individual Commitment to Diversity: recognizing the higher level of exposure to issues of conflict and/or harassment that On-wiki Admins and Movement Organizers experience, and considering bolstering the support available to them as community gatekeepers.

Calls to action

1. Dive deeper into the supplemental report’s data using the interactive report

We have posted aggregate data tables and interactive charts in Google Data Studio. There, you can be in charge of which measures and audiences you view and can view and download the aggregate data directly from the shared tables.

2. Use the report to drive change in your context

There are many ways to apply the data to develop interventions and changes to systems. It is our hope that this annual report might help Foundation and Movement stakeholders alike to:

• Examine products, services, and resource supports to identify how to address critical issues of inequity within our world and the movement to become a more inclusive and equitable movement.
• Budget for improvements to products, services, and resource supports to address critical issues of inequity within the movement by breaking down the systemic inequities that have been built into support infrastructures.
• Improve products, services, and resource supports to move the meter on equity and inclusion within the movement.
• Partner across affiliates, allied organizations, and open knowledge movement partners to bolster our movement efforts toward equity and inclusion across all languages, geographies, and demographic classes.

3. Tell us what you think

As we look to movement strategy and devise consistent metrics, how might these social climate factors help to guide the work you do? What, if any, additional questions come up for you in the context of your contributing communities? What excitements or concerns do the data bring for you related to the spaces you most often contribute? Tell us about it on the talk page!