Executive Summary[edit]

Wikipedia Community Values.

Editor Input Values: Safety, Privacy, and Participation.
Community Output Values: Neutrality, Consensus, and Access to Knowledge.

Research Community-Level Harms/Benefits.

Research benefits the community by building its capacity to work, volunteer, and contribute to more research: Advancing project understanding, proposing changes, and crediting community members (when desired).
Research harms the community when it impedes its ability to reach its goals: Imbalanced demand for effort, enabling others to do harm, and alienating editors.

Decisions about what research takes place.

Editors should wear hats appropriately.
Wikimedia Research should prioritize tools, policies, and guidelines for researchers.
External Researchers should consider community well-being, community norms, and reciprocity.

Instructions[edit]

This is an evolving document based on three research workshops with 16 editors, 5 Wikimedia Foundation Employees, and 4 Wikipedia researchers, conducted across 11/11/23, 04/11/24, 04/30/24. Given the small pool of participating stakeholders, please consider leaving your thoughts or feedback so we can revise this draft to better reflect the broader community.

Please use the talk page/discussion feature to share your thoughts. If you prefer to leave private feedback, please send an email to zentx005@umn.edu.
Open to all editors. We invite feedback from all editors, especially those who didn't attend the workshop.
All forms of feedback are welcome. This document is intentionally concise and your feedback can be too. A simple +1/-1 to any section or item on the talk page helps us know that you agree/disagree. Longer reflections are equally appreciated.

Preliminary Findings on Ethical Online Communities Research [Draft][edit]

What values are important to the broader Wikipedia community?[edit]

We asked community stakeholders to discuss and rank Wikipedia's community values by importance.

The community's values are interrelated and frequently build to other important goals in the community. For instance, editors feeling safe enables collaboration, which in turn allows the community to reach a consensus on important issues. Stakeholders were split on which end of the value chain was most important, but we characterize both perspectives below:

Editor Input Values (e.g., safety, privacy, and participation): This perspective acknowledges the volunteer aspects of contributing to the community and prioritizes the conditions that enable volunteer input.
Community Output Values (e.g., neutrality, consensus, and access to knowledge): This perspective focuses on the goals of Wikipedia (e.g., Five Pillars) and prioritizes the outputs of the community as the most important values.

Less important are the values relating to the bureaucratic systems that surround the Wikipedia community. Voting, rules, and shared governance are important to how the community operates but are implemented in practice by small groups. Similarly, there is increased individual autonomy and ownership in smaller language editions within the community. We highlight this to be cautious of an Is-Ought gap in the community. That is, what is happening in practice may not reflect the community's aspirational norms.

What benefits/harms should future research strive for/avoid on Wikipedia?[edit]

We asked community stakeholders about ways future research can benefit and harm the community. In broad strokes, research that uses community resources should give back to Wikipedia in direct ways. The Wikipedia community's bandwidth is finite. Research, in turn, should build the community's capacity to work, volunteer, and contribute to more research. These benefits can be felt in the community in a number of ways.

Advancing the understanding of projects.
Proposing policy or technical changes.
Crediting community members for their contributions (when desired).

Community-level harms are felt when research impedes the community's ability to reach its goals. Here, we outline a distinct set of community research harms salient to stakeholders.

Demanding more effort from the community than it contributes in value to the community.
Disseminating information with malicious use cases within the community.
Alienating, fatiguing, or misleading editors: Individual harms are felt by the community through opportunity costs.

How should people make decisions about future research on Wikipedia?[edit]

We asked stakeholders to discuss, in the ideal situation, how Wikipedia stakeholders should be involved in making decisions about what research is conducted with the Wikipedia community.

As a decentralized organization and community, the idea of the community making centralized research decisions is a complex issue. There is an idealistic draw to implement a centralized research committee composed of interested Wikipedians, Wikimedia Foundation Employees, and experienced researchers who are knowledgeable about Wikipedia research to gate access to the community. However, this structure raises issues with scalability, responsibility, and enforcement. Considering these ideals, we first define different stakeholder groups within the community. Then, we discuss practical insights for each group and reinforce existing practices supported by community stakeholders.

Core Editors refer to participating editors committed to "showing up" tomorrow. This group contributes to research decisions in an opt-in, interest-based capacity by discussing research policies and guidelines and providing feedback on research proposals. Below, we outline recommendations for this group.
1. Wear hats appropriately: Editors hold many different roles within the community, but groups don't inherently hold more power over research decisions than others. For research involving editors, core editors should contribute using their perspectives as editors, as opposed to other roles they may hold (e.g., Administrator).
Wikimedia Research refers to Wikimedia Foundation Employees who are knowledgeable about Wikipedia research. Members of this group hold an advising role and strongly influence research decisions. As stewards, they can help researchers interface with the community by connecting them with individuals and affiliate groups who need to know. Below, we outline recommendations for this group.
1. Prioritize tools, policies, and guidelines for researchers over project-level recommendations: Similar to the broader community, this group's resources are finite. Focus on sharing resources for new researchers to succeed that reinforce community norms (e.g., ensuring reciprocity or how to talk to community members).
2. Promote a central hub for research activity: By centralizing active and proposed research, it makes it easier for other community members (e.g., editors or legal teams) to engage. Consider how different types of research interface with the community (e.g., APIs or community pages) and link appropriate resources for users.
3. Publicize information for IRBs: Institutional review boards review and monitor human-subjects research but don't understand community-specific norms. This can both prevent researchers from satisfying desired outcomes (e.g., crediting participants upon request) and enable research that violates community norms. Create IRB-specific resources that outline additional community-specific norms applicable to human-subjects determination and human-subjects review.
4. Encourage desirable community research: Funding, sponsorship, and tail-end incentives are ways that all stakeholders can influence researchers' autonomy to promote community values. Consider publishing access requirements for resources like the Wikimedia Research Showcase based on community guidelines for research.
External Researchers refers to the people and groups proposing or carrying out research on or using resources from the Wikipedia community. The openness of projects and the community's anyone can contribute model means that external researchers are the ones who make the final decision about their research. This responsibility comes with an accountability to the community. Below, we outline recommendations for this group.
1. Consider community well-being, community norms, and reciprocity: When defining research goals and research methods, think critically about community-level benefits and harms defined by the community. Slow down and consider competing internal community and external demands that are in tension.
2. Pre-register research proposals on the community's central hub for research activity: Be transparent with all aspects of the research (Five Ws) and communicate them in a non-technical and accessible way. Be open to community member's perspectives and allow them to help make decisions.
3. Consider Power Dynamics: When engaging with community members, avoid jargon, be conscious of making demands, and actively reduce the power others may perceive you have. Use reciprocity to offset the burden placed on the community when extracting data.
4. Be transparent about community ethics practices: Document the procedures taken to evaluate the ethics research through the lens of the community in research proposals, papers, and review board submissions. Hold others accountable to the same standards during peer review.
5. Publish results in open-access venues and on the community's central research hub: The community benefits from having access to research results, including null results. When sharing results in Wiki, be concise, prioritize findings that are valuable to the community, and disclose limitations.