Community health initiative
Community health initiative
Helping the Wikimedia volunteer community to reduce the level of harassment and disruptive behavior on our projects
The Wikimedia Foundation's Community Tech and Community Engagement teams are underway on a multi-year project of research, product development, and policy growth to help the Wikimedia volunteer community to reduce the level of harassment and disruptive behavior on our projects.
This initiative addresses the major forms of harassment reported on the Wikimedia Foundation’s 2015 Harassment Survey, which covers a wide range of different behaviors: content vandalism, stalking, name-calling, trolling, doxxing, discrimination — anything that targets individuals for unfair and harmful attention.
This will result in improvements to both the tools on the MediaWiki software and the policies on communities suffering the most from disruptive behavior. From research and community feedback, the tools built in this initiative will focus on four areas: Detection, Reporting, Evaluating, and Blocking. Of course, these improvements need to be made with the participation and support of the volunteers who will be using the tools. As a product development team, we don't want to create new reporting, evaluation and blocking workflows that create more work for an already overburdened team of wiki administrators. We want to help make these tasks less grueling for admins, and able to more consistently produce effective outcomes.
- 1 Background
- 2 Goals
- 3 Community input
- 4 Anti-Harassment Tools
- 5 Policy Growth & Enforcement
- 6 Schedule of work
- 7 See also
- 8 References
Harassment on Wikimedia projects
On Wikipedia and other Wikimedia projects, harassment typically occurs on talk pages (article, project, and user), noticeboards, user pages, and edit summaries. Edit warring and wiki-hounding can also be forms of harassment. Conduct disputes typically originate from content disputes, such as disagreements about the reliability of a source, neutrality of a point-of-view, or article formatting and content hierarchy. These disputes can become harassment at the point when an editor stops thinking of this as a discussion about ideas, and starts associating the opponent with the idea -- turning the other editor into an enemy, who needs to be driven off the site. This unhealthy turn is more likely to happen when the content is closely associated with identity -- gender, race, sexual orientation, national origin -- because it's easy for the harasser to think of the target as a living representation of the opposing idea. This is especially true when the target is a member of a historically disadvantaged group, and has disclosed information about their identity during the course of their time on the projects.
The English-language Wikipedia community (and most other projects) have drafted conduct policies for their communities to follow including policies on civility, harassment, personal attacks, and dispute resolution. The spirit of these policies is right-hearted but enforcement is difficult given deficiencies in the MediaWiki software and the ratio of contributors to active administrators. The dispute resolution processes encourage users to attempt to resolve issues between themselves before bringing the situation to the attention of administrators on the Administrator's Noticeboard, and eventually ArbCom for extreme situations.
Online harassment is a problem on virtually every web property where users interact. In 2014, the Pew Research Center concluded that 40% of all internet users have been the victim of online harassment. In 2015 the Wikimedia Foundation conducted a Harassment Survey with 3,845 Wikimedia user participants to gain deeper understanding of harassment occurring on Wikimedia projects. 38% of the respondents confidently recognized that they had been harassed while 51% of respondents witnessed others being harassed. The most commonly reported types of harassment included:
- Content vandalism (26% of responses)
- Trolling/flaming (24%)
- Name-calling (17%)
- Discrimination (14.5%)
- Stalking (13%)
- Only 18% of all identified attacks on English Wikipedia resulted in a block or warning.
- 67% of attacks come from registered users.
- Nearly 50% of all attacks come from contributors with over 100 annual edits.
This research is illuminatory and one of the impetuses for this Community Health Initiative, but is only the beginning of the research we must conduct for this endeavor to be successful.
Community requests for new tools
The Wikimedia community has long struggled with how to protect its members from bad-faith or harmful users. The administrative toolset that project administrators can use to block disruptive users from their projects has not changed since the early days of the MediaWiki software. Volunteers have asked the Wikimedia Foundation to improve the blocking tools on a number of occasions, including:
- a 2013 survey of the German Wikipedia community
- as part of the Inspire Campaign on the Gender Gap, held in 2014, community members expressed a desire for better reporting systems for targets of harassment. This has been a recurring theme in surveys, consultations, and workshops on the topic.
- in the 2015 Community Wishlist survey
- and in a number of Phabricator requests, including "Send a cookie with each block" (November 2015), "Allow User agent (UA)-based IP Blocks" (May 2015), and "Throttle account creation and email sending per browser as well as IP address" (July 2015). See this ticket for coordination of many blocking-tools related tasks
In preparing for this initiative, we've been discussing issues with the current tools and processes with active administrators and functionaries. These discussions have resulted in requested improvements in several key areas where admins and functionaries see immediate needs — better reporting systems for volunteers, smarter ways to detect and address problems early, and improved tools and workflows related to the blocking process. These conversations will be ongoing throughout the entire process. Community input and participation will be vital to our success.
In January 2017, the Wikimedia Foundation received initial funding of US$500,000 from the Craig Newmark Foundation and craigslist Charitable Fund to support this initiative. The two seed grants, each US$250,000, will support the development of tools for volunteer editors and staff to reduce harassment on Wikipedia and block harassers. The grant proposal is available for review at Wikimedia Commons.
- Reduce the amount of harassing behavior that occurs on Wikimedia projects.
- Fairly resolve a higher percentage of incidents of harassment that do occur on Wikimedia projects.
Potential measures of success
There are challenges to measuring harassment but we still want to be sure our work has an impact on the community. Current ideas include:
- Decrease the percentage of identifiable personal attack comments on English-language Wikipedia, measured via Detox, Sherloq, or a similar system.
- Increase the confidence of admins with their ability to make accurate decisions in conduct disputes, measured via focus group or consultation.
- Decrease the percentage of non-administrator users who report seeing harassment on Wikipedia, measured in a follow-up to the 2015 Harassment Survey.
- ...add ideas here or on the talk page.
Gathering, incorporating, and discussing community input is vital to the success of this initiative. We are building features for our communities to use — if we design in a vacuum our decisions will assuredly fail.
The plans presented in the grant, on this page, and elsewhere will certainly change over time as we gather input from our community (including victims of harassment, contributors, and administrators,) learn from our research, and learn from the software we build. Community input includes, but is not limited to:
- Socializing our goals
- Generating, refining, validating, and finalizing ideas with community stakeholders
- Conversations about freedom of expression vs. political correctness. It’s very important that this project is seen as addressing the kinds of abuse that everyone agrees about (obvious sockpuppet vandalism, death threats) and the kinds of abuse that people will differ over (gender, culture, etc.). The project will not succeed if it’s seen as only a “social justice” power play.
Over the course of this initiative we plan to communicate with the community via regular wiki communication (talk pages, email, IRC) in addition to live-stream workshops, in-person workshops at hack-a-thons and Wikimanias, and online community consultations. At the moment, the best place to discuss the Community health initiative is on Talk:Community health initiative.
In short, we want to build software that empowers contributors and administrators to make timely, informed decisions when harassment occurs. Four focus areas have been identified where new tools could be beneficial in addressing and responding to harassment:
We want to make it easier and more efficient for editors to identify and flag harassing behavior. We are currently questioning how harassment can be prevented before it begins, and how minor incidents be resolved before they snowball into larger uncivil problems.
- Performance, usability, and stability improvements to AbuseFilter (meta notes)
- Reliability and accuracy improvements to ProcseeBot
- Anti-spoof improvements to pertinent tools
- Features that surface content vandalism, edit warring, stalking, and harassing language to wiki administrators and staff
According to Detox research, harassment is underreported on English Wikipedia. No victim of harassment should abandon editing because they feel powerless to report abuse. We want to provide victims improved ways to report instances that are more respectful of their privacy, less chaotic and less stressful than the current workflow. Currently the burden of proof is on the victim to prove their own innocence and the harasser's fault, while we believe the MediaWiki software should perform the heavy lifting.
- A new harassment reporting system that doesn't place the burden of proof on or further alienate victims of harassment.
Proficiency with MediaWiki diffs, histories, and special pages is imperative for admins to be able to analyze and evaluate the true sequence of events of a conduct dispute. Volunteer-written tools such as Editor Interaction Analyzer and WikiBlame help, but current processes are time consuming. We want to build tools to help volunteers understand and evaluate harassment cases, and inform the best way to respond.
- A robust interaction history tool, which will allow wiki administrators to understand the interaction between two users over time, and make informed decisions in harassment cases. (meta notes)
- A private system for wiki administrators to collect information on users’ history with harassment and abuse cases, including user restrictions and arbitration decisions.
- A dashboard system for wiki administrators to help them manage current investigations and disciplinary actions. (See existing dashboards here.)
- Cross-wiki tools that allow wiki administrators to manage harassment cases across wiki projects and languages.
We want to improve existing tools and create new tools, if appropriate, to remove troublesome actors from communities or certain areas within and to make it more difficult for someone who's blocked from the site to return.
Some of these improvements are already being productized as part of the 2016 Community Wishlist. See Community Tech/Blocking tools for more information.
- Per-page and per-category blocking tools to enforce topic bans, which will help wiki administrators to redirect users who are being disruptive without completely blocking them from contributing to the project; this will make wiki admins more comfortable with taking decisive action in the early stages of a problem.
- Make global CheckUser tools work across projects, improving tools that match usernames with IP addresses and user agents so that they can check contributions on all Wikimedia projects in one query.
- Sockpuppet blocking tools.
Policy Growth & Enforcement
In addition to building new tools, we want to work with our largest communities to ensure their user conduct policies are clear and effective and the administrators responsible for enforcing the policies are well-prepared.
Beginning with English Wikipedia, a large community from which can be obtained a wealth of data, we will provide contributors with research and analysis of how behavioral issues on English Wikipedia are a) covered in policy, and b) enforced in the community, particularly noticeboards where problems are discussed and actioned. We will provide research on alternate forms of addressing specific issues, researching effectiveness, and identifying different approaches that have found success on other Wikimedia projects. This will help our communities make informed changes to existing practices.
Schedule of work
This schedule will almost certainly change based on 1) Wikimedia annual plan processes 2) research conducted as part of this initiatives 3) community input 4) new opportunities that organically arise and 5) learnings from features built earlier in this initiative.
As of this writing (May 2017), the Community Tech and Community Engagement teams are in the process of hiring and on-boarding new team members, recruited specifically for this project. The goal is to have the new team in place by the end of May 2017. The remaining open positions is:
- A Back-end engineer to build the tools.
March 2017 - June 2017
Depending on team member start-dates and on-boarding, the first few months will include:
- Hire and onboard the new hires of the team.
- The start of new research projects to help the team gain a deeper understanding of the current needs and problems.
- Discussions with the volunteer community as a whole, as well as targeted discussions with wiki functionaries who handle disruptive behavior, people who have experienced and witnessed harassment on our wikis, and other important stakeholders.
- Development work on some of the tools that have already been identified by the community as needing improvement, most likely AbuseFilter and ProcseeBot.
July 2017 - June 2018
The first full fiscal year of this initiative is planned to include:
- Focused research on the Harassment Reporting System.
- Work with the community on potentially creating a new user group for volunteer administrators who want to work specifically on harassment cases. The community advocate will work with the Support and Safety team to provide training and support for these volunteers.
- Designing, building, releasing, and iterating on:
- usability & performance improvements to AbuseFilter
- accuracy improvements to anti-spoof tools across multiple pertinent tools
- a robust interaction history tool, similar to the edit interaction analyzer
- a private system for admins to discuss and evaluate incidents of harassment
- per-page blocking tools
- cross-project improvements to CheckUser tools
- robust sockpuppet identification and blocking tools
July 2018 - June 2019
Our plans will change based on our continual learnings, but at this moment in time the second fiscal year of this initiative is planned to include:
- Designing, building, releasing, and iterating on:
- the Harassment Reporting System
- tools that surface brewing situations of harassment and vandalism to community leaders before they become large-scale incidents
- a dashboard system for wiki administrators to help them manage current investigations and disciplinary actions
- cross-wiki tools that allow wiki administrators to manage harassment cases across wiki projects and languages
- Internationalizing all tools we've built to date
- Community health initiative/Links for more links.
- Community health initiative/Notes for team member notes.
- Community health initiative/The Team for a list of WMF staff working on this initiative.
- Community health initiative/English-language Wikipedia harassment policies & enforcement for analysis & documentation
- Community health initiative/Dashboards used by admins to track reports of harassment for analysis & documentation
- Community health initiative/User Interaction History tool for analysis & documentation
- Community health initiative/AbuseFilter
- Research:2015 Harassment Survey
- Inspire Campaign on "Addressing harassment" (June 2016)
- Board Statement on Healthy Community Culture, Inclusivity, and Safe Spaces (November 2016)
- "Wikipedia:List of administrators/Active". 2017-02-13.
- "Wikipedia:Harassment § Dealing with harassment.". 2017-02-12.
- Duggan, Maeve (2014-10-22). "Online Harassment". Pew Research Center: Internet, Science & Tech. Retrieved 2017-02-13.
- "Algorithms and insults: Scaling up our understanding of harassment on Wikipedia – Wikimedia Blog". Retrieved 2017-02-13.
- "Wikimedia Foundation receives $500,000 from the Craig Newmark Foundation and craigslist Charitable Fund to support a healthy and inclusive Wikimedia community – Wikimedia Blog". Retrieved 2017-02-13.
- "File:Wikimedia Foundation grant proposal - Anti-Harassment Tools For Wikimedia Projects - 2017.pdf - Meta" (PDF). meta.wikimedia.org. Retrieved 2017-02-14.