Research:Revision scoring as a service/Community engagement

Recentchanges filtering. A conceptual diagram shows how the work of reviewing recentchanges can be minimized by a machine learning classifier that can split edits into those needing review and those that are probably good.

Overall, most contributors to Wikimedia projects contribute in good-faith. By keeping our editing model open, we get to take advantage of this fact, but it also means that we have to make sure that edit review and curation work can keep up with the rate of new contribution.

One of the greatest advancements in building edit reviewing and curation capacity is the machine classifier. This tool helps minimize the number of edits that *need* review by large percentages. While these tools have been used extensively in English and German Wikipedia, they are not available most other large wikis and all smaller wikis. In the R:Revision scoring as a service project, we're working to bring useful machine learning models to any wiki community who will partner with us to do so. That's where our ask comes from -- to help us work effectively with as many communities as possible.

What community support is needed?

Right now, we're focused on providing access to "edit quality" models via the ORES service and mw:Extension:ORES for as many wikis as we can. These models are useful for counter-vandalism work and the identification of good-faith new editors. In order to stand up new prediction models, we need editors to help us:

1. gather language assets (e.g. lists of badwords [curses, racial slurs] and informal words ["hellooooo", "yolo", "wat"]). We can sometimes partially generate these, but they need a native speaker's review.
2. label edits as "damaging" and/or "good-faith" using our Wiki labels system. This helps us train the machine learning models to replicate human judgement.
3. act as liaisons to their local communities so that we can know their needs

In the past, we've taken advantage of the multi-lingual nature of the team. At one point, we had contributors who spoke English, Turkish, Portuguese and Persian on the team. So, those wikis have received the best support (as evident by the Objective Revision Evaluation Service/Support table). With some help to find and manage our interactions with community members in projects that have little or no support right now, we could bring these powerful tools to them more quickly.

Targeting emerging communities

Recentchanges patrolling is not that difficult for small wikis. For wikis that get fewer than 50 human (non-bot) edits per day, a single person can review Special:Recentchanges in a few minutes. However, as a community begins to grow in popularity, they'll likely experience an exponential growth curve. These are our emerging communities and during their emergence, editors will likely struggle with the sudden shift in patrolling workload. We want the ORES service and mw:Extension:ORES to be ready to help.

Further, there may be other issues that we (the Revision Scoring team) can help with that we're not currently aware of. For example, machine learning models can be very good at page topic categorization and other types of backlog processing work. Without a high bandwidth connection to those communities, we'll never know about such an opportunity.

Conclusion-summary

So, we'd like to work with Community Engagement in whatever capacity you can help us to gather collaborators from the communities most in need of our support.