Grants:Project/University of Virginia/Machine learning to predict Wikimedia user blocks/Final

Add links
From Meta, a Wikimedia project coordination wiki


Report accepted
This report for a Project Grant approved in FY 2018-19 has been reviewed and accepted by the Wikimedia Foundation.



Welcome to this project's final report! This report shares the outcomes, impact and learnings from the grantee's project.

Part 1: The Project[edit]

Summary[edit]

This pilot project had the following goals:

  1. use machine learning (ML) to develop a prediction model for identifying accounts on English Wikipedia which engage in misconduct
  2. engage the Wikipedia community in conversation on the ethics of addressing user misconduct, especially by using artificial intelligence (AI)
  3. produce documentation which would enable future researchers to do ML projects, both on misconduct and in general in Wikimedia data

The team accomplished all of these things as a pilots to establish precedents for future research. The research media here should orient anyone else doing committed research. Barriers to Wikimedia-style general consideration of this research include challenges understanding the Wikimedia datasets, the nature of machine learning, the spectrum of user misconduct, governance by community process, and financial costs of possible computation which could generate solutions. The near-future solution to these various challenges is more intense production of documentation and cultural products to increase understanding and conversation and to lower the barriers of additional analysis.

Project Goals[edit]

This pilot project had the following goals:

  1. use machine learning (ML) to develop a prediction model for identifying accounts on English Wikipedia which engage in misconduct
  2. engage the Wikipedia community in conversation on the ethics of addressing user misconduct, especially by using artificial intelligence (AI)
  3. produce documentation which would enable future researchers to do ML projects, both on misconduct and in general in Wikimedia data

Project Impact[edit]

Important: The Wikimedia Foundation is no longer collecting Global Metrics for Project Grants. We are currently updating our pages to remove legacy references, but please ignore any that you encounter until we finish.

Targets[edit]

  1. In the first column of the table below, please copy and paste the measures you selected to help you evaluate your project's success (see the Project Impact section of your proposal). Please use one row for each measure. If you set a numeric target for the measure, please include the number.
  2. In the second column, describe your project's actual results. If you set a numeric target for the measure, please report numerically in this column. Otherwise, write a brief sentence summarizing your output or outcome for this measure.
  3. In the third column, you have the option to provide further explanation as needed. You may also add additional explanation below this table.
Planned measure of success
(include numeric target, if applicable) 		Actual result Explanation
machine learning to research user misconduct produced research with a paper
 This research paper is the primary research output of the machine learning research.
develop project as a model for research The team developed a project page and an institution page.
 The team generated a community facing project page for this research and also the Wikimedia research portfolio of the university itself.
Collect lists of community ethical concerns produced essays
 The original proposal said "lists". It was a challenge to produce meaningful list media so we did essays for now.
Report ways to have casual conversation on the research. created on-wiki documentation
 English Wikipedia documentation pages are conventional channels for making policy available for community consideration.


Story[edit]

Looking back over your whole project, what did you achieve? Tell us the story of your achievements, your results, your outcomes. Focus on inspiring moments, tough challenges, interesting anecdotes or anything that highlights the outcomes of your project. Imagine that you are sharing with a friend about the achievements that matter most to you in your project.

  • This should not be a list of what you did. You will be asked to provide that later in the Methods and Activities section.
  • Consider your original goals as you write your project's story, but don't let them limit you. Your project may have important outcomes you weren't expecting. Please focus on the impact that you believe matters most.

Survey(s)[edit]

If you used surveys to evaluate the success of your project, please provide a link(s) in this section, then briefly summarize your survey results in your own words. Include three interesting outputs or outcomes that the survey revealed.

Other[edit]

Is there another way you would prefer to communicate the actual results of your project, as you understand them? You can do that here!

Automated detection of Wikipedia misconduct!

Methods and activities[edit]

Please provide a list of the main methods and activities through which you completed your project.

Project resources[edit]

Please provide links to all public, online documents and other artifacts that you created during the course of this project. Even if you have linked to them elsewhere in this report, this section serves as a centralized archive for everything you created during your project. Examples include: meeting notes, participant lists, photos or graphics uploaded to Wikimedia Commons, template messages sent to participants, wiki pages, social media (Facebook groups, Twitter accounts), datasets, surveys, questionnaires, code repositories... If possible, include a brief summary with each link.

Automatic Detection of Online Abuse in Wikipedia
Automated detection of Wikipedia misconduct!
SIEDS 2019 Poster
  • Automatic Detection of Online Abuse and Analysis of Problematic Users in Wikipedia
    Automatic Detection of Online Abuse and Analysis of Problematic Users in Wikipedia
  • An Ethical Reflection on User Privacy and Transparency of Algorithmic Blocking Systems in the Wikipedia Community
    An Ethical Reflection on User Privacy and Transparency of Algorithmic Blocking Systems in the Wikipedia Community
  • Ethics driven approach for implementing automated harassment blocks on the Wikipedia platform
    Ethics driven approach for implementing automated harassment blocks on the Wikipedia platform
  • Ethical considerations in Wikipedia automatic abuse detection system
    Ethical considerations in Wikipedia automatic abuse detection system
  1. Research Proposal
  2. Quarterly Research Progress Presentations
  3. Data Product - A model that detects abusive content in the Wikipedia user community and a model that predicts and flags users who are at risk of getting blocked in the future.
  4. Technical Paper published in the IEEE SIEDS 2019 journal
  5. Research Poster for IEEE SIEDS 2019
  6. Presentation of research at the IEEE SIEDS Conference in Charlottesville, Virginia
  7. Presentation of research at the Applied Machine Learning Conference,TomTom Summit 2019
  8. Other media
  9. Research Paper - File:Automatic Detection of Online Abuse and Analysis of Problematic Users in Wikipedia preprint.pdf
  10. Paper Detailing Ethical Implications of this research
  11. Data and Code Artefacts
  12. Powerpoint Presentation - Automatic Detection of Online Abuse in Wikipedia
  13. Video presentation Automated detection of Wikipedia misconduct!
  14. bluerasberry (31 July 2019). "Most influential medical journals; detecting pages to protect". The Signpost. 

Other[edit]

Learning[edit]

The best thing about trying something new is that you learn from it. We want to follow in your footsteps and learn along with you, and we want to know that you took enough risks in your project to have learned something really interesting! Think about what recommendations you have for others who may follow in your footsteps, and use the below sections to describe what worked and what didn’t.

What worked well[edit]

What did you try that was successful and you'd recommend others do? To help spread successful strategies so that they can be of use to others in the movement, rather than writing lots of text here, we'd like you to share your finding in the form of a link to a learning pattern.

  • Your learning pattern link goes here

What didn’t work[edit]

What did you try that you learned didn't work? What would you think about doing differently in the future? Please list these as short bullet points.

  • documenting Wikimedia's own underdocumented data sets and access processes
  • applying the prediction algorithm to active cases
  • having Wikimedia community conversation in the absence of documentation or this being a mainstream topic
  • archiving the datasets in this project
  • modeling Wikimedia research in machine learning for general engagement
  • incorporating non-English language experiments
  • identifying the best scholarly communication plan

Other recommendations[edit]

If you have additional recommendations or reflections that don’t fit into the above sections, please list them here. The Wikimedia community has extreme difficulty with the fundamentals of addressing misconduct. Useful developments include the following:

  • policy development
    • Code of conduct
    • Friendly space policy
    • Non-discrimination policy
  • machine readable reporting systems
    • anyone can make complaints
    • Wikimedia community gets public access to collect metatdata
    • multiple channels, including public in-wiki and private for Wikimedia community designated moderators

Next steps and opportunities[edit]

Are there opportunities for future growth of this project, or new areas you have uncovered in the course of this grant that could be fruitful for more exploration (either by yourself, or others)? What ideas or suggestions do you have for future projects based on the work you’ve completed? Please list these as short bullet points.

The path to establishing automated moderation tools probably includes a series of amateur projects like this one done as much as possible, perhaps 100 times, to replicate the project with varying focus and diversity of organizers. This project sets some precedent for this but there is still fundamental research infrastructure absent from the Wikimedia environment in addition to specific gaps for ML research.

Sooner or later, the following must happen, and it would better happen sooner:

more accessible documentation of the Wikimedia datasets
  1. rewrite mw:Manual:Database layout for the voice of Meta-Wiki
  2. make recommendations for how to post interesting datasets to a repository like datahub
  3. compile a practical narrative of the Wikimedia datasets for non-technical audiences, because this is an entry level project

If the data is more accessible and comprehensible then many types of research can follow.

Part 2: The Grant[edit]

Finances[edit]

Actual spending[edit]

Please copy and paste the completed table from your project finances page. Check that you’ve listed the actual expenditures compared with what was originally planned. If there are differences between the planned and actual use of funds, please use the column provided to explain them.

Expense Approved amount Actual funds spent Difference
Research sponsorship $5000
Total 5000 5000 0


Remaining funds[edit]

Do you have any unspent funds from the grant?

Please answer yes or no. If yes, list the amount you did not use and explain why.

If you have unspent funds, they must be returned to WMF. Please see the instructions for returning unspent funds and indicate here if this is still in progress, or if this is already completed:

  • no

Documentation[edit]

Did you send documentation of all expenses paid with grant funds to grantsadmin(_AT_)wikimedia.org, according to the guidelines here?

Please answer yes or no. If no, include an explanation.

  • No. This project was part of a 1-year university class machine learning research cohort. In this project, groups of 2-3 students collaborate for the term of the course to each complete a project of interest to a client. The client in this case was the Wikimedia community, supported in part by the Wikimedia Foundation Trust and Safety team. For this research the university requests US$5000 from nonprofit partners to offset research costs, but there is currently no system of itemized billing. The $5000 covered costs including computation in AWS, other university lab fees, university faculty and staff support outside the regular course fees, and media expenses.

Confirmation of project status[edit]

Did you comply with the requirements specified by WMF in the grant agreement?

Please answer yes or no.

  • yes

Is your project completed?

Please answer yes or no.

  • yes

Grantee reflection[edit]

We’d love to hear any thoughts you have on what this project has meant to you, or how the experience of being a grantee has gone overall. Is there something that surprised you, or that you particularly enjoyed, or that you’ll do differently going forward as a result of the Project Grant experience? Please share it here!


Retrieved from "https://meta.wikimedia.org/w/index.php?title=Grants:Project/University_of_Virginia/Machine_learning_to_predict_Wikimedia_user_blocks/Final&oldid=19833528"