Research:Patroller work load

From Meta, a Wikimedia project coordination wiki
Jump to: navigation, search
Nutshell.png
This page in a nutshell: The number of new pages that human editors patrol has been going down since 2007. This suggests that the workload of new page patrollers has also been decreasing.
Gnome globe current event.svg This project page documents a research project currently in progress.
Information may be incomplete and can change rapidly as science advances.
Research project
Patroller work load
Main contact
WMF contact not applicable
Start 2011-06
Status in progress Icon 66 percent.png
Field computer science
Open data This project has published open-licensed data
Open access This project has open access publications
WMF support
Wikimedia research projects Wikimedia research projects

When a new editor creates an article, how it and they are treated by the community has a large impact on whether they continue to edit or not. One factor that may have a large influence on whether these new article authors are bitten is if those doing new page patrol are overworked. Editors who are overworked and stressed are likely to feel less patient and accommodating towards new editors, so measuring the workload of new page patrollers is important if the movement is going to figure out how to be more open and welcoming to new editors.

Question[edit]

  • Has the workload for new page patrollers increased over time? (RQ1.16)

Background[edit]

New Page Patrol is a vital function of many Wikipedias as the front line of interaction between new authors and community members devoted to policing the quality of the project. It has variety of detailed, quite complex possible actions for patrolling pages in all namespaces. Below is a simplified flow of possible outcomes and actions in patrolling new articles in the main namespace.

EN New Page Patrol actions.png

Methods[edit]

This sprint analyzed all page patrolling activity for each calendar month of Wikipedia history from 2007 (when the patrol logs were started) up to June of 2011. We initially wanted to cover the activity of both new page patrol and vandalfighting, but due to downtime and time constraints we reoriented towards patrolling only. Note: This analysis only measures patrolling that appears in the log. If patrolling outside the queue occurred, it has been discounted.

Page patrolling activity was gathered from the Wikipedia database with the following query:

SELECT * FROM logging
WHERE log_type="patrol"
AND log_action="patrol"

Patrolling events were then grouped by the user who performed the action and the day the action took place. All plots and statistics come from this dataset.

Removing bots[edit]

Bots were removed by applying a regular expression based on en:WP:Bot_policy and en:WP:Username_policy.

/bot( |$)/i

Error bars[edit]

Standard error bars are included in each plot where applicable. In some plots they are too small to be visible due to the high number of observations.

Results and discussion[edit]

The distribution of page patroller's work is plotted for 2010. A distinct powerlaw curve can be seen. Very few editors perform the vast majority of page patrolling work.

Patrolling work is distributed across users as a power law (see figure above). This means that a very small proportion of English Wikipedia editors are doing the majority of page patrolling work. The top 50 patrollers plot to the right shows the change in editor trends since page patrol logging began in 2007. From 2008 onward, a small cohort of editors is responsible for the majority of page patrolling that takes place. The increasing activity of these top 50 editors suggests that the work load required for each page patroller is increasing.

2007
2008
2009
2010
2011*


Top 50 patrollers by their activity from 2007-mid 2011 for English Wikipedia.

The top 50 (non-bot) patrollers are plotted in order by the number of pages they had patrolled in a year. *This plot was generated in June of 2011, so the data for that year is not complete.

Patrolling activity by month vs year[edit]

When examining at the amount of patrolling that patrollers do in a year, the work load per patroller appears to be decreasing linearly.

Patrollers' monthly activities also appear to be decreasing linearly (β=-1.050 p<0.001). The Mean patrolling actions per user-month plot shows the decreasing trend in the amount of patrolling per editor-month. This trend suggests that editors are doing less patrolling on a monthly basis.

Both of these plots refute the hypothesis that patroller workload is increasing on a per user basis.

Workload for just the top 50[edit]

The result that workload was decreasing was surprising to say the least, so we re-focused our analysis on the work load of the top 50 users. Again we found that the per-patroller workload has been decreasing.

Proportion of bots in the top 50 new page patrollers
Year Number of bots in top 50 Percent of total
2011* 16 32%
2010 17 34%
2009 14 28%
2008 9 18%
2007* 0 0%

The number of bots in the top 50 new page patrollers and their proportion as a percent of the whole is presented. This table represents a ee-analysis to include bots and autopatrolled actions.

Summary[edit]

The results of this analysis refute the hypothesis that page patrollers' work is increasing. In fact, the workload has been decreasing and has been distributed among more patrollers than it used to be. A factor that may have added to the effect is the increased involvement of bots and autopatrolled editors to decrease the number of pages that patrollers must patrol, as well as a generally decreasing number of articles being created every month.[1]

Future work[edit]

  • It is quite easy to run these same queries across other languages, so replicating the study across Wikipedias other than English is our next step.
  • Are editors more or less specialized (performing only these tasks) than they used to be? Working harder over the years solely on new page patrol or vandalfighting may be leading to higher burnout rates among experienced editors, as well as a more hostile environment for new editors who are dealing with patrollers whose patience has worn thin.

References[edit]

  1. stats.wikimedia.org