Research:First edit session/Outline

From Meta, a Wikimedia project coordination wiki
The initial rejection and survival of new editors over time. See the write-up.

Researcher Aaron Halfaker (User:EpochFail) of the WMF's Summer of Research has discovered a strong predictor of new editor retention – the rejection they experience when first trying to edit the encyclopedia.

As a response to recent results that point to a decline in new editors retention since 2007[1][2], Aaron examined the work that new editors perform in their first few editing sessions and the community's reaction to that work, in order to build a model for retention. The results suggest that rejection of newbies' first few edits plays a strong role in newcomer retention. Moreover, the amount of initial investment (edits in the first session) exhibited by a new user exacerbates the effect.

"Wikipedia's climate has changed since the early days before and during the exponential growth. Back then, the community was driven toward building content. More recently, with popular articles becoming longer and more elaborate, a shift seems to have occurred for quality over quantity. I suspected it has become much more difficult for newbies to make edits that wouldn't be immediately rejected and that this would has an effect on their motivation to continue editing. I wanted to look for such an effect and find out how much it matters with respect to the decline in new editor retention."

The changing average length of articles new editors edit. See the write-up.

Aaron's work on the length of an articles the newbies are editing provided him with evidence that editors are editing longer articles, and that this is a strong predictor of being reverted, [3][4] presumably because of Wikipedia's increasingly stringent quality control mechanisms.[5]

To understand whether this increased rejection could explain the decline in editor retention, Aaron used a logistic regression model to explore factors that predict whether a new editor will survive or not. He found that the proportion of an editor's edits that are rejected by being deleted or reverted in their first three edit sessions[6] is a strong negative predictor of survival. This confirms the hypothesis that if a newbie's first experience editing Wikipedia is full of rejection, he or she be unlikely to continue working in the project. It turns out that this effect has existed throughout the history of Wikipedia and has been increasing over time, though it has decreased somewhat in recent years. What's more, while editors who show a high initial investment in the community (by making many edits in their first edit session) are more likely to survive in general, these highly invested new editors suffer even more from having their work rejected than editors who express a lower investment.

Total and surviving new editors by year of first edit.

These findings suggest that it is precisely the kind of newbies that Wikipedia needs – highly invested and prolific editors – who are being turned away by reverts and deletion.

Caveats[edit]

The proportion of new editors making more than 5 edits in their first session plotted over time.

The characteristics of newcomers are changing. Newcomers are expressing less initial investment, making fewer edits than they used to. This could be explained by an early/late adopter effect, or some other external factor.

A WMF report suggests that the number of editors who make acceptable contributions to the encyclopedia is still very high, but a more thorough analysis is needed to determine how much the increase in rejection can be attributed to changes in the quality of new editors' first contributions.

References[edit]

  1. The Editor Trends study [1]
  2. Suh et al., The singularity is not near: slowing growth of Wikipedia. WikiSym'09 http://portal.acm.org/citation.cfm?id=1641322
  3. Dalip et al., Automatic quality assessment of content created collaboratively by web communities: a case study of Wikipedia, JCDL'09 [2]
  4. Blumenstock, Size matters: word count as a measure of quality on wikipedia, WWW'08 [3]
  5. Stvilia et al., Information Quality: Discussions in Wikipedia. ICKM'05 [4]
  6. An edit session is defined as a sequence of edits saved in the encyclopedia separated by less than an hour. It's assumed that by grouping edits together this way, the amount and type of work an editor does in one editing session on Wikipedia can be analyzed.