Research:Testing effects of Impact Stats on newcomer retention
This page documents a proposed research project.
Information may be incomplete and may change before the project starts.
In surveys and interviews, Wikipedians say they started and continue editing primarily because they share Wikipedia's mission to make all knowledge free.[1][2] In other words, they are intrinsically motivated to make a difference. That motivation can be tested and tapped by seeing if giving editors information about the impact they have made encourages them to continue editing. Knowing the number of pageviews to articles they have edited (their "impact stats") is one way for newcomers to gauge that impact. If so, showing newcomers their impact stats may also be an effective way to retain new editors.
This study was one of the many research ideas that were generated by Wikipedians at CivilServant's 2019 Research Summit in Stockholm. We began fielding this study in December 2020, but it was put on indefinite hold in February, 2020.
Please see translations of this page in Bangla, German and Slovak.
Methods[edit]
In collaboration with Wikipedians in Bangla, German and Slovak Wikipedias, we began conducting a randomized controlled trial in which newly registered users ('newcomers') received a talk page message seven days after registering. That message either showed them their impact stats and encouraged them to continue editing or it was a control message that does not include their impact stats. Only newcomers and edits that satisfy criteria within their community were be included in the study. We planned to look at newcomer retention as our primary outcome. Up to 15,000 newcomers across the three Wikipedias were intended be included in the study.
Timeline[edit]
- August - September, 2020: Research design, in collaboration with liaisons in Bangla, German & Slovak Wikipedias.
- September - October, 2020: Research development and seeking community consent
- December, 2020: Study launch
- February, 2021: Study placed on indefinite hold
Policy, Ethics and Human Subjects Research[edit]
This study was reviewed by Cornell's Institutional Review Board to ensure it complies with research ethics. The study was also presented and discussed with Wikipedians in their public forums. While initially there was no objection to the study in each of our partnering Wikipedia communities, after initiating the study one community expressed concern about our process for seeking approval as well as some aspects of the design. After a discussion with the community, while the concerns resolving the design were resolved, we found there was not significant interest in moving the study forward.
About CivilServant[edit]
CivilServant (now Citizens and Technology (CAT) Lab at Cornell University) works with online communities to study the effects of technology on the public interest. Our studies are designed and conducted in collaboration with communities, independent of the WMF.
Results[edit]
If the study is resumed, results will be presented here.
References[edit]
- ↑ 2011 Wikipedia Editor Survey
- ↑ Bryant, S. L., Forte, A., & Bruckman, A. (2005). Becoming Wikipedian: Transformation of Participation in an Online Encyclopaedia. Proceedings of the 2005 International ACM SIGGROUP Conference on Supporting Group Work - GROUP ’05, 1.