Research:Testing effects of Impact Stats on newcomer retention
In surveys and interviews, Wikipedians say they started and continue editing primarily because they share Wikipedia's mission to make all knowledge free. In other words, they are intrinsically motivated to make a difference. That motivation can be tested and tapped by seeing if giving editors information about the impact they have made encourages them to continue editing. Knowing the number of pageviews to articles they have edited (their "impact stats") is one way for newcomers to gauge that impact. If so, showing newcomers their impact stats may also be an effective way to retain new editors.
This study is one of the many research ideas that were generated by Wikipedians at CivilServant's 2019 Research Summit in Stockholm.
In collaboration with Wikipedians in Bangla, German and Slovak Wikipedias, we will conduct a randomized controlled trial in which newly registered users ('newcomers') will receive a talk page message seven days after registering. That message will either show them their impact stats and encourage them to continue editing or it will be a control message that does not include their impact stats. Only newcomers and edits that satisfy criteria within their community will be included in the study. We will look at newcomer retention as our primary outcome. Up to 15,000 newcomers across the three Wikipedias will be included in the study.
- August - September, 2020: Research design, in collaboration with liaisons in Bangla, German & Slovak Wikipedias.
- September - October, 2020: Research development and seeking community consent
- October or November, 2020: Study launch
- April, 2021: Study completion
- Spring & Summer, 2021: Data collection, analysis and community debriefing.
Policy, Ethics and Human Subjects Research
This study will be reviewed by Cornell's Institutional Review Board to ensure it complies with research ethics. The study will also be presented and discussed with Wikipedians in their public forums. Only if there is community consensus to conduct the study will we move forward.
CivilServant (now Citizens and Technology (CAT) Lab at Cornell University) works with online communities to study the effects of technology on the public interest. Our studies are designed and conducted in collaboration with communities, independent of the WMF.
Results will be presented here.
- 2011 Wikipedia Editor Survey
- Bryant, S. L., Forte, A., & Bruckman, A. (2005). Becoming Wikipedian: Transformation of Participation in an Online Encyclopaedia. Proceedings of the 2005 International ACM SIGGROUP Conference on Supporting Group Work - GROUP ’05, 1.