Research:Measuring editor time commitment and workflow
Effort and the perception of effort do not always perfectly track each other. This is especially true online, were often the only visible evidence of work comes in your final result. The goal of this research project is to discover the intermediate steps that led to the final result, and to close the gap between effort and its perception. To do this we are asking Wikipedians to record their activities during the study period, the time that they commit to Wikipedia, and the process they follow as they work.
The end result of this study will be a data set of user work, time commitment, and work process. We intend to apply machine learning algorithms to this data set to predict time commitment involved in a given edit. We believe this stands to benefit the community by allowing more accurate recognition of time commitment to Wikipedia, as well as having possible applications in forecasting time commitment for various content development endeavors. Additionally, the data set will be released with user names removed to allow for other researchers and community members to build upon it.
Methods[edit]
We intend to use prompted diary study to collect this data.
Recruitment[edit]
We intend to use a combination of snowball sampling and targeted talk page messages for users that are highly active in the main namespace. For those that are receiving messages with out being refereed by fellow Wikipedian we intend to use the recruitment message provided below.
Hello <Wikipedia_username>,
My name is Kevin Schiroo and I am a graduate student at the GroupLens research lab at the University of Minnesota. We are working to better understand the work-flow Wikipedians follow as they develop their contributions to Wikipedia. We are currently running a diary study of Wikipedians to collect information on this topic and I would like to invite you to participate. The study period will last two weeks and should take between 5 and 20 minutes of your time per day. I’ve posted more information about this study [here] if you would like to learn more.
If this is something you are interested in please let me know by either responding here or on my talk page. If you aren’t interested that is perfectly okay also, this will be the last you will hear about it. Thanks for your time!
We are aiming for between 20 and 50 participants.
User Interface[edit]
We have implemented the diary study as a user script on Wikipedia. The study client is available at User:Another_Article/workflowstudyclient.js. Participants will enable it by adding importScript("User:Another_Article/workflowstudyclient.js"); to their common.js. A server at the University of Minnesota will identify when activity has taken place that we are interested in and the client script will prompt the user with a brief set of questions. Their responses will then be returned to the server.
Prompts are designed to cause minimal interuption, using a small notification box to ask for attention.
Upon selecting the notification box one of two prompts will appear. One focuses on user time commitment, while the other focuses on user process. Participants may answer one or more questions at a time through this interface.
To avoid burn out, participants will never be asked about more than 4 contributions in a 24 hour period and will never be asked about a contribution within 30 minutes of asking about an earlier contribution. Prompts will expire after 24 hours.
Interviews[edit]
After completion of the diary portion of the study a subset of participants will be invited to participate in phone or video interviews to provide additional insights into their specific responses and workflow. These interviews are expected to take less than 1 hour.
Policy, Ethics and Human Subjects Research[edit]
This study was declared exempt from review by the University of Minnesota IRB on January 28, 2016.
Consent Text[edit]
Consent form
Thank you for enabling our survey! But first, we ask that you read the following information and ask any questions you may have before agreeing to participate.
Purpose and Procedure
We are working to better understand the process through which Wikipedians contribute to Wikipedia. We would like to hear about the process you use as you contribute and invite you to participate in this study. If you agree to participate our survey will be integrated into your Wikipedia experience. As you go about your normal activities the survey will occasionally notify you that we have a question about some of your recent contributions. Each set of questions should take less than 5 minutes to answer and we will never ask you more than 4 question sets in a day. The survey period will last for two weeks.
Contacts and Questions
This survey is being conducted for research by the Grouplens Research Lab in the Computer Science and Engineering Department at the University of Minnesota. If you have any questions about this study, please feel free to contact the researchers at schir101@umn.edu. For general information about GroupLens, please visit our website at grouplens.org. If you have any questions or concerns regarding this study and would like to talk to someone other than the researchers, you are encouraged to contact the Research Subjects’ Advocate Line, D528 Mayo, 420 Delaware St. Southeast, Minneapolis, MN 55455; (612) 625-1650.
Confidentiality
Your participation in this study will be kept private and your off-line identity will never be recorded. If at any point in the future you wish to withdraw your responses from this study you may contact us and they will be removed.
You may volunteer to have your responses publicly released for the benefit of other community members and researchers. If you choose to do so your responses and their associated revision id will be included in the publicly released data set. Note that while this will not include your username explicitly, it will be possible to recover your username from this data. If at any point in the future you wish to withdraw your responses from this public data set they will be removed from our hosted data set, although we will not be able to remove them from copies we do not control.
Participation is Voluntary
Participation in this study is voluntary. Your decision whether or not to participate will not affect your relationship with the University of Minnesota. If you decide to participate, you are free to not answer any question or withdraw at any time without affecting this relationship.
Results[edit]
In progress