Research:Notifications user survey

From Meta, a Wikimedia project coordination wiki
Tracked in Phabricator:
Task T127257
Created
23:21, 18 February 2016 (UTC)
Duration:  2016-February – 2016-May
This project's code is open-source
This page documents a completed research project.


The goal of this survey is to understand how active Wikimedians use the Notifications feature, in order to inform the development (by the Collaboration team) of new Notifications features. In particular, the survey will focus on understanding the volume of notifications active users receive, how well the existing feature set serves their needs, and which types of notifications users consider to be most important to read and/or follow-up on.

Methods[edit]

Survey text

The survey will be developed in English and distributed on English-language wikis first. It will be distributed via a banner in the Notifications pane (Phabricator ticket). The goal is to receive responses from at least 1000 editors from the first round. Subsequent rounds will be distributed in at least one additional language in order to avoid relying solely on English-speakers.

Timeline[edit]

  • February 2016: draft and refine the survey with the Collaboration team, identify forums and list where the survey will be advertised, translate the survey (TBD)
  • March 2016: open the survey on English wikis. Translate to additional language and distribute.
  • April 2016: open the survey on French wikis.
  • May 2016: analyze and report results.

Policy, Ethics and Human Subjects Research[edit]

The survey will not elicit any personally identifiable information from respondents. Privacy statement & data use guidelines are available on FoundationWiki.

Results[edit]

Summary of findings presented to the Wikimedia Collaboration team.

We received a total of 158 responses to the survey.

Quantified results are from this Jupyter notebook.

Part I. Summary statistics & demographics[edit]

How many completed surveys do we have per language?
  • There were 122 responses in English
  • There were 36 responses in French


Q2. How active are the survey respondents?
  • 109 made more than 100 edits per month
  • 36 made more 10-100 edits per month
  • 8 made 2-10 edits per month
  • 5 made 0-1 edits per month


Q3. Q3. In the past three months, which of the following Wikimedia projects have you edited while logged in to your account?
  • 155 have edited Wikipedia in the past 3 months
  • 98 have edited Wikimedia Commons in the past 3 months
  • 58 have edited Wikidata in the past 3 months
  • 42 have edited Meta-Wiki in the past 3 months
  • 27 have edited MediaWiki in the past 3 months
  • 26 have edited Wiktionary in the past 3 months
  • 15 have edited Wikisource in the past 3 months
  • 8 have edited Wikiquote in the past 3 months
  • 8 have edited Wikibooks in the past 3 months
  • 6 have edited Wikivoyage in the past 3 months
  • 4 have edited Wikinews in the past 3 months
  • 3 have edited Wikiversity in the past 3 months
  • 2 have edited Wikispecies in the past 3 months


Q5. In the past three months, how frequently have you visited at least one Wikimedia project?
  • 120 logged in multiple times per day
  • 14 logged in once per day
  • 1 logged in multiple times per week
  • 12 logged in once per week
  • 3 logged in multiple times per month
  • 2 logged in once a month or less
  • 6 were not sure how frequently they logged in


Q6. In a typical day, how many new (unread) notifications do you usually get on the project you work on most?
  • 84 received 0-1 new notifications
  • 50 received 2-5 new notifications
  • 15 received 5-10 new notifications
  • 9 received more than 10 new notifications


Part II. Standard Notifications[edit]

Q7. Which of the following types of notification have you seen on Wikimedia projects?
  • 146 have seen user talk page message notifications
  • 146 have seen thanked notifications
  • 134 have seen mentioned notifications
  • 121 have seen reverted notifications
  • 92 have seen page linked notifications
  • 66 have seen page reviewed notifications
  • 45 have seen user rights notifications
  • 42 have seen email notifications
  • 4 have seen none of these notifications


Q8. Please rank the following list of notification types in order of how important it is that you read it right away?
  • 44.1% of respondents ranked 'user talk page message' notifications as their first reading priority
  • 32.6% of respondents ranked 'user rights' notifications as their first reading priority
  • 20.9% of respondents ranked 'reverted' notifications as their first reading priority
  • 17.1% of respondents ranked 'mentioned' notifications as their first reading priority
  • 7.5% of respondents ranked 'email' notifications as their first reading priority
  • 7.3% of respondents ranked 'thanked' notifications as their first reading priority
  • 6.7% of respondents ranked 'page reviewed' notifications as their first reading priority
  • 3.5% of respondents ranked 'page linked' notifications as their first reading priority


Q9. Please rank the following notification types in order of how important it is that you act on it right away?
  • 43.1% of respondents ranked 'user talk page message' notifications as their first action priority
  • 37.2% of respondents ranked 'reverted' notifications as their first action priority
  • 14.6% of respondents ranked 'mentioned' notifications as their first action priority
  • 14.0% of respondents ranked 'user rights' notifications as their first action priority
  • 11.1% of respondents ranked 'email' notifications as their first action priority
  • 5.0% of respondents ranked 'page reviewed' notifications as their first action priority
  • 3.6% of respondents ranked 'page linked' notifications as their first action priority
  • 2.3% of respondents ranked 'thanked' notifications as their first action priority

Part III. Flow Notifications[edit]

Q10. Have you used Flow on at least one of the Wikimedia projects you visit regularly?
  • 42 have used Flow
  • 42 have not used Flow because it is not enabled on projects they visit regularly
  • 28 have not used Flow even though it is enabled on one or more projects they visit regularly
  • 46 are not sure whether they have used Flow
  • 27 Enwiki respondents have used Flow
  • 15 Frwiki respondents have used Flow


Q11. Which of the following types of Flow notification have you seen on Wikimedia projects?
  • 33 have seen new comment on topic notifications
  • 27 have seen new topic on talk page notifications
  • 18 have seen mentioned notifications
  • 13 have seen thanked notifications
  • 7 have seen renamed topic notifications
  • 5 have seen none of these notifications
  • 4 have seen edited their post notifications


Q12. Please rank the following notification types in order of how important it is that you read it right away?
  • 66.7% of respondents ranked 'edited their post' notifications as their first reading priority
  • 53.3% of respondents ranked 'mentioned' notifications as their first reading priority
  • 37.5% of respondents ranked 'new comment on topic' notifications as their first reading priority
  • 20.0% of respondents ranked 'renamed topic' notifications as their first reading priority
  • 14.3% of respondents ranked 'new topic on talk page' notifications as their first reading priority
  • 10.0% of respondents ranked 'thanked' notifications as their first reading priority


Q13. Please rank the following list of Flow notifications in order of how important it is that you act on it right away?
  • 66.7% of respondents ranked 'edited their post' notifications as their first action priority
  • 60.0% of respondents ranked 'mentioned' notifications as their first action priority
  • 40.0% of respondents ranked 'new comment on topic' notifications as their first action priority
  • 19.0% of respondents ranked 'new topic on talk page' notifications as their first action priority
  • 10.0% of respondents ranked 'thanked' notifications as their first action priority
  • 0.0% of respondents ranked 'renamed topic' notifications as their first action priority


Q14. To what extent do you agree with the following statement?
"Notifications provides a good overview of relevant activity and help me keep up with what is happening on the Wikimedia projects I visit regularly."
  • 19 strongly agree that Notifications are useful
  • 14 agree that Notifications are useful
  • 0 neither agree nor disagree that Notifications are useful
  • 2 disagree that Notifications are useful
  • 2 strongly disagree that Notifications are useful

Free-text responses[edit]

See also[edit]

References[edit]