This document is a work in progress. Comments are appreciated but this is not a final draft.
A roundtable discussion on editor engagement took place at the Wikimedia Foundation in San Francisco on Saturday, June 22, 2013, from 10am to 5pm Pacific Time.
This page provides an overview of this group discussion, which involved 14 Wikipedia editors and 8 Wikimedia product team members for a day, as outlined below.
Learn more about this new program on the Roundtables page.
- 1 Goal
- 2 Topics
- 3 Agenda
- 4 Participants
- 5 Photos
- 6 Videos
- 7 Conclusions
- 8 Final Notes
- 9 Workflows
- 9.1 Workflow 1: Round up collaborators
- 9.2 Workflow 2: Welcoming New Users
- 9.3 Workflow 3: Deletion Notice
- 9.4 Workflow 4: Thanks
- 9.5 More Workflows
- 10 Next Steps
The goal of this roundtable was to improve Wikipedia's editor engagement tools with community participation -- and gain new insights from these conversations.
It was also intended as a first experiment with new forms of collaborations between Wikipedians and WMF product teams. Based on the success of this first event, we plan to host more roundtable discussions between the Wikipedia community and Wikimedia product teams: three more roundtables are being scheduled for the summer of 2013, two of which will take place at Wikimania in Hong Kong (see Roundtables page).
Together, we discussed interactive workflows that editors use every day on Wikipedia -- as well as ways these workflows could be improved with software like the Echo notifications and Flow discussion tools.
In the morning, we started with a large group discussion about common workflows (e.g. content collaborations, reversions, deletions and gratitude). We then broke up into small groups to step through and discuss these workflows in detail.
In the afternoon, we discussed two new tools (Echo notifications and Flow discussions), with interactive demos followed by conversations on how these tools can support workflows more effectively. We then brainstormed new ways to improve workflows with these tools -- through collaborative role-play and user interface sketches. Lastly, we will reviewed all these ideas as a larger group, and identified a few solutions that can create a better experience for all users -- as well as increase participation on Wikipedia.
Here are the slides that we presented throughout the day, to guide our discussion.
Here was our agenda for this daylong event:
- 10:00 am - Introductions + agenda
- 10:30 am - Discussion: what are your regular workflows on Wikipedia?
- 12:00 pm - Lunch: share 'aha moments' that made you more productive as an editor
- 12:45 pm - Echo / Flow demos + discussions: how can these tools support your workflows?
- 2:00 pm - Improving your workflows: go through each step, brainstorm solutions to key issues
- 3:30 pm - Review ideas for engaging users with these tools, identify promising solutions
- 5:00 pm - Roundtable ends
For more details on this roundtable format, check out this detailed facilitator script.
Fourteen community members joined this roundtable -- with an even mix of new editors, regular editors and power users. Invitees included Bay Area editors who participated in previous Edit-a-thons or meetups and/or were recommended by WMF team members, as well as a couple out-of-town editors.
- Kevin Gorman
- Max Klein
- Raymond Yee
- Krystle Chung
We shot extensive photos of this event. Here are a few samples of the discussions, sketches and ideas we captured.
Check out this photo gallery for more pictures.
We shot 5 hours of video footage during this roundtable and broke it into 10 different parts, in order to upload them in open source WebM format on Wikimedia Commons.
This footage can be downloaded from Commons and re-used under CC-BY-SA terms.
This documentation is intended for transparency reasons, as well as to insure better knowledge transfer with other community members who couldn't join these events.
At the end of the roundtable, participants were invited to summarize key issues and ideas that came out of their small group discussions. Here are some highlights of what they told us.
- It's hard to find a collaborator
- Newbies don't know where to go, what to do
- How to get experienced people to help newbies?
- Newbies don't get enough notifications, nobody talks to them
- Unify discussions so they are all in one place
- Don't want to check separate Feed page (in addition to Watchlist or Notifications)
- Need enough control on my feed so I can make sure it's relevant to me (proliferation)
- Mobile users should have the same basic features (notifications, talk page, etc.)
- Power users will have to agree on workflow logic rules (which could take a while)
- Twinkle users are unlikely to want to use a new tool (resistance to change)
- Welcome or help area on newbie user pages
- Big 'Help me' buttons for newbies (instead of cryptic 'Help templates')
- Help links should send newbies to TeaHouse/HelpDesk (not to policy pages)
- TeaHouse Notification should be sent automatically to newbies after x days
- Provide 'Refer to TeaHouse' button (that admins can post in response to newbie queries)
- Send Reply notification when someone responds to my post on another talk page
- Place the 'Thanks' notification link in more places (e.g.: watchlist, contribs)
- Add 'Thanks' link at the end of every talk message (to promote better dialog)
- Consider a 'Thanks' button like WikiHow (at the bottom of articles?)
- Add confirmation step when clicking 'rollback' on mobile phones
- Show Nav popups for linked articles, and users (with quick preview)
- Auto-sign my talkpage posts (so newbies don't have to learn about tildes)
- Use Transclusion, so a single discussion can appear in multiple places
- Use # hashtags to help tag posts, and add things to your feed
- Show 'Read' label when the recipient has read your message
How can we increase participation on Wikipedia?
- Pay people
- Improve new user experience
- Engage more readers (through feedback)
- Retain new users (anything that will engage them more)
Making Wikpedia better
What is one thing we could do for improving Wikpedia? We asked participants to write down their ideas, then took their picture.
- User Fun Is Serious Business
- Require User Registration
- Simple Friendly Welcomes
- Metadata: Good
- I Want Quick Feedback From Readers
- Be Welcoming. Make the Project User-Friendly
- Keep In Touch With Your Sister Projects
- Show The World TRANSCLUSION
- User Avatars
- Visual Maps of Key Stations
- Make It Friendly, Make It Helpful
- Global Watchlists
- Make It More Personal
We encourage participants to add their final notes about this roundtable on this discussion page (or in sub-pages related to this overview -- e.g.: Roundtable_1/Notes).
See also: personal observations from Wikimedia team members about this roundtable process.
In coming days, we will collaboratively expand this section to describe the workflows which each group discussed. We encourage participants to fill in any blanks, if you have time. To refresh you memory, you can check out the photos of our sketches here.
Workflow 1: Round up collaborators
Here are the participants and workflow for group 1. See photos of this group's sketches.
- I find a crummy article on a subject that I have no interest in, or knowledge of, and want to find someone relevant to help it
- Betsy finds crummy article (through watchlist, or NPP, or etc)
- Betsy wants to determine collaborators
- Betsy leaves a message on Jake's talkpage
- Jake gets a Notification, and reviews the article and its talkpage
- Jake leaves an @reply on his talkpage, and mentions that he "took a look at the article and made these changes"
- Betsy gets a Notification, and replies with a note to say thanks
- sometimes a "Mention" Notification is really a direct Reply.
- How does Betsy decide on a collaborator? How does Jake know to ask SPat and Saehrimnir?
- Uncertainty over where to reply when a question is asked at a usertalkpage.
- There's great potential here to let people register as topic experts so that they can be pinged for help. That could be done opt-in and manually per article. Or, it could be done opt-in, but by category (or WikiProject). Tools which automatically surface the most active recent contributors to a page, WikiProject, or Category could provide a base list of potential helpers. Finding out who is expert and experienced in what areas--and willing to help--would greatly improve possibilities for collaboration. This could be combined with notifications such that a person could put out a 'bat signal' for an article that needs attention, and any editors who opted-in to that particular area would receive a notification to look at it. The current method for this is a lame 'this article needs an expert' template and/or a WikiProject talk page post. WikiProjects being largely inactive and uncoordinated, are not really sufficient for most areas to generate a willing and able helper. The new notifications help a lot with pinging the right people to an article (using a mention), but don't help finding out who to ping. That'd be a great ability.
- While mostly semantic, the difference between a mention and a reply can be important. While mentions may be tangential, a direct reply to something you said is likely to be directly relevant and warranting a response. A reply could be defined specifically as a mention that happens immediately below your comment. Or it could be just a comment that follows your own but without a specific @ or User:Example ping. Or a reply could be any downstream comment in a thread you were involved in. These should all be configurable to moderate the number of updates a user receive depending on their personal preferences for volume/relevance.
- Editors vary greatly in their preferences for keeping conversations in one place versus making sure the relevant person reads them. Mentions greatly improve the functionality here since you don't have to worry about leaving talkback messages or pages being viewed on the Watchlist--due to the notifications that mentions generate. Better built-in functionality for mentions, such as @replies, username auto-complete would make this even better. One feature that might be useful would be the ability to 'move a thread' in the same way you can move a page. This would allow a conversation, for example, on a user's talk page which becomes irrelevant to the user, to be moved to a more appropriate location without having to copy-paste and lose history.
Workflow 2: Welcoming New Users
Here are the participants and workflow for group 2. See photos of this group's sketches.
- Express appreciation for a task the newcomer has completed.
- Incorporate newcomers into the community (activation)
- Offer tips for improvements, or pointers towards useful resources
- Krystle (new)
- Nick (experienced)
- Krystle makes edit to a page Nick watches
- Nick sees edit in Watchlist & sees talkpage is red
- Nick opens talkpage of Krystle in 1 tab & her contribs in another tab
- Nick looks at Krystle's contribs with nav-popups to see what she's done
- Nick uses personalized welcome-template (which apologizes for being generic)
- Krystle writes a note on Nick's talkpage, or on her own talkpage, as a reply.
- New contributor might never get noticed/welcomed
- New contributor might write on their own talk page (in which case new contributor might never get a response)
- New contributor is left in doubt as to where to reply (unless the experienced user specifies a preferred location somewhere on their talkpage, or in an editnotice)
- Everyone gets welcomed somehow
- "Reply to" link
- Collate welcomes via #welcome
- Optionally add (certain) welcomes to a "high-priority watchlist" (or highlight them in dense feeds, somehow)
Workflow 3: Deletion Notice
Here are the participants and workflows for group 3. See photos of this group's sketches.
- Notify & Retain (users and article)
- Amy (new)
- Bruce (experienced)
- Amy creates article (& it's insufficient)
- Bruce proposes-deletion (prod)
- Amy objects and removes template
- Bruce AfDs
- Amy goes to AfD and demands article be kept
- Invitation mechanisms (Can't be too broad, otherwise spammy) (Automated)
- E.g. "Three" (n-most) active editors get notified via "Mention", or auto-subscribed to AfD feed
- or: Wikiproject, Similar categories (maybe?), Personal recommendation, etc
- Article itself subscribes to the AfD feed.
- Subscribers, Invitations, Categories, Broadcasts, Motivation, States
- Automated steps, and Wait-for-human steps
Workflow 4: Thanks
Here are the participants and workflows for group 4. See photos of this group's sketches.
- Acknowledgement / Encouragement
- Fred (new)
- Mary (experienced)
- Fred makes an edit to the best of his ability
- Mary sees it
- Mary goes to Fred's talkpage and leaves a thank you note
- Next time Fred logs on, he gets the message notification, clicks, and sees message.
- Fred does more edits
- Mary clicks Thanks link (but maybe she wants to add a tiny specific message to the Thanks..)
- Mary wants to send a message, not edit a page
- Feed ideas
- Reply Notification (on someone else's talkpages)
- Autosign messages
- "Seen" label when a message has been read
- Welcome auto-post (lets new user know how to get help)
- Thank button below message (Acknowledgement)
- Thank Tool: Option to add a comment to the thanks.
- (Article thanks?) Heatmap for multiple Thanks. Gratitude = meaningfulness. Engage readers.
These additional workflows were discussed as alternates, but are generally not as fleshed out as the first workflows above.
Workflow 5: Warning
Prepared as a secondary workflow by Group 1.
- Unsourced info added to BLP by Saehrimnir
- SPAt sees change on watchlist, views diff, reverts if the info is very negative (and unsourced)
- SPat leaves a msg on Saehrimnir's talkpage, eith personal message or twinkle warning:
- ".... can you include your source?" - Saehrimnir adds source - SPat thanks Saehrimnir
- Twinkle warning - Unsourced BLP level x
- Option for user to respond with "I understand", some kind of acknowledgement that the warning was received, or an option to ask for further explanation. Also, this could be paired with a block/unblock workflow, since unblocks are currently handled through templates. A replacementy for this functionality could walk a blocked editor through basic unblocking thought processes '(a) do you understand why you were blocked? why? (b) how can you do it better next time?' Then any admin could review those answers.
Workflow 6: Deletion Notice 2
Prepared as a secondary workflow by Group 2.
- Let user know the article they started has been tagged for speedy-deletion
- Krystle (new)
- Maryana (experienced)
- Krystle starts article from red link
- Maryana goes to NewPagesFeed & sees article
- Maryana looks at page & decides not-notable. Tags article w/ db-a7
- Krystle gets template on her talkpage notifying her
- Krystle goes to article to see why it is 'speedied'
- Krystle comments on article talkpage (or on her own usertalkpage under the notification, or on a random helpdesk, or on Maryana's talkpage)
- Krystle notices mean edit summary in the wikilink
- Krystle replies within her own edit summary, while removing the template (or replies on Maryana's talkpage)
- Confusion over the multiple places one could start, or reply to, a discussion.
- Problems bringing in other editors who might have input (either for the article-topic, or guidance for the newcomer)
- More integration with Delsort and Wikiproject article-alerts
- Do usertalkpage discussions always need edit summaries? Or could the first line be excerpted instead?
Workflow 7: Why did you delete my article?
Prepared as a secondary workflow by Group 4.
- Retain good-faith new users
- Admin (overworked)
- Nuby discovers article deleted, has link to admin talkpage
- Nuby posts on admin's usertalkpage "why?!"
- Admin replies with "Read the Policy page"
- Options: Argue with admin / Go away and never return
- Only options for admin are:
- Give links to existing policy/guidelines
- Write a personalized note, with some or many details
- Delegate to others, who are more happy to take the time to explain things personally
- Refer to Helpdesk / Teahouse
Workflow 8: Content Collaboration
Here is the sample workflow we prepared to demonstrate how to do a workflow. It was presented by Howie and Maryana, based on initial work by Quiddity and Occassi. See photos.
Help another user improve an article
- New user: Jane
- Experienced editor: Paul
- Jane leaves a message on Paul's talk page to copyedit an article
- Paul responds on his talk page with suggestions for improvement
- Jane responds on Paul's talk page with a question about spelling
- Paul edits the article
- Paul leaves a message on Jane's talk page
- Jane gives a barnstar to Paul on his talk page
Other Workflow Ideas
At the beginning of the roundtables, we made a list of common workflows used by participants on Wikipedia. In coming days, we will list them in plain text below. Feel free to take a stab at transcribing [workflows from this photo].
This roundtable format seems like an effective way to bring together community and team members for a focused discussion on improving Wikipedia's user experience. This collaborative, user-centered design process is particularly helpful for in early stages of long-term projects like Flow or Multimedia.
Our first roundtable experience suggests that well-moderated face-to-face interactions like these can be more productive than virtual ones in generating new insights, because they humanize the interaction and provide a high-bandwith communication channel that makes better feature ideas possible.
Next, we'll distill the findings from this event and turn them into actionable recommendations for the Echo and Flow teams. Once we have collected final notes from participants, we'll update this report -- and write a blog post in early July. We will also host a couple more roundtables at Wikimania this summer, as outlined on the main Roundtables page.
If you are are interested in our work on editor engagement, we invite you to subscribe to our team's mailing list here, if you haven't already.
Thanks again to everyone who made this productive event possible! To be continued ...