Campaigns/Foundation Product Team/Event Discovery/ms
 | This page in a nutshell: The Campaigns team wants to make it easier to discover events on the wikis, and we are requesting feedback on how to do this. |
The event discovery project aims to create or improve tool(s) that make it easier for editors to learn about campaign events on the wikis. We hypothesize that, if more editors can stumble upon and learn about events, then more editors will join events and therefore improve the number of articles on high impact topics. This work is part of the effort in the 2023/2024 annual plan to address content gaps on the wikis.
Project introduction & problem analysis
What are the current problems with event discovery?
In the Wikimedia movement, it can be difficult to learn about campaign events. The current event promotion solutions have at least one of these problems:
- They are not available to all organizers.
- They are not available on all wikis.
- They do not reach many people.
We believe that issues related to event promotion and discovery create the following problems:
- Events receive lower participation rates.
- Events produce less contributions on high impact topics (due to lower participation).
- Participants and organizers may feel less motivated (due to lower participation).
- Editors miss out on opportunities to join impactful events and connect with other editors who care about similar topics, in turn feeling less integrated into the Wikimedia movement.
- There is limited exposure to the breadth of work being done and the types of events being created in the movement as a whole.
What are the current options available for promoting and discovering events on the wikis?
Right now, the main methods of promoting and discovering events are as follows:
CentralNotice: This is a banner that is displayed at the top of the wiki page. Organizers get their events displayed in the CentralNotice banner by issuing a request on the Request page. Additionally, organizers need to add their requests to the CentralNotice calendar. It is the responsibility of the organizer to ensure that there are no clashes with other banner campaigns targeting the same region and/or language. Requests are reviewed and banner campaigns are enabled by a very small number of active CentralNotice admins. The banner design is often done by the CentralNotice admins as well.
- Benefits: CentralNotice is easily discoverable, so it is very effective in bringing editors for some types of campaign events.
- Problems: CentralNotice only promotes some events. A huge number of events never make it onto the calendar or would be hard to support with Central Notice (i.e. events targeting very specific topics or geographies). In order to use CentralNotice, organizers need to understand complex technical and movement dynamics.
MassMessage: If an organizer has the MassMessage right (see MassMessage senders), they can send messages to a bulk of user talk pages about upcoming events. Many users receive automatic emails when they get a talk page message, so they may also be emailed when they receive the talk page message.
- Benefits: This is a mechanism for reaching people both on and off the wikis (if the editors have emails turned on for talk page messages).
- Problems: This only works for people who may already be “in the know,” to a certain degree, because they have signed up for updates from the organizing community on a topic (i.e. a WikiProject, Newsletter, etc). Organizers are required to integrate a complex technical workflow into their communication plan.
SiteNotice: This is a banner that is displayed at the top of the wiki page, and it is controlled by local site admins.
Benefitsː It is a simpler request process than CentralNotice, and it is often easier for non-English speakers to issue requests. Admins are local, which can be useful in some contexts.
Problemsː There is no targeting for different audiences, and it is not widely used in some wikis.
Geonotice: This is a notice on the Special:Watchlist page of English Wikipedia for users in a specific location similar to the global MediaWiki:Watchlist-summary. Organizers will add a request to have a geonotice.
- Benefits: This can be an effective strategy to let active editors (who are more likely to check their watchlists regularly) know about upcoming events, especially in person edit-a-thons and smaller scale events.
- Problems: It’s only on English Wikipedia, and it’s not useful for events that are online.
Homepage promotion: Some wikis promote events on their main page.
- Benefits: It reaches a lot of people.
- Problems: This isn’t a promotional strategy shared by all wikis, and it depends upon people actively and regularly going to a wiki homepage.
Calendars: There are currently many calendars in the movement that presumably cover events, but they vary in how much they cover. Some of these calendars include: Calendars in the project namespace (such as Wikipedia Meetup Calendar for English Wikipedia and Wikipedia:Editatón for Spanish Wikipedia), Events list on Meta-Wiki main page, Events link on Meta-Wiki, Events list on Mediawiki.org, and thematic calendars, like the International Women’s Day Calendar.
- Benefits: Editors can find events that may interest them (but that they don’t know about yet) on event calendars.
- Problems: There is no central or cross-wiki events calendar that is available on the wikis, many events never make it onto the calendars, the process to add events can be confusing, most calendars don’t have any search filters or notification/subscription systems.
What are the options for promoting and discovering events off the wikis?
Emails: If someone is on the email or mailing list of an organizer or affiliate, they may receive an email about an upcoming event.
- Benefits: They reach a targeted group of people who are more likely to be interested in the campaign.
- Problems: It’s easy to ignore or forget the emails, and they only reach the people who have already opted into communication. Organizers are required to understand a complex network of subscription lists and audiences to use successfully.
Chat groups: There are various chat groups on Whatsapp, Signal, Facebook and other off-wiki platforms that bring together Wikimedians. These chat groups can also be a place to promote events:
- Benefits: They reach a group of targeted people who have a high likelihood of being interested in the event.
- Problems: It doesn’t capture people who aren’t already in the group, and it’s easy to read the message and then completely forget about the event.
Social media promotion: Organizers may promote on external social media platforms.
- Benefits: This can help bring on newcomers who aren’t already present on the wikis, or it can help nudge who may be interested in an event but haven’t made the leap to sign.
- Problems: This method may not be very effective. Our data shows that social media is a minor recruitment engine for looking at event pages. Social media communication by affiliates and other programs tends to be self-reinforcing, only communicating among existing Wikimedia groups and not reaching new audiences.
Off-wiki organizer calendars (such as Diff Calendar, Art+Feminism calendar):
- Benefits: They can look great and may have a paid professional who is actively maintaining them, so they may be in pretty good shape overall
- Problems: They are off the wikis, so they tend to not capture a large range of activities. By being off-wiki and outside the larger movement ecosystem, visibility of the work gets lost to other movement contributors, especially among people who only contribute on the wikis.
What’s next?
As a next step, we want to hear from you! We want to know how people are currently finding (or not finding!) events on the wikis, and we want to know how we can make it easier for people to find events that interest them. Once we receive more feedback from people, we will summarize our findings and share our proposed next steps. In the meantime, we invite you all to answer the questions below, which will be critical in helping us understand what we can do to fix the problem.
Selepas menjawab soalan-soalan kami (lihat di bawah), anda dijemput untuk hadir ke pejabat kami semasa waktu pejabat pada 7 Oktober dan/atau 10 Oktober bagi bebincang akan projek ini bersama pasukan kami
Open questions
- How do you learn about Wikimedia campaign events? For example: banners, email lists, talk pages, social media, etc.
- Do you think it’s easy to find all of the campaign events that interest you? Why or why not?
- Ideally, how would you like to hear about campaign events?
- How do you decide which events to attend?
- Have you ever missed an event because you didn’t hear about it in time?
- How do you remember to attend the events that you registered for?
- Is there anything else you would like to add?
Status updates
April 16, 2024
In the last few months, we conducted an experiment to learn more about Event Invitations. We have now wrapped up the experiment, so we would like to now share our findings and next steps.
First, what was the experiment?
We created a model to generate an Invitation List (see T353459 and score calculation documentation). The Invitation List had two main parts: a list of editors to potentially invite, and a score assigned to each editor (i.e., high, medium, or low recommendation on whether to invite them to an event). The editors were identified because they had contributed to specific Wikipedia articles in the last three years. The score was determined based on the number of bytes they had contributed to the article(s), the number of edits they made to the article(s), their overall edit count on the wikis, and how recently they had edited on the wikis.
We decided to conduct a light-weight experiment to determine the usefulness of Event Invitations. This way, we could learn if Event Invitations were effective in bringing new audiences to events. We also wanted to learn what people liked, what they didn’t like, and what could be improved about the tool. With this information, we could make an informed decision about next steps, including whether or not to continue with the project.
Here’s how the experiment worked: We reached out to event organizers who were conducting events between December 2023 and March 2024. We collected the worklists of their events. From these worklists, we generated Invitation Lists (see T357007), which we shared with the organizers. The Invitation Lists did not display the scores, but we indicated if editors received higher scores or lower scores. Then the organizers made their own judgment calls about who to invite. The invitations were sent via talk page messages or wikimail tools such as massenmail. Then, the organizers shared data with us on who they invited, and we compared their finalized Invitation Lists with their registration lists.
To provide support, we held a 4-session office hour training. The first office hour introduced the project, the second office hour provided training on generating and sharing worklists (see some of the worklists shared), the third office hour provided training in sending out invitations via wikimail, and the final office hour focused on collecting user feedback on the tool.
What were the results & learnings?
First, we learned that organizers were interested in testing out the tool. We conducted a survey of organizers in the early stages of the project, and when we asked if they would be interested in testing out Event Invitations, we received the following results:
- 32 respondents said “Yes”
- 1 respondent said “No”
- 7 respondents provided no answer
Then, we launched the experiment, and had the following rates of participation:
- We worked with 19 different events to generate 20 Invitation Lists.
- One event focused on 2 wikis, so it received 2 lists.
- Note that 1 of the Invitation Lists (in December 2023) used an older model, and the remaining 19 (between January and March 2024) used an improved new model.
- 12 processes were completed, meaning the invitations were sent and we could compare invitation data to their registration list.
- 7 organizers did not complete the process (i.e., no invitations were sent or not all necessary data was shared with the team). This did not surprise us, since sending invitations and reporting on the invitation data is currently a labor-intensive process.
We were also able to learn what we wanted to learn through the experiment, which was: Do some editors respond to Event Invitations and join events—and, if yes, at what rates? Here is what we found (see T357827):
- 338 editors total were invited to 12 different events
- Of the invited editors, 42 of the invited editors registered for the events
- While this breaks down to 12.43% of the invited editors joining the events, there is large variation in the results from each event. Here is a breakdown of the performance of these 12 events:
| Event | Total number of editors invited | Total number of invitees who registered |
| Event 1 | 18 | 14 |
| Event 2 | 14 | 2 |
| Event 3 | 8 | 1 |
| Event 4 | 10 | 1 |
| Event 5 | 40 | 5 |
| Event 6 | 49 | 1 |
| Event 7 | 58 | 1 |
| Event 8 | 29 | 2 |
| Event 9 | 10 | 1 |
| Event 10 | 47 | 11 |
| Event 11 | 22 | 0 |
| Event 12 | 22 | 3 |
As displayed in the table above, some events had higher success rates and some had lower success rates with Event Invitations.
- The events with the highest success rates tended to have event pages with images, clear objectives, and clear timelines of the event.
- The events with lower success rates tended to have basic event pages, without images and/or clear objectives or timelines of the event.
On the qualitative side, when we talked to organizers who used Event Invitations, we generally heard positive feedback and interest in using the tool again for future events. Organizers told us that the tool provided an automated way of doing work that they previously did manually or did not have the time or resources to do at all. However, there were also requests for improving the experience, including:
- More information on the editors in the Invitation List
- Such as: links to their user page, their edit count
- We also heard requests to share more sensitive information on the editors, including their gender and location
- An easier way to send mass emails to invite editors
- A way for editors to opt out of receiving invitations
We also learned about some general user behaviors when using Event Invitations, such as:
- Organizers did not invite everyone from the Invitation Lists. They made their own judgment calls about who to invite.
- There was no one invitation method chosen by all organizers. Some organizers emailed invitations, some organizers used talk page messages, and some used a combination of methods.
- Some people in the Invitation Lists were already in the community contact lists of the organizers or were the organizers themselves. Some organizers also received their Invitation Lists after the event had started, so they found that some people on the Invitation Lists had already joined their events. For these reasons, the organizers did not invite these editors. This wasn’t necessarily a bad thing, since it indicated that the tool was identifying editors with interest in the event.
What are next steps?
We believe that Event Invitations could be a useful tool for event organizers who want to identify audiences to invite to their events, provided those events focus on Wikipedia articles. This is for two primary reasons:
- The quantitative data showed promise: The 12.43% average rate demonstrates that the tool can help identify some editors who are interested in attending editing events. Furthermore, we plan to incrementally improve the scoring model over time (see T358526 for early ideas), which could make registration rates in a similar test or experiment potentially higher in the future.
- The qualitative feedback generally showed interest in the tool among organizers, who said that they would like to test out the tool and/or use the tool again in the future
However, there is some potential sensitivity around the usage of the tool, since it allows organizers to generate lists of editors who edit certain articles (which could be articles on controversial or sensitive topics). For these reasons, we think it makes sense for Wikipedia communities to decide whether or not they want Event Invitations, if they have the CampaignEvents extension enabled.
Furthermore, we think it makes sense for usage of Event Invitations to be restricted to only those who are trusted to use it, which can be users with the Event Organizer right (which is controlled by local wiki admins). For example, here is the documentation of how the right is handled on Meta-Wiki. This way, we can empower organizers to have new ways of reaching audiences for their events, while also giving local admins the power to assess if and how such tooling is appropriate for their wiki communities.
Note that the CampaignEvents extension is currently only available on Meta-Wiki, but we are planning to begin releasing it on some Wikipedia wikis soon. Once it is enabled on Wikipedia wikis, we can engage with those communities to see if they are interested in enabling Event Invitations, when it is ready for general usage.
As next steps, we will develop some designs of how a simple version of Event Invitations can look. You can follow this work in T361029. We will share these designs on this project page and ask for feedback when they are ready. After we have collected feedback on the designs, we will start building the first early version of Event Invitations, in collaboration with the communities and organizers who are interested in the tool.
So, please do let us know what you think of our experiment, findings, and next steps. Do you think Event Invitations could be a useful tool for organizers? How can we potentially improve it? Please share your feedback on our talk page!
November 27, 2023ː Event Invitations Experiment
Hello, everyone! We are excited to share an update on the project. First, we have decided on the first focus area: Event Invitations.
With Event Invitations, we want to help event organizers reach out to new audiences of editors who may be interested in their events. This way, we hypothesize that more people can register for events and make impactful contributions. To do this, we plan to generate an Invitation List, which is a list of editors who the organizer can invite. The Invitation List may be based on some of the following criteria:
- Interest in the topical area: We can select editors who have made a significant contribution (for example, a certain minimum number of bytes) within a certain period of time (for example, within the last 2 years) on at least one of the articles being worked on during the event. Alternatively, we could also select editors who are watching a page indefinitely that is on the organizer's worklist.
- Recent editing activity: From this group, we can select editors who have made at least one unreverted edit on the wikis in the last 90 days, as an example.
- Productive history of editing: From this group, we can select editors who have made a minimum number of contributions (for example, 500 unreverted edits) on the project of the event, as an example.
With this list, the organizer can then choose to invite the editors to the event. The method of invitation has not yet been determined, but we may have organizers use communication methods already available to them as a first experiment (such as talk page messages or wikimail). The hope is that since this group of invited editors are productive, active, and potentially interested in the specific topics of the event, some of the invited editors may be interested in joining the event.
To assess whether Event Invitations are useful, we will be asking questions like:
- What percentage of invited editors register for the events?
- What percentage of invited editors make any contributions during the event?
- What percentage of invited editors make substantial contributions during the event?
- How do organizers feel about the Invitation List? Do they find it useful? Why or why not? Would they use the Event Invitation list again to help promote their events?
With this information, we will then determine what next steps we will take. If we see fruitful results, we may continue to explore Event Invitations. For example, we may investigate how we can improve the communication infrastructure available to organizers, so it is easier for them to message invited editors directly. We can also potentially look into how we can build a generalized tool for organizers, which could have ways for participants to opt out of event invitations (either in general or from a specific organizer). Alternatively, if we do not see fruitful results, we may choose to pursue a different Event Discovery project entirely.
The engineering team has begun to investigate how we can generate an Invitation List. You can check out our Event-Discovery board on Phabricator to see some of the work that we will be doing now and in the future.
In the meantime, if you would like to join us in the experiment as an organizer, please reach out to us! We want to connect with organizers who will be developing campaign events in 2024 who are interested in inviting new audiences to their events. If that is you, please reach out on our Talk page or in our chat group for organizers.
Finally, if you have any feedback on the Event Invitations idea, please share it on our talk page!