Grants:Project/Rapid/UG ELiSo/African month 2021/Report

Report under review

This report has been submitted by the grantee, and is currently being reviewed by WMF staff. Please create a discussion page for this grant report by following this redlink to add comments, responses, or questions, or by using the button below.

• To read the approved grant submission for this project, please visit Grants:Project/Rapid/UG ELiSo/African month 2021.

• Review the reporting requirements to better understand the reporting process.
• Note that if a grantee is unresponsive or uncooperative for 21 days or more, this report will be moved by WMF to incomplete.
• With questions about commenting on a report, or with questions about submitting a report, please Email rapidgrants at wikimedia dot org.
• Review all Rapid Grant reports under review.

Goals

Did you meet your goals? Are you happy with how the project went?

We have mostly not met our goals.

We are not fully, but reasonably happy about how the project went. The process of organising it was mostly smooth. We have also learned a lot about reaching out to the underrepresented community of Africans and engaging them. We have learned about both known and unknown unknowns. Currently we have more awareness about these unknowns, for some even kind of solution or at least several possibilities; solutions for others are still to be found. We have created and improved contacts, relations and infrastructure for future passive and active engagement, both in competitive and collaborative ways.

Outcome

Please report on your original project targets. Please be sure to review and provide metrics required for Rapid Grants.

Target outcome	Achieved outcome	Explanation
Number of participants: 15	13	12 direct participants of the competition, +1 additional participant of the project but not the competition. 12 participants is 44% Year-over-Year increase. We have aimed for 66% Year-over-Year increase, what we haven't reached. As this year we have put much more effort for engaging new Wikimedians from underrepresented community, less effort was put for engagement of long-term Wikimedians, who are usually the pillar of competition participation - in number of participants, in number of articles and in article quality. Some role may be played by the fact that for 2021 we have scheduled several competitions and campaigns, so long-term editors may decide to be more picky about them.
Number of articles created or improved: 100	80	80 articles is 41% Year-over-Year decrease. It is warning that despite the increase in the number of participants, the total number and total size of articles actually decreased. Similar as for number of participants, role may be played by our less reach for existing long-term editors (because of our much bigger reach to underrepresented community), and number of competitions scheduled for this year.

Target outcome	Achieved outcome	Explanation
Number of participants from Africa: 5	5	5 new African Wikimedians started to participate in Wikimedia wikis because of this project. From them, 4 directly participated in the competition.
Number of participants from Africa active (10+ edits in sum in all projects) 1 month after the competition: 2	0	From 5 new African Wikimedians, 2 have not continued editing after this specific competition, 2 have done several edits and 1 nearly reached the threshold set in this measurement of success. This one is still active and e.g. participate in the campaign WPWP 2021.
Number of new likes of the Facebook page of Esperanto Wikipedia: 300	71 (54 by paid ads + 17 by invitations)	As presented below, we have started the page likes campaign later than planned, likes acquisition was slower than expected and cost per like higher than expected. Likes invitations and acceptance were less effective than expected; probably we have overestimated the internet sociability of Africans, at least when it comes to likes.
Number of email addresses of Africans collected: 50	8	For collecting email addresses we have used 2 opt-in pages on our website. The first was quite long, presenting the contest and how to participate. The opt-in form was way below and supposedly many visitors haven't seen it. To counteract this we have created a much simpler opt-in landing page only for ads visitors. It had only opt-in form and explanatory video. Unfortunately, as we were notified later, many Africans have internet bandwidth usable only for text, and maybe pictures. As the instructions on what to do was explained in a video, supposedly many Africans couldn't follow them. Also there was a problem about targeting in Facebook ads. Despite the fact that we have targeted Esperanto speakers in Africa, many of the ads angagees were non-Esperanto speakers from outside of Africa (for unknown reason).

Three shared metrics:

1. Total participants: 13
2. Number of newly registered users: 5
3. Number of content pages created or improved, across all Wikimedia projects: 80

Learning

Projects do not always go according to plan. Sharing what you learned can help you and others plan similar projects in the future. Help the movement learn from your experience by answering the following questions:

• What worked well?
  • The competition was successful in general. It have provided new and improved articles in the Esperanto Wikipedia and some in Esperanto Wikivoyage (which is very new project, so we focus more attention to it in this year). Quality of contributions was mostly high.
  • Collaboration with the partners and (most of) prize providers was excellent! We were honored by auspices from the African Committee of Universal Esperanto Association and the African Committee of World Esperanto Youth Organization. These organisations are the highest representatives of Esperanto worldwide and have high status for Africans. Both Universal Esperanto Association and Flemish Esperanto League (as some of prize providers) worked excellently fast and we specially thanks them!
  • Both Universal Esperanto Association and World Esperanto Youth Organization helped us with the Facebook campaign. These organisations have one of the biggest Facebook followship in the Esperanto community. (however, there was some problems about Facebook campaigns, see below)
  • The project team collaboration was very good, tasks were divided and fulfilled.
• What did not work so well?
  • The project management was unplannedly carried out in heads, instead of some project management tool. We wanted to use a newly installed tool, but it was provided by our technician only after the project. So it was harder to have an overview of the project's tasks and deadlines.
  • Also tool for tracking expenses was not provided. So also tracking of expenses was done manually rather than automatically.
  • Using opt-in pages, especially with video, for capturing email addresses of Africans have had rather small impact. Lately we have switched our ads to the Wikipedia competition page instead, where we couldn't measure impact and so could't optimize the ads.
  • Targeting Facebook audience was very problematic. Despite our clearly set targeting for Africa, also people from other continents were included. This has reduced effectiveness of using the ads for finding potential Wikimedians in Africa.
  • Cost per Facebook page like was way higher than expected. We had expected it to be higher than the industry standard (because of a small sample of highly diverse people in the target group), but it was even higher. That was despite using the most recommended way - targeting people who already like Facebook pages (of partners).
  • Our campaign for new Facebook likes started later than planned because of other wiki and non-wiki duties. Consequently we haven't "warmed" our target audience enough for next steps (sending them to the opt-in / competition page).
  • Speed of gaining new Facebook likes was significantly slower than expected. Based on previous experience we thought the project time would be dozens-times enough. But at the end we have still unspent funds from the budgeted amount. (in addition, some unused funds from the "prizes" budget line was moved (with Grant team acceptance) to the outreach budget line, which is also unused)
  • In the last time Facebook have changed the way how A/B tests on ads works. So for some time (until we have learned the new way) we were using ads without A/B testing. (A/B testing enable to test several versions of ads which differ in some specific parameter, and finally elect the one with best performance, thus making use of funds most effective possible)
  • The way for engaging prospective new African Wikimedians was consisting from bit more than ideal number of steps with conflicting messages. That sourced basically from 2 things. 1) The conflict between more attention brought to Esperanto Wikivoyage (because it is a new project, so we support it more this year) and the fact that Wikipedia is the only one Wikimedia project which have at least a chance to be known by our target audience. 2) Participants should register their contributions on wiki, so we needed to send them there. But on wiki, we can not collect email addresses, which we want for long-term contact, engagement and relationships. So we tried to connect these 2 websites.
  • We have got a new team member who get trained on specific task (usable for many projects to come), but finally the task was fulfilled by someone else.
  • The concept of gift cards is quite unknown in some parts of Earth. Some participants thought to receive money and we were frequently requested to provide financial equivalent of the prizes instead.
  • It showed up too lately that many Africans have internet with very limited bandwidth (for texts and some images). It made our opt-in page less attractive, because the main message was delivered using video.
• What would you do differently next time?
  • Ideally, the project team should consist of more people who are super skilled about their narrow domain and can greatly collaborate on the shared goal working on their rather small tasks. E.g. in this project we would highly benefit from graphic designer, copywriter, Facebook ads manager, CiviCRM campaign manager, and online marketer with experience with creating A/B testable opt-in landing pages in WordPress. (Talent development and acquisition is already in our Annual plan 2021 and we are working on it.)
  • We would manage the project using some specialized PM software. (We already have such software installed and are using it for projects (including this one). It just wasn't in time) It is unclear what would be the best path in our situation, when provision was scheduled for goodenough time but it wasn't delivered. We have considered using another PM software for this specific project after initial delays but finally decided to not do so.
  • We would experiment more with A/B testable opt-in landing pages on our website. They are considered a great tool to catch email addresses of the target audience (in our case prospective African Wikimedians), and email is considered a great tool to cheaply stay in long-term contact and relationship (what is a part of the long-term goal of this project). But in this case it haven't worked. As of now, it is not known whether it was because of the specific setting of the opt-in and/or landing page(s), the Facebook campaign, mismatch with needs of the target audience, too small target audience, because of a different mentality of the target audience or other reason. The promise of this tool is still high, so we consider it to be worth of more testing.
  • We would consider how to provide more one-on-one support for the prospective new African Wikimedians. It showed up that the most high quality contributions came from new African Wikimedians with high one-on-one support. However, such support scales badly, so it needs more consideration and planning in advance.
  • We would collect learnings in the central place during the whole project.
  • We would improve our handbooks about organising competitions, so onboarding new project members would be easier. (work already have started and on the way)
  • Ideally we would trac project expenses in a tool which provides more control of spending, rather than controlling it manually.
  • We would try to make it much easier to officially join the competition. Some participants have at the beginning problems even with putting their username on the participants list.
  • We would more consider how to make the first steps for new Wikimedians easier. That is not an easy one. Many help pages on Esperanto wiki projects are outdated, and updating them would be serious undertaking, preferably done by a bigger part of the community, rather than couple of project members. Using video tutorials would be impractical for people with limited internet bandwidth.

Finances

Grant funds spent

Please describe how much grant money you spent for approved expenses, and tell us what you spent it on.

Total: 564,92 EUR

• outreach: 158,04 EUR
  • 48.94 EUR for Facebook page likes
  • 109.10 EUR for directing visitors to contest page
• prizes: 406,88 EUR
  • African newcomers: 226,89 EUR
  • regular contributors - writing: 89,99 EUR
  • regular contributors - translating: 90 EUR

Remaining funds

Do you have any remaining grant funds?

Remaining funds are retained by the grantee with WMF's permission until after the report review has been completed and report has been accepted, at which time the WMF program officer will also provide instructions regarding the grantee's reallocation or return of the unspent grant funds.

Yes, we have remaining 100,08 EUR.

We would like to use the money to continue outreach for prospective new African Wikimedians. That would help us to find / invite and engage new African Wikimedians for other competitions, including the next edition of African month Esperanto, and hopefully for regular non-competitive contributions.

As there was high costs per Facebook page like and less than estimated opt-ins with email address, we would like to experiment a bit more about the ads' settings to find more effective ones. We consider e.g.:

• setting the upper limit per Facebook page like to low amount and adjust to the situation;
• using Facebook campaign for receiving email addresses;
• creating a membership form specifically for this campaign;
• other options are possible as well.

Anything else

Anything else you want to share about your project?