Research:Engage native speakers to improve vital content in small language Wikipedias

Engage native speakers to improve vital content in small language wikipedias experiment
Engage native speakers to improve vital content in small language wikipedias experiment
	A test project to experiment content building and contribution pathways in small language wikipedias.
Group:	Wikimedia Language and Product Localization
Start:	2025-07
End:	2026-04
Team members:	Mary Munyoki; Satdeep Gill; Uzoma Ozurumba; Connie Chen; Huei Tan; Kirsten Stoller; Alex Stinson;
Lead:	Srishti Sethi (Hypothesis owner)

Translate this page; This page contains changes which are not marked for translation.

Tracked in Phabricator:
Task T399011

Background

The Language and Product Localization team, under the WE2 Vital Knowledge objective aims to experiment and evaluate interventions that help contributors improve the state of vital content on their Wikipedias. Among other interventions, we plan to implement and test a socio-technical intervention to support content creation activity for small language communities.

Rationale for this experiment

Learnings from the Language Onboarding experiment highlighted that the size of smaller language communities, often around 5-10 active editors might be quite minimal to have a significant effect on content creation taking into account communities burn out, volunteer time among other factors. We aimed to build a proof of concept on how content building and contribution pathways might look like for smaller wikis. The team identified this work as a pathway to help native speakers find smaller Wikipedias and tools for content creation. The goal is to reach these speakers where they are, starting from the high-traffic wikis as entry points. Followed by taking them through a guided approach with a call to action for micro-contributions for a start. Then we can later explore more ambitious tasks such as expansion of articles as a follow up hypothesis.

Hypothesis July 2025-Sep 2025

"If we invite native speakers of small wikis through a CentralNotice banner on a high-traffic Wikipedia in their region to contribute to SuggestedEdits and other Growth features, we can assess whether this approach attracts new native speakers and if they use these editing tools to improve vital content".

Summary

In Q1 of the fiscal year of the Wikimedia Foundation, we deployed CentralNotice banners on high-traffic Wikipedias in targeted regions to increase awareness about smaller Wikipedias. The goal was to observe the outcome on expanding our reach of native speakers where they are, invite them to start their journey using growth features, and ultimately improve vital content through suggested edits, such as adding an image, adding a link, and improving a section of an article. The banner included information about:

A small wiki to raise awareness and attract potential native speakers.
Newly enabled GrowthExperiments for easier feature discoverability.

This intervention complemented the efforts of GrowthExperiments to bring new contributors on board and encourage them to try out the feature. The features also included encouraging sharing resources, increasing accessibility of community events and allowing native speakers to see their direct impact.

Selected Wikipedias

Our selection was based on various criteria, using metrics from the Wiki Comparison Data 2025. It also considered:

the average number of monthly active editors and administrators in the past year
The availability of maintenance template in the Wikipedias
Having the Growth features enabled.

Through segmentation in three clusters of wikipedia of similar size and activity level. We randomly selected two Wikipedias from each cluster, resulting to the following Wikipedias:

Asturian (astwiki)
Luxembourgish (lbwiki)
Minangkabau (minwiki)
Moroccan Arabic (arywiki)
Punjabi (pawiki)
Zulu (zuwiki)

We determined the high traffic Wikipedias that native speakers from regions where the above languages are spoken visit.

English Wikipedia was the high traffic Wikipedia for speakers of Luxembourgish, Minangkabau, Moroccan Arabic, Punjabi and Zulu.
Spanish Wikipedia for the Asturian speakers.

Measurement plan

Our long-term primary metric is to analyse and compare values before and after the experiment across all six Wikipedias over a 3-month period. These results will help us understand whether the deployment of the CentralNotice Banners in this experiment:

Successfully increased awareness of small-language wikis.
Encouraged more page views and edits.
Increased engagement with newcomer tools.

Leading indicators tracked the following metrics and monitored the experiment’s progress before the experiment evaluation. The indicators helped determine whether intervention was needed or if a follow-up iteration of the experiment should be planned.

Daily impressions on banners
Click-through rate of the banners

The Experiment

The experiment ran from September 4th to September 18th. We used two "Call to Action" scenarios for the CentralNotice banner and randomly assigned each wiki to one.

Scenario A: Call to Action – Event registration

This scenario presents users with a call to action that directs them to a dedicated event registration page curated with contributors from the wikis.

This page provides information and resources on creating an account, logging in, and editing using the newcomer tools. The Minangkabau (minwiki), Moroccan Arabic (arywiki) and Punjabi (pawiki) were selected for this scenario.

Scenario B: Call to Action – Newcomer Homepage

When registered users clicked a call to action, they were shown either the newcomer homepage or a message prompting them to enable it in their user preferences. Non-registered users are directed to create an account. The Asturian (ast), Luxembourgish (lb), and Zulu (zu) were selected for this scenario.

The CentralNotice banner

It was essential to make the CentralNotice banner design and messaging appealing and targeted so they would be noticed by readers in the targeted regions. We restricted our messaging to a brief sentence letting them know about the wiki from their region, and inviting them to get started. We had the messages translated by community members in targeted regions, received approval to run CentralNotice, and informed the eight Wikipedia communities of our plans to run CentralNotice.

CentralNotice was launched across 8 Wikipedias on the 4th of September for 2 weeks.

Below is a sample banner displayed on the English Wikipedia for regions where Punjabi is spoken.

Experiment interim result

This part of the analysis was run immediately after the experiment. It directly evaluates and compares the two scenarios from the experiment: Scenario A and Scenario B based on the leading indicators and the primary measurement plan.

Impressions and clicks

Scenario A had 17,257,300 impressions and 8,150 clicks.
Scenario B had 1,819,086 impressions and 2,616 clicks.

Impressions and clicks
Wiki	clicks	Impressions	Scenario
arywiki	1,132	784,700	A
minwiki	3,059	7,552,600	A
pawiki	3,959	8,921,000	A
astwiki	694	369,200	B
lbwiki	1,689	1,381,086	B
zuwiki	233	68,800	B

Account Creations
Scenario A had a total of 164 account creations. Scenario B had 69 account creations.

Account Creations
Wiki	New Accounts	Scenario
arywiki	62	A
minwiki	19	A
pawiki	83	A
astwiki	39	B
lbwiki	21	B
zuwiki	9	B

Growth Features Engagement
Scenario A had 81 users engaged with the GrowthFeature homepage. Scenario B had 47 users engaged with the GrowthFeatures homepage.

Growth Features Engagement
Wiki	interactions	Scenario
arywiki	33	A
minwiki	11	A
pawiki	37	A
astwiki	29	B
lbwiki	13	B
zuwiki	5	B

Growth Features Engagement Rate Comparison

Overall, Scenario A had an engagement rate of 0.99%, and Scenario B had an engagement rate of 1.80%. And based on the test, the difference is statistically significant. Scenario B has a higher growth feature homepage engagement.
High activity language & size wiki (pawiki & lbwiki): pawiki had an engagement rate of 0.93%, and lbwiki had an engagement rate of 0.77%. According to the test results, the difference is not statistically significant. We cannot conclude which Scenario has higher homepage engagement than the other for this group.
Medium activity language & size wiki (minwiki & astwiki): minwiki had an engagement rate of 0.36%, and astwiki had an engagement rate of 4.18%. According to the test results, the difference is statistically significant. Scenario B has a higher growth feature homepage engagement than Scenario A for this group.
Low activity language & size wiki (arywiki & zuwiki): arywiki had an engagement rate of 2.92%, and zuwiki had an engagement rate of 2.15%. According to the test results, the difference is not statistically significant. We cannot conclude which Scenario has higher homepage engagement than the other for this group.

In conclusion, when comparing engagement rate, Scenario B is the more effective. Its overall performance was statistically significantly better than Scenario A. However, this effect was not consistent across all wiki segments; in medium-activity wikis, Scenario B was more effective. For both the High-Activity and Low-Activity wikis, there was no significant difference in performance between the two scenarios. It's possible that a real effect exists in high- and low-activity groups, but the test may not be powerful enough to detect it.

Edits by Newly Registered Users

During the experiment period, we found the following number of edits from newly registered editors:

Number of edits from newly registered editors
Wiki	Edit Count	Scenario
arywiki	23	A
minwiki	2	A
pawiki	24	A
lbwiki	6	B
astwiki	0	B
zuluwiki	0	B

Experiment final result

As part 2 of the evaluation, we collect the following data 3 months after the experiment:

Average monthly pageviews on content pages

Project	Monthly Pageviews	% Change vs. Pre-Experiment	% Change YoY
arywiki	201,803	13.41%	58.67%
astwiki	4,748,661	350.14%	834.06%
lbwiki	2,272,710	350.82%	924.66%
minwiki	1,165,182	114.69%	790.95%
pawiki	853,881	-11.81%	31.54%
zuwiki	295,205	54.81%	244.19%

The data reflects user-driven traffic specifically to content pages across our six target wikis. Compared with the average monthly pageviews from the pre-experiment period, we see an increase in most of the small wikis except pawiki. To account for seasonality, we compared current traffic to the same period in the previous year. Under this lens, all six wikis—including pawiki—showed a positive increase in pageviews.

Average monthly number of editors on content pages

Project	Monthly Editors	% Change vs. Pre-Experiment	Template:TuniT
arywiki	46	6.98%	12.20%
astwiki	99	3.13%	5.32%
lbwiki	63	0.00%	6.78%
minwiki	66	34.69%	6.45%
pawiki	100	1.01%	4.17%
zuwiki	25	-19.35%	19.05%

This data tracks average monthly unique, non-bot editors contributing to content pages. Most wikis saw a steady increase in editor activity compared to the pre-experiment period, while zuwiki experienced a 19.35% dip in editors relative to the pre-experiment period. Compared with the previous year, all six wikis achieved positive YoY growth.

GrowthFeatures usage by new users from the experiment

There are minimal number of new users from the experiment who return to use the Growth features in the three months after the experiment.

Total number of content edits in 3 months

Project	Total Edits	% Change vs. Pre-Experiment	Template:TuniT	Edits by new users from the experiment
arywiki	3,297	29.80%	76.88%	13
astwiki	8,805	-45.87%	10.59%	0
lbwiki	10,857	28.47%	6.16%	8
minwiki	1,910	107.16%	-35.03%	3
pawiki	4,815	11.74%	33.56%	19
zuwiki	874	19.07%	64.60%	1

When looking at total content edits in three months, although there are increases compared with both the pre-experiment period and the previous year, the number of edits by new users from the experiment remains negligible.

Total number of bytes added in 3 months

Project	Total Bytes Added	% Change vs. Pre-Experiment	Template:TuniT	Bytes Added by new users from the experiment
arywiki	3.90M	42.05%	242.48%	503
astwiki	2.43M	9.14%	169.65%	0
lbwiki	2.06M	10.28%	10.57%	636
minwiki	1.29M	173.24%	-42.16%	152
pawiki	5.11M	95.70%	-31.54%	6,220
zuwiki	0.15M	-62.82%	-22.86%	49

Content growth surged across most wikis relative to the pre-experiment period; however, year-over-year performance was split, with three wikis failing to match their previous year's volume. The contribution from the experiment users is still minimal across wikis.

Summary

In summary, data across pageviews, edits, and editors indicate a general trend of increasing engagement and contribution across most of the targeted small wikis compared to the pre-experiment period and the previous year. While the direct contribution from the experimental cohort that we can measure remains statistically low, the broad growth observed across these projects suggests the initiative may have positively influenced general community awareness.

Lessons learnt

This experiment has shown that through targeted regional and language outreach, it is possible to draw native speakers to smaller wikis.

In future iterations, we could focus on exploring:

How to address the gap between initial awareness and actual engagement activity and engage contributors in improving vital content.
How to provide tailored, guided onboarding to identify scalable approaches for language onboarding.

Target audiences:

Users who created accounts in the previous experiment.
Small wikis more broadly, learning from them where they are in their journey and which tools they need next (e.g., article expansion, easier edits, quality control, patrolling) to improve vital content on their wikis.

More longer-term projects could include:

Weaving key features into Wikimedia products to increase content awareness and discovery for small language wikis.
Emphasizing staged onboarding or milestone-based guidance;
- Start with core usability tools
- Progress to article expansion
- Eventually introduce quality monitoring tools
Integrating this guidance as part of the onboarding experience for new wikis beginning their journey in Wikimedia.

Hypothesis WE2.1.5

"If we surface content gaps contribution opportunities to native speakers and engage them in expanding articles using contribution tools (e.g content/section translation), a higher proportion of these contributors on smaller wikis will contribute to vital articles."

This hypothesis is based on the tailored onboarding for native speakers in small language wikis Idea and the work in progress.

Q2 (Oct 2025-Dec 2025)

In Q2 of the 2025-26 fiscal year, the team explored ways to identify content gaps and support native speakers in addressing them through tailored onboarding. As a result, we developed a three-week onboarding plan with weekly modules that provide clear instructions and short videos. These modules help new contributors to targeted Wikipedias learn the basics, expand sections of vital articles, and create missing articles using translation tools. This phase focused on designing and developing a method to highlight content gaps for new contributors.

Q3 (January-March 2026)

With the design and development phase complete, we will now execute and evaluate the hypothesis by surfacing content gaps to native speakers and engaging them through tailored, guided onboarding. Using a prioritized list of vital articles, and calls to action for various contribution types, we aim to determine whether this approach increases contributions to vital articles on small wikis. Below is an image showing the design.

Our approach

Create a section on the selected language Wikipedia home page that includes the following:
- A call to action inviting new contributors to register for a three-week tailored onboarding event.
- Onboarding materials with a step-by-step guide and weekly tasks, including:
- a list of vital articles to work on.
- Types of tasks to improve each article
Invite qualified members with the appropriate user rights from the selected Wikipedia to collaborate with us to:
- Set up the home page section using the script and instructions provided in the onboarding guide.
- Prepare for the onboarding event by completing the following:
  - Translating the onboarding materials
  - Creating the event registration page
  - Providing a prioritized list of articles for new contributors
  - Promoting the event or campaign to attract new contributors
Run the three-week onboarding program
Evaluate the onboarding program and gather feedback from Wikipedia community members

Selected Wikipedias

We have identified the following Wikipedias we will be collaborating with.

Nepali Wikipedia
Tongan Wikipedia

Experiment Final Result

These results are from the Nepali Wikipedia Vital Knowledge Drive campaign. For the Tongan community, there were no registrations during this campaign, so analysis is not applicable.

Campaign Metrics

Category	Metric	Baseline (3 weeks before campaign)	Campaign	Change
Awareness	Onboarding page views	88	1291	14.7×
Awareness	Main page (homepage) views	16900	20126	+19.1%
Awareness	Pageviews on 5 homepage-featured pages	147	576	3.9×
Awareness	Pageviews on full 241-title promoted list	6340	7304	+15.2%
Editing	Edits to promoted articles by existing editors (2 of 5 active; 4 of 241 pages touched)	0	16	new
Editing	Edits to promoted articles by new users (12 newcomers; 0 edited promoted content)	0	0	-
Editing	Edits via Section/Content Translation tool (SX), by registrants	0	0	-
Retention	Registrants still editing post-campaign (of 7 active during campaign)	-	3	43%

Summary

1. Visibility & traffic impact

Onboarding page views increased by 14.7×
The 5 homepage-featured pages saw nearly 4× more traffic
Main page views increased by ~19%
Page views on promoted articles increased by ~15.2%

2. Participation & editing behavior

Out of 17 registrants, only 2 experienced editors edited promoted articles
These edits covered 4 of 241 promoted pages
Among 12 newcomers, none edited promoted content; all 28 edits were on non-promoted pages
No edits were made using Section Translation or Content Translation tools

3. Content growth

+42,729 bytes added across 16 edits to 4 articles
Two homepage-featured articles showed strong expansion:
- अरस्तु 2,624 B → 33,249 B (~12.7× growth)
- कश्यप सागर: 2,311 B → 14,943 B (~6.5× growth)

4. Overall takeaway

The campaign improved visibility, with a ~15.2% increase in views on promoted articles
However, edits reached only ~1.6% of promoted articles, indicating limited conversion from awareness to contribution