List of experiments in Product and Technology

Experiments in Product & Technology

• Overview
• FAQ
• Archive

This list of experiments aims to become a centralized overview of experiments in the Product & Technology department.

Each experiment includes a brief description of the problem we're trying to solve, the type of experiment, selected population and other key aspects of the work. This page is a work in progress and will be updated regularly.

About this list

Experiments are listed according to their status:

• Upcoming: The experiment is being designed and configured for data collection. We're setting up what we want to test, how we'll measure it, and getting everything ready. It is not active.
• Live: The experiment has been deployed and is actively collecting user interaction data with specified traffic on defined wikis. The experiment is active for a pre-defined duration.
• Analysis: The experiment owner is reviewing results and community thoughts.
• Decision: The owner has reached a decision, to be documented in the Outcome field and typically falls into one of three categories:
  • "Ship it" – Recommend to roll out the winning version
  • "Don't ship it" – Results show the feature didn't perform as expected; don't implement
  • "We need to learn more" – Results are inconclusive; needs another test, qualitative feedback or further iteration

Experiments

Upcoming

Experiment	Team	Anticipated timing	Experiment type	Audience	Problem to solve / why we are doing this	Share your feedback
Guided article creation Help new editors easily understand and apply the existing practices and policies, and navigate the task of starting a new article with ease.	Language and Product Localization	TBD	A/B test	TBD % of users on Arabic, Bengali, Japanese, Persian, Portuguese, and Turkish Wikipedias	Currently, editors need to make a significant effort to create new content from scratch. As a result, reviewers have to process many low quality contributions and revert/fix them. Finally, readers see content creation as a mystery, which makes it harder for them to decide to become contributors in the future.	Discuss
Redesign account creation form on mobile web Redesign the mobile web account creation form to follow Codex guidelines and reduce visual and cognitive complexity. The updated design shortens explanatory text, improves layout, and allows users to view all required fields on a typical mobile screen without scrolling.	Growth	April - June 2026	A/B test	TBD % of users on English Wikipedia	As a mobile web user, I want a cleaner and less verbose account creation form so that I can view all required fields at a glance without scrolling, and complete registration with reduced information overload.	Discuss
Encouraging users with unconfirmed emails to confirm them	Product Safety and Integrity team	TBD	A/B/C test	Logged-in users with unconfirmed emails on English, French, German, Polish, Japanese, Hebrew, Wikidata, Commons, Mediawiki, Meta	Wikimedia projects do not currently require email confirmation in order to edit and take other actions as a registered account. As a result, many user accounts have associated emails that have never been confirmed. This is contributing right now to users being locked out of their account, with no safe way for them to regain access. This experiment will measure the effectiveness of persistent communication that encourages user accounts that have an unconfirmed email address to confirm it, or set up a new confirmed email address.	Discuss
Mobile page previews	Reader Growth	April 20 - May 18	A/B test	10% of logged-out readers on Arabic, French, Italian, Polish, and Vietnamese Wikipedias and 0.1% of logged-out readers on English Wikipedia	We want to test Mobile Page Previews — a pop-up bottom sheet that appears when a reader taps a blue link, showing the thumbnail, lead paragraph, and an option to open the article — on mobile web article pages. Page Previews already exist on the Wikipedia desktop experience and both mobile apps.	Discuss
Explore Feed Refresh Testing a refreshed Explore Feed with a more modern design, clearer structure, and lightweight personalization, with a goal of making it easier for readers to discover interesting content and come back to the app more often.	Wikimedia Apps Team	TBD	A/B test	New and casual Wikipedia app readers (especially logged-out users)	The Explore Feed is not engaging enough; most users don’t come back to it, don’t customize it, and don’t clearly understand its purpose.	Discuss

Live

Experiment	Team	Anticipated Timing	Experiment Type	Audience	Problem to solve / why we are doing this	Share your feedback
A/A test on retention rate for logged-in readers baseline Establish a baseline retention rate for logged-in readers.	Reader Experience	March 24 to May 30, 2026	A/A test	Randomized users across top 50 trafficked wikis	We do not currently have an existing measurement of user retention rate for logged-in Wikipedia readers. This test is intended to etablish a baseline will be useful in helping us set key result goals for improving logged-in and active user retention on web next FY.	Discuss
Add user account button to header on mobile web Improve the discoverability of account creation on mobile web.	Growth	Start Date: 2026-03-31 End Date: 2026-06-28	A/B test	10% of users on Hindi, Indonesian, Bengali, Thai, and Hebrew Wikipedias	Account creation is not easily discoverable on mobile web. Users must open the hamburger menu to find "Log in," then close the auto-focused keyboard on the login page, and scroll to the bottom to discover the "Create account" option. A analysis of the top 20 most visited websites shows that nearly all platforms display a way to create a account directly from the homepage (T410558). We think this will significantly improves discoverability and reduces friction in getting readers to create accounts.	Discuss
Improve the logged-out warning message on mobile web Reduce the sense of friction or alarm when users encounter the logged-out warning upon editing, and highlight the benefits of creating an account (e.g., edit attribution, watchlist, communication tools, and reputation building).	Growth	March – June 2026	A/B test	10% of users on Arabic, French, Spanish, German, Russian, Chinese, Italian, Portuguese, Persian, and Polish Wikipedias 0.1 % of users on English Wikipedia	When users attempt to edit while logged out, they encounter a jarring warning message that emphasizes editing as an IP or temporary account. This interface can feel abrupt and discouraging, and it frames logged-out editing as the primary path rather than encouraging users to create an account. There is an opportunity to soften the logged-out warning message and guide users toward account creation in a more supportive, motivating way. By improving the tone and visual hierarchy, we can better communicate the benefits of creating an account and make the overall experience less intimidating.	Discuss
Mobile Table of Contents Offer a table of contents on mobile web.	Reader Growth	March - April 2026	A/B/C test	Mobile web users on Arabic, French, English, Indonesian, and Vietnamese Wikipedias	Readers struggle to navigate our mobile interface without wayfinding help. We want to explore whether a table of contents supports readability.	Discuss
Hybrid search Show richer results to readers when they search for something in natural language	Wikimedia Apps Team	February - April 2026	A/B test	Android app users on Greek, English, French, and Portuguese Wikipedias	Readers struggle to find specific information on Wikipedia using internal search, and hybrid search (keyword search plus semantic search) shows potential for improving navigation and wayfinding.	Discuss
Revise Tone structured task Support newcomers by introducing a Suggested Edit that focuses on identifying and improving non-neutral language in articles. [includes a machine learning model]	Growth	Start Date: 2026-03-31 End Date: 2026-04-16	A/B Test	50% of users on English, French, Arabic, and Portuguese Wikipedias	Writing in a neutral tone is a pillar of Wikipedia, but is a practice many new volunteers find to be unintuitive. An October 2024 analysis of English Wikipedia found that 56% of new content edits from newer volunteers contained peacock words, with 22% of these edits being reverted. Revise tone structured task is a way that we can help identify and improve non-neutral language in articles, alongside other implementations of the tone check model.	Discuss

Analysis

Experiment	Team	Experiment Timeline	Experiment Type	Audience	Problem to solve / why we are doing this	Share your feedback
Reader → Contributor Attribution Baseline Experiment Measure how many readers create contributor accounts and see whether the rate differs depending on how people came to the site	Various teams involved in Key Result WE5.3	March 22nd - April 2nd	Invisible (no discernable UI change)	Sampled anonymous readers across Wikipedia projects, sampling the top 50 largest Wikipedias	WMF would like to understand how different types of readers become contributors. This experiment puts additional code on existing interfaces that tracks how many readers become contributors across a variety of characteristics (like which device a person uses, how they found the site, etc.)The goal is to test the code to see if such experiments can be carried out going forward and gain insight into how readers become contributors. The data gathered from this experiment will be aggregated or discarded once the experiment is complete, in accordance with the data collection policies.	Discuss

Decision

Experiment	Team	Experiment Timeline	Experiment Type	Audience	Problem to solve / why we are doing this	Share your feedback	Outcome
hCaptcha A/B Test Test a bot detection service (hCaptcha) as a possible replacement for the current CAPTCHA system.	Product Safety and Integrity	2025 Oct – 2026 Jan	A/B Test	French Wikipedia: 10%	The current CAPTCHA system is not equipped to defend against modern malicious automated activity. If we prove the usability of the new service, we’ll be able to better protect Wikimedia projects from malicious bots while potentially improving usability and accessibility.	Discuss	Ship it: We plan to expand hCaptcha to more editing interfaces and more wikis. In the A/B test, the rate of completing the account creation process went up by ~10% and we observed a 31% reduction in block rates for user accounts where hCaptcha was involved, indicating that automated bad actors were more likely to fail in registering accounts when they encountered hCaptcha compared to FancyCaptcha.
Image browsing A feature that will make discovery of images and videos easier on Wikipedia.	Reader Growth	2025 Nov–Dec	A/B Test	10% of mobile web readers on Arabic, Chinese, French, Indonesian, Vietnamese; 0.1% on English Wikipedia	In WMF's surveys of global internet users, “more images/photos” is the most asked-for improvement to Wikipedia - and by creating richer, more visual experiences, we aim to attract new readers and encourage them to return.	Discuss	We need to learn more: Do another experiment focused on image carousel and default full-screen display, with aim to gather more data before rollout.
Reading lists Allow readers to save articles to a list for reading later.	Reader Experience	2025 Nov–Dec	A/B Test	Logged-in users with zero edits on Arabic, Chinese, English, French, Indonesian, Vietnamese	We aim to learn whether offering curation options helps readers engage more deeply with Wikipedia - and ultimately stay longer and return more often.	Discuss	Ship it: Start with a beta rollout release, with the aim of eventually shipping globally.
Expanded mobile sections Testing autoexpanded mobile web sections.	Reader Growth	2025 Dec	A/B Test	10% of mobile web readers on Arabic, Chinese, French, Indonesian, Vietnamese	Sections on mobile web are currently collapsed by default, which makes browsing difficult.	Discuss	Don't ship. Instead, run another experiment including a mobile Table of Contents with expanded sections.
Section editing dead-end on mobile web Allow editors to navigate between sections in edit mode on VisualEditor in mobile	Editing	2025 Dec - 2026 Jan	A/B test	10% of mobile web edits on the 100 largest Wikipedias, 0.1% for en.wiki	This feature is intended to make it easier for people on mobile web to edit any article section, regardless of which section-edit icon they tapped to begin.	Discuss	Ship it! Full-page editing affordance on mobile web has been released to all Wikipedias. 2.6% increase in constructive edits for logged-out users.
Tone Check Show new editors a warning when their writing is considered to be NPOV by a transformer model [includes a machine learning model]	Editing	2025 Sept-Jan	A/B test	0.1% of edit attempts eligible for Tone Check on fr.wiki, ja.wiki, pt.wiki (limited to edits by unregistered users and users with 100 or fewer edits)	Tone Check is meant to address two core issues: Newcomers publishing edits to Wikipedia that contain promotional, derogatory, or otherwise subjective language because they lack the awareness that this kind of editing is not aligned with Wikipedia policies. Experienced volunteers being burdened by the effort and attention they need to allocate towards patrolling and reviewing preventable damage made in good faith. This can come at the expense of identifying and addressing more subtle and complex forms of vandalism.	Discuss	Ship it! Tone Check will be released to all Wikipedias (T416406) and included in Suggestion Mode. Overall: Tone Check improved the rate of constructive edits by +6.2% [4.4] percentage points. We observed improvements in overall edit quality at each of the three partner Wikipedias. Platform: on desktop, constructive edit rate increased by +6.4% while we observed no statistically significant change in mobile web constructive edits. Experience level: Tone Check appears especially effective at increasing the constructive edit rate of a registered Junior Contributors, where we observed a +14.8% increase [10.2 pp] in constructive edit rates.
Improve verification email Simplify the email that asks users to confirm their email	Growth	2025 Dec - 2026 Jan	Release to pilot wikis	en.wiki, Wikidata, Commons	The current verification (confirmation) email can be perceived as alarming, especially with language about the user's IP address. This may deter users from confirming their email or engaging further. We aim to revise this email to make it more welcoming and informative, improving the overall onboarding experience and increasing email confirmation rates.	Discuss	Ship it! Improved verification email has been released to all wikis (T416748). Email verification rates increased across all three pilot wikis (English Wikipedia, Commons, Wikidata) in both month-over-month and year-over-year comparisons. Same-week verification rate rose from 41.6% to 45.9%, a 4.3% absolute increase (~10% relative improvement). Observed largest gains on Commons; improvements are consistent across both app and non-app registrations.
Paste Check	Editing	2025 Oct-Dec	A/B test	50% of users editing a desktop or mobile main namespace page using Visual Editor on 22 Wikipedias	This check is an effort to increase the likelihood that the new content people are adding to Wikipedia is aligned with the Movement’s commitment to offering information under a free content license.	Discuss	Ship it! Paste Check is now available to all Wikipedias (T405127). We observed a -18% relative decrease [10.5% → 8.6%] in the revert rate for edits shown Paste Check compared to edits eligible but not shown Paste Check. However, the data is insufficient to statistically confirm that this effect was not due to random noise.
Reference Check	Editing	2025 Nov-Dec	A/B test	38% of new content edits on English Wikipedia	If you add a large amount of text without any citations, the reference check will remind you to add a citation. This only works for edits made using the visual editor. This check is shown when you click on the Publish button. Introduced in late 2023.	Discuss	Ship it! Reference Check has been enabled for all newcomers on English Wikipedia (T367343) , and is now available to all Wikipedias. When Reference Check was shown, edits were far more likely to add a reference or acknowledge/explain why they did not: Mobile web: ~17.5× more likely (2.8% → 48.9%) Desktop: ~2.2× more likely (30.7% → 68.2%) We observed the following impact on constructive edits (not reverted within 48 hours) Mobile web: 56.4% → 66.7% (+18.2% relative) Desktop: 75.5% → 77.9% (+3.2% relative)
Donate Button The donate button will be changed to a more eye-catching design.	Readers	2025 Aug–Sep	A/B Test	English Wikipedia: 0.1% — Test Wikipedia: 10%	Through this experiment, we want to see whether changing the “Donate” button entry point in English Wikipedia would encourage more clickthroughs from logged-out users and ultimately increase the number of donations.	Discuss	Ship it! — Proposal to scale to all users. More information
Watchlist: Group By Toggle The "group results by page" feature will be shown more prominently for a subset of contributors.	Moderator Tools	2025 Aug–Sep	A/B Test	Greek, Bengali, Urdu, Czech, Portuguese, Chinese, French Wikipedias: 100%	We want to improve feature discoverability and make it easier and faster for contributors to locate the edits they are interested in, and as a side effect, motivate their return.	Discuss	Discontinued due to inconclusive results.
Wikipedia Year in Review	Wikimedia Apps Team	October 11 – December 31, 2025 (with release in early December)	A/B testing (with different variations on Android and iOS)	Wikipedia app users (readers, editors and donors)	the apps team wants to help users feel more connected to Wikipedia by showing them their reading habits and activity. This can make them more engaged, come back more often, and even encourage donations.	Discuss	Ship it! — The feature showed strong engagement: Many users viewed multiple slides Users shared their results on social media It increased donations compared to the previous year
"Which came first?" Game	Wikimedia Apps Team	May 2025 (A/B test for 20 days)	A/B Test	Wikipedia Android app users (mainly logged-out readers)	The team wants to increase engagement and retention by offering a fun, daily game that encourages users to come back to the app and explore more content.	Discuss	Ship it! Users who played the game returned 7.3% more 71.4% of users who started the game completed it 10% engaged further (shared score, read, or saved articles) 169 app installs came from shared game links 95% of users were satisfied or neutral