Research and Decision Science/Data glossary

Research and Decision Science

This page collects definitions of essential and core metrics that teams in the Wikimedia Foundation use to guide tactical and strategic decisions. The sibling Data Dictionary page documents various data sources.

Definitions for core metrics and essential metrics

Essential metrics

Essential metrics are metrics maintained by the Wikimedia Foundation in production which meet specific requirements. This includes the Core annual plan metrics as well as other key indicators of health or product engagement.

Click here to review the Essential Metric criteria

Essential metrics must be:
1. trustable;
2. accessible to decision makers;
3. transparent towards those affected by the decisions made by the usage of the metrics;
Measurements of essential metrics must:
1. be relevant for one or more of the following cases:
  1. frequent decision-making in situations where erroneous decisions can have department, Foundation, or Movement-level impact;
  2. frequent decision making about the Wikimedia Foundation's strategic direction;
  3. frequent monitoring of important operations.
2. trustable;
3. The definition and measuring, storage, use, and potential publishing of the metric and its measurement must meet relevant WMF policies and guidelines. More specifically: WMF's Privacy Policy, Data Collection Guidelines, Data Retention Guidelines, Data Publication Guidelines, and Human Rights Policy.

Please refer to the FAQ and more details to learn more about essential metrics.

Core metrics

Core metrics (also called "Core Annual Plan metrics") are a small, well-defined set of measurements that we use to guide strategic decisions. Core metrics are a special subset of essential metrics. These are the metrics the Foundation highlights in its annual plan. For FY26-27, the four goals are:

(IR) Increase our reach: Diversify our traffic beyond traditional search.
(DE) Deepen engagement: Support our contributors and turn casual readers into repeat readers, editors and donors.
(PP) Protect our projects: Sustain and defend our model.
(ST) Build speed and resilience: Support the people and systems behind this work.

Core & Essential Metrics

(Note: This table includes metrics that are part of annual plan, or monthly movement health, monitoring and reporting as well as essential metrics of the multi-year measurement strategies for the Consumers- and Contributors-focused product areas.)

In addition to the Core and Heartbeat metrics measured for the Annual Plan, there is a set of Movement Health metrics regularly calculated for monitoring overall. The code that calculates Movement Health metrics can be found in the movement-metrics repository public reporting is available monthly as Movement Metrics, read the latest report.

In the table below, all essential metrics and their public access points are outlined.

Metric	Definition	Relevance	Public Code	Public Data Access
Retained Editors^[1]	Registered users who made at least one edit during both the given calendar month and the month before. This includes all edits (including edits to content, talk, and other namespaces) whether or not the edits have been reverted.	Core Metric (DE 1) & Movement Health	_GitLab	_GitLab
Constructive Editors	Registered users who publish at least one constructive edit (edit was not reverted within 48 hours) within the experiment (or defined timeframe). This includes Constructive Mobile Editors: Beginner editors who use Visual Editor to publish at least one constructive edit on mobile web (DE 1.1) Dimensions: Surface (Desktop, Mobile web, Mobile apps) Experience (based on lifetime edits): Beginner < 100 Learner 100 - 499 Experienced 500 - 999 Very experienced 1000+ Editor interface (Visual Editor, Source Editor)	OKR Metric (DE 1.1)	_GitLab
Weekly engagement count	Average edits+non-edit actions per user to any namespace in the past calendar week, whether reverted or not.	OKR Metric (DE 1.2)
Editor second-week retention	Percentage of contributors who return to publish at least one edit (any namespace) during their second week (day 7-14) after their initial constructive edit on a main namespace after experiment exposure.	OKR Metric (DE 1.3)
Retained Newcomers^[1]	Newly-registered users who: constructively activated (i.e. made at least one content edit during their first 24 hours after registration which was not reverted within 48 hours) and then made another edit (in any namespace, whether reverted or not) during days 2-15 after their constructive activation	OKR Metric (DE 1.3) & Movement Health	_GitLab	_GitLab
30-day survival rate of articles	Proportion of articles not deleted within 30 days of publication by editors with <100 edits who were presented with a guided article creation flow through controlled experiments, compared to editors in the control group.	OKR Metric (DE 1.4)
Monthly edits on Abstract	Number of monthly edits to existing Abstract Wikipedia content pages	Core Metric (DE 2)
Monthly active editors on Abstract	Number of monthly active editors (defined as editors who did 10 or more edits in the past month ) on Abstract Wikipedia	Core Metric (DE 2)
Daily Active Users on Wikipedia mobile apps	Unique Users of Wikipedia apps who send the "app open" event on a given day, where User: uniquely identified by app install id in imported events, and App Open: anytime the screen of the app is in the foreground of the user's device (not backgrounded or displayed as a secondary widget - see tasks mentioned above for details).	Core Metric (DE 3)	_GitLab
Weekly logged-in Wikipedia readers on web	The number of logged-in web users who have never edited and who viewed a page on any Wikipedia during a given 7-day period, including both desktop and mobile web.	Core Metric (DE 3)	_GitLab
Casual & Active Retained Readers	Casual: 2nd week retention on Wikipedia apps: The percentage of users, regardless of login or edit status, on Wikipedia apps who visit once during a 7-day period, then again between 7 and 14 days after the first visit. (DE 3.1) Logged-out Wikipedia 21-day retention on mobile web: The percentage of logged-out users on Wikipedia mobile web who return at least once after the first 24 hours and before the end of day 21 after their first visit within a given 7-day cohort week. (DE 3.1) Logged-in Wikipedia reader 2nd week retention on web: Logged-in users, on any Wikipedia, with 0 edits on web who visit once during a 7-day period, then again between 7 and 14 days after the first visit (DE 3.2) Active: 2nd day Wikipedia retention on apps: The percentage of users, regardless of login or edit status, on Wikipedia apps who visit once during a 7-day period, then again between 1 and 2 days after the first visit (DE 3.2)	OKR Metric (DE 3.1 - 3.2)	_GItLab
Number of newly-created Wikipedia active reader accounts	Number of newly created accounts on web during a month that made zero edits within the first 7 days after registration.	OKR Metric (DE 3.3)	_GitLab
New installs of the Wikipedia app	Count of unique Wikipedia app install events in a given week. (NOTE: for Android, these are generated by our app; for iOS they're not available except for with manual import, see task)	OKR Metric (DE 3.4)	_GitLab
Same-day handling of bad-faith content	Of edits reverted by a user with extended rights (UWER) within 30 days, the percentage handled within 24 hours ("same day")	Core Metric (DE 4)
Signal-Driven Handling of Bad-Faith Accounts	The proportion of confirmed bad-faith accounts that are efficiently identified through automated signals, and then mitigated either by Users with Extended Rights (UWERs) or the platform. Metric will be evaluated using a 4-week rolling average monitored on a weekly basis.	Core Metric (DE 4)
Value per donor	The total revenue (USD) from all sources divided by the total number of unique donors who gave during the referenced time period (typically the fiscal year).	Core Metric (DE 5) & Heartbeat
2nd month recurring donor retention	Count of on-platform recurring donors who give in the second month.	OKR Metric (DE 5.3)
Invited Contributions	Number of edits through collaborative contributions	Core Metric (DE 8)
Reaching new contributors through volunteering programs	Count of new contributors making first edits during events	Core Metric (DE 8)
Access to Reliable Sources	Number of monthly users of The Wikipedia Library	Core Metric (DE 8)
Marginal gains in referred pageviews	Marginal gain in external search-referred pageviews on areas of intervention (IR1) Marginal gain in non-search-referred pageviews on segments affected by interventions (IR2)	Core Metrics (IR 1 & 2)
Reuser value	Reuser Value = (Value of Referral Pageviews ± Value of Off-platform Impressions ± Financial Contracts Value) - Cost of Infrastructure	Core Metric (PP 1)
Enterprise revenue	Total Revenue from Enterprise subscriptions and professional services for the reporting period.	Core Metric (PP 2) & Heartbeat
Percentage of identified automated traffic through responsible pathways	Percentage of identified automated traffic (request count) on responsible pathways / total automated traffic on responsible pathways	Core Metric (PP 3)
Weekly Human Pageviews	Total count of page view requests made by human users for the contents of a wikimedia project page aggregated weekly, from Monday through Sunday.	Heartbeat	_GitLab
Weekly Unique Devices	The count of distinct devices visiting wikimedia project pages each week.	Heartbeat
Weekly Account Registrations	Weekly count of sign-ups for new Wikimedia global accounts.	Heartbeat	_GitLab
Weekly Active Editors	Weekly count of unique named users making at least 1 edit to a content article that is not reverted within 48 hours.	Heartbeat	_GitLab
Weekly Active Users with Extended Rights (UWERs)	A user is an "active UWER" for the week if they performed at least one action that is afforded to a users-with-extended-rights(UWER) group, in the 7-day window The list of actions that are included in this definition and the types of users that can perform them are listed here for clarity. Administrators: block, delete, protect, edit-filter and interface edits Global sysops: (same as Administrators applied cross-wiki) Bureaucrats: grant/revoke user groups, rename users CheckUsers: check IPs, accounts Ombuds Comm: global CheckUser investigations Stewards: global account lock/unlock, global rights, global block, global rename Oversighters: suppress revisions, suppress log entries, hideuser	Heartbeat	_GitLab
Clicks in Google Search	The number of times a user selected your website's link in the search results and successfully navigated to your site each day.	Heartbeat
Impressions in Google Search	The number of times a user saw a link to Wikimedia project in Google Search, Discover, or News results each day. How it works: A single impression is logged whenever your listing loads on a search results page. For standard searches, this counts regardless of whether the user actually scrolled down to see the result. Meaning: It measures your search visibility and potential reach.	Heartbeat
Donors	Total Number of donors in a given fiscal year, as represented by a single record in our donor CRM (civicrm)	Heartbeat
Revenue	Total revenue raised in a given fiscal year, as recorded in our donor CRM (civicrm). Note this would not include lines like investment income or enterprise income.	Heartbeat
Working capital ratio (in months)	End of year net assets / next fiscal year projected expenses * 12	Heartbeat
Expenses	Total expenses in a given fiscal year, as recorded in our Financial Planning & Reporting software, Adaptive.	Heartbeat
Monthly Account Registrations	Monthly count of sign-ups for new Wikimedia global accounts.	Movement Health	_GitLab	_GitLab _wikistats
Monthly Active Editors^[1]	Registered users who made at least 5 content edits in the given calendar month. See meta:Research:Active editor for the full definition. Dimensions: New: Active editors who registered during the given month. Returning: Active editors who registered before the given month. Experience: number of content edits made in the 12 months ending with the given month^[2] Junior: < 100 Experienced: 100-299 Very Experienced: 300+	Movement Health	_GitLab	_GitLab _wikistats
Monthly Total Active Moderators	Registered users who did 5+ moderator actions in a given month, with bots excluded.	Movement Health	_GitLab
Monthly Retained Active Moderators	Users who were Active Moderators during a given month and during the preceding month (i.e.: Users who performed 5+ moderator actions during a given month, and who also performed 5+ moderator actions during the preceding month.)	Essential Metric (Contributors)	_GitLab
Monthly New Active Moderators	Users who were Active Moderators during a given month, and who were not Active Moderators during any of the 12 preceding months. (i.e.: Users who performed 5+ moderator actions during a given month, and who performed 0-4 moderator actions during each of the 12 preceding months).	Essential Metric (Contributors)	_GitLab
Monthly Active Administrators	Users who performed at least one administrative action (page deletion or undeletion, user block or unblock, page protection or unprotection, or user rights change) in the given calendar month. Full definition at meta:Research:Active administrator.	Movement Health	_GitLab	_GitLab
Net New Content	The number of content pages added since the previous month, excluding deleted pages and redirects.^[3] Wikipedia articles: The number of Wikipedia articles added since the previous month, excluding deleted articles and redirects.^[4] Commons files:The number of Wikimedia Commons content pages added since the previous month, excluding deleted pages and redirects.^[5] The number of Wikidata entities^[6] added since the previous month, excluding deleted ones and redirects.^[7]	Movement Health	_GitLab	_GitLab _wikistats
Mobile Edits	The number of edits tagged as having been made using the mobile website (which can be used by desktop computers) or the mobile apps.	Movement Health	_GitLab
Content Gap	Is a metric that quantifies knowledge gaps^[8] in Content by estimating the distribution of pieces of content (e.g., Wikipedia articles, Wikidata items) across different categories (e.g., gender, geographic distribution, cultural background). A complete list of mappings for content can be found here	Movement Health	_GitLab	_GitLab
Monthly Unique Devices	Full definition: R:Unique Devices Data source: unique_devices_per_project_family_monthly Monthly unique devices for all Wikipedias (includes desktop and mobile web), based on calendar month.	Movement Health	_GitLab	_GitLab
Content Interactions	Monthly pageviews (all platforms) based on calendar month + desktop previews corrected for spurious IE views from some countries.^[9] Full definition: R:Page view Data source: pageview_hourly Dimensions: Desktop: Same limited to the desktop domains (e.g. en.wikipedia.org) Mobile Web: Same limited to the mobile web domains (e.g. en.m.wikipedia.org, does not include apps) Desktop previews: Seen page previews, defined as previews popups that remain visible for at least one second. (Data source: virtualpageview_hourly)	Movement Health	_GitLab _pageviews _previews	_GitLab _wikistats
Clickthrough Rate	Click-based measure of engagement with interface elements and features.	Product Analytics Key Indicator	_meta
Retention Rate	Repeated usage-based measure of user engagement and satisfaction with products and features.	Product Analytics Key Indicator	_meta
Campaign Sign-up Rate	Conversion of users into event participants.	Product Analytics Key Indicator	_meta
Constructive Activation	Newcomers making their first edit to an article in the main namespace of a Wikipedia project within 24 hours of registration, and that edit not being reverted within 48 hours of being published.	Product Analytics Key Indicator	_GitLab
Constructive Edit	Also known as "Productive edits" or "survived edits" are edits which are not reverted within the measurement window	Product Analytics Key Indicator	_GitLab
Programmatic Expense Ratio^[10]	Average Program Expense Percentage = Average Program Expenses ÷ Average Total Expenses (When Calculating Using Form 990) = Average of Part IX line 25B ÷ Average of Part IX line 25A ^[11]	Core Metric

Note: If you are a Wikimedia Foundation staff or contractor, you may find additional internal access information in this spreadsheet.

Dimensions

Important dimensions used to break down the metrics listed above include:

Developed and Emerging Wikipedias: used with content gap and contributors metrics

Diversity

A set of contributors, content, or readers metrics (defined above), restricted to:

Global regions: Grants:Regions, canonical_data.countries
Gender: definition for mapping gender to content

Footnotes & Citations

↑ ^a ^b ^c Unlike Wikistats, this metric includes edits to pages which were later deleted.
↑ Content edits on any wiki are included, but Wikidata edits are counted as 1/10 edit each due to their greater granularity.
↑ Produced by subtracting last month's total content pages metric from this month's.
↑ A subset of net new content. Produced by subtracting last month's total Wikipedia articles metric from this month's.
↑ A subset of net new content. Produced by subtracting last month's total media files metric from this month's.
↑ An entity is anything represented by a page in a content namespace, namely an item, property, lexeme, or entity schema.
↑ A subset of net new content. Produced by subtracting last month's total Wikidata entities metric from this month's.
↑ Knowledge gaps are major differences in participation or coverage of a specific group of readers, contributors,or content.
↑ The current calculation includes user agents and automated agents. We use the User Agent to define whether we assign a mobile or desktop experience; User Agents we consider to be mobile are captured in this code on Github. Most tablets are classified as mobile.
↑ The percentage of the budget spent on program expenses, called the Programmatic Expense Ratio, is an important financial metric that we track. Each year, we set a target for our programmatic expense ratio is to align with nonprofit sector best practices. Independent charity assessment organizations like Charity Navigator help establish these best practices. Charity Navigator sets its benchmark for highest-scoring nonprofits as exceeding >70% programmatic expenses. Charity Navigator provides the following explanation and formula to calculate this ratio: "Charities exist to provide programs and services. They fulfill the expectations of givers when they allocate a good portion of their budgets toward their stated missions. While administration expenses are necessary for efficient charity operations, organizations that grossly underspend on their programs and services will most likely not have as strong an impact on their charitable missions. We calculate the nonprofit's average program expense percentage over its three most recent fiscal years and then assign a numeric score based on an established scale.
↑ "Our Methodology". Charity Navigator (in en-US). Retrieved 2024-08-06.

[:0-1] Unlike Wikistats, this metric includes edits to pages which were later deleted.

[2] Content edits on any wiki are included, but Wikidata edits are counted as 1/10 edit each due to their greater granularity.

[3] Produced by subtracting last month's total content pages metric from this month's.

[4] A subset of net new content. Produced by subtracting last month's total Wikipedia articles metric from this month's.

[5] A subset of net new content. Produced by subtracting last month's total media files metric from this month's.

[6] An entity is anything represented by a page in a content namespace, namely an item, property, lexeme, or entity schema.

[7] A subset of net new content. Produced by subtracting last month's total Wikidata entities metric from this month's.

[8] Knowledge gaps are major differences in participation or coverage of a specific group of readers, contributors,or content.

[9] The current calculation includes user agents and automated agents. We use the User Agent to define whether we assign a mobile or desktop experience; User Agents we consider to be mobile are captured in this code on Github. Most tablets are classified as mobile.

[10] The percentage of the budget spent on program expenses, called the Programmatic Expense Ratio, is an important financial metric that we track. Each year, we set a target for our programmatic expense ratio is to align with nonprofit sector best practices. Independent charity assessment organizations like Charity Navigator help establish these best practices. Charity Navigator sets its benchmark for highest-scoring nonprofits as exceeding >70% programmatic expenses. Charity Navigator provides the following explanation and formula to calculate this ratio: "Charities exist to provide programs and services. They fulfill the expectations of givers when they allocate a good portion of their budgets toward their stated missions. While administration expenses are necessary for efficient charity operations, organizations that grossly underspend on their programs and services will most likely not have as strong an impact on their charitable missions. We calculate the nonprofit's average program expense percentage over its three most recent fiscal years and then assign a numeric score based on an established scale.

[11] "Our Methodology". Charity Navigator (in en-US). Retrieved 2024-08-06.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]