Wikimedia Foundation Annual Plan/2024-2025/Product & Technology OKRs
 | The Annual Plan draft is now available. Translations are underway. |
This is part of the Wikimedia Foundation's Product and Technology departments drafting process for the 2024–25 Annual Plan.
This document represents the first part of the 2024-25 Annual Planning process for the Wikimedia Foundation's Product & Technology department. It describes on the departments' draft "objectives and key results" (OKRs). This is a continuation of the structure of work portfolios (called "buckets") that began last year.
.jpg)
I spoke with you all back in November about what I believe is the most pressing question facing the Wikimedia movement: how do we ensure that Wikipedia and all Wikimedia projects are multigenerational? I’d like to thank everyone who took the time to really consider that question and to respond to me directly, and now that I’ve had the chance to spend some time reflecting on your responses, I’ll share what I’ve learned.
First, there is no single reason volunteers contribute. In order to nurture multiple generations of volunteers, we need to better understand the many reasons people contribute their time to our projects. Next, we need to focus on what sets us apart: our ability to provide trustworthy content as disinformation and misinformation proliferate around the internet and on platforms competing for the attention of new generations. This includes ensuring we achieve the mission to assemble and deliver the sum of all human knowledge to the world by expanding our coverage of missing information, which can be caused by inequity, discrimination or bias. Our content needs to also serve and remain vital in a changing internet driven by artificial intelligence and rich experiences. Lastly we need to find ways to sustainably fund our movement by building a shared strategy for our products and revenue so that we can fund this work for the long term.
These ideas will be reflected in the Wikimedia Foundation’s 2024–2025 annual plan, the first portion of which I’m sharing with you today in the form of draft objectives for our product & technology work. Like with last year, our entire annual plan will be centered around the technology needs of our audiences and platforms, and we’d like your feedback to know if we’re focusing on the right problems. These objectives build off ideas we’ve been hearing from community members over the past several months through Talking:2024, on mailing lists and talk pages, and at community events about our product and technology strategy for the year ahead. You can view the full list of draft objectives below.
An “objective” is a high level direction that will shape the product and technology projects we take on for the next fiscal year. They’re intentionally broad, represent the direction of our strategy and, importantly, what challenges we’re proposing to prioritize among the many possible focus areas for the upcoming year. We’re sharing this now so community members can help shape our early-stage thinking and before budgets and measurable targets are committed for the year.
Feedback
One area in which we’d particularly like feedback is our work grouped under the name “Wiki Experiences.” “Wiki Experiences” is about how we efficiently deliver, improve, and innovate how people directly use the wikis, whether as contributors, consumers, or donors. This involves work to support our core technology and capabilities and making sure we can improve the experience of volunteer editors — in particular, editors with extended rights — through better features and tooling, translation services, and platform upgrades.
Here are some reflections from our recent planning discussions, and some questions for all of you to help us refine our ideas:
- Volunteering on the Wikimedia projects should feel rewarding. We also think that the experience of online collaboration should be a major part of what keeps volunteers coming back. What does it take for volunteers to find editing rewarding, and to work better together to build trustworthy content?
- The trustworthiness of our content is part of Wikimedia’s unique contribution to the world, and what keeps people coming to our platform and using our content. What can we build that will help grow trustworthy content more quickly, but still within the quality guardrails set by communities on each project?
- To stay relevant and compete with other large online platforms, Wikimedia needs a new generation of consumers to feel connected to our content. How can we make our content easier to discover and interact with for readers and donors?
- In an age where online abuse thrives, we need to make sure our communities, platform, and serving system are protected. We also face evolving compliance obligations, where global policymakers look to shape privacy, identity, and information sharing online. What improvements to our abuse fighting capabilities will help us address these challenges?
- MediaWiki, the software platform and interfaces that allow Wikipedia to function, needs ongoing support for the next decade in order to provide creation, moderation, storage, discovery, and consumption of open, multilingual content at scale. What decisions and platform improvements can we make this year to ensure that MediaWiki is sustainable?
Draft Objectives
Currently published are the highest planning level - the "Objectives".
The next level - the draft "Key Results" (KR) for each finalised objective are provided below.
The underlying "Hypotheses" for each KR will be updated on the relevant project/team's wiki pages throughout the year to be updated throughout the year as lessons are learned.
| Wiki Experiences (WE) draft objectives | ||||
|---|---|---|---|---|
| Objective | Objective area | Objective | Objective context | Owner |
| WE1 | Contributor experience | Both experienced and new contributors rally together online to build a trustworthy encyclopedia, with more ease and less frustration. | In order for Wikipedia to be vibrant in the years to come, we must do work that nurtures multiple generations of volunteers and makes contributing something people want to do. Different generations of volunteers need different investments -- more experienced contributors need their powerful workflows streamlined and repaired, while newer contributors need new ways to edit that make sense to them. And across these generations, all contributors need to be able to connect and collaborate with each other to do their most impactful work. With this objective, we will make improvements to critical workflows for experienced contributors, we will lower barriers to constructive contributions for newcomers, and we will invest in ways that volunteers can find and communicate with each other around common interests. | Marshall Miller |
| WE2 | Encyclopedic content | Communities are supported to effectively close knowledge gaps through tools and support systems that are easier to access, adapt, and improve, ensuring increased growth in trustworthy encyclopedic content. | Encyclopedic content primarily on Wikipedia can be increased and improved through continuous engagement and innovation. Tools and resources (both technical and non-technical) that are available for contributors to use for their needs can be made more discoverable, and reliable. These tools should be better supported by WMF, through feature improvements achievable in short cycles. In view of recent trends around AI assisted content generation and changing user behaviour, we will also explore groundwork for substantial changes (e.g. Wikifunctions) that can assist scaled growth in content creation and reuse. Mechanisms to identify content gaps should be easier to discover, and plan with. Resources that support growth of encyclopedic content, including content on sister projects, projects such as Wikipedia Library, and campaigns can be better integrated with contribution workflows. At the same time, methods used for growth should have guardrails against growing threats, that can ensure that there is continued trust in the process while staying true to the basic tenets of encyclopedic content as recognised across Wikimedia projects.
Audience: Editors, Translators |
Runa Bhattacharjee |
| WE3 | Consumer experience (Reading & Media) | A new generation of consumers arrives at Wikipedia to discover a preferred destination for discovering, engaging, and building a lasting connection with encyclopedic content. | Goals:
Retain existing and new generations of consumers and donors. Increase relevance to existing and new generations of consumers by making our content more easy to discover and interact with. Work across platforms to adapt our experiences and existing content, so that encyclopedic content can be explored and curated by and to a new generation of consumers and donors. |
Olga Vasileva |
| WE4 | Trust & Safety | Improve our infrastructure, tools, and processes so that we are well-equipped to protect the communities, the platform, and our serving systems from different kinds of scaled and directed abuse while maintaining compliance with an evolving regulatory environment. | Some facets of our abuse fighting capabilities are in need of an upgrade. IP-based abuse mitigation is becoming less effective, several admin tools are in need of efficiency improvements, and we need to put together a unified strategy that helps us combat scaled abuse by using the various signals and mitigation mechanisms (captchas, blocks, etc) in concert. Over this year, we will begin making progress on the largest problems in this space. Furthermore, this investment in abuse protection has to be balanced by an investment in the understanding and improvement of community health, several aspects of which are included in various regulatory requirements. | Suman Cherukuwada |
| WE5 | Knowledge platform I (Platform evolution) | Evolve the MediaWiki platform and its interfaces to better meet Wikipedia's core needs. | MediaWiki has been built to enable the creation, moderation, storage, discovery and consumption of open, multilingual content at scale. In this second year of Knowledge Platform we will take a curating look at the system and begin working towards platform improvements to effectively support the Wikimedia projects core needs through the next decade, starting with Wikipedia. This includes continuing work to define our knowledge production platform, strengthening the sustainability of the platform, a focus on the extensions/hooks system to clarify and streamline feature development, and continuing to invest in knowledge sharing and enabling people to contribute to MediaWiki. | Birgit Müller |
| WE6 | Knowledge platform II (Developer Services) | Technical staff and volunteer developers have the tools they need to effectively support the Wikimedia projects. | We will continue work started to improve (and scale) development, testing and deployment workflows in Wikimedia Production and expand the definition to include services for tool developers. We also aim to improve our ability to answer frequently asked questions in the field of developer/engineering workflows and audiences and make relevant data accessible to enable informed decision making. Part of this work is to look at practices (or lack of such) that currently present a challenge for our ecosystem. | Birgit Müller |
| Signals and Data Services (SDS) draft objectives | ||||
|---|---|---|---|---|
| Objective | Objective area | Objective | Objective context | Owner |
| SDS1 | Essential metrics | Our decisions about how to support the Wikimedia mission and movement are informed by high-level metrics and insights. | In order for us to effectively and efficiently build technology, support volunteers, and advocate for policies that protect and advance access to knowledge, we need to understand the Wikimedia ecosystem and align on what success looks like. This means tracking a common set of metrics that are reliable, understandable, and available in a timely manner. It also means surfacing research and insights that help us understand the whys and hows behind our measurements. | Kate Zimmerman |
| SDS2 | Experimentation platform | Product managers can quickly, easily, and confidently evaluate the impacts of product features. | To enable and accelerate data informed decision making about product feature development, product managers need an experimentation platform in which they can define features, select treatment audiences of users, and see measurements of impact. Speeding the time from launch to analysis is critical, as shortening the timeline for learning will accelerate experimentation, and ultimately, innovation. Manual tasks and bespoke approaches to measurement have been identified as barriers to speed. The ideal scenario is that product managers can get from experiment launch to discovery with little or no manual intervention from engineers and analysts. | Tajh Taylor |
| Future Audiences (FA) draft objective | ||||
|---|---|---|---|---|
| Objective | Objective area | Objective | Objective context | Owner |
| FA1 | Test hypotheses | Provide recommendations on strategic investments for the Wikimedia Foundation to pursue – based on insights from experiments that sharpen our understanding of how knowledge is shared and consumed online – that help our movement serve new audiences in a changing internet. | Due to ongoing changes in technology and online user behavior (e.g., increasing preference for getting information via social apps, popularity of short video edu-tainment, the rise of generative AI), the Wikimedia movement faces challenges in attracting and retaining readers and contributors. These changes also bring opportunities to serve new audiences by creating and delivering information in new ways. However, we as a movement do not have a clear data-informed picture of the benefits and tradeoffs of different potential strategies we could pursue to overcome the challenges or seize new opportunities. For example, should we...
To ensure that Wikimedia becomes a multi-generational project, we will test hypotheses to better understand and recommend promising strategies – for the Wikimedia Foundation and the Wikimedia movement – to pursue to attract and retain future audiences. |
Maryana Pinchuk |
| Product and Engineering Support (PES) draft objective | ||||
|---|---|---|---|---|
| Objective | Objective area | Objective | Objective context | Owner |
| PES1 | Efficiency of operations | Make the Foundation's work faster, cheaper, and more impactful. | Staff do a lot in their regular work to make our operations faster, cheaper, and more impactful. This objective highlights specific initiatives that will both a) make substantial gains toward faster, cheaper, or more impactful; and b) take coordinated effort and change of formal and informal practices at the Foundation. Essentially, the KRs included in this objective are the hardest and best improvements we can make this year to operational efficiency of work touching our products and technology. | Amanda Bittaker |
Draft Key Results
The draft "Key Results" (KR) for each finalised objective are here. They correspond to each of the objectives, above.
The underlying "Hypotheses" for each KR will be updated on the relevant project or team's wiki pages throughout the year as lessons are learned.
| Wiki Experiences (WE) Draft Key Results
[ Draft Objectives ] | |||
|---|---|---|---|
| Key Result shortname | Key Result text | Key Result context | Owner |
| WE1.1 | Develop or improve one workflow that helps contributors with common interests to connect with each other and contribute together. | We think community spaces and interactions on the wikis make people happier and more productive as contributors. Additionally, community spaces help onboard and mentor newcomers, model best practices of contributing, and help address knowledge gaps. However, the existing resources, tools, and spaces that support human connection on the wikis are subpar and do not meet the challenges and needs of the majority of editors today. Meanwhile, the work of the Campaigns team has demonstrated that many organizers are eager to adopt and experiment with new tools with structured workflows that help them in their community work. For these reasons, we want to focus on encouraging and promoting a sense of belonging among contributors on the wikis. | Ilana Fried |
| WE1.2 | Constructive Activation: #% YoY increase in the percentage of newcomers who publish ≥1 constructive edit in the main namespace on a mobile device. | Current full-page editing experiences require too much context, patience, and trial and error for many newcomers to contribute constructively. To support a new generation of volunteers, we will increase the number and availability of smaller, structured, and more task-specific editing workflows (E.g. Edit Check and Structured Tasks).
Note: Baselines will only be established towards the end of Q4 current FY, after which our KR target metric percentage will also be established. |
Peter Pelberg |
| WE1.3 | Increase user satisfaction in 4 moderation products by X%. | Editors with extended rights make use of a wide range of existing features, extensions, tools, and scripts to perform moderation tasks on Wikimedia projects. This year we want to focus on making improvements to this tooling, rather than undertaking projects to build new functionality in this space. We're aiming to touch a number of products over the course of the year, and want to make impactful improvements to each. In doing so we hope to improve the experience of moderating content overall.
We will define X% upon measuring baselines for some common moderator tools that we may target with this workstream. The Community Wishlist will be a substantial contributor to deciding on the priorities for this KR. |
Sam Walton |
| WE2.1 | By the end of Q2, support organizers, contributors, and institutions with specific tools, insights, & organizing approaches that increase the coverage of quality content in key topic areas [TBD] by X%. | This KR is about improving topic coverage towards reducing existing knowledge gaps. We’ve established that communities benefit from effective tools paired with campaigns targeted at increasing the quality of content in our projects. This year we want to focus on improving existing tools and experimenting with new ways of prioritizing key topic areas that address knowledge gaps.
Our target X% increase in coverage will be determined by looking at existing baselines of quality content creation. Also, the topic areas we’ll be focusing on with communities and institutions will be determined by next quarter. |
Purity Waigi & Fiona Romeo |
| WE2.2 | By the end of Q2, implement and test two recommendations, both social and technical, to support languages onboarding for small language communities, with an evaluation to analyze community feedback. | There are editions of Wikipedia in about 300 languages. And yet, there are many more languages that are spoken by millions of people, in which there is no Wikipedia and no Wiki at all. This is a blocker to fulfilling our vision: that every single human being can freely share in the sum of all knowledge. The Wikimedia Incubator, is where potential Wikimedia project wikis in new-language versions can be arranged, written, tested and proven worthy of being hosted by the Wikimedia Foundation. The Incubator was launched in 2006 with the assumption that its users would have prior wiki editing knowledge. This problem is exacerbated by the fact that this process is supposed to be mostly performed by people who are the newest and the least experienced in our movement.While editing on Wikimedia wikis has significantly improved since then, the Incubator hasn't received these updates due to technical limitations. Currently, it takes several weeks for a wiki to graduate from the Incubator and only around 12 wikis are created each year, showing a significant bottleneck.
Existing research and materials reveal technical challenges in every phase of language onboarding: adding new languages to the Incubator, complexities in developing and reviewing content, and slow process in creating a wiki site when a language graduates from Incubator. Each phase is slow, manual, and complex, indicating the need for improvement. Addressing this problem will allow creating wikis in new languages more quickly and easily, and allow more humans to share knowledge. Various stakeholders, existing research and resources have highlighted proposed recommendations both social and technical. This key result proposes testing two recommendations both social and technical and evaluating the community feedback. |
Satdeep Gill & Mary Munyoki |
| WE2.3 | By the end of Q2, 2 new features guide contributors to add source materials that comply with project guidelines, and 3-5 partners have contributed source material that addresses language and geography gaps. | To grow access to the quality source material that’s needed to close strategic content gaps, we will:
|
Fiona Romeo & Alexandra Ugolnikova |
| WE2.4 | By the end of Q2, enable Wikifunctions calls on at least one smaller language Wikipedia to provide a more scalable way to seed new content. | To reduce our knowledge gaps effectively, we need to improve workflows that support scalable growth in quality content, especially in smaller language communities. | Amy Tsay |
| WE3.1 | Release two curated, accessible, and community-driven browsing and learning experiences to representative wikis, with the goal of increasing the logged-out reader retention of experience users by 5%. | This KR focuses on increasing the retention of a new generation of readers on our website, allowing a new generation to build a lasting connection with Wikipedia, by exploring opportunities for readers to more easily discover and learn from content they are interested in. This will include explorations and the development of new curated, personalized, and community-driven browsing and learning experiences (for example, feeds of relevant content, topical content recommendations and suggestions, community-curated content exploration opportunities, etc).
We plan on beginning the fiscal year by experimenting with a series of experiments of browsing experiences to determine which we would like to scale for production use, and on which platform (web, apps, or both). We will then focus on scaling these experiments and testing their efficacy in increasing retention in production environments. Our goal by the end of the year is to launch at least two experiences on representative wikis and to accurately measure a 5% increase in reader retention for readers engaged in these experiences. To be optimally effective at achieving this KR, we will require the ability to A/B test with logged-out users, as well as instrumentation capable of measuring reader retention. We might also need new APIs or services necessary to present recommendations and other curation mechanisms. |
Olga Vasileva |
| WE3.2 | 50% increase in the number of donations via touch points outside of the annual banner and email appeals per platform. | Our goal is to provide a diversity of revenue sources while recognizing our existing donors. Based on feedback and data, our focus is on increasing the number of donations beyond the methods the Foundation has relied upon in the past, specifically the annual banner appeals. We want to show that by investing in more integrated donor experiences, we can sustain our work and expand our impact by providing an alternative to donors and potential donors that are unresponsive to banner appeals. 50% is an initial estimate based on the decreased visibility of the donate button on Web as a result of Vector 2022, and the increase in the number of donations from FY 2023-2024's pilot project on the Wikipedia apps to enhance donor experiences (50.1% increase in donations). Evaluating this metric by platform will help us understand trends in platforms and if different tactics should be deployed in the future based on a difference in behavior based on platform audience. | Jazmin Tanner |
| WE4.1 | Provide a proposal of 3 countermeasures to harassment and harmful content informed by data and in accordance with the evolving regulatory environment by Q3. | Ensuring user safety and well-being is a fundamental responsibility of online platforms. Many jurisdictions have laws and regulations that require online platforms to take action against harassment, cyberbullying and other harmful content. Failing to address these may expose platforms to legal liability and regulatory sanctions.
Right now we do not have a very good idea about how big these problems are or the reasons behind them. We rely heavily on anecdotal evidence and manual processes which leave us exposed both to legal risks as well other far-reaching consequences: underestimation of the problem, escalation of harm, reputational damage and erosion of user trust. We need to build a strong culture of measuring the incidence of harassment & harmful content and proactively implement countermeasures. |
Madalina Ana |
| WE4.2 | Develop at least two signals for use in anti-abuse workflows to improve the precision of actions on bad actors by Q3. | The wikis rely heavily on IP blocking as a mechanism for blocking vandalism, spam and abuse. But IP addresses are increasingly less useful as stable identifiers of an individual actor, and blocking IP addresses has unintended negative effects on good faith users who happen to share the same IP address as bad actors. The combination of the decreasing stability of IP addresses and our heavy reliance on IP blocking result in less precision and effectiveness in targeting bad actors, in combination with increasing levels of collateral damage for good faith users. We want to see the opposite situation: decreased levels of collateral damage and increased precision in mitigations targeting bad actors.
To better support the anti-abuse work of functionaries and to provide building blocks for reuse in existing (e.g. CheckUser, Special:Block) and new tools, in this KR we propose to explore ways to reliably associate an individual with their actions (sockpuppetting mitigation), and combine existing signals (e.g. IP addresses, account history, request attributes) to allow for more precise targeting of actions on bad actors. |
Kosta Harlan |
| WE4.3 | Reduce the effectiveness of a large-scale distributed attack by 50% as measured by the time it takes us to adapt our measures and the traffic volume we can sustain in a simulation. | The evolution of the landscape of the internet, including the rise of large-scale botnets and more frequent attacks have made our traditional methods of limiting large-scale abuse obsolete. Such attacks can make our sites unavailable by flooding our infrastructure with requests, or overwhelm the ability of our community to combat large-scale vandalism. This also puts an unreasonable strain on our high privilege editors and our technical community.
We urgently need to improve our ability to automatically detect, withstand, and mitigate or stop such attacks. In order to measure our improvements, we can't rely solely on frequency/intensity of actual attacks, as we would be dependent on external actions and it would be hard to get a clear quantitative picture of our progress. By setting up multiple simulated attacks of varying nature/complexity/duration to be run safely against our infrastructure, and running them every quarter, we will be able to both test our new countermeasures while not under attack, and to report objectively on our improvements. |
Giuseppe Lavagetto |
| WE5.1 | By Q3, complete at least 5 interventions that are intended to increase the sustainability of the platform. | The MediaWiki platform sustainability is an evergreen effort important for our ability to scale, increase or avoid degradation of developer satisfaction, and grow our technical community. This is hard to measure and depends on technical and social factors. However, we carry tacit knowledge about specific areas of improvements that are strategic for sustainability. The planned interventions may help increase the sustainability and maintainability of the platform or avoid its degradation. We plan to evaluate the impact of this work in Q4 with recommendations for sustainability goals moving forward. Examples of sustainability interventions are: simplify complex code domains that are core to MediaWiki but just a handful of people know how it works; increase the usage of code analysis tooling to inform quality of our codebase; streamline processes like packaging and releases. | Mateus Santos |
| WE5.2 | Identify by Q2 and complete by Q4 one or more interventions to evolve the MediaWiki ecosystem’s programming interfaces to empower decoupled, simpler and more sustainable feature development. | The main goal of KR 5.2 is to improve and clarify the interaction between MediaWiki's core platform and its extensions, skins, and other parts. Our intent is to provide functional improvements to MediaWiki’s architecture that enable practical modularity and maintainability, for which it is easier to develop extensions, and to empower the requirements from the wider MediaWiki product vision. This work also aims to inform what should exist (or not) within core, extensions, or the interfaces between them. The year will be divided into two phases: a 5-month research and experimentation phase that will inform the second phase where specific interventions are implemented. | [TBD] |
| WE5.3 | By the end of Q2, complete one data gathering initiative and one performance improvement experiment to inform followup product and platform interventions to leverage capabilities unlocked by MediaWiki’s modeling of a page as a composition of structured fragments. | The primary goal here is to empower developers and product managers to leverage new MediaWiki platform capabilities to meet current and future needs of encyclopedic content by making possible new product offerings that are currently difficult to implement as well as improve performance and resiliency of the platform.
Specifically, at a MediaWiki platform level, we want to shift the processing model of MediaWiki from treating a page as a monolithic unit to treating a page as a composition of structured content units. Parsoid-based read views, Wikidata integration, and Wikifunctions integration into wikis are all implicit moves towards that. As part of this KR, we want to more intentionally experiment with and gather data to inform future interventions based on these new capabilities to ensure we can achieve the intended infrastructure and product impacts. |
[TBD] |
| WE6.1 | Resolve 5 questions to enable efficiency and informed decisions on developer and engineering workflows and services and make relevant data accessible by the end of Q4. | “It’s complicated” is a frequent response to questions like “which repositories are deployed to Wikimedia production”. In this KR we will explore some of our “evergreens” in the field of engineering productivity and experience - recurring questions that seem easy but are hard to answer, questions that we can answer, but the data is not accessible and require custom queries by subject matter experts, or questions that are cumbersome to get a response on for process gap or other reasons. We will define what “resolve” means for each of the questions: For some this may just mean to make existing, accurate data accessible. Other questions will require more research and engineering time to address them. The overarching goal of this work is to reduce the time, workarounds and effort it takes to gain insights in key aspects of the developer experience and enable us to make improvements to engineering and developer workflows and services. | [TBD] |
| WE6.2 | By Q4, enhance an existing project and perform at least two experiments aimed at providing maintainable, targeted environments moving us towards safe, semi-continuous delivery. | Developers and users depend on the Wikimedia Beta Cluster (beta) to catch bugs before they affect users in production. Over time, the uses of beta have grown and come into conflict—the uses are too diverse to fit in a single environment. We will enhance one existing alternative environment and perform experiments aimed at replacing a single high-priority testing need currently fulfilled by beta with a maintainable alternative environment that better serves each use case's needs. | Tyler Cipriani |
| WE6.3 | By Q2, introduce a sustainability scoring system for the Toolforge platform across a variety of technical and social factors. By Q4, improve one of its key technical factors by 50%. | Toolforge, the key platform for Wikimedia’s volunteer-built tools, plays a crucial role from editing to anti-vandalism. Our goal is to enhance Toolforge usability, lower the barriers to contribution, improve community practices, and promote adherence to established policies. To this effect, we will introduce a scoring system by Q2 to evaluate the Toolforge platform sustainability, focusing on technical and social aspects. Using this system as a guide, we aim to improve one of the key technical factors by 50%. | Slavina Stefanova |
| Signals & Data Services (SDS) Draft Key Results
[ Draft Objectives ] | |||
|---|---|---|---|
| Key Result shortname | Key Result text | Key Result context | Owner |
| SDS1.1 | Leaders of 2 programs or KR driven initiatives have produced widely shared documentation explaining the logical link between their team’s work and impact on one or more core Foundation metrics. | Our core organizational metrics serve as the basis for assessing the Foundation's progress toward its goals. As we allocate resources to programs and design key result (KR) oriented workstreams, these high-level metrics should guide how we link these investments to the Foundation's overarching goals as defined in the annual plan.
The work in this key result acknowledges that the Foundation as a whole is at an early stage in its ability to quantitively link the impacts of all planned interventions to high-level, or core metrics. In pursuit of that eventual goal, this KR aims to develop the process by which we share the logical and theoretical links between our initiatives and our high-level metrics. In practice, this means partnering with initiative owners throughout the Foundation to understand how the output of their work at a project level is linked to and impacts our core metrics at a Foundation level. Impact mapping frameworks and exercises like ‘Theory of Change mapping’ and causal graph construction will be employed to ensure consistency and rigor in documenting the potential impact of work. To execute on this key result, we will also need to develop supporting materials that help initiative owners understand organizational metrics and understand how to construct theories of change associated with their work. |
Omari Sefu |
| SDS1.2 | Answer 2 strategic open research questions by December 2024 in order to provide recommendations or inform FY26 annual planning. | There are many open research questions in the Wikimedia ecosystem, and answering some of those questions is strategic for WMF or the affiliates. The answers to these questions can inform future product or technology development or can support decision-making/advocacy in the policy space. While some of these questions can be answered by utilizing purely research or research engineering expertise, given the socio-technical nature of the WM projects arriving at trustworthy insights often requires cross-team collaboration for data collection, context building, user interaction, careful design of experiments, and more. Through this KR we aim to prioritize some of our resources towards answering one or more of such questions.
The work in this KR includes prioritizing a list of strategic open questions, as well as doing experimental work to find an answer for X number (currently estimated 2) of them. The ideal type of questions we tackle in this KR are questions that once answered can have an unlocking effect by enabling multiple other teams or groups to do (better? informed) product, technology, or policy work. We intend the work in this KR to be complementary to the following KRs:
|
Leila Zia |
| SDS1.3 | Achieve at least a 50% reduction in the average time required for data stakeholders to understand and trace data flows for 3 core and essential metrics | Required for Data Governance standards.
Tracing back the transformation and source of datasets is difficult and requires knowledge of different repos and systems. We should make it easy to understand how data flows around our systems so that data stakeholders can work in a more self service way. This work will support workflows where data is transformed and used for analytics, features, API’s and data quality jobs. There will be a follow up KR around documenting metrics. |
Luke Bowmaker |
| SDS2.1 | By the end of Q2, we can support 1 product team to evaluate a feature or product via basic A/B testing that reduces their time to user interaction data by 50%. | We think using shared tools will increase product teams' confidence in data-driven decision making, improve efficiency and productivity, and enhance product strategy and innovation.
We will look at adopting team's individual time to user interaction data baselines and improve it by 50%. We will also investigate how we can contextualize these gains in the fuller context of all product teams. We expect to learn how we can improve the experience and identify and prioritize capability enhancements based on feedback from the adopting team and results of SDS2.2. |
Virginia Poundstone |
| SDS2.2 | By end of Q2, we will have 2 essential metrics for analyzing experiments (A/B tests) to support testing product/feature hypotheses related to FY24-25 KRs. | When a product manager (or designer) has a hypothesis that a product/feature will address a problem/need for the users or the organization, an experiment is how they test that hypothesis and learn about the potential impact of their idea on a metric. The results of the experiment inform the product manager and help them make a decision about what action to take next (abandon this idea and try a different hypothesis, continue development if the experiment was performed early in the development lifecycle, or release the product/feature to more users). Product managers must be able make such a decision with confidence, supported by evidence they trust and understand.
A major hurdle to this is that product teams currently formulate their hypotheses with custom project-specific metrics which require dedicated analyst support to define, measure, analyze, and report on them. Switching to a set of essential metrics for formulating all testable product/feature hypothesis statements would make it:
We think that a set of essential metrics which are widely understood and consistently used – and informed/influenced by industry standard metrics – would also improve organizational data literacy and promote a culture of review, experimentation, and learning. We are focusing on essential metrics that (1) are needed for best measurement and evaluation of success/impact of products/features related to 2 Wiki Experiences KRs – WE3.1 and WE1.2 – and (2) reflect or map to industry-standard metrics used in web analytics. |
Mikhail Popov |
| Future Audiences (FA) Draft Key Result
[ Draft Objectives ] | |||
|---|---|---|---|
| Key Result shortname | Key Result text | Key Result context | Owner |
| FA1.1 | As result of Future Audiences experimental insights and recommendations, by the end of Q3 at least one objective or key result owned by a non-Future Audiences team is present in the draft for the following year's annual plan. | Since 2020, the Wikimedia Foundation has been tracking external trends that may impact our ability to serve future generations of knowledge-consumers and knowledge-contributors and remain a thriving free knowledge movement for generations to come. Future Audiences, a small R&D team, will:
|
Maryana Pinchuk |
| Product and Engineering Support (PES) Draft Key Results
[ Draft Objectives ] | |||
|---|---|---|---|
| Key Result shortname | Key Result text | Key Result context | Owner |
| PES1.1 | Culture of Review: Incrementally improve scores for P+T staff sentiment related to our delivery, alignment, direction, and team health in a quarterly survey. | A culture of review is a product development culture based on shorter cycles of iteration, learning, and adaptation. This means that our organization may set yearly goals, but what we do to achieve these goals will change and adapt over the course of the year as we learn. There are two components to building a culture of review: processes and behaviors. This KR focuses on the latter. Behavior changes can grow and strengthen our culture of review. This involves changes in individual habits and routines as we move towards more iterative product development. This KR will be based on self-reported changes in individual behaviors, and measuring resulting changes, if any, in staff sentiment. | Amy Tsay |
| PES1.2 | By the end of Q2, the new Wishlist better connects movement ideas and requests to Foundation P+T activities: items from the Wishlist backlog are addressed via a 2024-5 KR, the Foundation has completed 10 smaller Wishes, and the Foundation has partnered with volunteers to identify 3+ areas of opportunity for the 2025-26 FY. | The Community Wishlist represents a narrow slice of the movement; approximately 1k people participate, most of whom are contributors or admins. People often bypass the Wishlist by writing feature requests and bug reports via Phabricator, where it’s hard to discern requests from WMF or the community. For participants, the Wishlist is a costly time investment with minimal payoff. They still engage with the Wishlist because they feel it is the only vehicle to call attention to impactful bugs and feature improvements, or signal a need for broader, strategic opportunities. Wishes are often written as solutions, vs problems. The solutions may seem sensible on paper, but don’t necessarily consider the technical complexity or movement strategy implications.
The scope and breadth of wishes sometimes exceeds the scope and capacity of Community Tech or a single team, perpetuating the frustration, leading to RFCs and calls to dismantle the Wishlist. Whereas community members prefer to use the Wishlist for project ideas, teams at the Foundation look at the Wishlist and other intake processes for prioritization, in part because wishes are ill-timed for Annual Planning and are hard to incorporate into roadmaps / OKRs. The Future Wishlist should be a bridge between the community and the Foundation, where communities provide input in a structured way, so that we are able to take action and in turn make volunteers happy. We’re creating a new intake process for any logged in volunteer to submit a wish, 365 days a year. Wishes can report or highlight a bug, request an improvement, or ideate on a new feature. Anyone can comment on, workshop, or support a Wish to influence prioritization. The Foundation won’t categorize wishes as “too big” or “too small.” Wishes that thematically map to a larger problem area can influence annual planning and team roadmaps, offering strategic directions and opportunities. Wishes will be visible to the Movement in a dashboard that categorizes wishes by project, product/problem area, and wish type. The Foundation will respond to wishes in a timely manner, and partner with the Community to categorize and prioritize wishes. We will partner with Wikimedians to identify and prioritize three areas of improvement, incorporated in the Foundation’s 2025-26 Annual Plan, which should improve the adoption rate and fulfillment of impactful wishes. We will flag well-scoped wishes for the volunteer developer community and Foundation teams, leading to more team and developer engagement and more wishes fulfilled, leading to community satisfaction. Addressing more wishes improves contributor happiness, efficacy, and retention, which should generate more quality edits, higher quality content, and more readers. |
Jack Wheeler |
| PES1.3 | Run and conclude two experiments from existing exploratory products/features that provides us with data/insights into how we grow Wikipedia as a knowledge destination for our current consumer and volunteer audiences in Q1 and Q2. Complete and share learnings and recommendations for potential adoption for future OKR work in the Wiki Experiences bucket by the end of Q3. | This work is a counterpart to the Future Audiences objective, but focuses instead on uncovering opportunities to increase and deepen engagement of our existing audiences (of Wikipedia consumers and contributors) through more nimbly testing more on-platform product ideas.
It lives in PES1 as it is an energiser and multiplier - channelling the time individuals and teams have already devoted to hacking/experimenting on side projects to bring more promising features into focus. Instead of these side projects languishing (not a good use of our limited resources), this KR provides a path for some of these ideas to potentially make it into larger APP setting through proven experiments, thus more efficiently using staff time and motivating their creativity and productivity. By shepherding more of these smaller, shorter projects into play, we also diversify our spread of ‘bets’ for more learnings and trials of ideas that may transform Wikipedia in line with the changing needs and expectations of our current audiences. This will make our work more impactful and faster as it helps the foundation to align on the correct goal in less time. |
Rita Ho |
| PES1.4 | Learn how to: set, monitor, and make decisions on SLOs. Pick at least one new thing to define SLOs for as we release it. Collaborate with the respective team(s) (typically: product, development teams, SRE) to define that SLO. Reflect and document guidelines for what releases should have SLOs in the future and how to set them. | FUTURE KR:
Set up process and rudimentary tools for setting and monitoring SLOs for new releases. Report on a quarterly basis, and use it to make decisions on when to (and not to) prioritize work to fix something. Share report with the community. WHY: We don’t know when we need to prioritize work to fix something. And we have a lot of code. As this footprint continues to grow, there are more situations where we may need to decide between addressing issues or focus on innovation, and more uncertainty around when we should. Also, not clear to staff and community what our level of support/commitment on reliability and performance is for all the different features and functionality they interact with. If we define a expected level of service, we can know when we should allocate resources to it or not. |
Mark Bergsma |
Explanation of buckets
Wiki Experiences
The purpose of this bucket is to efficiently deliver, improve and innovate on wiki experiences that enable the distribution of free knowledge world-wide. This bucket aligns with movement strategy recommendations #2 (Improve User Experience) and #3 (Provide for Safety and Inclusion). Our audiences include all collaborators on our websites, as well as the readers and other consumers of free knowledge. We support a top-10 global website, and many other important free culture resources. These systems have performance and uptime requirements on-par with the biggest tech companies in the world. We provide user interfaces to wikis, translation, developer APIs (and more!) and supporting applications and infrastructure that all form a robust platform for volunteers to collaborate to produce free knowledge world-wide. Our objectives for this bucket should enable us to improve our core technology and capabilities, ensure we continuously improve the experience of volunteer editors and moderators of our projects, improve the experience of all technical contributors working to improve or enhance the wiki experiences, and ensure a great experience for readers and consumers of free knowledge worldwide. We will do this through product and technology work, as well as through research and marketing. We expect to have at most five objectives for this bucket.
Knowledge is constructed by people! And as a result our annual plan will focus on the content as well as the people who contribute to the content and those who access and read it.
Our aim is to produce an operating plan based on existing strategy, mainly our hypotheses about the contributor, consumer and content "flywheel". The primary shift I’m asking for is an emphasis on the content portion of the flywheel, and exploration of what our moderators and functionaries might need from us now, with the aim of identifying community health metrics in the future.
Signals and Data Services
In order to meet the Movement Strategy Recommendations for Ensuring Equity in Decision Making (Recommendation #4), Improving User Experience (Recommendation #2), and Evaluating, Iterating and Adapting (Recommendation #10), decision makers from across the Wikimedia Movement must have access to reliable, relevant, and timely data, models, insights, and tools that can help them assess the impact (both realized and potential) of their work and the work of their communities, enabling them to make better strategic decisions.
In the Signals & Data Services bucket, we have identified four primary audiences: Wikimedia Foundation staff, Wikimedia affiliates and user groups, developers who reuse our content, and Wikimedia researchers, and we prioritize and address the data and insights needs of these audiences. Our work will span a range of activities: defining gaps, developing metrics, building pipelines for computing metrics, and developing data and signals exploration experiences and pathways that help decision makers interact more effectively and joyfully with the data and insights.
Future Audiences
.svg)
The purpose of this bucket is to explore strategies for expanding beyond our existing audiences of consumers and contributors, in an effort to truly reach everyone in the world as the essential infrastructure of the ecosystem of free knowledge. This bucket aligns with Movement Strategy Recommendation #9 (Innovate in Free Knowledge). More and more, people are consuming information in experiences and forms that diverge from our traditional offering of a website with articles – people are using voice assistants, spending time with video, engaging with AI, and more. In this bucket, we will propose and test hypotheses around potential long-term futures for the free knowledge ecosystem and how we will be its essential infrastructure. We will do this through product and technology work, as well as through research, partnerships, and marketing. As we identify promising future states, learnings from this bucket will influence and be expanded through Buckets #1 and #2 in successive annual plans, nudging our product and technology offerings toward where they need to be to serve knowledge-seekers of the future. Our objectives for this bucket should drive us to experiment and explore as we bring a vision for the future of free knowledge into focus.
Sub-buckets
We also have two other “sub-buckets” which consist of areas of critical functions, which must exist at the Foundation to support our basic operations, and some of which we have in common with any software organization. These “sub-buckets” won’t have top level objectives of their own, but will have input on and will support the top level objectives of the other groups. They are:
- Infrastructure Foundations. This bucket covers the teams which sustain and evolve our datacenters, our compute and storage platforms, the services to operate them, the tools and processes that enable the operation of our public facing sites and services.
- Product and Engineering Support. This bucket includes teams which operate “at scale” providing services to other teams that improve the productivity and operations of other teams.