Research:Growth and diversity of Technology team audiences
Program 4 of the Technology Department's 2017-18 Annual Plan involves improving the support that the Wikimedia Foundation provides to technical communities inside and outside the Wikimedia Movement who build upon and use our software products and platforms. These users include "code contributors, documentation contributors, bug reporters, API consumers, volunteers who build innovative solutions to on-wiki workflow issues, researchers who examine the data generated by the Wikimedia projects, value-added vendors who provide services and support based on Wikimedia free and open-source software products, and true 'third parties' who install and use FLOSS software produced by the Wikimedia movement on their own computers for various reasons[1]."
Overview
[edit]Summary of findings
[edit]Key observations
[edit]- Awareness of audiences
- Tech teams are aware of their external audiences, and work hard to support them.
- Audience overlap
- There is a lot of audience overlap between teams.
- Ad-hoc communication
- Communication with external audiences is ad hoc.
- Importance of researchers
- Researchers are recognized as an important, and diverse, external audience.
- Value of offline interactions
- Offline interactions--e.g. participation in hackathons, meetups, and professional and academic conferences--are seen as effective ways of learning the needs of existing audiences and introducing Wikimedia Technology platforms to new audiences.
Challenges
[edit]- Shared platform ownership
- Shared ownership of technology platforms across teams can create ambiguity around who is responsible for external audience support.
- Limited bandwidth
- Teams have limited bandwidth for performing outreach and maintaining end-user documentation.
- Diverse communication channels
- The diversity of communication channels used to interact with audiences can hinder outgoing and incoming communication.
- Limited data
- Limited data about platform users and usage can make it difficult to track adoption by audiences.
Recommendations
[edit]The following list of recommendations is not intended to be exhaustive. It is focused on addressing the kinds of issues over which the Wikimedia Technology teams have some degree of direct control, and which can plausibly be addressed with existing resources (or near-term resource growth projections).
- Prioritization and goal-setting
- Develop a Technology-wide strategy for prioritizing audiences and formalizing growth goals on a per-audience basis.
- Documentation and communication support
- Provide teams with increased support for end-user documentation and collecting input and feedback from audience members on an ongoing basis.
- Define responsibilities and processes
- Clarify audience support responsibilities and processes, within and across teams.
- Increase offline communication
- Increase tech team participation in conferences, workshops, and events attended by current and potential audiences.
Definitions
[edit]- Technology product or platform: technological infrastructures or outputs of technology teams that are intended to be used directly by external audiences.
- External audience: Individuals or organizations that use particular products or platforms developed and/or maintained by Wikimedia Technology. The current research focuses solely on external audiences--those who are outside the Wikimedia Foundation.
- Usage: In this research, usage means more than simple consumption. Usage involves building something with or on top of a Wikimedia Technology product or platform. By this definition, Wikipedia readers are not included in the class of users we are focused on, nor are Wikipedia editors whose only interaction with Wikimedia Technology products is through directly editing the content of Wikipedia websites.
- New/potential audience: External individuals or organizations that are not currently using Wikimedia Technology platforms, but who may be interested in adopting our platforms in the future
- Adoption: the use of Wikimedia Technology platforms by new audiences for specific purposes, or the increased use of our platforms by existing audiences.
Problem statement
[edit]Outcome 2 of Technology's 2018-19 Departmental Program #4 involves measuring the adoption of Wikimedia technology by external audiences. The Technology department currently lacks a set of broadly-accepted, reliable, and meaningful measures for tracking the adoption of our technology by members of these audiences, which limits our ability to:
- measure the current and historical usage of our technology platforms outside of WMF,
- identify opportunities to reach new audiences and/or serve our existing audiences better, and
- measure adoption of Technology platforms (on a per-team or per-product basis) over time, including evaluating the impact of any future interventions we make to grow or more fully support our external audiences.
Research questions
[edit]This project is the first step towards developing these kinds of metrics. For each of the teams we work with on this project, we seek to answer the following questions:
- Which audiences are you currently serving? Who else do you believe you should be serving?
- How does your work support the audiences you currently serve? How could you serve these audiences better?
- How are you currently measuring adoption from these external consumers? If you don't, do you have any notions of size/adoption rate?
- What kinds of audiences are you serving (e.g. individuals vs institutions/organizations)? What is the geographical distribution of these audiences?
Methods
[edit]This study will consist of interviews with team leads in Tech involved in building platforms with an externally-facing audience and get them to articulate who their core audience is, and how to characterize adoption of their platforms.
At least initially, this research will not involve interviews with members of all Tech department teams. Rather, we will focus on teams that have well-defined external audiences, beyond other WMF teams/departments, and in some cases beyond the Wikimedia Movement altogether. We are conceiving external audiences for Tech teams as researchers, developers, data consumers who build other platforms or products on top of the software products (e.g. MediaWiki), research/analytics resources (including publications, specifications, dashboards, data APIs and datasets), and platforms (e.g. ORES, Toolforge, WikiData Query Service) we develop and maintain.
The anticipated output of this research will be an audience map for the audiences identified in the interviews, along with how the team that identified this audience currently communicates with and/or tracks tech adoption from this audience. Common communication channels and adoption metrics used or proposed by multiple teams will be highlighted for further investigation and evaluation as possible KPIs.
Timeline
[edit]- October 2017: Develop interview protocol
- November 2017 - January 2018: conduct interviews with team leads
- January - March 2018: Analyze and report results
Teams involved
[edit]This list includes teams for which we have already identified products or platforms that have external audiences. This list may change during the course of our study, as we refine our definitions of audiences, products, and adoption.
- Analytics
- MediaWiki Platform
- Research
- Scoring Platform
- Search Platform
- Services
- Cloud Services
Results
[edit]See Research:Growth and diversity of Technology team audiences/Report for the full report of findings.
Stakeholder matrix
[edit]Audience types | ||||||
Team | Platforms discussed | Communication channels | Researchers | Platform developers | Platform consumers | Platform partners |
---|---|---|---|---|---|---|
Research | Research papers, curated datasets, code libraries, WM data dumps, APIs, Analytics dashboards | public mailing lists, Research newsletter, research.wikimedia.org, personal correspondence, metawiki, @Wikiresearch, blog posts, conferences and lectures. | academic researchers, data storytellers, NGOs | Code library contributors | Media entities, research educators | External collaborators, Grantmaking organizations |
Scoring Platform | ORES, JADE, WikiLabels | Wiki-ai mailing list, mediawiki, village pumps, GitHub issues, IRC, Phabricator, conferences, lectures, and hackathons. | academic research analysts, model builders | direct contributors to ORES platform, developers of client applications ('local technologists'), WikiLabels campaign organizers | Orgs like Google and Kiwix who want to know which versions of an article to index | , |
Cloud Services | ToolForge, Toolsadmin, CloudVPS | public mailing lists, IRC, wikitech, Phabricator. | academic researchers, Wikimedian researchers, the 'open knowledge-curious' | Bot and tool maintainers, documenters on wikitech | ||
MediaWiki Platform | MediaWiki | MW Stakeholders Group meetings and mailing list, mediawiki, public MW mailing lists, conferences. | Researchers, research educators, data storytellers, NGOs | Volunteer extension developers, Mediawiki consultants | Semantic MediaWiki project, Mediawiki Stakeholders Group, WikiApiary project | |
Analytics | WikiStats2, RESTbase data API endpoints, datasets, EventStream, Browser Stats dashboard, Reportcard dashboard, WikiMetrics | Phabricator, wikitech, IRC, public mailing lists, personal correspondence. | academic researchers | Wikimedia movement volunteers, Wikimedia chapters & user groups, Orgs that use Wikimedia data (Google, Facebook, Microsoft | ||
Security | Security reviews, vulnerability reports | Phabricator, IRC, conferences, hackathons, personal correspondence. | Developers in the community, independent developers, GSOC participants, MW consultants. | WMDE | ||
Services | RESTbase, REST API | mailing lists, Phabricator, IRC, mediawiki. | Researchers who use the data API endpoints | RESTbase service developers | External MediaWiki users who want "the full Wikipedia Experience", Orgs like Google that use the REST API to crawl Wp | |
Search | MW Search API, WikiData Query Service, ElasticSearch | mailing lists, mediawiki, Phabricator, IRC, conferences | Wikimedian researchers ("editors who are trying to pull more information together about certain domains") | Open Source Connections (contracted to improve search models) | External MediaWiki users, organizational Search API users (e.g. Google), Orgs with SPARQL-compatible databases (e.g. museums) | WMDE, BlazeGraph |
See also
[edit]- Technology department on WikimediaFoundation.org
- Wikimedia Foundation 2017-18 Annual Plan - Technology Department programs
- Wikimedia Foundation Audience research initiative
References
[edit]- ↑ "Wikimedia Foundation Annual Plan/2017-2018/Final/Programs/Technology - Meta". meta.wikimedia.org. Retrieved 2018-03-23.