Wikimedia Foundation Report, July-September 2015

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search
Correct.svg This page is currently a draft. More information pertaining to this may be available on the talk page.

Translation admins: Normally, drafts should not be marked for translation.


See also the original PDF version of this report



Information

You are more than welcome to edit this wiki version of the report for the purposes of usefulness, presentation, etc.



Wikimedia Foundation

Quarterly Report

Wikimedia Foundation RGB logo with text.svg

July — September 2015
Q1 of the fiscal year

Foreword[edit]

We are pleased to bring you the Wikimedia Foundation’s Quarterly Report for Q1 of the 2015/16 fiscal year, a comprehensive summary of how we did on the objectives defined earlier in our quarterly goal setting process. We are continuing to optimize the report’s format and the organization’s quarterly review process based on the feedback that we have received.

This issue includes some new pieces of information and a few format changes. Teams have been starting to highlight one key performance indicator (KPI) each - with ongoing efforts to identify the best possible metrics - and to estimate how much time fell into each of the three categories from the 2015 Call to Action (strengthen, focus and experiment). We have reorganized the content to present all the information that is related to a particular objective in one place (description of the goal, measures of success, how we did on achieving the objective, and what we learned from working on it), and changed these slides to a cleaner, more effective layout.

As before, we are including an overview slide summarizing successes and misses across all teams. In a mature 90-day goal setting process, the “sweet spot” is for about 75% of goals to be a success. Organizations that are meeting 100% of their goals are not typically setting aggressive goals.

Terry Gilbey, Chief Operating Officer


2

Quarterly metrics scorecard (beta)[edit]

WikiFont uniE600 - userAvatar - blue.svg        Participation
Active editors
(5+ edits/month, July-August)
75.2k/mo -4.7% from Q4
-0.7% y-o-y
New editor activation
(top 5 Wikipedias)
6.1% +38% from Q4
+45% y-o-y
Edits via mobile
(on WP, percentage of total)
3.0% Q4: 2.0%
Q1 2014-15: 1.5%
WikiFont uniE018 - cog - blue.svg        Site reliability
Read uptime 99.942% -0.002% from Q4 (May-June)
y-o-y: N/A
Write uptime 99.935% -0.007% from Q4 (May-June)
y-o-y: N/A
Read latency
Median first paint time
1.1 seconds Q4: 1.3 seconds
Write latency
Median page save time
1.0 seconds Q4: 1.6 seconds
Wikifont uniE010 - eye - blue.svg        Readership
Page Views
Crawlers excluded
15.3B/mo -12.5% from Q4
approx. -7%  y-o-y
WikiFont uniE100 - article - blue.svg         Content
New articles 7.6k/day +14.8% from Q4
-43.5% y-o-y
Edits (in WP articles) 11.6M/mo +0.8% from Q4
+13.4% y-o-y
WikiFont uniE033 - heart - blue.svg        Fundraising
Amount raised $7.9M
(missed $8M target)
-$3.6M from Q4
-$3.6M y-o-y
On mobile 12% Q4: 10%

3

Q1 2015/16: Successes/misses by team[edit]

WMF 2015-16 Q1 successes and misses by team.svg
4

Q1 2015/16 objectives overall

133 objectives
36 teams

WMF 2015-16 Q1 successes and misses overall - pie chart.png

   81 successes (61%)
   52 misses (39%)

Misses include anything not delivered as planned, including “yellow” goals and those that were intentionally revised or abandoned due to changing priorities.


5

Community Engagement[edit]

Quarterly review

Community Engagement
Q1 - 2015/16


6


For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department


Q1 - CE Department


Objective: Strategy and Priorities

 


Objective Measure of success Status
Goal 1. FOCUS: Define overall strategy and prioritize current work into the available resources
  • Define an overall community strategy
  • All projects will appear on a community Master Project List
  • Identify gaps and map against existing capabilities


Projects on MPL, and gaps mapped (informally) as part of strategy.


Strategy defined, but did not complete all steps requested by ED (definition of each team’s mission statement, top priorities)

7


Q1 - CE Department


Objective: Strengthen community health

 


Objective Measure of success Status
Goal 2. STRENGTHEN community health
  • CA defines approach to harassment on the projects with community input and prepares Q2 broad community consultation and surveys
  • Consult with community on funding process and structures
  • Clarify membership criteria and engagement channels for Wikipedia Education Collaborative


Successful. Details in each team’s report.
8


Q1 - CE Department


Objective: internal support

 


Objective Measure of success Status
Goal 3. STRENGTHEN support of other WMF departments
  • Liaisons effectively support VE deployment
  • Liaisons effectively support creation of product development process
  • CA team develops support framework and response workflow for non-engineering teams, inc. Partnerships, Office of the ED


Product development process was not successful.


Other steps successful or deliberately reprioritized (details in each team).

9


Q1 - CE Department


Objective: Strengthen community health

 


Objective Measure of success Status
Goal 4. EXPERIMENT with new community tools and approaches
  • Investigate new tools for global ban enforcement
  • Resources develops pilot plans for Community Capacity Development Framework (see CR Goal #4)
  • Create process and roadmap for Community Tech team


Successful. Details in each team’s report.
10


Q1 - CE Department


Objective: Experiment

 


Objective Measure of success Status
Goal 4. EXPERIMENT with new community tools and approaches
  • Resources develops pilot plans for Community Capacity Development Framework
  • Create process and roadmap for Community Tech team
  • Investigate new tools for global ban enforcement


Global ban enforcement experiments were deprioritized.

Other experiments were mostly successful; details in each team’s report.

11


Q1 - CE Department


Objective: Core

 


Objective Measure of success Status
Goal 5. CORE: support existing workflows.
  • Continued support of community needs.


Successful. Details in each team’s report.
12

Developer Relations[edit]

Quarterly review

Developer Relations
Q1 - 2015/16


Approximate team size during this quarter: 3.5 FTE
Time spent: strengthen 30%, focus 10%, experiment 60%


Key performance indicator

 


Users of Wikimedia Web APIs N/A (T102079) N/A from Q4 N/A YoY
13


Q1 - Developer Relations  


Objective: Wikimania Hackathon

 


Objective Measure of success Status
Goal 1: EXPERIMENT

First Wikimania Hackathon fully integrated with the main program and the editor community


Team members involved: 2


Hackathon Showcase features at least 8 demos produced at the event in front of a mixed audience of developers and editors.


All newcomers and editors joining the hackathon get a tech savvy buddy.


23 hackathon projects showcased on Wikimania’s first official day.


About 50% of the attendance in the showcase had not participated in the hackathon.


All participants requesting a buddy got one, making a total of 22 buddy pairs.

Volunteer Siebrand Mazeland was a very helpful co-organizer supporting Rachel Farrand.

A smaller Hackathon space was available during all Wikimania, hosting small meetings and work.

Survey: 87% of participants had a positive/very positive experience, 0% negative/very negative.

Lessons learned


14


Q1 - Developer Relations  


Objective: Web APIs hub

 


Objective Measure of success Status
Goal 2: EXPERIMENT

Integrate the new Web APIs Hub with mediawiki.org


Team members involved: 2


New Web APIs bub available in mediawiki.org with basic content and coherent user experience.


Community process to propose and contribute documentation.


New Web APIs hub is ready for 3rd party developers.


A process for contributing documentation exists.


BUT Blueprint skin available only in prototype, not mediawiki.org, due to lack of UX resources and community resistance.

S Page (previously in Engineering Community) was on loan from Reading to work on this goal.

Volker Eckl (Reading Design) went beyond call of duty volunteering many fixes to Blueprint skin.

Plan to keep writing documentation based on explicit needs from developers using our APIs.

UX refresh priority demoted as long as UX resources are not committed.


15


Q1 - Developer Relations  


Objective: Gerrit Cleanup Day

 


Objective Measure of success Status
Goal 3: STRENGTHEN

Organize a Gerrit Cleanup Day


Team members involved: 1


All WMF developer teams join the Day.


All patches contributed by volunteers, and all patches from the past 3 months have at least one review.


All WMF developer teams using Gerrit participated, although with different degrees of engagement.


The queue of changesets without any review was reduced by 18% (total) and 24% (last 3 months).


BUT 752 changesets were still unreviewed (was 910), 314 from the past 3 months (was 406).

We committed to the goal of 100% without a calculator; we are still happy about the 18%-24%.

Lessons learned are on their way, but first impression is that the experience was positive and it helps sensibilizing our teams and our technical community about improving our code review practices.


16


Q1 - Developer Relations  


Objective: GSoC and Outreachy

 


Objective Measure of success Status
Goal 4: STRENGTHEN

All completed GSoC and Outreachy projects have code merged and deployed by the end of September


Team members involved: 1


All the GSoC and Outreachy interns that pass the program in August have their code reviewed and deployed by the end of September.


From the 9 projects that passed the mid-term evaluation, 6 are merged and deployed, and 3 are fully functional and showcased in Labs as planned.


It is the first time that we achieve such performance.

(blog post with details)

Hall of Fame: Jan Lebert & Cross-wiki watchlist tool, Sumit Asthana & Wikidata PageBanner extension, Alexander Jones & Flow support in PyWikiBot, VcamX & OAuth support in PyWikiBot, Tina Johnson & Newsletter extension, Vivek Ghaisas & SmiteSpam extension, Frédéric Bolduc & Graph for VisualEditor, Ankita-ks & Language Proofreading for VisualEditor, Dibya Singh & Translation Search.

Niharika Kohli (former intern, from India) started as stellar volunteer org admin and is now a contractor at the Community Tech team.

Frédéric Bolduc (ferdbold, from Canada) is now a contractor at the VisualEditor team.


17


Q1 - Developer Relations  


Objective: Wiki Loves Open Data

 


Objective Measure of success Status
Goal 5: EXPERIMENT

Establish a framework to engage with data engineers and open data organizations


Team members involved: 1


Publication of basic documentation and community processes for open data engineers and organizations willing to contribute to Wikidata.


Ongoing projects with one open data org.


Wiki Loves Open Data offers a framework and is the result of a collaboration with the Wikidata team and community, including some chapters and projects.


BUT even if WMF Strategic Partnerships and some chapters are in talks with organizations, we cannot count that as “ongoing projects” yet.

Sylvia Ventura (Strategic Partnerships) started promising talks with World Bank, OECD, and others, but the requirement for CC0 licensing is the main obstacle for quick collaborations.

The involvement of Lydia Pintscher (Wikidata), Liam Wyatt (Europeana), Susanna Ånäs (WMFI), Yair rand (volunteer developer) and Jens Ohlig (WMDE) among others has been very valuable and puts this first step in a promising direction inspired by the GLAM precedent.


18

Community Resources[edit]

Quarterly review

Community Resources
Q1 - 2015/16


Approximate team size during this quarter: 8.75 FTE
Time spent: strengthen 65%, focus 25%, experiment 10%


Key performance indicators

 


People supported


Global metrics from reports by resourced initiatives this Q

39,988

total individuals involved

9,188 new editors

(24% of total)

9,308 active editors

(24% of total)

 


Grants to

Global South

approved this Q

95 grants
$328,371
61%
of total #
66%
of total $
+533%
# from Q4
-6%
$ from Q4
+25%
# YoY
+21%
$ YoY
19


Q1 - Community Resources


Objective 1: Resources Consultation

 


Objective Measure of success Status
Goal 1. STRENGTHEN

Prioritize updates to funding programs in consultation with community


Team members involved: 8 (CR), 2 (L&E)


  • Input gathered from at least 200 community participants via community consultation
  • Public plan complete on meta-wiki with milestones for rest of year
  • Annual plans pilot process for small affiliates ready for opt-in use


245 responses from 101 countries
  • 39% Global South,  21% women, all Wikimedia projects

Outcomes reported

  • 6 milestones, implementation of 33 actionable suggestions

Annual plan simple process pilot on-track for 30 Sept launch

Success: Consultation a useful first step in building alignment. Using as a model for future consultations. Planning to run survey questions again in future to track improvement on pain-points.

Learning: Offering a private space to give feedback + usual on-wiki discussion brought in many diverse and constructive perspectives. (198 survey respondents vs 34 on-wiki)

People: Big thanks to Winifred Oliff, Chris Schilling, and Edward Galvez for ensuring high participation

Learn more: Blog, Slides on key findings


20


Q1 - Community Resources  


Objective 2: Maintain Grants Core

 


Objective Measure of success Status
GOAL 2. STRENGTHEN

Maintain full coverage for

Annual Plan (APG),

Project and Event (PEG),

Individual Engagement (IEG) grant program workflows


Team members involved: 5 (CR)


  • All PEG, IEG and APG workflows are fully staffed for H1 2014/15
  • Meeting commitments for 10 of 10 grantmaking workflows on-time
  • Kacie Harold onboarded to maintain PEG during Program Officer’s leave
    • PEG funded 4 new user groups (Korea, Latvia, Tec de Monterrey, Wikisource)
  • Marti Johnson onboarded for IEG, open call in progress, improved criteria/support for software proposals
  • APG round 1 grantees in active support, round 2 proposals on-track
    • 3 APG orgs received site visits to review programs & org capacity & tailor support

Learning: 75-95% of PO time is spent on these core workflows.


21


Q1 - Community Resources


Objective 3: Grow Wikidata

 


Objective Measure of success Status
Goal 3. STRENGTHEN

Develop plan to support Wikidata’s growth


Team members involved: 1


Plan is developed with Wikimedia Deutschland and delivered to C-team to the satisfaction of ED, COO, and VP Engineering
  • Plan for initial restricted APG grant achieved with committee and WMDE
  • Exploration of further growth developing in consultation with many stakeholders, but not yet finalized or approved by C-team.

Miss: APG PO’s core workflows haven’t left sufficient time for taking on a significant extra project like this.

Learning: FTE .25-.5 is needed for managing this project.


22


Q1 - Community Resources


Objective 4: Community Capacity

 


Objective Measure of success Status
Goal 4. FOCUS

Implement Community Capacity Development in partnership with emerging communities


Team members involved: 2

At least 3-4 emerging communities participate in framework discussions.


5 emerging communities (+2 others) engaged and indicated interest in building capacities
Scope and plan 2 pieces of community-building work for launch in Q2 2 communities identified areas for pilots, but still under discussion on-wiki. Expect pilot plans to be scoped in first 2 weeks of Q2

Success: Strong interest from communities in building partnerships, technical, communication and conflict-resolution capacities.

Miss: Planning stage, including volunteer translation of materials, continues at community’s own pace.

Learning: Respecting community process more important than WMF timeline for co-creating workplans. In-person discussions in Ukraine helped forward on-wiki interest.


23


Q1 - Community Resources


Objective: Increase Open Innovation

 


Objective Measure of success Status
Goal 5. EXPERIMENT

Define open innovation (IdeaLab) team based on available resources


Team members involved: 2

JD approved and delivered to recruiting


Community Organizer JD delivered, candidate identified, hire in progress to lead IdeaLab campaigns

Learning: In Resources Consultation, community ranked as #1 priorities:

  • connections to others
  • support for applicants

IdeaLab Community Organizer will help us provide better support in both areas.


24

Community Advocacy[edit]

Quarterly review

Community Advocacy
Q1 - 2015/16
Approximate team size during this quarter: 6
Time spent: strengthen 30%, focus 50%, experiment 20%


Key performance indicators

 


SLA for Trust & Safety correspondence: Resolving 95% of emergency@ within three hours 100% +/- 0 change from sample (100%) +/- 0 change YTD (100%)
SLA for public correspondence: Resolving 95% of answers@ and business@ within two business days 97% +/- 0 change from sample (97%) +/- 0 change YTD (97%)
25


Q1 - Community Advocacy  


Objective: Internal collaboration

 


Objective Measure of success Status
Defined pathways for communication with non-Engineering teams


Team members involved: 3


  • Assigned staff to each non-engineering department designated
  • Staff will have attended at least one meeting with that department


Objective redefined shortly after start of quarter to permit focus on shifting priorities and emerging tasks, as per instruction. CA staff continue to liaise with non-engineering departments on “as needed” basis with defined pathways but no assigned staffing.

CA created a preliminary plan for internal consultancies, but ceased developing this project in July when instructed to focus on other tasks. This could still be developed further in future, but for now “as needed” consultancies continue. We have refined and updated our process for that on our internal wiki for staff access.


26


Q1 - Community Advocacy  


Objective: Harassment strategy

 


Objective Measure of success Status
Support community health through creation of a community-informed harassment strategy.


Team members involved: 6


  • Harassment strategy project plan complete
  • Plan for broad community consultation complete
  • Community consultation and UX survey designed and out for translation.
Quarterly objectives met towards this cross-quarter goal.

CA held discussions at Wikimania, assembled small community workgroups, consulted with outside organizations, and began compiling extensive research (one, two, three) to begin preparing a survey and community consultation for Q3. Patrick and Kalli were instrumental in reaching out internally and externally to craft the consultation and survey. Their work benefitted greatly from the invaluable assistance of Edward Galvez, Abigail Ripstra and the harassment workgroup.


27


Q1 - Community Advocacy  


Objective: Global ban enforcement

 


Objective Measure of success Status
Define and explore further ideas for scalability of global ban enforcement


Team members involved: 1


  • Investigation into computer aided enforcement for global bans underway. Carry out at least three experiments for global ban enforcement (e.g. abusefilter)
  • If tenable, requirements definition complete.


Objective postponed due to shifting priorities and emerging tasks.

Learnings: Meeting the unpredictable resourcing demands for trust & safety makes committing resources for scalability experiments currently challenging. Contractor services may be useful still in achieving this important goal in future quarters, but focus has shifted to strengthening critical trust & safety processes and policies, with an emphasis to be placed on improving efficiency later.


28


Q1 - Community Advocacy  


Objective: Community communication.

 


Objective Measure of success Status
Monitor and reach out to community on various community fora.


Team members involved: 3


  • Obtain approval for channel maintenance SLA and publish on Meta.
  • Staff identified public channels 3 days a week, with a response time to appropriate conversations within 48 hours.


Objective superseded by community hub (Q2 goal)

Learnings: Difficulties in identifying and determining staffing requirements for public channels highlighted the need for a more centralized approach, which we are seeking to explore in Q2 with a community hub project.


29


Q1 - Community Advocacy  


Objective: Maintain the core.

 


Objective Measure of success Status
Maintain the core.


Team members involved: 6


  • Execute and maintain core workflows with 95% of inquiries responded to within 2 business days.


CA met KPIs related to defined inquiry paths and maintained core workflows in spite of staffing transitions.
30

Learning & Evaluation[edit]

Quarterly Review

Learning & Evaluation
Q1 - 2015/16


Approximate team size during this quarter: … 7.5
Time spent: 60% Strengthen, 20% Focus, 20% Experiment


Key performance indicators

 


Resource pageviews 61,354 Q4 Not yet available YoY Not yet available
Resource page editors 239 +99% from Q4 +149% YoY
Learning Patterns created 75 +63% from Q4 +97% YoY
Community leaders engaged 185 not applicable not applicable
31


Q1 - Learning & Evaluation   


Objective: Focus

 


Objective Measure of success Status
Goal 1. FOCUS

Identify priority workflows and processes for project mgmt.


Staff involved: 7


  • Priority research projects, workflows, KPI’s and tools mapped ✓  
  • Project charter system for all projects requiring over 5 hours of staff resourcing. ✓  view example


Complete



Learning:  Use of the “Pixar Pitch Process” to creating the Master Project List (MPL) was effective  and resulted in a more strategic approach to prioritization of staff resourcing of projects. Team members prepared a 10 minute pitch of projects with required resources and  impact statements  which were then challenged in a collaborative process. Pixar Pitch Process

Including community voice in the retreat incredibly valuable. To be expanded in the Annual Plan  via community notes process.


32


Q1 - Learning & Evaluation   


Objective: Strengthen

 


Objective Measure of success Status
Goal  2. STRENGTHEN

Encourage and support the community to better self-evaluate


Staff involved: 6.5

  • 85% of Wikimania Learning Day participants    increased skills/competency; 50 + community  conversation and consults; Virtual podcast with 200 downloads. Global Podcast ✓   
  • 1 new toolkit added on Photo Contests ✓    
  • Central Eastern European Train the Trainer Workshop Workshop Toolkits ✓  


Wikimania and online engagement targets exceeded

Report releases well received in peer and community review.  

current timeline

CCD Scoping postponed until after pilots

Learning: Partnering with Community Leaders is key to scale and teach  best and promising practices and co-author toolkits.

Decision to focus on a Train the Trainer Model and Community Knowledge Stewards.  The first one  pilot happened in Estonia.  Thanks to Jaime Anstee, Amanda Bittaker, Maria Cruz and Community Member, Nikola Kalchev.   Additional user testing on tools including translation required.


33


Q1 - Learning & Evaluation   


Objective: Experiment

 


Objective Measure of success Status
Goal 3. EXPERIMENT

Quantify the impact of grants and community projects.


Staff involved: 2


  • Global coordination of Wikimania Scholarship programs: Wikimania scholarships are built into proposals of Round 1 APG organizations  ✓
  • Conduct analysis of PEG and IEG programs, measuring content-related impact and exploring & measuring qualitative impact (e.g. dev. of people, programs)  ✓


Complete

FY 13-14 Impact Analysis completed

Proposal created for grantees to plan for Wikimania Scholarships



Successes & Learning:

-Survey Specialist, Edward Galvez has jumped into his new role with attention to streamlining foundation surveys and avoiding “Community Survey Fatigue”.  Edward is also doing an inventory of all WMF survey tools, creating a survey resource center for community and WMF.


-Scoping of a solution to fix to pain of gathering Survey Metrics by volunteers and staff  completed by Amanda  Bittaker.

 


34


Q1 - Learning & Evaluation   


Objective: Measure

 


Objective Measure of success Status
Goal 4. MEASURE

Set-up large scale data collection system to measure community health and activities


Staff involved: 4


  • Run Annual Evaluation Pulse Survey (Target: Increase responses by 25% to 122 completed) ✓
  • Launch Phase 1 Community Health Research with a series of internal/external interviews as part of the community health research. Virtual engagement strategy around topic of community health (See Appendix) ✓
  • Run Wikimania Scholarship Survey ✗


Complete in conjunction with expanded Community Consultation.

Health Phase 1 Research Launch-DC Conference Health Clinic

Wikimania Scholarship Survey postponed to 2016.

Successes and Learnings: Haitham Shammaa is leading the research work into community health. This includes curating practices, research and initiatives across communities and by other WMF teams.


35


Q1 - Learning & Evaluation   


Objective: CORE

 


Objective Measure of success Status
Goal 5. CORE

Continue development of the online resources and tools, and tracking of community activities interacting with those resources.


Staff involved: 7.5


  • Track, monitor and increase use of Evaluation Portal and Tools  (Increase by 5%)
  • Strategic virtual meet-up sessions (1-2 monthly) ✓  
  • Strategic Communications plan(via the comms form) and executed for all key community research, tools and key community engagement activities ✓  


Complete --Data inserted at the end of the quarter.



Success & Learning:  Just in Time Video Training is important for busy volunteers. This quarter, the team launched “bite size videos” which review Wikimedia programs highlights in under 3 minutes.  Look for future innovations in shared learning  as we work to make program data actionable and use more video for Wikimedia Learning. https://www.youtube.com/watch?v=5ER0YCdRDyk

Communications Liaison, Maria Cruz leading both user testing of tools and exploration of best practices in community knowledge.sharing (peer to peer learning)


36

Wikipedia Education Program[edit]

Quarterly review

Wikipedia Education Program
Q1 - 2015/16


Approximate team size during this quarter: 3.8 FTE
Time spent: strengthen 70%, focus 25%, experiment 5%


Key performance indicator

 


Interactions with program leaders worldwide 187 individuals in 61 countries + 110% individuals from Q4

+ 53% countries from Q4

YoY n/a
37


Q1 - CE / Education  


Objective: Maintain core & improve Collab

 


Objective Measure of success Status
Maintain the core


Team members involved: 4


Maintain resources, tools, mentoring, support and communications of education programs worldwide (see list of 20 main workflows on office wiki page) On track for mentoring, support and communication.

Tech tools require additional WMF support.

Strengthen the Education Collaborative to form the frontline of education mentorship, fostering personal global relationships


Team members involved: 2

Improve Collab model:
  • Better define membership criteria
  • Assess WMF needs and Collab capacity

Start planning meeting for fall 2015:

  • Determine date & location
  • Assess which members can attend
On track for improvements to Collab model and membership.

On track to determine date, location and attendance criteria.

38


Q1 - CE / Education  


Objective: Strengthen Arab world programs

 


Objective Measure of success Status
Strengthen Arab world education programs


Team members involved: 1


  • Embed program in Egypt at administrative and professorial level
  • Fall pilots are ready to start (Oman, Palestine)
  • Egypt: Education activities supported through User Group
  • Oman and Palestine pilots on track
  • Integrate current programmatic activity in Jordan with newly established UG
In conversation with stakeholders and leaders

Oman: Working on building a strong pilot program by connecting three different groups: government sponsors, classroom professor and community members.

Jordan: Interest by different parties to join forces, but more conversation is warranted.


39


Q1 - CE / Education  


Objective: Refine data gathering

 


Objective Measure of success Status
Refine data gathering efforts to produce more, better and consistent data


Team members involved: 1

At least 10 programs report metrics * 16 programs reported metrics for Wikimania session

Highlights from initial data gathering efforts:

  • + 12,500 students
  • + 1,700 teachers
  • + 178 volunteers
  • + 70 universities
  • + 122 secondary schools
  • + 21 primary schools
  • + 5 adult schools
  • + 12 Wikimedia projects
  • + 21 languages
  • + 88,000 articles created
  • + 1,500 articles translated
  • + 925 articles improved
  • + 4,500 files uploaded
  • + 283 million bytes added
40


Q1 - CE / Education  


Objective: Engagement in regional meetings

 


Objective Measure of success Status
Regional meetings as a venue for sharing experiences, improving learning and fostering personal connections.


Team members involved: 3


Wikimania:
  • Engaging sessions for 40 - 60 attendees of education pre-conf
  • Host 4 sessions during pre-conference
  • Host and facilitate 2 education-related sessions at Wikimania
  • 70% CE booth coverage during conference hours by Ed team
  • 60 attendees to pre-conference (>90 sign-ups)
  • More than half very satisfied with pre-conference program
  • Almost 50% of attendees were Mexican
  • Facilitated 2 education related sessions
  • ~75% booth coverage by team

What we learned:

  • Due to the language gap between English speakers and Spanish-only speakers and an experience gap, it was hard to keep everybody equally interested. Next time: plan ahead on dividing up the groups in smaller groups by experience level, and pre-plan different language tracks, or arrange for translation.
41

Wikipedia Library[edit]

Quarterly review

WIKIPEDIA LIBRARY
Q1 - 2015/16


Approximate team size during this quarter:  2.6 FTE
Time spent: strengthen 48%, focus 36%, experiment 16%


Key performance indicator

 


Accounts distributed (total) 4707 +8.4% from Q4 +45% YOY

 


Global branches (total) 11 +37% from Q4 +267% YOY
42


Q1 - Wikipedia Library  


Objective : Editor Access

 


Objective Measure of success Status
GOAL 1: STRENGTHEN

Enhance editor access to research with new major partners and by extending the reach of those accounts


Team members involved: 3


# Add 8 partners (at least 4 non-English) ✗
  1. 300 accounts distributed, 75 new users ✓
  2. Add 1 new coordinator contacting publishers ✓
  3. Improve citation-impact data and plan ✓
  4. Signup efficiency metrics ✓


# 7 partners (4 non-English)
  1. 370 accounts and 120 new users
  2. Added 6 contacting publishers
  3. Created plan: next step is dependent on WMF analytics
  4. Ran metrics: avg is 27 days


Quite good: picked up 7 partners at ALA conference including largest donation *ever* from EBSCO, not reflected in partner count metric. We have most major English publishers and global outreach is still ramping up. Can’t yet query per-user citation, our most critical performance indicator. Signup speed and more accurate metrics are our major motivators for building the Library Card Platform.


43


Q1 - Wikipedia Library  


Objective: Global Branches

 


Objective Measure of success Status
GOAL 2: FOCUS

Continue scaling the English Library model by starting-up unique, locally-run global branches


Team members involved: 4


# Contact with 20+ interested language leaders ✓
  1. 6 new consultations, 4 new page setups ✗
  2. Advance activity 1 step with 8 existing branches ✓
  3. Deploy TWL-lite model for 2 emerging communities ✗
  4. 14 total TWL branches with pages ✗


# Success, robust Wikimania engagement
  1. 7 consultations, 3 new page setups (Catalan, Finnish, Vietnamese)
  2. Maintained active engagement with all existing branches
  3. Deployed on Vietnamese
  4. 11 total TWL branches with pages


Wikimania was fantastic for building connections and momentum, but movement on branch development was greatly delayed by long summer holidays.  Expect ramp-up in Q2 with focus on Spanish, Italian, Hebrew, and Dutch.


44


Q1 - Wikipedia Library  


Objective: Curate Innovation

 


Objective Measure of success Status
GOAL 3: FOCUS

Acts as a best practices hub by expanding and disseminating the Global Branch Project Menu


Team members involved: 2


# Place 5 new visiting scholars and complete transition of US program to Wiki Ed ✓
  1. Wrapup Interns Class and handoff to coordinator ✓
  2. Document new project models from Wikimania ✓
  3. Continue to consult on WVU Library WIR Grant ✓


# Successful transition to WikiEd
  1. Interns wrap-up successful, transition strategy in place including Volunteer Engagement plan
  2. Documentation in progress
  3. Done


Our idea of being a clearinghouse for global Wikipedia best practices has solidified, engaging volunteers from international networks in consultation and documentation. Our high-level talks with major library IFLA (International Federation of Library Associations) and ARL (Association of research Libraries) are quickly advancing. Piloting ALISE library education reference-desk participation and reference videos project.


45


Q1 - Wikipedia Library  


Objective: Expand Network

 


Objective Measure of success Status
GOAL 4: FOCUS

Extend our network of influence with universities, libraries, consortia, publishers, and research leaders


Team members involved: 3


# Present 2 talks and host meetup at Wikimania ✓
  1. GLAM leaders feedback on WMF role description ✓
  2. Host 3 more WMF Partnerships Brunches ✓
  3. Complete draft of WMF partnership guideline ✓


# Presented 5 talks, hosted TWL Wikimania meetup
  1. Distributed GLAM strategy
  2. Held 3 successful meetings
  3. Draft posted on Office Wiki; pilot ongoing internally for next 1-2 quarters


Increasingly shifting our networking role towards large scale international library networks (International Librarians Network, IFLA, ARL, SPARC). Major wins in sending volunteers to conferences; we’re planning 4-affiliate attendance at Frankfurt Book Fair and volunteer attendance at Arabic Publishing Conference.  Advising WMF on Sustainable Development Goals overlap with libraries network, with growing potential to collaborate with large NGOs.


46


Q1 - Wikipedia Library  


Objective: Develop Tools

 


Objective Measure of success Status
GOAL 5: EXPERIMENT

Build for a more efficient, robust, inter-connected TWL platform for the future


Team members involved: 4


# Identify technical coordinator and/or contractor ✗
  1. Begin coding library card platform ✗
  2. Complete alpha ‘reference tooltip literacy guide’ ✓
  3. Complete Template:Cite archive template ✗


# Call for developers was extended: 7 total applicants, 3 to be interviewed in October
  1. Delayed by developer hiring
  2. Done, and consulted with Readership team on next steps
  3. Slower progress due to time reduction of CA-embedded project lead


Good progress here despite miss.  We had 7 very strong applications and are interviewing 4 in early October.  We received excellent consultations from Asaf, Dario, and Jon Katz with regards to the development of the Library Card Platform and mapped out a fully-featured 4-phase roadmap.


47

Community Liaisons[edit]

Quarterly review

Community Liaisons
Q1 - 2015/16
Approximate team size during this quarter: 7 FTE (6 FT, 2 PT)
Time spent: Strengthen 60%, focus 30%, experiment 10%


Key performance indicators:

 


Individual contributors with 2+ edits in product pages Q1/J - 887

Q1/A - 894

Q1/S - 898

N/A from Q4 N/A YoY
Page Views Q1/J - 29,887

Q1/A - 22,773

Q1/S - 21,173

N/A from Q4 N/A ToY
48


Q1 - Community Liaisons


Objective: Defining process

 


Objective Measure of success Status
Define and integrate community engagement process into Engineering


Team members involved: 8


  • Clear definition of CL Tools and activities within product development, including
    • refined Workflows definitions and communication channels/SLAs
  • Adding CL projects into MPL documentation


  • Product development and community process still evolving through Q2.
  • CL projects have been submitted to the MPL
  • Most CL initiatives are part of product team MPLs, and the FTEs are counted through product team line items.
  • Learning: Product process and CL’s tools within it is an integrative process taking longer than anticipated. It is currently in-flight, and has been added to CL’s Q2 goals
49


Q1 - Community Liaisons


Objective: Supporting VisualEditor Launch

 


Objective Measure of success Status
Finish launch of VE to new account holders at enwiki


Team members involved: 4


Support VE rollout to enwiki (RfC, enwiki feedback channels, triaging requests) Discussion at en.wp concluded with consensus to begin slow roll-out to new accounts. Initiative completed.
  • Discussion did not become a formal RfC.
  • Learning: Announcing A/B announcement/testing/follow-up and results followed careful product process, which may have been helpful in holding a successful conversation.
  • Sherry, Erica and Nick were integral to advising on and executing discussions around this discussion and launch.
50


Q1 - Community Liaisons


Objective:  Wikimania session

 


Objective Measure of success Status
Complete Wikimania session with new ideas for community collaboration in product development


Team members involved: 2


  • Community roundtable at Wikimania
  • Follow up report and conversation on-wiki
  • Ideas are realistic possibilities within product development methodology


  • Roundtable included about 20 community members, notes uploaded.
  • 2 users engaged in conversation onwiki, one of whom attended the event.
  • Did not meet objective due to low turnout in on-wiki conversation.
  • Learning: Immediate posting (prior to end of Wikimania) and more promotion of conversation online may have increased engagement in conversation.
  • Requested during session were more clearer communication channels with Foundation, product prioritization inclusion, recruitment of more volunteers to help, and considering language localization whenever possible.
51


Q1 - Community Liaisons  


Objective: Adding Tech/News to workflows

 


Objective Measure of success Status
Adoption of Tech/News into Community Liaison tasks


Team members involved: 1


Intake of Guillaume’s weekly Tech/News, incl. any needed modifications/ adjustments Completed intake of workflow by first week of August.
  • Tech/News is translated to 15 languages weekly, has 430 individual subscribers and 71 project page subscriptions (including en.wp’s Technical Village Pump), and is featured weekly in The Signpost.
  • Johan successfully adopted weekly workflow and collaborates with tens of volunteers who help with translating and making the information accurate and accessible every week.  
52


Q1 - Community Liaisons  


Objective: Core Workflows

 


Objective Measure of success Status
Maintain presence in product teams that CLs are currently assigned to


Team members involved: 7


  • Active filing and updating reporting for products that CLs work with
  • Providing designers, PMs, eng with product feedback


  • 233 Phabricator tickets filed for/on behalf of teams
  • Teams assigned into: VE, Flow, Comm Tech, Reading, Analytics
53

Discovery[edit]

Quarterly review

Discovery
Q1 - 2015/16


Approximate team size during this quarter: … 12 FTE
Time spent: strengthen 40%, focus 40%, experiment 20%


Key performance indicators

 


User satisfaction Start Q1: -- End Q1: 15% -- YoY
Zero Results Rate Start Q1: 33% End Q1: 33% -- YoY
54


For more detail, see the slide deck and minutes from the quarterly review meeting of the Discovery department


Q1 - Discovery


Objective: Reduce zero results rate

 


Objective Measure of success Status
Enhance the experience for our users that search by reducing the number of queries that return zero results.


Team members involved: 7


Zero results rate cut in half, from 25% to 12.5%.


No decrease in user clickthrough rate from search results.


Zero results rate unchanged.


Actual human zero results rate is around 12% for full text search, and 33% for prefix search. (research)

  • Automata, like crawlers, spiders and Wikimedia bots, account for sometimes up to 30% of total search traffic with up to 80% zero results rate, and fluctuate wildly. (research)
    • Action: need to represent this on our dashboard. Better infrastructure is needed. (task)
  • Created A/B test infrastructure and ran around five tests on search parameters; none were better than the defaults.
  • We’re trying something more radical: a complete replacement for prefix search. Work still ongoing, as the new system needs a lot of validation that it supports existing use cases.
55


Q1 - Discovery


Objective: Production beta of Wikidata Query Service

 


Objective Measure of success Status
Allow users on our cluster to run arbitrary queries on the data contained in Wikidata by deploying the Wikidata Query Service.


Team members involved: 3


Wikidata Query Service is deployed and usable from within our cluster.


Wikidata Query Service keeps with Wikidata update stream.


Define metrics and KPIs for service.


Display metrics and KPIs on Discovery Department dashboard.

Launched 7th Sept 2015.
  • Usage is ~1000 SPARQL queries per day.
  • What we do next is dependent on usage metrics and user input.
56


Q1 - Discovery


Objective: Production beta of Maps Tile Service

 


Objective Measure of success Status
Allow users on our cluster to build features using maps by deploying a Maps Tile Service.


Team members involved: 5


Wikimedia Maps Tile Service is deployed and usable from within our cluster.


Define metrics and KPIs for service.


Display metrics and KPIs on Discovery Department dashboard.

Launched 17th Sept 2015.
  • Usage is on on our dashboards.
  • What we do next is dependent on usage metrics and user input.
57


Q1 - Discovery


Objective: Quantitative search satisfaction KPI

 


Objective Measure of success Status
Further our understanding of whether our search is giving our users relevant results by finding, implementing, and deploying a quantitative metric to measure user satisfaction with search.


Team members involved: 4


Define and communicate search satisfaction metric.


Implement data collection to measure metric in production.


Visualise metric on Discovery Department dashboard.


Iterate until metric can be used as a KPI for Discovery Department in Q2 2015-16.

Launched on 30th Sept 2015.
  • Success: Public dashboarding has been a big success, very positively received.
  • Learn: Metric needs qualitative validation and testing for further iteration.
  • People: Oliver and Mikhail have trained other teams and organisations on analysis and dashboarding.
58

Reading[edit]

Quarterly review

Reading
Q1 - 2015/16


Approximate team size during this quarter: 27 FTE
Time spent: strengthen 60%, focus 20%, experiment 20%


Key performance indicator

 


Global Pageviews (provisional*) 15.3 B -12.4% from Q4 approx. -7% YOY
* cross project, across desktop/mobile, across web/apps


59


For more detail, see the slide deck and minutes from the quarterly review meeting of the Reading department


Q1 - Reading  


Objective: Apps Engagement

 


Objective Measure of success Status
Apps:

Increase engagement via improved link preview (Android)


Team members involved: 5

10% increase in Android app pageviews + link previews:

(Full Pageviews + TextExtracts retrievals with link preview feature) >= 1.1 × (Pageviews prior to link preview)

Objective as stated has been achieved (increase of about 16%), but we didn’t see nearly the amount of increased engagement we were expecting in Production.
  • The Beta app should not be relied upon as a consistent indicator of user behavior in production (although the other metrics surrounding link previews are similar between beta vs. prod).
  • Differences between Beta / Production tests indicate more opportunities for testing / improvements to user experience / better onboarding of users to this feature.
  • Great press after strong collaboration with Communications (see appendix for details)
60


Q1 - Reading  


Objective: Web Engagement

 


Objective Measure of success Status
Web:

Improve engagement via page load time decrease (mobile)


Team members involved: 3


Time to first render on mobile does not take more time than desktop (as measured by graphite), given equal network speed


Objective achieved! (data in appendix). The impact was partially driven by a change made by the Performance team that mobile web was able to benefit from due to some changes made last quarter.  
  • Though it is hard to verify, there is some directional evidence to suggest that increased speed led to traffic boosts.  We saw a ~5-10% increase in mobile web traffic in all continents except North America, but there is no evidence that it is related.
  • Ultimately it is hard to nail down the impact of changes on users without being able to measure users or sessions.
  • We benefit from dashboards and need checks to prevent regressions!  Already caught and reverted some site-slowing changes made by another team.  
  • We are working with the performance team to ensure maximum ROI on additional speed improvements.
61


Q1 - Reading  


Objective: MediaWiki Security

 


Objective Measure of success Status
Infrastructure:

Improve MediaWiki security and extensibility by updating authentication framework


Team members involved: 3

Creation of a metrics dashboard to track improvements to system.


Dashboard has been created.
  • Implementation of AuthManager is ongoing. These are developer/sys admin facing changes that will increase the configurability of the on-wiki authentication process.
  • When work finishes in Q2 the WMF will be able to start planning follow on work (e.g. two-factor auth for Stewards, password hash storage and verification service).
  • Graphite infrastructure largely unowned and nearing breaking point for adding new measurements
  • Operational monitoring in general is an ad-hoc effort by individuals and not a core service owned by WMF teams
62

Advancement Team[edit]

Quarterly review
Advancement Team
Q1 - 2015/16


Approximate team size during this quarter: 18 FTE
Time spent: strengthen 40%, focus 40%, experiment 20%


Key performance indicator

 


Amount Raised $7.9 million -$3.6 million from Q1 2014 -$3.6 million YoY
For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department
63


Advancement - Fundraising (general)[edit]

Q1 - Advancement - Fundraising  


Objective: $8 million

 


Objective Measure of success Status
FOCUS- Raise $6 million in Online campaigns and $2 million from Foundations and Major Gifts


Amount raised $2.2 million raised from Major Gifts; $5.7 million raised from the online team.  We ended $100K under due to moving the Italy campaign to October to support Wiki Loves Monuments.
STRENGTHEN - Reintegrate Amazon Payments


Delivering a high functioning product on-time


Done
EXPERIMENT - Collaborate with the Readership Team to test the use of banners to direct traffic to the app in Finland


# of app downloads; Learning how this converts to active users


Done

OPTIONAL: Objective details (screenshots, explanations), successes and misses, learnings


64

Online Fundraising[edit]

Q1 - Online Fundraising


Other successes and misses

 


Objective Measure of success Status
2014-15 Fundraising Report


Report publicly posted Donehttps://wikimediafoundation.org/wiki/2014-2015_Fundraising_Report
Weekly banner tests 20% improvement from July Done
Community & WMF campaign coordination High quality campaigns running on the site -- with metrics and coordinated scheduling We want to work with the community to create a more clear process. We are looking to hire someone to help.
Test Drive ChargeBack Alert Service Test service and assess impact Done
Fraud Research Project Finding improvements on fraud filters to scale back false positives Done & to be continued
65

Fundraising Tech[edit]

Q1 - Fundraising Tech  


Objective: Make Q2 Fundraising Totals Possible

 


Objective Measure of success Status
STRENGTHEN - Amazon upgrade We do not lose Amazon as a payment processor when they deprecate our original integration in Q2. Done
STRENGTHEN - PCI gap analysis and Improvements Increase our level of PCI compliance to SAQ A-EP in time to run a campaign in France in October.


DEFERRED (Q3)
FOCUS - Prepare for Big English fundraiser Complete all fundamental restructuring and initial deployment of critical components for the December fundraiser in time to begin fundraising code freeze process on October 1 Done
66


Q1 - Fundraising Tech


Other successes and misses

 


Objective Measure of success Status
STRENGTHEN - Worldpay “Enhanced Silent Post” Technically capable of running a campaign in France, in October, via new integration type. Done
STRENGTHEN - Astropay Integration Support fundraising creative team in running a successful Brazil campaign for the first time in two years, via new payment integration. Done
STRENGTHEN - Hired CRM expert (contractor) Hire and onboard a team member who is an expert in the CRM used by the online fundraising team (CiviCRM). Done
EXPERIMENT - Banner History project Able to collect and analyze new information about the total banner experience, from the perspective of individual donors In Progress
67

Major Gifts & Foundations[edit]

Q1 - Major Gifts & Foundations


Other successes and misses

 


Objective Measure of success Status
Implement new event registration & payment system


System ready for use/being used Done
Onboard new staff Staff up to speed and contributing at or close to 100% capacity Done
Complete profiles of all $10k+ donors Profiles of all $10k+ donors created and entered into CiviCRM In Progress
68

Strategic Partnerships[edit]

Q1 - Advancement - Strategic Partnerships  

 


Objective Measure of success Status
Goal 4. EXPERIMENT - Secure preinstall deals with a commitment to install 1 million apps; Secure one deal in the Global South


# of app preloads; Learning of how this converts to active users (# of sessions) We are in process of finalizing the contract for our first app pre-install deal.  
Goal 5. FOCUS -             (A) Secure at least two new Zero Rating Deals.

(B) Have our app featured by Apple, Google, and/or Amazon in their app stores.


Number of readers covered and pageviews resulting from the deals. Number of new app downloads when featured


Done.  

(A) The team has secured four new Zero rating deals this quarter.  (B) App was featured in Google Play’s “Back to School” promotion and they have nominated us for “New and Updated”.

69


69

Editing[edit]

Quarterly review

Editing Department
Q1 - 2015/16


Approximate team size during this quarter: 32 FTE
Time spent: strengthen 70%, focus 20%, experiment 10%


Key performance indicator (provisional metric; * – N.B. data for Q1 is for the first two months only)

 


Monthly active editors on all wikis 75.2k average in Q1* -4.7% from Q4 (78.9k) -0.7% YoY (75.7k)
70


For more detail, see the slide deck and minutes from the quarterly review meeting of the Editing department

Collaboration[edit]

Q1 - Collaboration Team  


Objective: Flow opt-in

 


Objective Measure of success Status
Make Flow available to communities that want it, reduce deployment burden


Team members involved: 8

Release opt-in Beta feature for users to enable/disable Flow on their user talk page


Launched on MediaWiki.org on 29 September, coming to various language Wikipedias in early October

Flow on user talk (Beta feature screenshot).jpg

During this quarter, we manually converted on request about 90 user talk pages on French, Catalan and Hebrew Wikipedias, Wikidata, and MediaWiki.org. To satisfy the growing user demand for Flow talk pages, we released the opt-in feature, which will be enabled on wikis that want to use it.


71


Q1 - Collaboration Team  


Objective: LQT de-deployment

 


Objective Measure of success Status
Reduce the department’s technical debt


Team members involved: 3

Convert existing uses of LiquidThreads to Flow, where the community is willing MediaWiki.org converted in mid-August, se.wikimedia in late September, pt.wikibooks TODO

Flow topic on wikimedia.se (screenshot).jpg Flow topic on Wikilivros (screenshot).jpg
pt.wikibooks could not be converted due to a technical issue with Portuguese translations


72


Q1 - Collaboration Team  


Objectives: Technical debt

 


Objective Measure of success Status
Align with UI standards


Team members involved: 7

Convert common UI elements in Flow to use OOjs UI Editor system rewritten with OOjs UI, other things converted where relevant.
Remove urgent blocker to Operations


Team members involved: 3

Switch Flow to work with the “External Storage” cluster’s compression needs Collaboration Team’s script ready in early August, no further blockers.

We were the first team to convert large, existing features to OOjs UI, instead of being written in it from the start. We learned that OOjs UI isn’t yet ready to be used on every page; trying to do so caused performance problems. We’re working with the Performance team to find a solution.


73

Language Team[edit]

Q1 - Language Team  


Objectives: CX availability and adoption

 


Objective Measure of success Status
Move Content Translation out of beta feature for a subset of wikis.


Team members involved: 3

In Wikipedia in at least one language Content Translation is enabled by default.

A conversion process from beta status to default status is defined.

De-prioritised to attend to high priority bugs being reported after wider deployment last quarter.

Wider availability surfaced several bugs and the added responsibility to ensure better uptime of the tool.

  • Learning - The need to consider the follow-up work created for patrollers and support them better by generating cleaner wikitext. This would mean better handling of tags, templates etc. from within the tool.
  • Learning - Ensuring reliability of the platform can be a challenge as the tool is now available to more users. Besides quick responses, we need to invest time to identify interruptions like publishing failures, monitor closely and  provide long term fixes.
74


Q1 - Language Team  


Objectives: CX availability and adoption

 


Objective Measure of success Status
Engage with the translators to keep them translating helped by notifications appreciations, campaigns and article suggestions.


Team members involved: 3

At least 33% of the users who published anything with CX, made at least 5 translations.


At least 20,000 articles are published using CX.

50% achieved:

✘ About 11% of users with 5+ translations

✔ 20,000+ articles created at the end of the quarter

(see appendix for details)

[[Image:]]

Article suggestions were made available towards the end of the quarter in 11 Wikipedias and it is too early to be able to measure the impact. (See image for the 1st week.)

  • Learning -  Cannot measure the impact of features completed late in the quarter. For Q2 we will adjust the metrics to practical sub-goals mapped to delivery schedule for features.
75


Q1 - Language Team  


Objectives: CX technical improvements

 


Objective Measure of success Status
Translation tool enhancements for better links, reference, dictionary support and provide the parallel corpora to enhance MT engines.


Team members involved: 3

Parallel corpora implemented, so that third party machine translation developers can use the output of users of Content Translation.


Access to at least 3 more dictionaries implemented.

Deprioritized due to no immediate requests from 3rd parties and to focus on high priority bugs

✔ Increased machine translation support through Apertium for more languages

Have an initial exploration for mobile support


Team members involved: 3

An initial exploration plan written and reviewed. Available at: https://www.mediawiki.org/wiki/Content_translation/Mobile_exploration
76

Multimedia Team[edit]

Q1 - Multimedia Team  


Objectives: In-edit-upload and graph features

 


Objective Measure of success Status
Uploading within the visual editor


Team members involved: 3

Users can upload files within the editor without abandoning their edit Not completed by EOQ; done but not live by time of QR. Follow-on work forthcoming in Q2.
Reduce performance drag of multiple technologies


Team members involved: 3

UploadWizard is completely converted to OOUI with no loading of jQuery UI code, technical debt Completed for main-sequence experience; one edge case tool needs a single library, for Q2.
Graph integration within the visual editor


Team members involved: 1

Users can modify charts from the Graph extension inside the editor


[GSoC mentored project.]

Completed in early September. Led to contracting the GSoC student for further work on this.
77

Parsing Team[edit]

Q1 - Parsing Team  


Objectives: Support VisualEditor roll-out

 


Objective Measure of success Status
Fix VE blocker bugs


Team members involved: 3

Zero Parsoid-related VE deployment blocker bugs in Phabricator All VE blocker bugs resolved.

Reduction in <nowiki> insertions.

  • Monitored VE feedback page and promptly addressed Parsoid-related issues.
  • Promptly addressed bugs that affected VE deployment or reception.
  • Strong focus on addressing <nowiki> complaints (making fixes to VE code where necessary)
    • Current status: about 1 in 100 VE edits has <nowiki> added to them (for fr, it, he, en)
    • # edits / day with nowiki additions
      • fr: 30 → 21; it: 15 → 12; he: 8 → 4; en: 14 → 40
      • fr, it, he: Noticeable reduction in # edits / day that had nowikis added
      • en: Increase because of VE rollout (anecdotally, many due to use of wikitext in VE mode), but it is still 1/ 100 edits that get a <nowiki>
78


Q1 - Parsing Team  


Objectives: Improve parsing performance

 


Objective Measure of success Status
Improve parsing latency


Team members involved: 1

Speed up PEG tokenizer by 25% Change in parse latencies (†)

✔  mean : 0.95s → 0.81s

✔  p95    : 3.5s → 2.5s

✘   p99    : 49.5s → 52.9s

PEG tokenizer is just one part of entire parse pipeline => it definitely sped up more than 25%

Parsing latency:

  • We forked an upstream library (pegjs) and improved its performance; deployed on 22 July.
  • Load on Parsoid cluster depends on edit activity of templates (and can vary quite a lot at times)

† – Picked 3-week period before/after 22 July (when fix was deployed) to measure impact, but somewhat difficult to accurately correlate perf changes with deployment, because of load variability.


79


Q1 - Parsing Team  


Objectives: Parsoid HTML for read views

 


Objective Measure of success Status
Reduce Parser : Parsoid HTML rendering diffs


Team members involved: 3

At least 10% test pages render with pixel-perfect accuracy. (4.6% on 6/30/15)


At least 75% render with "< 1% diffs" (63% on 6/30/15)

✔ 16% test pages render with pixel-perfect accuracy

✘  63% test pages render with   “< 1% diffs”; lot of false positives in the other 37%

  • Fixed several bugs that brought Parsoid’s rendering closer to PHP parser’s rendering.
    • Goal did not get sustained focus -- bug fixes for VE, Flow, CX often help this indirectly.
    • Q4 2014/15 work on CSS-based customization of Cite helped as well.
  • Worked on visual diffing infrastructure to analyze rendering differences
    • Generalized it for use in other scenarios (testing correctness of the Parsoid Batching API, testing impact of replacing Tidy in core parser with HTML5 tools)
  • Insufficient work on visual diffing infrastructure towards this goal
    • Lot of false positives because of rendering-irrelevant pixel shifts.
    • Test set very small (~750 pages) and needs to be expanded and updated.
80

VisualEditor Team[edit]

Q1 - VisualEditor Team  


Objective: English Wikipedia deployment

 


Objective Measure of success Status
Successfully complete deployment of the visual editor on the English Wikipedia


Team members involved: 5

The visual editor is enabled for all new accounts on the English Wikipedia


New account availability ramp-up started post-Wikimania, reached 100% on 1 September.
  • Uptake varies widely between wikis with otherwise-identical configurations; we do not know why
  • Typically 16% of article edits are made with VisualEditor on big wikis where it’s available to all
    • French 18.0%, Italian 16.5%, Polish 14.3%, Russian 16.1%, Hebrew 15.2%
    • Outliers: Portuguese is 34.0%; Swedish is 11.3%.
  • Typically 0.7% of article edits are made with VisualEditor on big wikis where it’s not
    • Spanish 0.9%, German 0.8%, Dutch 0.5%
  • English is now at 3.9%, increased from ~0.6% at the start of the quarter

Numbers reflect the proportion using the visual editor out of all article edits made by IPs and accounts excluding unattended bots in the last week of September. Use of attended bots & tools not excluded.


81


Q1 - VisualEditor Team  


Objective: Code quality

 


Objective Measure of success Status
Secure and build upon the foundational work for comprehensive mobile support


Team members involved: 4

VisualEditor on a mobile phone secures positive feedback in pre-deployment user testing


Prototype found some issues in heuristic evaluation and is being tweaked before further user testing.
Improve stability; maintain or increase test coverage


Team members involved: 4

Code coverage is maintained at or increased from current levels.


Overall coverage increased from 75% to 76%; for dm, from 87% to 90%; for ce, from 66% to 67%.
Hiring a mobile-expert engineer


Team members involved: 4

Team has a mobile-expert engineer join, to bring the engineering team up from 3.0 to 4.0 FTE. Our new Senior Software Engineer David Lynch started on 1 October.
82

Infrastructure / CTO[edit]

Quarterly review

Infrastructure / CTO
Q1 - 2015/16


Approximate Infrastructure size during this quarter: 35 FTE
Approximate CTO size during this quarter: 15.5 FTE


83


For more detail, see the slide deck and minutes from the quarterly review meeting of the Infrastructure and CTO departments

Infrastructure/CTO: Overview


Infrastructure:

  • ANALYTICS
  • RELEASE ENGINEERING
  • SERVICES
  • TECHNICAL OPERATIONS

CTO:

  • DESIGN RESEARCH
  • RESEARCH & DATA
  • PERFORMANCE
  • SECURITY
84

Analytics[edit]

Quarterly review

ANALYTICS
Q1 - 2015/16


Approximate team size during this quarter: 7 FTE
Time spent: strengthen 75%, focus 25%


Key performance indicator

 


Velocity July: 151  (MoM -32%) August: 145   (MoM -4%) September: 388 (MoM 167%)
85


Q1 - Analytics  


Objective: PAGEVIEW API

 


Objective Measure of success Status
Deploy an API for pageviews in production.


Team members involved: 5

Analytics metrics will provide data on usage of the API by the community (for example, daily queries & visitors). Missed

The API has not been deployed yet.

Success:

  • We evaluated several storage options and decided to use Cassandra.
  • We sent a survey to the analytics list and collected a comprehensive set of use cases.
  • All code is ready to deploy (hadoop, restbase and puppet).

Miss:

  • Reaching consensus with the Services and the Operations teams on system topology and repository structure took longer than expected.
86


Q1 - Analytics


Objective: EVENTLOGGING AUDIT

 


Objective Measure of success Status
Complete an audit of EventLogging data.


Team members involved: 2

All EL Schemas have been reviewed, documentation has been updated and new data purging/sanitizing has been automated. Missed

Part of data purging/sanitizing implementation is still in progress.

Success:

  • Reviewed status and privacy of 137 schemas.
  • Negotiated purging strategies with 28 schema owners.

Miss:

  • We didn't get enough DBA cycles to complete the database purging/sanitizing.
87


Q1 - Analytics


Objective: PAGEVIEWS IN VITAL SIGNS

 


Objective Measure of success Status
Publish daily pageview counts per project in Vital Signs.


Team members involved: 3

"Pageviews" metric is available in vital-signs.wmflabs.org, using new pageview definition and on a per-project basis. Met

Success:

  • Vital Signs graphs the new Pageview definition.
  • It includes “Totals” - an aggregate of all wikis.
88


Q1 - Analytics


Objective: EL USING KAFKA

 


Objective Measure of success Status
Migrate EventLogging to use Kafka.


Team members involved: 3

EventLogging’s capacity increases by one order of magnitude (1k to 10K msg/s). Workers are parallelized. Met

Success:

  • Now, one single machine can process more than 7.5k msg/s.
  • EventLogging capacity scales linearly: the more machines used, the more capacity.
  • If an outage in EventLogging occurs, the data is queued in Kafka (it is not lost).
89

Release Engineering[edit]

Quarterly review

Release Engineering
Q1 - 2015/16


Approximate team size during this quarter: 6 FTE
Time spent: strengthen 13%, focus 31%, experiment 0%, maintenance/interrupt 56%


Key performance indicator

 


WMF Log Errors (open task count) 138 +31% from Q4 (105) +202% YoY (68 on 2014-11-21)
90


Q1 - Release Engineering


Objective: Reduce CI wait time

 


Objective Measure of success Status
Isolate our Continuous Integration instances

Reduce CI wait time


Team members involved: 3 (~0.8 FTE)

  • Boot CI instances from OpenStack API - T109913


  • CI Instances are booting automatically


  • Subset of CI jobs run in disposable instances - T109914


  • Python CI jobs and the CI config repository itself are running on disposable instances


[[Image:]]

91


Q1 - Release Engineering


Objective: Reduce number of deploy tools

 


Objective Measure of success Status
Improve deployment tooling

Reduce number of deploy tools from 3 to 2


Team members involved: 4 (~1.4 FTE)

  • Implement the needed features in scap for RESTBase deploys - T113119


  • All needed features implemented or close to being merged (iow: in code-review).
  • Backwards compatible
  • Ready to use for Beta Cluster
  • Did not get RESTBase migrated by EOQ.


92


Q1 - Release Engineering


Objective: Retire Gerrit and Gitblit

 


Objective Measure of success Status
Migrate Gerrit to Differential


Team members involved: 2 (~0.45 FTE)

  • Allow repository cloning from Phabricator - T128


  • Cloning over https done
  • via ssh still in-progress (working with Ops)


  • Prototype integration with our Continuous Integration infrastructure - T103127


  • Conceptual work done
  • [scap3 work took priority over this]


93

Services[edit]

Quarterly review

SERVICES
Q1 - 2015/16


Approximate team size during this quarter: 4 FTE
Time spent: strengthen 40%, focus 35%, experiment 25%


Key performance indicator

 


REST API usage, mean req/s 216 req/s mean 31% up from 165 req/s Q1
94


Q1 - Services  


Strengthen: Mobile app service deployment and API

 


Objective Measure of success Status
Reduce app load times and duplicated development effort with an app API, providing content and metadata suitable for app consumption, backed by a service built by the app team.


Team members involved: 3


  • App service deployed in production.
  • Provide public REST API end points with documentation.
  • Integration with our CDN infrastructure.


- App service deployed in production.

- Public REST API deployed and documented.

- Full edge caching prepared, pending format finalization.

- Android app is still testing this. Current ETA for switch is end of October. IOS app to follow.

  • The content section API is also needed for progressive page loading in mobile web. We are working with the parsing and reading teams to generalize the API to work for both.
95


Q1 - Services  


Focus: Scale RESTBase, replicate across datacenters

 


Objective Measure of success Status
Add the storage and throughput we'll need to store more types of content. Establish redundancy at the DC level for site availability and disaster recovery.


Team members involved: 3


- Expand the existing cluster by at least three new nodes.


- Set up a second RESTBase cluster in the Dallas datacenter.


- Set up ongoing replication between the clusters.

- Eqiad cluster expanded from six to nine nodes.

- Added multi-DC support in RESTBase.

- Six-node codfw cluster operational, using larger HW.

- Replication fully operational, incl. encryption for cross-DC traffic.

  • Much of this work was done in cooperation with the operations team, especially Filippo Giunchedi. On the services side, this project was led by Eric Evans.
  • Other Cassandra users (maps, analytics, possibly sessions) can reuse multi-DC support.increased density thanks to multi-instance work
96


Q1 - Services  


Objective: Make a plan for change / event propagation

 


Objective Measure of success Status
Changes to structured data and other events need to be propagated through storage and cache layers. Our current systems are not very reliable, exclusive to MediaWiki, and don't work across projects.

We aim to make change propagation more reliable, general, and manageable for developers.


Team members involved: 3


- Establish requirements.


- Research solutions (both existing and possible development).


- Establish a clear plan of action.

- Requirements worked out in several tasks: T102476, T88459.

- Broad agreement on using Kafka in production.

- Prototyped producer API, and established a plan for phased implementation:

  1. Event bus providing common set of edit / change related events.
  2. Simple change propagation between services.
  3. (Q3) Complex change propagation and dependency tracking.


97

Technical Operations[edit]

Quarterly review

TECHNICAL OPERATIONS
Q1 - 2015/16


Approximate team size during this quarter: 18 FTE
Time spent: strengthen 40%, focus 50%, experiment 10%
79% maintenance, 21% new


Key performance indicator

 


Availability 99.942% -0.002% from Q4  
98


Q1 - TechOps


Objective: codfw

 


Objective Measure of success Status
Focus: Reduce missing cluster dependencies in codfw data center


Team members involved: 5, (+ 5 from other teams)


Deploy HTTPS caches in codfw, complete successful failover of eqiad caching traffic;

Deploy RESTbase with Cassandra replication;

Deploy ElasticSearch cluster with replicated search indexes

codfw HTTPS caches deployed in production, successful failover of all eqiad cache traffic performed.

RESTbase cluster in codfw set up successfully with Cassandra replication.

ElasticSearch codfw cluster deployed and ready. Discovery is still finishing MediaWiki multi-DC search index support.

Delays in the establishment of the new equipment leasing process severely impacted our hardware procurement and pushed availability of hardware to the last two weeks of the quarter.

Learning: We should make more explicit agreements on ETAs (with buffer time) for dependent work.


99


Q1 - TechOps


Objective: Mail tech debt

 


Objective Measure of success Status
Strengthen: Update Wikimedia mail systems


Team members involved: 4


Upgrade Mailman to version (2.1.18) on a supported distribution and new hardware;

Upgrade OTRS Investigate OTRS upgrade to 4.0.9 and provide trial system for OTRS users;

Enable TLS encryption for inbound and outbound e-mail

Objective was redefined early in the quarter around OTRS. A beta installation of version 4.0.13 was made available to OTRS users.

Mailman was upgraded to 2.1.18 and migrated to a new VM.

TLS has been enabled on all inbound and outbound e-mail.

Early on in the quarter we established that due to high amount of dependencies upon us, some changing circumstances (reduced DBA capacity) and shifting priorities (fr-tech), we would not be able to perform a full upgrade of OTRS this quarter. In collaboration with the CA team we decided to redefine the goal.

Learning: Migrations with a strong impact (change of functionality/use & feel) on users need more careful planning and coordination with liaison teams.


100


Q1 - TechOps


Objective: Security

 


Objective Measure of success Status
Focus: Reduce our security attack vectors


Team members involved: 1+10


Implement organized package upgrades (debdeploy);

>80% of hosts protected by firewalls, or explicitly assessed as unwanted and not needed

debdeploy has been successfully implemented and tested on a subset of cluster hosts, and we achieved 86% firewall coverage.

860 hosts now have firewalls, and 148 have been evaluated as explicitly not needing them. 1008 out of 1176 hosts/VMs are covered, amounting to 86%.


101

Labs[edit]

Q1 - TechOps


Objective: Labs reliability

 


Objective Measure of success Status
Strengthen: Improve reliability of Wikimedia Labs


Team members involved: 4


Meet or exceed 99.5% uptime for each Labs infrastructure service;

Remove Labs support host SPOFs;

Migrate NFS to RAID10 with sharding;

Audit Labs projects on NFS dependencies and support migrating off

Cumulative availability in Q1 was 99.742%. Redundant hosts have been set up for Labs NFS storage, networking, DNS and databases. NFS storage was migrated to RAID10 with 3 shards. 134/176 (76%) of Labs projects have been made independent of NFS storage.
102


Q1 - TechOps


Objective: Distributed Cluster Environment

 


Objective Measure of success Status
Experiment: Evaluate and test a new distributed cluster environment for Tool Labs


Team members involved: 2


Establish evaluation criteria for cluster solutions and perform an evaluation;

Set up the selected distributed cluster environment in ToolLabs to users to experiment with their web services

Criteria were established, and Apache Mesos and Kubernetes were evaluated.

Kubernetes was selected and made available in ToolLabs, and two Tools were migrated onto it for testing.

103

Design Research[edit]

Quarterly review

DESIGN RESEARCH
Q1 - 2015/16


Approximate team size during this quarter: 4 FTE
Time spent: strengthen 15%, focus 65%, experiment 20%


Key performance indicator

 


No one KPI due to challenges in quantifying qualitative research and dependencies on product teams for project work. Open to suggestions.      
104


Q1 - Design Research


Objective: Collaborate with Product Teams

 


Objective Measure of success Status
Support human centered product design and development by including Design Research in teams' product development process.


Team members involved: 4

Design Research projects are completed and output is relied upon to generate and iterate workflows, tools and products. Completed 8 design research projects for 6 teams:

4 Generative / Exploratory

1 Usability testing

3 heuristic evaluations



First time/New Editor discovery research informed design of new editor education for visual editor.

Workflow research informed understanding of user workflows to iterate workflow tools.

Mobile contribution research (31:11) is the start of informing us how to design / build for mobile contributions.

Link preview usability testing informed pre release iteration of link preview.


105


Q1 - Design Research


Objective: Mentoring

 


Objective Measure of success Status
Mentor non-design researchers to do design research to the standards of the Design Research team.


Team members involved: 1


STRETCH

Two non-designers researchers have done 2 or more usability tests to the standards of the Design Research team.


Non-designers learning to implement quality design research.

Mentoring:
  • May, visual designer
    • exploratory research on MW engineers’ use of style guides
  • Sherah, prototyper
    • usability study on link inspector
  • Both projects met standards of Design Research team


Link preview usability testing informed iteration of link preview before release.

Exploratory research for UI Standardization: Conversation on wiki and interviews on the same topic at Wikimania and post Wikimania are informing UI Standardization


106


Q1 - Design Research


Objective: Personas

 


Objective Measure of success Status
Iterate pragmatic personas from information gathered in persona interviews.


Team members involved: 3


12 persona interviews completed


  • Interviewed 7 new editors
  • 5 failed to show up to scheduled interviews
  • Continuing to recruit to complete this set of interviews.


The interviews we have done so far are already informing our thinking on the new editor persona.


107


Q1 - Design Research


Objective: Participant Database

 


Objective Measure of success Status
Grow database of opted-in design research participants beyond 2000.


Team members involved: 3


STRETCH

Database now contains over 2000 participants. New opted-in participants came from:
  • social media postings
  • Craigslist
  • outreach at Wikimania
  • other


108

Research and Data[edit]

Quarterly review

RESEARCH AND DATA
Q1 - 2015/16


Approximate team size during this quarter: 4.5 FTE, 2 research fellows, 10 collaborators
Time spent: strengthen 20%, focus 30%, experiment 50%


Key performance indicator

 


No one KPI due to the nature of the work      
109


Q1 - Research and Data


Objective: Revscoring productization

 


Objective Measure of success Status
Productize revscoring and wiki labels

Team members involved: 1

Collaborators: 9

Stable version of production-ready ORES service up and running on Labs


  • scores cached in real time for all new revisions
  • 10M scores generated so far  
  • Fast response: cache miss: 0.1-5s, hit: 0.05s


  • revscoring makes automated edit and article quality assessment available as a service to our contributors
  • at scale, revscoring can effectively reduce human curation effort by up to 93%
  • 4 models in production (reverted, wp10, damaging, goodfaith), 6 languages (en, es, fa, fr, pt, tr)
  • Improved AUC for enwiki (.82 → .90)
  • early community adoption: revscores now integrated into top semi-automated anti-vandal tool (Huggle)

[[Image:]]

https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service


110


Q1 - Research and Data


Objective: Article Recommendation tests

 


Objective Measure of success Status
Run controlled tests of article translation recommendations

Team members involved: 2Collaborators: 2

  • Complete WMF staff pilot
  • Complete and analyze the results of live tests on Wikipedia


  • completed pilot and live test in French Wikipedia
  • analyzed/presented results
  • wrote and submitted paper


  • Personalization is computationally expensive but matters: we can increase contributor activation rate by 82% over control
  • Recs boost article creation rate: 78% with random recs and 220% with personalized recs, over control

[[Image:]]

https://meta.wikimedia.org/wiki/Research:Increasing_article_coverage


111


Q1 - Research and Data


Objective: Value-added research

 


Objective Measure of success Status
Measuring value added

Team members involved: 1Collaborators: 1


STRETCH

  • Perform historical analysis on who adds value to Wikipedia


Preliminary analysis in progress

Full study deferred to Q2

[[Image:]]

  • Survival of content has been measured.
  • The basic quality of a contribution is evident after a few edits/hours.  

https://meta.wikimedia.org/wiki/Research:Measuring_value-added


112


Q1 - Research and Data


Objective: Article recs improvements

 


Objective Measure of success Status
Re-train and productize article recommendations


Team members involved: 2Collaborators: 2


STRETCH

  • Improve algorithm based on user feedback
  • Integrate the algorithm in ContentTranslation


  • Integration in CX paused
  • Pivoted to stand-alone API and client on Labs for rapid iteration
  • Work staged with Editing


  • Set up a public API on Labs
  • Developed an instance on Labs with article recommendation service offered for 11 language pairs.

[[Image:]]

https://meta.wikimedia.org/wiki/Research:Increasing_article_coverage


113

Performance[edit]

Quarterly review

PERFORMANCE
Q1 - 2015/16


Approximate team size during this quarter: 4 FTE
Time spent: strengthen 40%, focus 40%, experiment 20%


Key performance indicator

 


Time to first paint 675ms -49% from Q4 ~ -55.73% YoY
114


Q1 - Performance  


Objective: Subsecond page save time

 


Objective Measure of success Status
Make editing Wikipedia fast. Sub-second page save time.


Miss. We hit the goal in August, but a regression in September wiped out gains.


-28% reduction in page save time.

[[Image:]]

Q1 start


Regression; cause not known yet. (T114339)


115


Q1 - Performance  


Objective: Sub-900ms first paint time

 


Objective Measure of success Status
Faster delivery of content to readers. (T105391) Achieve sub-900ms median first paint time. Hit. Median first paint is now 675ms, a 49% reduction.

[[Image:]][[Image:]][[Image:]]

116

Security[edit]

Quarterly review

SECURITY
Q1 - 2015/16


Approximate team size during this quarter: 2 FTE
Time spent: strengthen <10%, focus <10%, experiment 40%


Key performance indicator

 


Critical and High Priority Security Bugs   -66.6% / +15.9% from Q4 **  
117


** - Close approximation, to the extent that phabricator allows us to reconstruct history


Q1 - Security  


Objective: Automated Dynamic Scanning

 


Objective Measure of success Status
Run dynamic security scanning tool against beta every week.

Team members involved: 1


  • Pick tool to implement
  • Configure weekly automated scanning from labs of beta
  • Record baseline scan results for core and one extension


Security team chose OWASP’s ZAP for scanning. Weekly scans are running from scanner.security-tools.eqiad.wmflabs. Baseline measurements are recorded at mw:Wikimedia_Security_Team/ApplicationScanning.
118


Q1 - Security  


Objective: Metrics for Security Bugs

 


Objective Measure of success Status
Document and report initial metrics for security bug handling

Team members involved: 1


  • KPI metric chosen
  • Metrics tracked monthly


Until we can better understand the subtleties of our backlog, we will use the number of Critical and High priority security bugs as our KPI mw:Wikimedia_Security_Team/Metrics

Learning: The number of open bugs is a poor metric for overall health of an application, but is the most indicative metric that we can practically track at this time.

All metrics are tracked privately. Historically, the number of open security issues in mediawiki has been considered private to the organization.


119

Legal Team[edit]

Quarterly review

Legal Team
Q1 - 2015/16


Approximate team size during this quarter: 10 FTE Humans & 1 Stuffed Tiger*


Time spent: strengthen 65%, focus 20%, experiment 15%


KPIs set out in Appendix A


*Temporary staff this quarter: 8 legal fellows/interns.
Public Version


120


For more detail, see the slide deck and minutes from the quarterly review meeting of the Legal department


Q1 - Legal  


Objective: Core

 


Objective Measure of success Status
Core

Top notch, quick legal advice and support on wide host of issues constituting our 34 legal workflows. Categories include:

  • Transactional
  • Litigation
  • Privacy
  • Public Policy
  • Trademarks
  • Governance
  • Technology
  • Fundraising
  • Training


KPIs


  • Turn-around rate for contracts exceeded KPI of 95% w/i 7 days (99%).
  • Turn-around rate for legal@ exceeded KPI of 95% w/i 7 days (99%).
  • Core legal advice and daily operations to the satisfaction of ED and C-levels per KPI.


  • Board work completed but not always on time per protocol.
121


Q1 - Legal  


Objective: NSA Litigation

 


Objective Measure of success Status
NSA Litigation

Protection of global users: readers and contributions


No missed dates for affidavits and motions.  High quality documents and credible showing at hearings.


Remains at an early stage of litigation, but proceeding as expected:

• All motions filed on time with strong WMF research and input on opposition to gov't motion to dismiss.  

• Hearing held Sept. 25 in Alexandria, VA.

• Work continues on time to prepare staff and expert declarations.

122

Appendix A


Key Performance Indicators

 


KPI Goal Actual (no QoQ yet) YoY
Contract Turnaround 95% w/i 7 days 99% YoY N/A
Legal@ Turnaround 95% w/i 7 days 99% YoY N/A
Core legal satisfaction of ED and C-levels expectations met YoY N/A
Board Support satisfaction of Board deadlines did not meet expectations per Board protocol; substantive work met expectations YoY N/A
123

Talent & Culture[edit]

Quarterly review

Talent & Culture
FQ1 - 2015/16


Approximate team size during this quarter: 7
Time spent: strengthen 45%, focus 45%, experiment 10%


124


For more detail, see the slide deck and minutes from the quarterly review meeting of the Talent & Culture department


FQ1 - Talent & Culture


Objective

 


Objective Measure of Success Status
Goal 1 FOCUS

Domestic PEO (for temps) replacement selection

Team members involved: 3

-Meetings with 3 alternatives

-Recommendation done

-Met with iWG/Upwork, PRO Unlimited, Namely

-Stay w/Upwork but have them move WMF from Coretechs to iWG

Goal 2. - FOCUS

Process improvements in workflows for ATS

Team members involved: 3

All recruiting and hiring workflows occur inside GreenHouse without external email approval chains -Posting approval workflow done

-Offer approval workflow done

Goal 3 - FOCUS

Learning and training tool with tracking and testing

Team members involved: 2

Tool selection & implementation

5 modules in place, including:

2 Onboarding & 2 Manager

-Mindflash selected & implemented

-5 modules in place (Legal 101, unconscious bias, constructive feedback, benefits orientation, recruiting training for HMs)

125


FQ1 - Talent & Culture


Objective

 


Objective Measure of Success Status
Goal 4 - STRENGTHEN

HRIS replacement selection

Team members involved: 5

-Meetings with three ADP alternatives

-Recommendation done

-Met w/Namely, Workday, Zenefits

-Recommend Namely

-NOTE: To be reviewed by new VPHR to confirm proceeding

Goal 5 - STRENGTHEN

Manager interviewing & recruiting training

Team members involved: 3

-Deliver course material

-Integrate course material into learning tool

-3 training sessions done

-Materials loaded on Office Wiki.  -Video of session done

Goal 6  - STRENGTHEN

Hire (replace) OD Mgr

Hire (new) Coach

Team members involved: 7

Successfully fill both positions -On hold for VP of HR per Terry

-JDs have been posted

-Interviewing

126

Communications[edit]

Quarterly review

COMMUNICATIONS
Q1 - 2015/16


Approximate team size during this quarter:
5 FTE, 4 interns, 2 volunteers
Time spent: strengthen 50%, focus 20%, experiment 30%


Please see the scorecard appendices of the team’s quarterly review documentation for further information.


127


Q1 - Communications


Objective: Wikimania awareness

 


Objective Measure of success Status
Advance regional objectives through media promotion at Wikimania MX. Raise Wikimedia profile in Latin America.


Team members involved:

3 FTE, 3 contractors

  • 3 AAA broadsheets
  • 15+ press hits
  • 2 major broadcast interviews


  • 17 AAA broadsheets
  • 408 press hits
  • 8 major broadcast interviews including CNN (x2), Ibero 90.9 (x2), Milenio TV (x2), Canal 22, W Radio
  • 20+ other broadcast interviews

Complete with all measures of success met. See media report.

128


Q1 - Communications


Objective: Visual editor plan

 


Objective Measure of success Status
Build public VE rollout plan and internationalized content marketing strategy for user adoption and awareness


Team members involved: 3 FTE

  • Executed against product timeline
  • In-product messaging
  • Zero negative press
  • Content marketing campaign for user adoption and awareness


Complete with all measures of success met.

  • In-product messaging includes revised welcome message and tooltip language
  • 3 videos in user testing: How to Edit Wikipedia, How to Add Links and Citations, and The History Page
  • Ongoing research into new editor experience to inform campaign
  • Campaign will leverage video, social, and more to encourage/support new editors  
129


Q1 - Communications


Objective: Wikipedia 15

 


Objective Measure of success Status
Create strategic messaging opportunity around 15th birthday to engage new Wikimedia stakeholders and audiences.


Team members involved:

3 FTE, 3 contractors

  • Finalize calendar for WP15 activities
  • Contract any relevant key resources


  • Retained creative agency (Mule Design)
  • Community outreach and capacity building
  • See the celebration mark on the following pages

Complete with all measures of success met.

  • 21 community celebrations in 18 countries planned and counting
  • Campaign launch to include community-driven announcement, influencer engagement, user-generated content, launch of microsite, digital platform engagement, and social promotion

[[Image:]]

130


Q1 - Communications


Objective: Staffing

 


Objective Measure of success Status
Fill critical gaps in core communications roles.


Team members involved:

4 FTE

  • Recruit two planned FTEs.
  • Recruit fall class of 5 interns.


  • 3 FTEs retained
  • 5 interns retained

Complete with all measures of success met.

FTEs:

  • Ed Erhart - Editorial Associate - 9/1 (est. 9/1)
  • Jeff Elder - Digital Communications Manager - 9/28 (est. 8/15)
  • Zack McCune - Global Audience Manager - 10/12 (est. 9/15)
131


Q1 - Communications


Objective: Workflows

 


Objective Measure of success Status
Timely, professional, first-rate support on core and reactive communications workflows.


Team members involved:

4 FTE, 3 contractors

Evaluative chart of core workflows


  • See Core workflows and metrics (slide 13).

Complete with all measures of success met.

132

Finance, Administration, OIT[edit]

Quarterly review

Finance, Administration, OIT
Q1 - 2015/16


Approximate team size during this quarter: … 20
Time spent: strengthen 50%, focus 40%, experiment 10%
For more detail, see the slide deck and minutes from the quarterly review meeting of the Finance, Administration and Office IT department
133

Finance[edit]

Q1 - Finance  


Objective: Implement Concur Purchase Order

 


Objective Measure of success Status
Purchase order will be implemented and utilized by departments and teams to make purchases.


100% of all contract orders will have been processed via the purchase order system.  


Project was not started, due to delays in implementing Concur Expense, which was a dependency. Also, this is put on hold until we hire a Purchasing Specialist and finalize a purchase order workflow/process.



Dependency on completing Concur purchase order was not clearly identified when goal was set.  New projected completion date of Q3.


134


Q1 - Finance  


Objective: Financials by the 15th

 


Objective Measure of success Status
Improve consistency in having financial data ready by the 15th of the month.


Team members involved: 4


During the quarter financial data will be ready for input in the scorecard by the 15th of the month.


For each month of the quarter, financial data was ready by the 15th of the month.

One of the key dependencies for meeting this object is the reconciliation of purchase card statements on time.


135


Q1 - Finance  


Objective: Test of Adaptive Discovery

 


Objective Measure of success Status
Develop platform for holding scorecard data and making it accessible to managers.

Team members involved: 3


Test group of managers will have access to their scorecard data via an platform other than Google sheets.


Project was not completed due to vendor (Adaptive) inability to set up test accounts for project.
136


Q1 - Finance  


Objective: Proposals for Audit & Tax Services

 


Objective Measure of success Status
Obtain audit and tax services proposal for 3 years starting with FY15-16 from accounting firms with non-profit, technology, and international grants experience with global presence.

Team members involved: 3


Selection of a new Accounting Firm that meets our needs by October 31, 2015.


Received 9 proposals - 1 from Big 4 (KPMG) and 8 from mid-sized national accounting firms with global presence. Have narrowed down to 2 and will reach out to the references provided before recommending to our Audit Committee.
137

Administration[edit]

Q1 - Administration

Admin   


Objective: Complete RFP for real estate broker

 


Objective Measure of success Status
Launch and complete RFP for real estate broker for WMF

Team members involved: 3


Outline success features of brokerage house and complete interview and selection process with 18 month plan drafted by 9/30/2015


Chose CBRE Real estate broker services out of six proposals and in depth interview process. Draft plan and workplace strategy are complete.

Learning: Using success of prior projects; sincere, open, inclusive discovery based on human  centered design principles. Separate fact (data) from opinion (narrow). Inclusion of diverse stakeholders = Clear goals, informed decisions. VITAL will continue to partner with WMF and CBRE. Director of Administration and VP of HR will work very closely on this project. Project is launching 10/8/2015 with COO and VP of HR meeting and reviewing plan with CBRE.


138


Q1 - Administration


Objective: Office wiki project

 


Objective Measure of success Status
Improve format and relevance of office wiki information


Team members involved: 6

Cross org team members: 4

Community member: 1


Increase staff involvement by hours and number of staff collaborating and delivering on project and number of quality pages.


Increase community involvement by one team member


Have inclusive, cross org and input and review.

Our cross org work can be see here. OfficeWikiProject.


Knowledge management flow and Report of findings from survey and interviews w/overview and ranked next steps guiding our work here



Learnings: Cross org system of design, review, support and training tested for inclusive, paced tested, successful implementation.


139


Q1 - Administration


Objective: Implement best practice guidelines

 


Objective Measure of success Status
Create best practice guidelines format


Team members involved: 4

Cross org team members: 3


Complete Design standard format for all guidelines complete.


Prioritize guidelines needed, assign staff to review, revise and implement accordingly.


Draft team offsite guidance and expense planning ready for inclusive input phase.


Standard format devised

Example: Mobile device guidelines

Summary of draft guidelines

Draft on wiki off site guidelines



Learnings: Thoughtful, open, inclusive review and input from stakeholders & users together with education from staff that have performed the work leads to greater adoption. All guidelines will continue to be revised, improved, posted and integrated into training for all new staff and managers.


140


Q1 - Administration


Objective: Implement SLA  

 


Objective Measure of success Status
Create SLA for all services under management


Design standard format for SLA


Complete list and inventory of services, assign staff to review,  implement on a timeline.


Where service inventory and refinement have been done formal SLA have not been completed.


Draft will be complete by end of Q2



Due to competing priorities, transitioning of CFA as well as additional work created with Q1, this goal was worked on, but not complete.


141

OIT[edit]

Q1 - OIT  


Objective: Implement Blue Jeans

 


Objective Measure of success Status
Setup large group video conference solution.

Team members involved: 2


Large teams ( 25 or more) will be able to have a video conference.


Large teams are able to meet via video conference.

With the implementation of this video conference solution, it was hoped that we would also have an alternate video conference solution for Metrics meetings, but testing showed that the current solution was prefered for now.


142

Team Practices Group[edit]

Quarterly review

Team Practices Group
Q1 - 2015/16


Approximate team size during this quarter: 7 FTE
Time spent: strengthen 60%, focus 20%, experiment 20%


Key performance indicator

 


Positively impacting value delivered by supported teams 4.2 out of 5 (Likert scale) baseline metric
Positively impacting sustainability of supported teams 4.6 out of 5 (Likert scale) baseline metric
143


For more detail, see the slide deck and minutes from the quarterly review meeting of the Team Practices Group


Q1 - Team Practices Group


Objective: Team Health Checks

 


Objective Measure of success Status
Identify organizational pain points for software engineering and  supporting teams; foster self-awareness of team health to support continual improvement


Team members involved: 6


  • Deliver Team Health Check survey to majority of engineering teams
  • Publish results, trends, recommendations to officewiki
  • Present findings to management and key stakeholders


  • 1 CE, 13 Eng teams, and TPG participated
  • Results published
  • Biggest takeaway findings:
    • Sense of lack of support from mgmnt and other teams
    • Struggling with community involvement
    • Lack of clarity around org goals; disconnect between team and org goals


144


Q1 - Team Practices Group  


Objective: MPL

 


Objective Measure of success Status
Support the prioritization of current and upcoming work across the WMF, and balancing it against available resourcing.


Team members involved: 2


  • All current and projected projects across the WMF listed, prioritized, and estimated on the Master Project List (MPL)
  • All current projects with a priority outside of the WMF’s resource capacity will have a “sunset” or “stall” plan


Scope reduced to encompass only projects in Community Engagement and Engineering, and to not include sunset/stall plan measures of success. Engineering and Community Engagement projects now documented and up to date.
145


Q1 - Team Practices Group  


Objective: Vision and Strategy

 


Objective Measure of success Status
Establish TPG vision and strategy for FY16


Team members involved: 7


Publish document outlining TPG’s high-level vision and strategy for FY16


  • Miss this quarter due to two extended unplanned absences on the team & emergent priorities
  • Carrying over for Q2


146