Wikimedia Foundation Report, April-June 2015

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search
See also the original PDF version of this report



Information

You are more than welcome to edit this wiki version of the report for the purposes of usefulness, presentation, etc.


Wikimedia Foundation

Quarterly Report

Wikimedia-logo.svg

April — June 2015
Q4 of the fiscal year

Foreword[edit]

We are pleased to bring you the Wikimedia Foundation’s Quarterly Report for Q4 of the 2014/15 fiscal year. This is the third report since we switched from a monthly cycle, to align with our quarterly goal setting process. We are continuing to optimize the report’s format and the organization’s quarterly review process that the report is based on, to bring you better information at a lower overhead for the teams that take out time from their work to tell you how they have been doing. Participation in the review process is good and growing.

This issue includes some new pieces of information, e.g. the approximate size of each team (in FTE, on average during this quarter), and for each objective, the number of team members who were involved with a significant amount of their time. The overall metrics scorecard now contains new, more reliable uptime numbers for both readers and contributors.

As before, we are including an overview slide summarizing successes and misses across all teams. In a mature 90 day goal setting process, the “sweet spot” is for about 75% of goals to be a success. Organizations that are meeting 100% of their goals are not typically setting aggressive goals.

Terry Gilbey, Chief Operating Officer


2

Quarterly metrics scorecard (beta)[edit]

WikiFont uniE600 - userAvatar - blue.svg        Participation
Sign-ups 1,329k -11.8% from Q3
+36.6% y-o-y
New editors 15.8k/mo -7.9% from Q3
-1.5% y-o-y
Active editors
(5+ edits/month)
79.4k/mo +1.7% from Q3
+4.1% y-o-y
WikiFont uniE018 - cog - blue.svg        Site reliability
Read uptime
(May-June)
99.944% Q3: N/A
y-o-y: N/A
Write uptime
(May-June)
99.942% Q3: N/A
y-o-y: N/A
Read latency
Median first paint time
1.3 seconds Q3: 1.4 seconds
Write latency
Median page save time
1.6 seconds Q3: 2.0 seconds
Wikifont uniE010 - eye - blue.svg        Readership
Page Views
Crawlers excluded
17.8B/mo Q3: N/A
y-o-y: N/A
Visitors Unique visitors to come comScore desktop UVs (deprecated):

417M/mo in Q4

WikiFont uniE100 - article - blue.svg         Content
New articles 6.7k/day -14.9% from Q3
-41.3% y-o-y
Edits (in WP articles) 11.55M/mo +21.7% from Q3
+17.7% y-o-y
WikiFont uniE033 - heart - blue.svg        Fundraising
Amount raised $9.3M
(exceeded $7M target)
Q3: $9.9M
On mobile 10% Q3: 21%
3

Q4 2014/15: Successes/misses by team[edit]

WMF 2014-15 Q4 successes and misses by team.svg
4

Q4 2014/15 objectives

118 objectives
35 teams
87 successes (74%)
31 misses (26%)

Misses include anything not delivered as planned, including “yellow” goals and those that were intentionally revised or abandoned due to changing priorities.


5

Community Engagement (CE) department[edit]

  • Learning & Evaluation
  • Community Resources
  • Wikipedia Education Program
  • Community Liaisons
  • Community Advocacy
  • Engineering Community
  • Wikipedia Library
6

CE: Learning & Evaluation (7.2 FTE)[edit]

Objective Measures of success [Team members] Status
1. Promote effective program design and resourcing:


Programs Reports


Increase knowledge about Wikimedia programs, their design, and impact through

(1) Development and publication1 of 4 of the remaining Programs Evaluation reports [2-6]

(2) To be followed with announcements, RFC, and socialization (see timeline for rollout) [2-5]

MET TARGET

All targeted reports are in final draft and within various stages of their soft release2 within internal or community review (see timeline)

2. Promote effective program design:  


Program Toolkits


(1) Conduct 7-8 program leader interviews for each programs toolkit for contests: (a) Writing contests (Target: Completion) and (b) Photo events (Target: Initiation only) [2-4]

(2) Launch writing contest toolkit with at least 5 new learning patterns each and initiate calls to action to engage community [3-5]

MET TARGET

Toolkit soft-release with 16 new learning patterns shared to program leaders 6/26/2015. Full launch set for 7/5/2015. Interview contacts for photo events initiated.

1“Publication” involves initial soft-release on meta followed by blog announcement and larger community RFC and input to final sign off of publication and release of peer review template. Each of the reports will be in various stages of publication as noted in the timeline."  On-wiki Writing Contests, Edit-a-thons, Editing Workshops, all in soft-release stages while, as noted in the original Q4 objective, the GLAM Content Release Partnerships soft-release is on hold for the drafted report for pairing and roll-out with Wikimedians in Residence report in Q1.

2 Based on first reports we had to revisit design, with feedback from across Community Engagement, and a revised communications plan early in the quarter we slowed roll-out of the reports to allow each report a more clear community review and input time and process.


7

CE: Learning & Evaluation (7.2 FTE)

Objective Measures of success [Team members] Status
3. Evaluation Capacity development


Program Leader Facilitation, Training, and Support


(1) GLAM-Wiki Conference Learning Circles
  • At least 10 community members initiate involvement3 [2]

(2) WMCON Pre-Conference

  • At least 30 program leaders, 80% or more indicate progress pre to post (5)


SURPASSED TARGET

51 Unique participants across the different conference events. 87% of pre-conference participants indicated progress in their understanding of key terms and concepts. (Read the blog for more!)

4: Internal WMF support


(1) On-going internal support: Emerging communities research completed with 16 communities/countries [1]

(2) Evaluation of Round 2 APG grantee impact [2]

MET TARGET

Community research identified 7 high-impact capacities for development in Q1 and Q2

3 Learning Circle goals are to develop  learning pattern frames for best practices for pitching, and tracking and reporting, GLAM-Wiki partnerships  in order to work with GLAM community to develop toolkit patterns


8

CE: Learning & Evaluation (7.2 FTE)

  • Program Evaluation Reports got a 2.0 update to include Community Voices, Executive  Summary and a Focus on the So What? (e.g. How do we use evaluation reports to  improve program design and performance across communities and serve as an initial data point for guiding investments?)
    • Writing Contest Toolkit feature refinements based on community feedback. Pilots a new feature for peer to peer knowledge sharing.
  • Released report on community capacity needs, documenting for the first time on-wiki and off-wiki capacities for development, as reported by emerging communities. Seven high-impact areas were identified for further discussion and development in Q1, setting the stage for community specific engagement and action in following quarters.
  • Focus on Team Systems put into place by new director include a strategic communications plan for all product launches to ensure we maximize product reach and impact; project charter system correctly scopes and sanity checks projects to prioritize  investments.  This work is directly aligned with workflow analysis and creation of the master project list.
9

CE: Learning & Evaluation (7.2 FTE)

Communicating our Work and Framing the Message for Community Value:  Release of our program report on “Wiki Loves Monuments” identified the need for an improved messaging strategy as well as report design changes. It also identified community sensitivity to the importance and need for qualitative data references, which extends beyond easily retrievable data such as quantity of photos or articles added.


Learning:  

  • Make reports easier to read in bite size chunks. This will include a 4-minute presentation on how to use the reports - for those without time for extended conversations - as well as executive summaries of key findings.
  • Enhance L&E and WMF brand by partnering evaluation reports with tools and resources for knowledge sharing; shifts our reputation from “Auditor” and more of a knowledge sharing “Partner”
  • Community engagement prior to any release is key. This is now part of a strategic communications planning process, that carves out time for crafting product messaging to communities.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department


10

Q4 - Community Resources  (7 FTE)

 


Objective Measure of success Status
1. Publish and socialize Capacities Development Framework (CDF) for emerging communities

Team members involved: 1 (CR), 1 (L&E)

* Insights report, CDF framework published on meta

* 5 actions across 3 communities planned & committed to for Q1

Report and framework published, with 7 areas identified & prioritized for development into action


Community-building actions planning pushed to Q1

2. APG: Ensure round 2 funding decisions are made based on impact analysis

Team members involved: 2 (CR), 2 (L&E)

Majority of FDC members indicate using past achievement reports in at least 70% of funding decisions. The FDC’s Round 2 funding decisions were based on impact analysis of programs for 100% of submitted proposals.
3. PEG: Provide guidance materials for conference organizers

Team members involved: 1

Publish conference program resources on meta-wiki First draft published for review by GAC.
11

Q4 - Community Resources[edit]

Objective Measure of success Status
4. Inspire: Support more new gender-focused projects with impact potential

Team members involved: 4 (CR), 1 (CA)

* 20 grants for $250k or less

* At least 50% of project leaders supported are women (increase from Q1&2’s <30%)

* 16 new gender-focused grants for $120k, increase PEG+IEG gender portfolio from 9% (H1 2014) to 18% (H1 2015)

* 88% led by women, 81% first-time grantees.

5. Evaluate feasibility of proactive grants campaign strategy

Team members involved: 3

Report published on meta with actionable recommendations to enable a go/no-go decision for running future campaigns * Report and blog post published.

* Planning to run 2 campaigns in next fiscal, if board approves annual plan w/ added community organizer

6. Roll-out Friendly Space Expectations to all grants spaces on meta-wiki

Team members involved: 1 (CR), 1 (CA)

Consultation with CA and meta admins leads to expectations published for the grants namespace In progress: review by staff complete, under review by grant committees.

CA & CR to complete review w/ admins and roll out in first month of Q1

12
Quarterly Objectives
Successes

Q4 - Community Resources

  • APG Funding decisions were correlated to impact, thanks to the analysis provided on outcomes achieved and expected targets proposed.Learning: Analysis is resource intensive for CE but critical to good funding decisions. Coaching grantees on SMART objectives was effective.   People: Katy, Winifred, Sati, Garfield & Jaime were instrumental in driving this
  • Inspire has doubled our gender-focused funding portfolio (dollars spent on projects via PEG and IEG)Learning: Focused campaigns can bring in new grantees and projects aimed at strategic issues.People: Alex Wang and Marti Johnson worked overtime to make this happen
  • Wikimedia Conference messaging continued to rebuild trust between WMF & affiliates. “I saw many changes towards a better, more open and more collaborative Foundation” - WMIT board memberLearning: Talking points were an effective tool. Coordination with ED’s office needs to be improved.  People: Asaf, Katy, Winifred and Alex have been working on affiliate relationships for several years, and their contributions to key messages was invaluable
13

Q4 - Community Resources

Quarterly Objectives:
Misses
  • Publication of the report on community capacity needs was pushed out as staff time had to be reprioritized for community emergencies, leaving insufficient time for next step action-planning with specific communities.
Learning: By pushing back our deadline to give communities adequate time to provide feedback before publication, and openly communicating the rationale for being behind schedule, we seem to have improved the ultimate reception of the research & its conclusions. In future timelines, we aim to build in extra buffer time to account for other responsive workflows.
  • Implementation of the Friendly Space Expectations was pushed out to early Q1 because of inadequate review time and conflicting staff schedules between CA and CR.
Learning:  In future we will do more to clarify and coordinate priorities between teams, provide extra time for staff and community review, and build in extra time to allow for community emergencies.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department


14

Q4 - Education (4.3 FTE)[edit]

Objective Measure of success Status
1. Actively mentor program leaders worldwide

Team members involved: 5

* 50% of interactions are initiated by community.

* Connect 5 volunteers with Collab

* Analyze current bottlenecks to engagement for Collab

* 5 new programs report metrics

* Interactions in 40 countries

* 67% of interactions initiated by community

* Connected 6 people with Collab

* Collab analysis complete

* 6 programs reported metrics

2. Add content to Arabic Wikipedia through expanding ed program

Team members involved: 2

* Determine viability of Omani education program pilot after visit to Oman Deemed viable. Q1 focused on getting ready for pilot in the fall semester.
3. Add content to Spanish Wikipedia (and sister projects)

Team members involved: 2

Co-organize 2 day education pre-conference at Wikimania:

* Invite global audience

* Ensure engaging sessions

* Host 4 sessions during pre-conference

On track for a successful pre-conference at Wikimania.

* 94 people signed up (target: 40 - 60) in one week

* 25 signups are international

15

Q4 - Education


Objective Measure of success Status
4. Provide materials for program leaders and encourage community involvement

Team members involved: 1

* 10 community members add to / endorse Learning Patterns (LP) in Q4 * 14 volunteers edited 10 LPs

* 1 new LP by community member

* 1 LP translated into 2 languages

* 5,000 views of Ed Toolkit pages (includes LPs)

5. Share lessons learned with education and Wikimedia audiences and facilitate discussion and networking

Team members involved: 2

* Publish monthly newsletter, with at least 10 contributions from community

* Write one blog post about education per month

* Facebook group:

** 50% community posts

** one post per week from Ed team

** 25% member growth in Q4

* 12 contributions from community to newsletter (April and May)

* 3 blog posts written (1 published)

* Facebook group:

** 55% community posts

** ~2 posts per week from Ed team

** 85% member growth in Q4

16


Q4 - Education

Quarterly Objectives:
Successes

Facebook group: Grew much more than expected (85% in one quarter).

Learning: There’s a deep-rooted desire for people to connect, share their stories and learn from others, also through non-wiki channels.

Berlin WMCON: Great education team presence, great conversations.

Learning: Presence at in-person events goes hand in hand with resource development. It’s where we socialize the resources we have, where we hear new stories and get a deeper understanding of needs.

Wikimania pre-conference: 94 people signed up in one week’s time, greatly exceeding expectations.

Learning: Be prepared for anything when organizing a pre-conference :o)


17

Q4 - Education

Quarterly Objectives:
Misses

Collab: Q4 focused on analyzing Collab bottlenecks to engagement and finding solutions. Limited capacity on team means Collab did not have the benefit of team defining improvements in early stage.

Learning: Our goal to go “light” on the organizational aspect of the Collab has left it with unclear expectations about what to do and how. We are redefining membership criteria, focusing on concrete actions that are transparent to the community. Stay tuned for updates in Q1.

July 7th update: Updated Collab main page, new membership request page and new open task list page.


For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department


18

Q4 - Community Liaisons (6+1 FTE)

 


Objective Measure of success Status
1. VisualEditor: Support communities and product team in discussion launch

Team members involved: 1

VisualEditor is prepared to launch with confidence from community and product team perspectives.


Community Engagement holds successful conversations and notifications around VE development and launch.

The team has been supporting conversation around a graduated release of VisualEditor after the product team and A/B test resulted in a proposal to release VE to new accounts.
2. Complete responses to ideas in[1]Product Process ideas page

Team members involved: 2


Response to community ideas, supporting product inclusion process.


Setting expectations on what is reasonable or not reasonable on product process

Not completed.
19

Q4 - Community Liaisons (6+1 FTE)[edit]

Objective Measure of success Status
3. Complete SUL finalization

Team members involved: 1


Single Unified Login complete, nearly three million accounts renamed, nearly five million accounts made global.


The objective was completed by the given ETA
4. Document Workflows

Team members involved: 2

Ensuring clarity of team tasks for ongoing development (internally and to community) This is drafted but not finalized.



20

Q4 - Community Liaisons

Quarterly Objectives:
Successes
  • VisualEditor: Message management is resulting in a civil, positive conversation
  • SUL: An initiative announced in 2004 has been completed. Over 200 volunteers were engaged on this project.
    • Learning: Community supported initiatives, early volunteer engagement and consideration of workflows was key to this project’s success
  • Two new team members onboarded
    • Benoît (France) and Johan (Sweden), bringing 18 combined years of onwiki experience to the team
  • Successful support of unplanned work, including the https project
21

Q4 - Community Liaisons

Quarterly Objectives:
Misses
  • Process ideas: We were unable to finish triaging and responding to the Process Ideas due to changes in engineering & product.
    • Learning: This may have been an example of putting the cart before the horse; the process needs to be defined enough internally and consistently first. CLs aim to define a consistent community collaboration process in Q1, which will allow us to take another look at these ideas.
  • Workflows: We began documenting these, but are working with the rest of the department to make them consistent.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department


22

Q4 - Community Advocacy (6 FTE)[edit]

Objective Measure of success Status
1. Stewards visit to San Francisco.

Team members:  1

* Develop an agenda for meetings and coordinate dates with stewards.

* Establish a date for this event by 6/5/15.

* Book office facilities as needed

* Coordinate with travel to provide travel/lodging

Completed late (6/20).  Goal was 6/5.
2. Board and FDC Election:


Team members: 2

* Form a community election committee to ensure that the FDC Election (5/3) and Board Election (5/17) are executed.

* Ensure ease of candidacy, submission, and voting.

* Increase turnout and diversity of the candidate pool.

Completed on time.
23

Q4 - Community Advocacy


Objective Measure of success Status
3. Event bans and banned users


Team members: 3

* Produce a (legal-approved) list of global and event banned users

* Create a protocol for appearance (or threat of it) at events by banned users

* Supply to Conference Coordinators for events beginning in Q1 (6/30)

* Gather feedback and iterate.


Completed late (7/2).  Goal was 6/30.



4. Iterate on global ban process


Team members: 2

* Produce a proposed legal-vetted protocol and checklist by 4/30. Completed on time.
24

Q4 - Community Advocacy


Objective Measure of success Status
5. Onboarding Kalli to emergency and legal-reports


Team members: 3

Start date: 4/15,

Shadowing by 5/4 and

On call rotation by 6/1

Completed on time.
6.  Execute on core workflows


Team members: 6

Ensure that 95% of contacts are handled within two working days of receipt; and that other workflows perform as required. Completed on time.
25

Q4 - Community Advocacy

Quarterly Objectives:
Successes and Learnings
  • Elections: Broader translation and engineering’s timely help with SUL and SecurePoll paid off in FDC and board election turnout.  
  • Capacity issues on the team meant we were late on two of our goals (one by several weeks, and one by three days.)  With our now-increased team capacity and better goals this won’t be an ongoing problem.
  • New CA staff onboarding: Experiment in hiring externally (with transferrable and directly applicable skills) paid off.  She’s learning the wiki-specific things, but already knows Trust & Safety and was able to hit the ground running, decreasing ramp-up time.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department


26

Q4 - Engineering Community (3-4 FTE)[edit]

Objective Measure of success Status
1. Most welcoming and productive Wikimedia Hackathon

Team members involved: 2

33 projects showcased

At least 6 worth supporting

19% of participants were new volunteers

100% of newcomers had a buddy

Our bet on demoable projects and buddies was a success. Survey results available soon.
2. Successful start of all GSoC & Outreachy projects

Team members involved: 1

100% passed the community bonding period and received their first payment Participants are meeting expectations better thanks to public evaluation criteria.
3. Most basic Tech Community metrics are published

Team members involved: 1

Basic community metrics agreed on, implemented, and publically available. Finally we have KPIs and a foundation for more sophisticated metrics.
4. Plan to focus on the developer audience

Team members involved: 1

Developer Relations proposal agreed and ready to start implementation ECT to start conversion in Q1 through related quarterly goals.
27

Q4 - Engineering Community

Quarterly Objectives:
Successes
  • Prototype of Developer Hub: Dropped a quarterly goal on delivering a Data & Developer Hub prototype, due to the move of S Page to Reading.
  • Hackathon projects: Goal of hackathon was 8 projects; had 33 instead! It didn't require more work, just the willingness to work differently, and of course dozens of highly engaged developers.
  • Google Summer of Code/Outreachy: Last winter, Niharika Kohli volunteered to help as co-org admin of Google Summer of Code and Outreachy in her volunteer capacity. This has allowed Quim Gil to focus on other tasks without any negative impact in these programs.
28

Q4 - Engineering Community

Quarterly Objectives:
Misses

Collaboration with Wikimedia France: They were key for Hackathon’s success, but their introduction of own ways of working increased our effort for no clear benefit.

Learning: We need to be more systematic defining expectations while still encouraging local initiative and a local style of event. We have shared this learning with Wikimedia Israel already, and we have agreed on documenting a guide for hackathon organizers as we go.


For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department


29

Q4 - Wikipedia Library (2.3 FTE)[edit]

Objective Measure of success Status
1. Enhance editor access to research

Team members involved: 4

1) 8 new publishers; 2) 3 non-English publishers; 3) 400 accounts distributed; 4) 100 new users; 5) one new coordinator; 6) Plan to measure citation-impact; 7) Define metric for signup speed 1)Hit        6) Hit

2)Hit        7) Hit

3)75%

4)Hit

5)Hit

2. Scale the English Library model globally

Team members involved: 4

1) 20 interested languages; 2) 4 branches in consultations, 3) 3 branches in page setup; 4) Renew activity with 3 existing branches; 5) develop TWL-lite model; 6) 8 total branches All Hit
3. Share best practices and project models

Team members involved: 2

1) 5 New Visiting Scholars; 2) US WVS to Wiki Ed; 3) Complete Interns model; 4) New Branches Project Menu and FAQ; 5) Initiate documentation on 4 community project models; 6) Consult on WVU Library WIR Grant All Hit
30

Q4 - Wikipedia Library (2.3 FTE)


Objective Measure of success Status
4. Extend our network of influence

Team members involved: 2

1) Present at 4 major conferences; 2) Exhibit at ALA; 3) scope GLAM-Wiki role within Foundation; 4) 3 WMF Partnerships Brunches; 5) Outline draft of WMF partnership policy 1) Hit

2) Hit

3) Hit

4) 67%

5) Hit

5. Plan for a future TWL platform

Team members involved: 4

1) Write development roadmap for LibraryCard Platform, Citation metrics, IP authentication (OAuth), and Reference tooltip guide; 2) Scope timeline for URL resolver, Link Rot, Echo notifications, and Citoid support All Hit
31

Q4 - Wikipedia Library

Quarterly Objectives:
Successes

Global branches are starting up at scale

  • French language volunteers blitzed to set up 3 signups, redesigning the signup, train 2 coordinators, and write a blog post… in 1 week
  • Turkish WP started a branch without even telling us, just using our setup kit
  • Discovering new project models across local languages, helping us build a menu of projects for communities to try, especially smaller wikis

Great networking presence

  • GLAM-Wiki, DPLAFest, Wikimedia Conference, Project MUSE Keynote, and ALA where we exhibited and presented

Added 11,931 links to partner resources in Q4


32

Q4 - Wikipedia Library

Quarterly Objectives:
Misses

Despite nearly 40 partnerships, we have only 300 to 400 quarterly signups. (290 received access this quarter with another 158 awaiting distribution)

Cause -> Solution

  • Lack of direct invitation & publicity  ->  need to use Echo or central notice
  • Individual signups  ->  need to make simultaneous-signup possible*
  • Hacky password distribution  ->  need to streamline access transfer*
  • Some partnerships are low value  ->  need to focus on impactful donations
  • Unfamiliarity with donated content  ->  need integrated search exploration*
  • Global rollout still ongoing  ->  need more global branches with partners
  • High requirements  ->  we only lowered them mid-way through the quarter

Starred (*) solutions will be incorporated into our Library Card Platform


For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department


33

Advancement department[edit]

Approximate team size during this quarter: 12

  • Fundraising (general)
  • Online Fundraising
  • Fundraising Operations
  • Major Gifts & Foundations
  • Fundraising Tech
  • Wikipedia Zero
  • Strategic Partnerships
34

Q4 - Advancement: Fundraising (general)[edit]

Objective Measure of Success Status
Raise $7 million in Q4 primarily through:

1) Banner and Email campaigns in Latvia, Romania, and Slovakia. Testing in Ukraine, China and South America.

2) Transfers from Chapters.

2) Foundations, Major Gifts: Send proposals to four new institutional donors and secure a multi-year commitment from one new funder


Team members involved: 10


Raise $58.5 million in FY 2014-15; $7 million this quarter On Track; $8 million raised; Received transfer from Chapter; New Major funder secured.


(Described further in Online Fundraising and Major Gifts sections)

35

 Q4 - Advancement: Fundraising (general)


Objective Measure of Success Status
Endowment Prep: Be ready to start fundraising for an endowment by the start of the new fiscal year. Select entity; Develop messaging and strategic purpose; Recruit at least 3 advisors; Develop fundraising plan

Team members involved: 1

Develop a strategy for the long term financial health of the organization; Get signoff from the board on a strategy Postponed; Endowment contractor ended contract; JD for Endowment Director is ready to post. We have completed research for an entity selection and we have proposals from 3 marketing firms for messaging.
Implement Astropay

Team members involved: 5

Expand our fundraising infrastructure Contract signed; Tech work has begun
Partnerships:
  1. Develop Partnership Plan
  2. Secure at least one non-profit partnership


Develop a strategy for the long term financial health of the organization NOTE: This has been moved to Partnerships.
36


Q4 - Advancement

Quarterly Objectives:
Successes
2014-2015 Fiscal Year Rough Totals

The fundraising team exceeded the goal of $58.5M by $16.5M, raising ~$75M** this fiscal year.

Online $58.0
Major Gifts $10.5
Chapters   $6.5
Total $75.0
**Preliminary approximate totals.  These numbers will change after fiscal year reconciliation.  


37

Q4 - Advancement

Quarterly Objectives:
Misses

Endowment

  • Endowment contractor ended contract
  • JD for Endowment Director is ready to post
  • We have completed research for an entity selection and we have proposals from 3 marketing firms for messaging.

Learning: We decided to move forward with looking for a permanent (FTE) Endowment Director instead of another contract to hopefully have someone in this role long-term.


For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department


38

Q4 - Online Fundraising (4 FTE)[edit]

Objective Measure of success Status
Raise $7 million in Q4 through:


Banner and Email campaigns in Latvia, Romania, and Slovakia. Testing in Ukraine, China and South America.


Team members involved: 4

$7M raised Exceeded FY1415 goal by $13M
Implement Astropay to expand our fundraising infrastructure


Team members involved: 3

Test donations with local donors We were able to test donations via credit card in Brazil
39

Q4 - Online Fundraising

Quarterly Objectives:
Successes
  • Raised the quarterly (and annual) goal ahead of schedule
  • $2.5 million in Q4
  • Desktop, mobile, and email campaigns in Spain, Latvia, Romania, Slovakia
  • World-wide low-level campaigns
  • Additional banner tests to help with tech prioritization and overall strategy

Learning: Setting up tests as early as possible for new methods that require significant work greatly improves our prioritization and annual planning process.

Learning: Banner campaigns run by community members can significantly interrupt the staff’s workflow (fundraising team, legal, community, others).  We need support coordinating and evaluating community campaigns.

40


Q4 - Online Fundraising

Quarterly Objectives:
Misses

All of the prep work to begin the Astropay integration was complete (contract negotiation, legal/finance sign off, acceptance criteria, documentation). With the Astropay integration, we were able to QA the Brazilian donation form and test with individual volunteers.

Without the technical auditing and listening pieces complete, we were unable to run a banner test in Brazil.

Learning: When setting goals that are dependent on multiple teams, we should continuously check in on milestones and getting firm deadlines at each step in the process so that each team can plan accordingly.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department


41

Q4 - Fundraising Operations (3 FTE)[edit]

Objective Measure of success Status
Implement payment processor for Latin America

Team involved: 1 + tech

Process payments in Brazil Setup complete//processing is not live (objective was redefined for just Brazilian Credit Cards and met)
Preparing to re-integrate Amazon in early Q1.

Team involved: 2 + tech

Scope the effort, understand the changes, add to roadmap Reintegration is on the roadmap
Re-launch the Wikipedia store website

Team involved: 2 + volunteers

Improve website layout

Release 5 new products Launch Marketing Plan


Website launched with new products and gross sales increased by 249% compared to last quarter.
Donor Services ready for Astropay


Team involved: 8

DS team ready to help Brazilian donors Training, new processes, translations and documentation complete. Waiting for launch.
42

Q4 - Fundraising Operations

Quarterly Objectives:
Successes

Wikipedia Store relaunch

Products: 2 shirts, pencils, notebook and baby onesies

Gross Sales: ~$17k in Q4

(growth of 249% over Q3 and 144% over last year’s Q4).

Strategy: Website redesign, name change (from Wikimedia Shop to Wikipedia Store), new products and marketing efforts (Facebook, Twitter, Instagram, Banners,...)


Wikipedia Store Sales
Gross Sales

Learning:

  • Website improvement and name change were critical for brand recognition and partnerships
  • New mission aligned products have become bestsellers (Free knowledge T-shirt).
  • Store banners don’t have as much conversion as Fundraising banners.

Victoria Shchepakina project managed this effort and was involved from end to end. Many other people contributed to this project and were instrumental in this launch, including Heather Walls, Michael Guss, May Galloway, Nirzar P., Michelle N. and Gustavo O. and other volunteers.


43

Q4 - Fundraising Operations

Core workflows
and metrics

Donor Services:

- 4,874 total tickets received (25% increase in tickets in Q4 2014)

- Avg. response time 8 hours


Most common emails:

- 12.9% (632) Help on payments or donation flow

- 11.5% (563) Response to nicest compliments (tagged "we_love_you_too")


Most common complaint/problem tickets:

- Bulk email  (5%)

- Credit Card Rejections (1.99%)

- Unintended recurring donations (1.1%)

- Geolocation (0.90%)

- Banner complaints: (0.73%)


For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department


44

Q4 - Major Gifts & Foundations (3.2 FTE)[edit]

Objective Measure of success Status
Objective 1 - Send proposals to four new institutional donors

Team members involved: 3

Four proposals sent


Goal achieved and surpassed: Proposals out to six institutional donors



Objective 2 - Secure a multi-year commitment from one new funder

Team members involved: 3

Multi-year commitment secured Goal achieved: Teterev Foundation committed to 3 year, $151,000 grant
Objective 3 - Meet year-end fundraising goal

Team members involved: 4

Raise $7 million in FY 14-15 Goal achieved and surpassed: $10.2 million raised as of May 30th
45

Q4 - Major Gifts & Foundations

Quarterly Objectives:
Successes
  • $99k in revenue in first months of Humble Bundle agreement
    • Approach: Willingness to explore new types of partnerships & make time to cultivate these relationships
  • 3 year - $151k gift (an upgrade) from Teterev Foundation
    • People: Jonathan Curiel wrote custom proposals and reports
    • Approach: Cultivate existing donors throughout grant period, not just at the time of renewal
  • Donor Cultivation (i.e., no $ ask) event in NYC at Print Wikipedia exhibition
    • People: Artist & Gallery Owner asked to host us, donors felt flattered to be invited
  • Successful transition to Development Analyst maternity leave fill-in
    • People: Rosie Lewis planned and implemented the transition
46

Q4 - Major Gifts & Foundations

Quarterly Objectives:
Misses
  • Civi upgrades and reporting improvements still not done.  Dependent on FR-Tech capacity.  Overall, a good decision to push down the work queue but still leaves MGF with limited/clunky reporting (ie donor prospecting) capabilities within CiviCRM in the new FY.

Learning: We need to continue to improve our ability to plan an achievable amount of tech work each quarter.


47

Q4 - Major Gifts & Foundations

Core workflows
and metrics

Restricted Grants Discussions

  • Restricted grant process begun with Knight Foundation about funding Search & Discovery

Tool & process improvements (with assistance from Fr-Tech & Fr-Ops)

  • Donation processing improvements for efficiency and accuracy
    • DAFs, international checks, stock & wire donations, batch imports
  • CiviCRM features turned on for the first time or fixed or improved
    • automatic reminders and donor prospect information fields
  • New event management tool selected and implementation in progress
    • streamlines invitations, RSVPS, donations, and guest info tracking

Also

  • Grant reporting, donor & prospect research, early-stage event planning, legacy donor cultivation, and more!

For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department


48

Q4 - Fundraising Tech (6 FTE)[edit]

Objective Measure of success Status
Expand reach of fundraising capability to new donors and provide better experience for existing donors in Latin America

Team members involved: 3

New payment processor integration with key metrics:
  • Can process donations from Latin American cards/accounts (Brazil, Mexico, Chile, Peru, Colombia, Uruguay, Argentina)
  • Donation information (including donations, refunds, and chargebacks) appears in CiviCRM through realtime and nightly audit as appropriate


Original objective not met as written. Main objective requirements were quickly redefined in Q4, to only include Credit Card payments in Brazil. The new minimum requirements were met in Q4.


Other countries and methods through AstroPay will be re-prioritized based on projected donation numbers, and the planned timing of the relevant country campaign(s).

49

Q4 - Fundraising Tech

Quarterly Objectives:
Successes

Successfully made two new hires in early May, increasing the team size from four to six.

Lesson: Hiring and subsequent onboarding efforts could be made much more visible in all planning and tracking systems used by Engineering.

Adam Wight has once again demonstrated his excellent teaching abilities with the new team members; Though they still have a lot to learn about various fundraising systems, they have become useful members of the team much faster than we anticipated.

Paid down a significant amount of technical debt in the donation pipeline, with minimal disruption.

Lesson: Complex incremental refactoring is much easier to work in with the rest of the goals, when the team is both strict with itself about keeping their refactoring patches completely separate from other work, and fanatical about writing backend integration tests before the refactoring patches are submitted.

Elliott Eggleston has been instrumental both in the actual refactoring (particularly in the DonationInterface extension) and in deploying the refactored code with surprisingly few disruptions.


50

Q4 - Fundraising Tech

Quarterly Objectives:
Misses

The single reported fr-tech goal for Q4 was not completed as it was originally written.

The fundraising team noticed early in the quarter that it was unrealistic for tech to support all countries and payment methods available through AstroPay before the end of Q4, and internal expectations were adjusted accordingly.  This “miss” is generally considered by the team as more of a success than a failure because we hit the redefined goal.

Lessons:  It would be helpful to break out multiple milestones throughout a large project to help coordinate planning across fundraising teams.

One binary goal for the whole quarter does not adequately represent the fr-tech team’s work (Already fixed going forward: Engineering teams are no longer constrained to one goal per quarter). Additionally, in the future, published goals will be written in such a way that they represent something closer to an MVP, with all stretch goals and associated KPIs removed.


For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department


51

Q4 - Wikipedia Zero (6 FTE)[edit]

Objective Measurement of Success Status
Expand reach of Wikipedia Zero in the GS


Team members involved: 4

Reach extra 20 million page views per month - 94 total PVs Dropped to 70M page views a month. Possible causes: seasonality and HTTPS migration (requires further study)
Keep our service in sync with our partners IP updates


Team member involved: 6

Allow partners to edit their IP ranges on Zero self-serve portal Engineering work has been completed enabling partners to utilize the Portal for IPs updates.
Increase Wikipedia’s footprint and availability


Team members involved: 4

Launch total of 50 partners Zero team has launched 68 partners - 2 new launches in Q4
52

Q4 - Wikipedia Zero


Objective Measurement of Success Status
Android preload


Team members involved: 4

Get agreement of at least one OEM in the GS


Multilaser - local Brazilian OEM was re-engaged and confirmed the agreement to preload our app but has not signed a formal agreement yet.
Preserve free Wikipedia after HTTPS rollout


Team members involved: 4

Convert all Wikipedia Zero partners zero-rating to IPs and enable HTTPs to provide secure access after the HTTPS by default rollout. Zero partners that have the ability to do IPs based zero-rated were converted.
53


Q4 - Wikipedia Zero

Quarterly Objectives:
Successes

Marketing redesign for Portal: The Zero team successfully designed a new marketing support section that helps partners to create post-launch campaigns based on global and local event promotions. This is part of effort to increase awareness in local markets and also organic page view growth.

Wikipedia awareness survey: The Zero team worked with VotoMobile to create a survey for measuring awareness and usage of Wikipedia and Wikipedia Zero targeted to at least one African market.  This survey is conducted via phone, and it's not directed toward Wikipedia users. The goal is to learn more about new and potential users of Wikipedia. The survey is ready to be conducted in Q1 with proper budget approval.

Improving user interface: The Zero team received partner feedback regarding confusion about possible charges when using our service. We jointly designed a better interface that took into account the unique needs of the GS subscribers. This design work is nearing completion, and should be implemented in Q1 pending engineering support.

Highlights: Dan Foy lead the major efforts on the portal and survey efforts. Anne Gomez also provided invaluable support to the team with improving the user interface project.


54

Q4 - Wikipedia Zero

Quarterly Objectives:
Misses

Page view miss: We went from 75M in May to 70M in June without adding or removing partners. It appears to be seasonal variance, although the HTTPS migration may have reduced some traffic from low end smartphones. Internet.org’s expansion in the GS markets derailed many of our deals in process for the quarter.

Pre-load miss: The Zero team followed business leads in the GS to ship a first Android preload. Even though we engaged local OEMs in the GS, we didn’t understand how the app preload business works. As a consequence, we were unable to complete deals in our goal timeframe.

Learnings: We have identified that financial costs and Wikipedia awareness are the two main challenges to the success of our app preload. There are significant costs to get the app preloaded, and then additional costs for in-phone promotion and offline marketing.  Also, significant segments of GS markets have limited knowledge about WP, and often are not familiar with app usage on smartphones.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department

55

Q4 - Strategic Partnerships (1 FTE)[edit]

Objective Measurement of Success Status
Secure at least one non-profit partnership


Team members involved: 1


(seed a framework for strategic collaborations to grow the quality, quantity and diversity of content made available across all our projects)

Increased diversity, quality and quantity  of content across projects Completed
56

Q4 - Strategic Partnerships

Quarterly Review:
Successes

Successes:

Non-profit partnership: Secured partnership between a global organization (World Bank) and Wikimedia Foundation. This collaboration will open access to new structured content to be added and curated on Wikidata.

Generation Z outreach:  Secured Wikipedia Zone in the new SFPL Teen Tech Center (launching September 2015 for Back to School). This is part of a global opportunity with Mozilla’s HIVE Learning Network (see appendix of the Advancement team’s quarterly review slides for their global footprint).

NGO partnership: OECD agrees to review their copyright policy on selected data sets to accommodate WP’s CC0 needs. They seek deep engagement with the projects. Partnership agreement under development. Implementation expected 2Q-16

Learnings:

(1) NGOs respond positively to the fact that we are both a high visibility platform for their content as well as a valuable resource for their own projects.

(2) The pace at which we onboard new content partners is contingent on community support and participation - something we are building into this new framework.


57

Discovery (13 FTE - 3 new hires)[edit]

Quarterly Objectives
Summary
Objective (FTE) Measure of success Status
Search Instrumentation (5) Initial construction of a dashboard and establish core KPI set Alpha Launched May 12th,

Key KPI’s presented June 19th

Search Research & Analysis (3) Create initial feedback paths and initial survey on search Launched June 23rd, 1200 responses in 7 days
Hire and form team for search and Discovery (6) Hire relevancy engineers, data analysts, ux, backend. Create channels of communication and process. Hired and recruited internal for key positions (3 new)

Created lists, irc, phabricator boards, team page

Wikidata Query Service beta (2) Launch initial beta version of service on labs. Launched May 30th

Presented in lightning talks

Maps service prototype (2) Create prototypes of tile services. Demoed at hackathon, State of the map, WMF lightning talks
58

Q4 - Discovery

Quarterly Objectives:
Successes
  • Oliver Keyes and Dan Garry worked aggressively on our Search instrumentation goal to prep for our initial presentation June 15th.  We support community engagement and transparency in our product roadmap as a rule so we created a public dashboard to measure our key KPI’s.  (User satisfaction, User-perceived load time, No results rate, API usage).
  • As part of our research and analysis effort, Moiz Syed and team crafted user satisfaction surveys on key flows for search. 1200 responses in 7 days.  Mobile and Wikipedia.org surveys are in process. Summary results will be presented at Wikimania.
  • Tomasz Finc managed hiring well and led the effort around our core competencies of data analytics, relevancy, back-end support, UX front-end and key leadership roles. He also handled transitioning in from current internal employees for a smooth transition. We also created the channels needed for communicating externally and internally.
  • Stas Malyshev established and rolled out on time our initial beta for Wikidata query service
  • Yuri Astrakhan and Max Semenik prototyped the Maps Tile Service and presented at the Hackathon in Lyon, Open Maps Conference in NY and in subsequent discussions at WMF events.  We also reviewed initial work to get to an initial production service.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Discovery team


59

Reading (team size: 27)[edit]

Quarterly Objectives
Summary
Objective Measure of success Status
Android and iOS:

Provide readers with the information they are looking for more quickly via a prototype that making taps on Wikipedia links show article information (more info)


Team members involved: 8

Engagement:

Link taps per user per session will increase (more than page views will decrease)

Link previews out on Android beta with 30% increase in links clicked. 60% of clicks continue on to new article. (More clicks but 20% fewer articles read)
Link taps followed through to destination per user per session will not decrease. This is not a good metric because this behavior is actually desirable per our impact theory (reducing links followed indicates faster resolution)

(Clarified Pre-reorg Mobile goals)


60

Q4 - Reading

Quarterly Objectives
Summary
Objective Measure of success Status
Mobile Web:

Allow users to create individualized collections of articles and share them with others by launching Gather on English Wikipedia, Mobile web


Team members involved: 5

Contributions:

>10,000 creators/month rate on production

>1000 shares per month rate

Current numbers are 20x goal (extrapolated from beta)

Drove users to ‘beta’ and avoid the cost of testing a new feature on production.

We would have exceeded numeric goals on stable, but have since determined this bar was too low to warrant promotion at this stage, and our focus has changed.

(Clarified Pre-reorg Mobile goals)

61

Q4 - Reading

Quarterly Objectives:
Successes

Q1 Design > Android Link Preview > Prototype

Link preview prototype for Wikipedia Android app.png
62

Q4 - Reading

Quarterly Objectives:
Successes

Gather: public feed

Collections (Gather) screenshot, Wikipedia (Beta).png
63

Q4 - Reading

Quarterly Objectives:
Successes
Full Q4 ‘retrospective’ here

Team

  • Team dealt with re-org extremely well, adjusting to new scope and loss of product resources with updated planning and management (Kristen, Jon Katz, Adam, Dmitry, Joaquin, Corey, Bryan)
  • Design integrated with core workflows (Vibha, Kaity, Sherah, iOS & Android teams)
  • Learning: Team is flexible, motivated and able to collaborate in difficult circumstances

Distribution

  • Android: 15% increase in installs (up to 13.3M from 11.5M) via PR, new app features & Google Play Features
  • Learning: We can drive downloads via PR & Google Play at limited scale

Engagement

  • Gather beta performance exceeded initial expectations dramatically
  • Learning: New features need to have more substantial impact

Quality & Developer Productivity

  • Improved QA, test environment and release process led to 5 iOS App releases in Q4 v 1 in Q3
  • Learning: Investment in testing and productivity pays off
64

Q4 - Reading

Quarterly Objectives:
Successes
Full Q4 ‘retrospective’ here

Infrastructure

  • Updated API to improve JSON format, better support for “continuing” queries and versioning
Wow, this list of changes fixes a lot of things that I've hated about our API for a long time (no more star properties! no more setIndexedTagName errors! consistent JSON formatting!). I feel like it's Christmas! Thanks for all your hard work on this Brad! It is much appreciated.[2]

Community Tech

  • Hired 2 developers and 1 Manager (Welcome Frances, Niharika and Ryan)

UX Standardization

  • OOJS UI: We continued work in expanding the library. Some of the designed components include date picker, dropdown menu, stepper, toggle. Implementation pending resources.
  • Living Style Guide (Uses Blueprint skin that Standardization team built) Data & Developer hub started using Blueprint skin and has hopes to integrate the skin with Mediawiki. Currently going through the list of blockers and addressing them before integration review.
65

Q4 - Reading

Quarterly Objectives:
Misses
Full Q4 ‘retrospective’ here

Team

  • Reorg uncertainty hurt velocity and direction (see success above)
  • Learning: We did well in managing impact but it was still disruptive

Distribution

  • iOS: Installs are declining (went from ~180K day to ~140K day) with no PR bump
  • Learning: We need to understand differences in marketing across platforms

Engagement

  • Page views and other metrics flat at best
  • Learning: Need long term strategy to increase our impact on the world
  • We exceeded Gather success metrics but they weren’t the right metrics
  • Learning: Metrics need to be aligned to goals (to be fair, the goals changed)

Quality

  • iOS ratings fell as a result of crashes (3.9 to 2.6 out of 5, since recovering)
  • Learning: Quality matters to our users! (Quality improvements called out in Success slides
66


Q4 - Reading

Quarterly Objectives:
Misses
Full Q4 ‘retrospective’ here

Infrastructure

  • Sharing a manager and an engineer with other teams has kept new team from working together effectively as a unit.

Community Tech

  • Budget woes and focus stopped us from moving forward as fast as we would have liked.

UX Standardization

  • We need some exec focus on resolving design leadership issues.

Learnings: We needed to address lingering re-org issues more effectively as an organization

For more detail, see the slide deck and minutes from the quarterly review meeting of the Reading team

67

Editing department[edit]

Approximate team size during this quarter: 30 FTE

  • Collaboration
  • Language Engineering
  • Multimedia
  • Parsing
  • VisualEditor
68

Q4 - Collaboration team, Editing[edit]

Objective Measure of success Status
Make Flow a discussion system that can handle all of the core Discussion use cases, to a level that can be used on active project and article talk pages


Team members involved: 9

(+ 1 from other departments)

One Wikipedia language using Flow on every talk page in the project and/or article talk namespace, enabled with the active support of the community on that language, by the end of June 2015 – T99117 Catalan Wikipedia is using Flow on all talk pages in the project namespace (Viquipèdia Discussió), as of June 30. Catalan is also using Flow on the five Viquipèdia:La taverna (village pump) pages.

Example link: https://ca.wikipedia.org/wiki/Viquip%C3%A8dia:La_taverna/Propostes

69

Q4 - Collaboration team, Editing

Quarterly Objectives
Successes

This quarter we converted a namespace on an active wiki to use Flow instead of wikitext talk pages. We worked with the community at Catalan Viquipèdia to make sure we addressed all of their use cases, and we built several features -- Mark as resolved, edit patrolling and moving a Flow board.

We built a new side rail feature to hold the metadata templates (aka “the yellow boxes”) common on talk pages. The side rail can be collapsed to allow full-width viewing of Flow pages, a much-requested feature.

We began conducting generative research with experienced editors to plan our approach to Workflows, an important feature set for our most active users.


70

Q4 - Collaboration team, Editing

Quarterly Objectives
Successes
Screenshot from Catalan, showing new features requested by the community -- marking a discussion as resolved, and the side rail for template boxes. Not pictured: Clicking the X at the top right of the side rail collapses the rail, making the page full-width.
Screenshot: https://ca.wikipedia.org/wiki/Viquip%C3%A8dia:La_taverna/Tecnicismes of GPL and MIT software, and content used under CC-BY-SA
Template image: https://ca.wikipedia.org/wiki/Fitxer:Icona_de_la_Cafetera_de_la_Viquip%C3%A8dia.png used under LGPL.


71

Q4 - Collaboration team, Editing

Quarterly Objectives
Misses

We’re in the process of converting LiquidThreads pages to Flow boards on MediaWiki.org, an important step towards retiring a previous discussion system that we’re not working on anymore. MediaWiki.org is our first large conversion, with 1700 LQT pages. As we’ve been converting pages, we’ve found bugs in the process that have slowed down completion of this task. We’ve made good progress (250 pages converted), but we’d hoped to be done by now.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Editing team

72

Q4 - Language team, Editing[edit]

Objective Measure of success Status
Strengthen the Content Translation infrastructure


Team members involved: 6

Facilitate the creation of more types of links in a reliable way – T76455 Core improvements done. Advanced cases planned for next quarter.
Add [[Special:CXStats]] with selected article publishing statistics – T99230 Done
Make Content Translation available to more users and maintain a high quality of service availability


Team members involved: 6

Deployed on all Wikipedias as a beta-feature – T88408 Complete; all done in-quarter except English (done on 7th July).
Prep. release from beta to production on one Wikipedia – T102107 We compiled the Catalan needs for graduation without major blockers.
Design CX integration into the editing workflow and provide entry points to involve readers – T87867, T99071 Done — Effectiveness measured and positive impact on metrics.
73

Q4 - Language team, Editing

Quarterly Objectives
Successes
Wikipedia CX Stats March vs June 2015 - Articles published.png
  • Content Translation was made available in 266 further Wikipedias this quarter. All Wikipedias now have CX.
  • 8000 total articles have been created with CX. At present, 1000 new translations are created per week (last quarter: 100), after CX was enabled on the top 10 Wikipedias in the last few weeks. Deletion rate for the tool’s articles at 7% (down from 8% in May) is favourable.
  • Notification system was introduced, for increased interaction with users. Community interactions throughout this period have been positive, constructive and participatory.
  • We partnered with the Research team about article suggestions; article suggestion is an intended way users can begin translations and results from this experiment will guide the development of this feature.
74
Wikipedia CX Stats Q4 2014-15 - Users per number of translations.png

Q4 - Language team, Editing

Quarterly Objectives
Misses
  • Last mile experience for users can be even better; sporadic publishing failures (working with Parsing team on this) and other issues (like local scripts, gadgets etc.) that get in the way of CX are a challenge.
  • Most users published less than 5 articles. This is expected from a newly introduced tool, but we need to continue the momentum on developing features that engage users early and retain interest.
  • No clarity on analytics support; more so after the reorg.
  • Team members are overworked, especially with the hiring cuts and removal of existing positions. Non-CX projects are now included in sprints but actual execution of maintenance work is far from reality (small maintenance OK, bigger is not).


For more detail, see the slide deck and minutes from the quarterly review meeting of the Editing team

75

Q4 - Multimedia team, Editing[edit]

Objective Measure of success Status
Make uploading media an easy, integrated process


Team members involved: 4

Users can upload media to Commons by clicking upload or using drag-and-drop inside VisualEditor and the wikitext editor whilst they edit, by end September 2015 – T40030 Underway, on track
76

Q4 - Multimedia team, Editing

Quarterly Objectives
Successes

We are on track with our work. This quarter, we made solid progress on the modernisation of UploadWizard in planned incremental improvements. Our focus is on code quality and architecture, paying down our technical debt.


We decided to concentrate more on the ‘core’ product by generalising our approach. We are designing, planning, and making some code progress on a new MediaWiki upload API and modernised UX for the core upload tool.


This will replace parts of our uploading process, so that official uploading integrations into VisualEditor and mobile, and community tools like campaigns, can be served more flexibly and simply.


77

Q4 - Multimedia team, Editing

Quarterly Objectives
Misses

Getting UploadWizard fully refactored was more of an effort than anyone initially imagined, so there is still work to do there. We expected this to be done by the end of the coming quarter (two quarters’ work), but it may slip. Some of the projected effort will be obviated by the new upload APIs which we took on as part of this effort.


Outside our objectives, work on the longer-term priority of structured data (Wikibase) on Commons is not currently possible, though there is a plan in the works. We have yet to solidly establish the barriers or deliverables and thus a timeline.


For more detail, see the slide deck and minutes from the quarterly review meeting of the Editing team


78

Q4 - Parsing team, Editing[edit]

Objective Measure of success Status
Top goal: Identify & fix the most prominent remaining semantic roundtripping diffs


Team members involved: 3

At least 99.95% of the ~160K test pages roundtrip (wikitext → HTML → wikitext) without semantic errors in full roundtrip testing, by end June 2015. – T92643 Met goal as of June 23rd, 2015.

We were at 99.82% at start of quarter, and 99.75% end-June, 2014.

Announcement on wikitech-l

(Implicit objective)

Fix VE Q4 blocker bugs

Team members involved: 4

(+1 from other department)

VisualEditor 2014/15 Q4 blockers tasks depending on Parsoid are resolved. Met goal. There is currently one unclosed task that requires more discussion and clarification -- moved to FY15-16 Q1.
Improve Parsoid performance Handle large / pathological pages on which Parsoid is currently timing out (T75412,[3]T88915) Minimal work identifying reasons for timeouts on some pages.
79

Q4 - Parsing team, Editing

Quarterly Objectives
Successes

Based on gradual progress over the last year, at the start of the quarter, we expected we would be able to meet 99.95% semantically accurate round-tripping goal. But, it was not as easy as anticipated.


We hand coded ~100 test failures in terms of what caused the failure. This was critical to identify non-edge case failures that would be most useful to fix.


We had a lot of false semantic failure reports because our error classification algorithm was 2+ year old code and was overly complex and buggy in parts. It required a rewrite of pieces of that code to fix this.


80

Q4 - Parsing team, Editing

Quarterly Objectives
Misses

We were primarily focused on meeting our top goal and fixing VE blocker bugs (Arlo, Scott, Subbu, Marc). In addition, we undertook a lot of unplanned, but necessary, code cleanup (Arlo, Scott, Subbu). We also wanted to get some other longstanding work (CSS-based customization of Cite) finished which we did (Marc).


All the above, in combination with the reorg, some team changes, and a small team meant that we did not have any bandwidth to work on fixing problems with pages that were timing out, but we made progress improving perf. of the PEG tokenizer (Tim).


As for other sub-goals listed in T92643 that were dependent on VE’s goals and priorities, we made some progress towards supporting section editing (Arlo). That requires additional engineering work in VE, Parsoid, and RESTBase and is deferred for later in FY2015/16. Same with switching between HTML / wikitext editing modes in VE.


81

Q4 - Parsing team, Editing

Core workflows
and metrics
  • Ongoing deploys (up to 2 times a week)
    • 0.6 developer months of our time per quarter
  • Code maintenance, cleanup, easing technical debt, documentation
    • This quarter, we focused on this part more than usual and spent up to 3 developer months of our time doing maintenance and easing technical debt.
  • Maintenance of the OCG PDF backend (primarily Scott’s responsibility)
    • Up to 1 developer month of work spent doing this.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Editing team


82

Q4 - VisualEditor team, Editing[edit]

Objective Measure of success Status
Improve the editing experience for new and anonymous users on the English Wikipedia by giving them VisualEditor

Team members involved: 7

(+ 5 from other departments)

Maintain VisualEditor production service at our quality criteria. – Project 1113 Met
Run test of availability for new accounts on the English Wikipedia. – T90666 Completed
Gradual ramp-up of availability on the English Wikipedia for new users, from 5% to 100%. – T90664 Delayed by result 2. Have begun community discussion; on track for next quarter.
Make VisualEditor available on the English Wikipedia for anonymous users. – T90663 Delayed by result 3. Will begin once result 3 is secured.
83

Q4 - VisualEditor team, Editing

Quarterly Objectives
Successes

We met our on-going principal task of maintaining the quality of our production editing service against our quality criteria whilst suffering a significant reduction in resourcing.

We made significant alterations to the workflow of citations and link editing based on user research. These usability improvements have notably simplified and enhanced the user experience.

We continued to trial new ways of working to involve the editing and developer communities more deeply in our planning and prioritisation of our work to be aligned with their wants and needs.


84

Q4 - VisualEditor team, Editing

Quarterly Objectives
Misses

User research uncovered big usability misses in some of the new functionality added last quarter, as well as in existing areas of focus. This led to significant proposed re-designs of these features being generated. Most of these were accepted as blockers which could cause the A/B test to fail if unaddressed. The time taken to build, re-test & deploy these, albeit rapid, collectively delayed the A/B test by two months. This had consequent impacts on the post-test tasks.

Action: We in Editing have set up processes to involve User Research more consistently and earlier in the design and development process. We expect that this change will avoid a recurrence of this kind of rapid re-re-design in future.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Editing team

85

Infrastructure department[edit]

  • Analytics
  • Release Engineering
  • Services
  • Technical Operations, Labs
86

Q4 - Analytics (6 FTE)[edit]

Objective Measure of success Status
Provide metrics, a dashboard and ad-hoc support for Editing Team The dashboard for Editing is used over time (measured in Pageviews for the dashboard) GREEN - The objectives were completed, and we see an average 1.9 of unique visitors per day (114 visits in Q4)



87

Q4 - Analytics

Quarterly Objectives:
Successes

Dan Andreescu worked with Aaron Halfaker from the Research & Data team to vet and sanitize the data collected in EventLogging from the Visual and Wikitext Editors.

We added “Failure Types by User Types” to the dashboard.

Learning:  The Editing Dashboard isn’t getting used as often as we expected.  More work needs to be done with stakeholders to define the right metrics to visualize for the dashboard to be useful on a daily basis.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Infrastructure department


88

Q4 - Release Engineering (6 FTE)[edit]

Objective Measure of success Status
Release MediaWiki 1.25


Team: 1

The software that runs Wikipedia and sister projects has a successful release for third-party users. Success! (Announcement)
Deployment Tooling: Assessment and Planning


Team: 4

A plan for how to attack the problem of multiple deploy tools that don’t meet everyone’s needs. Success! Read the assessment and plan for next quarter.
Create the Staging cluster


Team: 4

A cluster that more closely mimics production with fewer differences than the Beta Cluster. Paused. We were unable to complete this during the quarter and paused it mid-way through so other goals wouldn’t be jeopardized.
Isolated CI Infrastructure


Team: 1 (+1 external)

Working Proof of Concept (instances booting with jobs running) Delayed. However, the Labs team has improved their infra. and there is a proof of concept of spawning instances.
89

Q4 - Release Engineering

Quarterly Objectives:
Successes


In late May the team had its first team offsite (directly before the Lyon Hackathon) 10 months after the team’s inception. This activity had multiple successes:

1) solidifying team morale and identity,

2) group assessment of our team’s needs with help of Team Practices Group

3) a plan of action to address those needs.


The team made the right (but hard) decision to pause the staging project in an effort to accomplish other goals.


90

Q4 - Release Engineering

Quarterly Objectives:
Misses

The team broadly estimated the work needed to accomplish our goals. Implementation needed to complete each project was largely siloed.

We’re addressing this issue by pairing regularly on triage and implementation so that no one team member is tasked with moving a project forward—if a “baton” is dropped there will be someone else on the team with the right knowledge to pick it up. See: the staging and isolated CI instances projects.

The team lost a backfill position (aka: total team size was reduced from 7 to 6 for the next fiscal year). This will cause us to not do or delay project requests from other teams. Full list in-progress.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Infrastructure department


91

Q4 - Services (3 FTE)[edit]

Objective Measure of success Status
API support for mobile VE

Team members involved: 2

Develop and deploy section retrieval and save API


Built and partially deployed API

VE team discovered additional requirements, might tackle those next quarter.

Support Parsoid and VE teams in reducing HTML size

Separate metadata from HTML.

Team members involved: 0

Provide storage and API support once Parsoid & VE are ready to work with separate metadata Deferred: VE and Parsoid teams did not have time to work on this.
Streamline and mentor service development / deployment

Team members involved: 2

Half the number of steps needed during service code deployment


Guide and support service developments

Reduced number of code deploy steps from 11 to 3 (using Docker, Ansible)

Graphoid deployed & exposed through RESTbase

Mobile App service ready for deployment

92


Q4 - Services

Quarterly Objectives:
Successes

Early in the quarter, we Introduced a global /api/ namespace, and exposed the REST API at /api/rest_v1/. This greatly reduced time to first byte by

  • eliminating extra DNS lookups,
  • using geo-distributed HTTPS termination, and
  • supporting SPDY connection sharing with HTTPS.

Marko Obrovac helped Marielle Volz of the editing team clean up and improve the Citoid service used to convert URLs to citation metadata in VisualEditor.


Towards the end of the quarter, we hired Petr Pchelko (Russia) as a Software Engineer (contractor), bringing the team size to four FTE.


93

Q4 - Services

Quarterly Objectives:
Misses

The VisualEditor team did not have as much time for mobile editing work as expected, which made our VisualEditor goals difficult to achieve. We did create the planned section edit APIs, but more work is needed in VisualEditor and Parsoid before we can reduce HTML size further and enable section editing in VisualEditor.

Expanding the Cassandra cluster turned out to be harder than expected, as we ran into stability issues with larger instances. Going forward, we will keep instance sizes moderate with more aggressive horizontal scaling.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Infrastructure department


94

Q4 - Infrastructure / TechOps (18 FTE)[edit]

Objective Measure of success Status
Streamline deployment steps for SOA services


Team members involved: 5

Reduce the number of individual manual service deployments steps from currently (typically) 11 to <= 5 using consolidation, increased abstraction and automation A typical deployment is now a 5 step, guided process
Implement a configuration discovery system to aid fully automatic configuration of load balancing, Varnish caching and availability and metrics monitoring in one automated process etcd discovery system has been deployed and integrated with PyBal and Varnish
Evaluation of container systems and deployment system options Early evaluation of deployment systems (with RelEng) is now available, but container systems were not evaluated
95
Quarterly Objectives:
Successes

Q4 - Infrastructure / TechOps

After some initial struggles, we were able to successfully define and implement our quarterly objective by focussing on the current obstacles around deploying and managing new services, rather than starting with technology choices. Thanks to Marko (Services) for helping with this observation.

We managed to hit our goal of reducing the number of deployment steps for new services by:

  • defining a clear process for acquiring all required information from the developers early on
  • developing a new layer of abstraction for services in the form of the generic service::node Puppet module
  • providing a new_wmf_service.py script that guides service developers through many steps and creates a template to work with.

The introduction of the new etcd service discovery system takes care of many challenges people inside and outside Operations have in deploying new services or nodes to load balancing and monitoring, by making these changes automatic and machine editable. It’s an important building block for future tooling to be developed by Ops, RelEng and Services.


96

Q4 - Infrastructure / TechOps

Quarterly Objectives:
Misses

Although we assisted the Release Engineering team in their goal of formulating a plan for a generic software deployment tool, we did not also start the (related) evaluation of container systems - we redefined the goal as it was deemed too large in scope, and we lacked the necessary bandwidth. The new processes for coordination and definition of goals that are being instated should avoid this in the future.

A related experimental goal for next quarter is to investigate and implement a container based distributed cluster environment in ToolLabs (initially). This work will be done with production use in mind as well, and should prove to be useful for both environments.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Infrastructure department

97

Q4 - Infrastructure / Labs[edit]

Quarterly Objectives
Summary
Objective Measure of success Status
Improve reliability of ToolLabs


Team members involved: 3

ToolLabs has at least 99.5% provable, measured uptime for each individual 'service' that ToolLabs provides its users Cumulative average availability from May 1 - June 30 was an appalling 97.589% after a long storage outage.
Wikimedia ToolLabs availability May-June 2015.png
98

Q4 - Infrastructure / Labs

Quarterly Objectives:
Successes

Although this has clearly not yet resulted in acceptable availability for (Tool)Labs due to problems at the lower levels, many significant improvements on especially the higher layers of ToolLabs have been made: The ToolLabs nodes & services have been made redundant across compute nodes, and been Puppetized more thoroughly. Automatic service monitoring & restarts for user tools have been reimplemented and made reliable. Proper monitoring of ToolLabs has been implemented using our new external monitoring service, giving us much better insight into the problems & performance issues.

Near the end of the quarter, the focus has been on reducing unneeded dependencies on underlying (storage) infrastructure while its problems were being addressed, and this has made many Labs projects much more resilient.


99

Q4 - Infrastructure / Labs

Quarterly Objectives:
Misses

Due to several severe outages on underlying Labs compute node & NFS storage infrastructure, ToolLabs had disastrous availability of 97.6% over the period of May 1 - June 30. This was caused by:

  1. hardware and kernel issues on the old & on the new replacement hardware (now resolved)
  2. the fragility of the Labs NFS storage setup, presenting a SPOF for nearly all Labs instances.

The suboptimal NFS storage layout provided performance only just adequate for regular use, and greatly complicated and delayed the creation of backups and the migration to a better architecture - almost any additional disk operations caused service degradation. This storage migration also triggered the corruption of the Labs NFS file system, resulting in the large scale outage on June 18th.

However, a positive side effect of this very unfortunate downtime event was our ability to cut through the remaining storage migration steps, and we came out with a significantly more performant and resilient storage setup at the end. We’re now much better positioned to provide a high reliability setup, and will continue to make this the main focus of the Labs team in the upcoming quarter.

Through more hands-on involvement of multiple team members and more explicit discussion about these system design decisions and implementation details with expertise of the entire team, we will avoid such problems in the future.


For more detail, see the slide deck and minutes from the quarterly review meeting of the Infrastructure department


100

CTO (Chief Technical Officer) department[edit]

  • Performance
  • Security
  • Research & Data
  • Design Research
101

Q4 - Performance[edit]

Objective Measure of success Status
Hire additional team members Make sure the team is staffed to fulfill its mandate. Recruited Peter Hedenskog, Timo Tijhof
102

Q4 - Performance

Quarterly Objectives
Summary
Median page save time on Wikimedia sites, May 21 - June 30, 2015.png
103

Q4 - Performance

Quarterly Objectives
Summary
  • Reduced page save time from 3.5s to 1.1s (104% improvement)
    • 20m edits a month = 48m seconds saved
    • Saves cumulative 1.5 years of waiting time every month. :)
  • Created performance metrics portal https://performance.wikimedia.org/
  • Provided instrumentation to support HTTPS migration.
  • Added profiling / stats reporting capabilities to MediaWiki core.

For more detail, see the slide deck and minutes from the quarterly review meeting of the CTO department


104

Q4 - Research and Data (5 FTE)[edit]

Quarterly Objectives
Summary
Objective Measure of success Status
Design a series of controlled tests and analyze them in order to support the Editing team in preparing for a successful launch of Visual Editor


Team members involved: 1

Provide guidance to the Editing team based on these tests to help them decide if the software meets the requirements to ship Success!
105

Q4 - Research and Data (5 FTE)

Quarterly Objectives
Other achievements
Objective Measure of success Status
Article creation recommendations

Team members involved: 2 (+ 1 research fellow)

* Design and evaluate an algorithm for personalized content translation recommendations.
  • Complete a pilot experiment


Success! Model design / evaluation completed. Conducted one pilot and completed a first test in production in French Wikipedia. See Report.
Revscoring

Team members involved: 1


* Prototype service online for 5 wikis
  • Labeling service online
  • Article quality model deployed
  • Caching layer deployed


Success! See Report
Improving linking structure

Team members involved: 1

(+ 1 research fellow)

* Release dataset of browsing traces


Paused Postponing data release to end of Q1 to allow for paper submission. See Report.
Impact of HTTPS rollout
Team members involved: 1
* Produce a report on the impact of the HTTPS rollout on traffic


Success! See Report
106
Quarterly Objectives:
Successes

Q4 - Research and Data

The team successfully completed two major experimentation projects such as the VE and the content translation A/B tests with complex dependencies (Legal, Comms, CL, Design Research, Product, Analytics Dev). TPG was instrumental in making this possible.


Outcomes

  • Clear product recommendations for the VE launch
  • Promising first results on the impact of article translation recommendations on engagement and content growth
107


Q4 - Research and Data

Quarterly Objectives:
Successes


Other achievements beyond our stated goals:

108

Q4 - Research and Data

Quarterly Objectives:
Misses

Blockers and dependencies for article recommendation experiment: running experiments takes a lot of time. (thanks to Ori, Dan, Michelle + Legal, Echo team, LangEng and CL for their support)

Our mandate and engagement model, post reorg, is still in flux: team receives requests that should in principle be honored by the audience teams.

109

Q4 - Research and Data

Appendix

For more detail, see the slide deck and minutes from the quarterly review meeting of the CTO department


110

Q4 - Design Research (3.5 FTE)[edit]

Objective Measure of success Status
Objective 1:

Evaluate the usability and user experience of editing tasks for VE launch.

Team members involved: 2

Basic editing tasks tested and clear communication of any usability issues to be addressed. Iteration support. Successful iteration:

Two rounds of research done: Round 1:On Wiki, Deck. Round 2: On Wiki, Deck

More to come for Link inspector and new user education.

Objective 2:

Start REFLEX : user segment based tasks to evaluate effects of changes to products and services on users experience over time.

Providing a better understanding of the user experience and impacts of design changes in functionality on users over time.   Objective redefined during quarter (was not funded for future, so we refocused on pragmatic personas.)
111

Q4 - Design Research

Quarterly Objectives:
Successes
  • Set of Pragmatic Personas and a workshop to iterate the pragmatic personas and align with designers, PMs and track leads. Posters / Hand outs
  • Evaluative Research for mobile:
  • Generative Research for Flow:
    • Flow workflow research.
  • Re-aligning with product teams on how to work most effectively together in the new post re-org structure.
  • Began managing our work flow in Phabricator so our work is more visible to the rest of the organization.
  • Working to address the gap in design leadership post re-org with UX team (researchers, interaction and visual designers) and executives.
112

Q4 - Design Research

Quarterly Objectives:
Misses
  • The REFLEX (benchmarking) goal focused on understanding user experiences with basic tasks within Wikipedia would have helped to better understand, over time, how various user types (personas) experience Wikipedia, and the impacts on experience our changes to the UI have on the users’ experiences. We refocused work on pragmatic personas instead.
  • Participant recruiter contract ended and we are figuring out how to fill this much needed position moving forward.

For more detail, see the slide deck and minutes from the quarterly review meeting of the CTO department


113

Q4 - CTO / Security (1.5 FTE)[edit]

Objective Measure of success Status
Begin building the security team, while continuing ongoing auditing and response work.

Team members involved: 1

Application Security Engineer hired Done (Darian Patrick hired and onboarded)
Initial risk assessment of engineering groups Done (https://www.mediawiki.org/wiki/Wikimedia_Security_Team/WMF_Engineering_risk_assessment)
114


Q4 - CTO / Security

Quarterly Objectives:
Successes


Significant improvement in Ops security, and user account security through Ops’ effort (https / ipsec).

Learning: Hiring dedicated security people works. Let’s discuss hiring onto specific teams next year.


115

Q4 - CTO / Security

Quarterly Objectives:
Misses

Metrics for ongoing, operational work has not been established, making it hard to determine success for the majority of our work.

Learning: Goals for Q1 include establishing metrics for security bugs (review metrics will be worked on in Q2).


116

Q4 - CTO / Security

Core workflows
and metrics
  • Security audit and response
    • Security issues: 21% fewer issues fixed than opened
    • Reviews: 9 requested, 10 completed
  • Training
    • Secure Code and Secure Architecture and Design trainings at Lyon Hackathon

For more detail, see the slide deck and minutes from the quarterly review meeting of the CTO department


117

Q4 - Legal Team (10 FTE)[edit]

Objective Measure of Success Status
Core


Top notch, quick legal advice and support on wide host of issues constituting our 33 legal workflows

Onboard two new attorneys and meet SLAs on core workflows. Done (about 80% of our resources are Core dedicated)
  • 33+1=34 core workflows managed well (see Appendices A & B).
  • Legal@ - of 136 requests: 100% turnaround w/i 7 days, 96% within 3 days.
  • Contracts@ - of 88 contracts: 99% reviewed/negotiated/approved by Legal within 7 days.
  • Zhou and Vickie - who came with strong experience - onboarded with smooth transition.


NSA litigation

Protection of global users: readers and contributions

No missed dates for affidavits and motions.  High quality documents. Done
  • All deadlines met.
  • Filed strategic amended complaint.


118

Q4 - Legal Team

Quarterly Objectives
Success

1) HTTPS by default. This multi-functional rollout was a success that provides long-awaited protection for our users.  It enables private and secure connections and minimizes censorship opportunities on all our projects.  Partnership included Tech, CE & Comms.

Learning - Coordinated efforts can ensure successful implementation on long-term projects.

2) NSA lawsuit.  Secured high-quality pro bono representation to help support our workloads (Cooley), provided solid research, and hit all deadlines with filing of strategic amended complaint. Strong partnership with Tech and CE.

Learning - With purposeful coordination of complicated projects, we work multi-functionally within WMF and across legal teams efficiently (WMF, ACLU, Cooley).

3) Open Access Policy. Successful release in support of global free, open knowledge. Partnership included Tech and Comms.

Learning - Commitment to mission values can speed pace of innovation around the globe.

4) 33 Core Workflows.  We delivered 33 workflows on time and added a new one ("Training"), despite other unanticipated legal demands.

Learning - Efficient, working core provides stable base and flexibility to address unplanned needs and innovation.


119

Q4 - Legal Team

Quarterly Objectives
Miss
  1. Budget planning.  We should have had a better internal accounting method to track previously approved variables
Learning -  New practices in Legal should improve monthly line-item tracking, modeling, and prediction.


120

Q4 - Legal Team

Appendix - Scorecard
Legal Scorecard Q4 QoQ YoY
Legal@ Requests 136 9% ↓ (136/149) TBD
Contract Requests 88 6% ↓ (88/93) 1% ↓ (88/89)
Trademark Permission Requests 30 30% ↓ (30/43) 12% ↓ (30/34)

For more detail, see the slide deck and minutes from the quarterly review meeting of the Legal department


121

Q4 - HR (7 FTE)[edit]

Objective Measure of Success Status
Fast Hiring of Candidates


Team members involved: 2

90% of domestic offers to manager in 2 business days of receiving a completed hiring packet 100% of 24 new hire offers to manager in 24hrs of approvals (58% in less than 6hrs)

(for c-level approvals: 100% in 2 days, 54% same day)

Annual Performance Review


Team members involved: 5

3 completed sprint support sessions for mgrs/staff

90% review completion by June 19

11 sprints and 6 training sessions done


98% of reviews completed by 6/19 deadline

International Payroll & Benefits

Team members involved: 2

95% eligible intl. staff enrolled or business entity established by 6/30 (not including FR) 44 of 45 set - 98%  

(final one set for 8/1, due to LOA)

ADP Upgrade

Team members involved: 2

ADP system upgrade for payroll, HRIS & Time & Attendance by 4/30.  Add Analytics module. Upgrade completed 4/18 (due 4/30)

Analytics added 6/3 (due 6/30)

Comp Review

Team members involved: 2

100% of worksheets out to c-level Delayed due to lack of data from c-level on managers & depts.  

100% rolled out 6/12 (the adjusted deadline, was 5/31/15)

Benefits & Wellness

Team members involved: 2

Survey data analyzed & shared

Benefits materials streamlined

5/1 data shared in comp/benefit deck (due 5/29)

6/11 materials streamlined (due 6/30)

122

'Q4 - Recruiting


Objective Measure of Success Status
Recruit Top Talent


Fill top 10 req positions in the organization


Team members involved: 4

Hire Director of Recruiting

Hire 50% of open recruiting Dept FTE

Outsource contract in place to handle headroom

Hired  Director of Recruiting

91% FY14-15 new positions filled

58% of FY14-15 replacements filled

Hired contract recruiters


Build University Recruiting Program


Team members involved: 2

Hire top engineering, legal, and communication interns and fellows.   15 interns hired in FQ4 for engineering, legal & comms

Career fairs at Stanford & California College of the Arts

123

Q4 - Talent & Culture

Quarterly Objectives:
Successes

Fast Hiring of Candidates: Turn around time from completed hiring packet to offer letter is streamlined to an average of one day, which is half the time mentioned on office wiki. The longest turn-around was two business days, due to c-level approval speed.  Learning: HR can do an incredibly fast turn around, but gathering approvals can take a lot of pinging. Delays in getting the hiring packet to HR occurred mostly due to missing data in the packet.  Approvals should be faster as they are now in Greenhouse.

International Payroll & Benefits: Across 19 countries, onboarded 44 staff, with another 15 thru a business entity, greatly reducing risk and increasing in-country payroll & benefits.  Learning: The process was complex and required taking into account personal detail: preference on timing, concerns about in-country contract wording, making sure benefits used SF as a baseline.

ADP Upgrade: Completed software upgrade from ADP’s oldest version to their current, with improved user interface and information accessibility.  Added an Analytics module.  Learning: The upgrade helped but ADP still sucks.  The upgrade did not fix all existing issues, and the analytics module data needs further cost and work to fix ADP illogical processes that screw with data like hire and exit count.

Annual Review: Completed the annual review process, which includes self reviews, 360s and manager reviews for 184 people.  Support included a massive 17 training & sprint sessions (which included unconscious bias and difficult conversations), and resulted in a whopping 98% on-time completion by managers. Learning: Staff continue to value 360s highly, but the time and logistics is difficult for managers to schedule, and completing mass 1:1 conversations is a stressful challenge.


124

Q4 - Talent & Culture

Quarterly Objectives:
Successes

Compensation Review: We created a tight and streamlined plan for the annual review & compensation processes for June, and were able to meet the adjusted deadline to rollout the comp sheets and meet with managers individually about their plans.  Learning: Do not roll-out a re-org right before year end and before annual reviews and compensation. Managers needed an extra month to settle their change data, which affected/delayed several HR workstreams.

Recruiting: We successfully hired the new Director of Recruiting, as well as a replacement engineering contract recruiter and a contract recruiting consultant to help with interim support until the Director could start. This provided uninterrupted support for hiring managers, and achieved 91% of new positions being filled.  Learning: We need stability with the recruiting team. 100%+ turnover on the team in the past year has resulted in significant time required to find and train new recruiting staff.


125

Q4 - Talent & Culture

Core workflows
and metrics

- WikiLead - 3rd cohort of 12 people finished the 7 month long leadership skills program on 6/24

- Wellness management for fiscal year end, including multiple reminders to all staff and individuals with full balance; wellness office hours; 260 claims processed this quarter (161 for June alone)

- Re-Org for Community Engagement, and then later Engineering, changes & updates for HR systems

- 33 people onboarded

- 10 people offboarded, including release agreements

- Employee relations management with Legal and outside counsel (metrics withheld for confidentiality)

- Hire of PT temp for Payroll Clerk, including training

- Ongoing support of the WikiWomen’s Group

- Contractors & renewals: 77 new and renewed contracts

- Immigration: 42 active cases (H1B, F1, J1, green card)

- Successful 401k audit completion

- Update of the Recruiting Corner on Office Wiki

- Rolled out Recruiting training sessions for managers


For more detail, see the slide deck and minutes from the quarterly review meeting of the HR department


126

Q4 - Finance and Administration (17 FTE)[edit]

Objective Measure of success Status
Objective 1 - Intacct changes to allow for project accounting.

Team members involved: 2

Intacct is ready to do project accounting.   Completed.  Intacct is now ready to do project accounting.



Objective 2 - 18 Month Rolling Budget.

Team members involved: 2

18 month rolling budget in place to track changes in budget. Project not implemented in this quarter.  Project stalled after planning phase.
Objective 3 - Annual Plan

Team members involved: 3

Annual Plan provided to Board of Trustees for vote by June 30, 2015. Completed.  Annual Plan sent to Board on June 25, 2015 for their vote.
127

Q4 - Finance and Administration

Objective Measure of success Status
Objective 4 - Monthly data for reporting ready by the 15th instead of the 25th.

Team members involved: 4

Financial data for the scorecard ready by the 15th of the month. Completed.  Financial data is ready for input into the scorecard by the 15th of the month.
Objective 5 - Purchase Order implementation.

Team members involved: 2

WMF can use purchase orders. Delay in implementation of Concur Expense delayed project.
128
Quarterly Objectives:
Successes

Q4 - Finance and Administration

Completed six versions of the annual plan, when annual plan schedule was planning on only three revisions.

Learning:

Implementation of the Adaptive software enabled this type of rapid iteration of the annual plan, so the investment in new software has provided excellent ROI.

Tony and Amy did an exceptional job managing each new version, maintaining the timeline and responding to changing requirements for the annual plan.


129

Q4 Finance and Administration

Quarterly Objectives:
Misses

Implementation of purchase order process and software should not have been scheduled until a implementable timeline for the implementation of Concur Expense was in place, as this was a dependency item.

Learning: Need to get complete information from software vendors, even cloud providers on what all dependencies are for implementation.


130

Q4 - Finance and Administration

Core workflows
and metrics

Finance and Administration has ongoing workflows which are the primary activities of the team.

Here are some additional metrics from Administration operations:

FY Q4 (April, May, June 2015)

Number of:

“On hours” hosted events:   8     Events (approx 32 hours of support in Admin and OIT)
“Off hours” hosted events: 20    Events (approx 60 hours of support in Admin and OIT)
Front office orientations: 19 Multiple staff attend these “Day One” orientations
Office moves: 67 Partnered with directors and VP’s to implement new seating behavior
Ergo evaluations: 15 Partnered with HR on ergo evals and procured necessary equipment
Facilities orientations: 18   Multiple staff attend these “Day One orientations
New users: 27    New users came through WMF onboarding process
Scholarship WMF visa letters: 23 Approx 10 hours of support to Wikimania Scholarship recipients
Packages shipped: 74 Packages shipped to 24 different countries
Packages  received: 534     Packages received for business, personal and remote staff holds
Contracts prepared & executed: 135 Contracts, prepared for Legal’s review, tracked, and executed
Tickets booked: 470 Flights booked
Room nights booked:         1969 Hotel rooms nights booked for Wikimania and other events


For more detail, see the slide deck and minutes from the quarterly review meeting of the F&A department


131

Q4 - Communications (5 FTE)[edit]

Objective Measure of success Status
Successful VisualEditor roll-out in collab. w/ CE and Engineering   Minimal negative response (25% or less), increase awareness w/ public Postponed. Messaging criteria identified key blockers. 2 FTE.
Build executive and org strategic messaging platform. Clarify/unify org message for primary audience groups Interviewed two top firms, but canceled per ED direction. 1 FTE.
Redesign WMF.org and rebuild high value main pages Update/align WMF brand. Clarify value proposition, narrative, employer brand. Indefinitely postponed due to insufficient cross-departmental resources post-reorg. 1 FTE.
Finalize WP15 project and marketing/branding plan. Engage new Wikimedia stakeholders and audiences. Execute on brand messaging. Proposed budget and plan. Postponed due to budgeting, calendar, approval. 2 FTE.
Finalize budget, identify director, begin shooting for ‘Future of Knowledge’ documentary. Signal strategy, position brand as knowledge leader, introduce brand to new audiences. Finalized budget and spec. Canceled due to budget considerations. 1 FTE.
GOALS REASSESSED; CANCELLED 5/1 DUE TO INTERDEPENDENCIES.
132


Q4 - Communications (5 FTE)

Revised Quarterly objectives summary
Objective Measure of success Status
Minimally disruptive and maximally clear change management communications. *Reorg communication strategy, internal and external announcements. (Docs, emails, FAQs, charts, reports.) Completed/shared on time. Staff feedback that information increases understanding.
Grow team capacity to improve support for internal and external demands.   *Secure Wikimania PR agency

*Launch summer intern program

*Fill Junior Associate position

Complete. Converted intern to FTE, reducing JA onboarding overhead.
Advance Wikimedia movement goals through Executive Director visits and speeches *Zero negative press or feedback

*1 identifiable programmatic outcome per staffed trip

Complete. Detailed executive travel process in development.
Strategic product communications support: positioning and messaging *Zero remaining critical communications blockers for VE Criteria identified for VE led to significant UX improvement. Support for iOS, Collections, etc.
Improve social and blog content to build brand and community engagement. *Finalize audience research report

*Develop user personae

*Outperform organic social growth

Audience review and personae complete, new content under development, met social metrics.
REVISED AND RESET GOALS INTRODUCED 5/1
133

Q4 - Communications

Quarterly objectives:
Successes
  • Drove product messaging and communications for key WMF-developed products (desktop and mobile)
    • Developed VisualEditor public and in-product messaging and advised on UI changes, partnering with user research to identify and recommend a key change that resulted in improved user performance
    • Promoted iOS app launch with PR and Share a Fact video; resulted in 56 tech press articles in 18 countries
    • PR coverage resulted in iOS download spike, Android “halo” effect download spike.
  • Improved blog and social content, UX, research with focus on quality
    • Developed content strategy for serving diverse personae and communications channels
    • Created/tested new content formats (Wikipedia highlights, In the news, multimedia)
    • Developed new blog features for better visibility and discoverability (email subscriptions, updated UI)
    • Established research framework to track content impact (through metrics, surveys, competitive studies)
  • Engaged with community stories and affiliates to raise positive PR awareness of Wikimedia
    • 100+ articles for Print Wikipedia including NY Times, Washington Post, USA Today
  • Scoped community communications needs for future programming and support
    • Held workshop at Wikimedia Conference to engage with community capacity building needs
  • Extensive support for unscheduled organizational change management, including WMF reorg
    • Created FAQ for staff and community communications, and 15 hyperlinked org charts.
  • Supported ED in 7 international trips including 3 meetings with high level officials
    • Briefing documents, logistics, media management, presentations, staffing
  • Secured and onboarded 7 interns to support team capacity in design, research, PR, and editorial
134

Q4 - Communications

Quarterly objectives:
Misses
  • VisualEditor rollout was postponed, pushing back timelines
    • Identified critical communications and UX blockers resulting in a delayed VE rollout on enWP and postponed public launch and social marketing campaign.
  • Executive Director messaging platform postponed
    • Initial planning and evaluation completed, including vetting, proposals, and interviews with 2 world class agencies. Postponed due to ongoing strategy development.
  • Updates to Wikimedia Foundation.org were postponed
    • Personnel changes and resource reallocations following reorg required that we reevaluate this priority.
  • Planning for Wikipedia 15 birthday and brand research
    • Scouted venues, sourced quotes from vendors, developed birthday campaign and event proposal. Postponed and subsequently revised following updated WMF budget forecast.
  • Planning for a Future of Knowledge documentary
    • Developed budget and a draft spec proposal. Project was canceled due to revised WMF budget forecast.
  • Internal coordination for executive travel
    • Lack of clarity around roles and responsibilities with departments led to difficult coordinating important travel and preparation materials; new travel process proposal development underway.

For more detail, including core workflows and metrics, see the slide deck and minutes from the quarterly review meeting of the Communications team


135

Q4 - Team Practices Group (4 → 6 FTE)[edit]

Objective Measure of success Status
Develop repeatable best-practices for WMF engineering teams (1) List of practices defined by editing team and stakeholders that directly supported release of VisualEditor in Q4. Done, process review published on mediawiki.org, June 2015
136
Quarterly Objectives
Success

Q4 - Team Practices Group

Joel Aufrecht from the TPG led this work with Neil Quinn from the Editing team.

They went above and beyond the scope of the goal by fully and collaboratively documenting the current processes and practices of the editing team with the team itself and their stakeholders. They also identified the most pressing challenges to the team and made recommendations for process improvements; top recommendations under way now.

Learning: Collaboratively documenting existing processes and pain points with a team and its stakeholders creates shared awareness of areas for improvement, which leads to shared buy-in around improvement efforts.


137
Core workflows
and metrics

Q4 - Team Practices Group

RelEng team development off-site workshop Kristen Lans

  • Goal: Create shared understanding about how the team works, what they work on, and how they want to work

Burnup/Burndown charts Kevin Smith

  • Functioning charting targeted for end of July, 2015

TPG internal continual improvement

  • Unable to provide desired level of support with current operating model and resourcing
  • Q1 goal of devising new strategy and reimagining team operating model
138


Q4 - Team Practices Group

Core workflows
and metrics
TPG Stats
Q4
QoQ (Q4/Q3)
Teams receiving dedicated support
9
+80% (+4)
Teams receiving or with individuals receiving  ad-hoc coaching
6
0%
New hires to TPG
2
0%
Teams participating in Health Check Survey
12
+20% (+2)

For more detail, see the slide deck and minutes from the quarterly review meeting of the Team Practices Group

139