Wikimedia monthly activities meetings/Quarterly reviews/Community Engagement, October 2015
Please keep in mind that these minutes are mostly a rough paraphrase of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material
Present (in the office): Lila Tretikov, Jake Orlowitz, Siko Bouterse, Rachel diCerbo, Rosemary Rein, Terry Gilbey, Luis Villa, Sarah Malik, Geoff Brigham, Floor Koudijs, Tilman Bayer, James Alexander, Toby Negrin, Kevin Leduc, Sylvia Ventura, participating remotely: Quim Gil, Wes Moran, Asaf Bartov, Maggie Dennis, Katy Love, Edward Galvez, Katherine Maher, Jaime Anstee, Trevor Parscal, Haitham Shammaa
Intro, overall department objectives
1.: (define overall strategy) Lila asked us to clarify this
Geoff: when will this be published/shared with rest of org?
Luis: mid-quarter, as planned earlier
2.: STRENGTHEN community health
3.: (STRENGTHEN support of other WMF departments)
4.: (EXPERIMENT with new community tools and approaches)
(EXPERIMENT with new community tools and approaches, continued)
5.: core workflows; hit these regularly
Quim: still cannot measure our KPI, but Engineering is helping to work on it, since this data point is useful for more teams. Hopefully ready by the next quarterly review.
Terry: is it that we don't know how to do it, or we haven't done it?
Quim: there are different ways of doing this, and we are working on the implementation.
Toby: know how to do it, but need to write software, Readership and Discovery are working on it.
Lila: should it be on this team's quarterly goals then?
Terry: lack of UX resources because of resources moved away during quarter, or because of planning shortcomings?
Quim: we knew the UX resources were not committed, the discussion happened when Engineering was recovering from the reorg. We still wanted to try, and in fact Reading Design went beyond call of duty.
Lila: current situation?
Quim: this quarter focus on improving content
not going to get more UX resources in the short term, but that's fine
Toby: need to sync on Blueprint, UX people see it as a reference design, not as an implementation.
--> Lila: let's talk [about Blueprint]; you should be able to get the right resources at the right time
Quim: OK, taking that as action item
Quim: in the Appendix we are including code review metrics that help seeing the impact of this Gerrit Cleanup Day, and also the dimension of the problem we have handling code reviews.
Quim: even if we didn't achieve the goal as initially defined, we are happy with the result and we are going to recommend the celebration of Cleanup Days every quarter until the queues are at reasonable levels -- now they are not.
GSoC and Outreachy
Lila: background on GSoC?
Quim: students, one intern, two mentors
Luis: summer of code casts a broad net, so typically the interns come from outside our community.
Lila: do we hire from that pool?
Quim: yes, many of today's employees came from there
Trevor: after this round, I just hired one from GSoC and one from Outreachy
Quim: Wiki Loves Open Data
the initial framework was created with the Wikidata team and community, but we failed on the objective of ongoing work with one open data org. Big orgs don't move very fast.
Terry: what dials will this move?
Quim: GLAM initiative (decentral) was big success
WMF should help, but if chapters do it, great
Luis: hope it feeds into Knowledge Engine (initative)
--> Lila: figure out a full framework
I asked to use one of the partnerships as model for rest
but this is cross-functional. sync with Sylvia, Wes
want data that's pushed/pulled to remain fresh
complex ecosystem to manage
in next(this) quarter, get much clearer on what goals are, and how this will be managed
Sylvia: CC0 (=Wikidata) is high bar, they usually release on CC BY-SA. Similar with OECD
--> Lila: need working group and a leader
iterate on this, give exec team more insight
Sylvia: OK. Wes is involved with that
Terry: need to know about success of experiment
Luis: yes, as experriment it was successful
this new slide format is triggering questions on every goal ;)
that's a good thing, but need to keep time
Quim: chapter like WMDE or WMBE might be ahead on such partnerships; we need to including in any working group hat it is formed.
For WMDE involving open data orgs is strategic, and they have the Wikidata team.
Lila: does Wikimania count as Global South funding?
Siko: yes, in case of support for GS individuals attending
Luis: and since next q there is no Wikimania, GS ratio might drop
Siko: number of people supported is kind of the KPI taken from community-reported Global Metrics
did not track this previous q, so no QoQ
it's a lot of work to generate that number
Lila: these are your KPIs, if it's not helpful / takes too much time, then scrap it
Siko: looked at amount of hours team spent collecting data, and the work community members spend to gather Global Metrics - it's huge
Lila: so where was that private space?
Siko: besides 1. wiki: 2. survey, 3. smaller focus groups/conversations
survey was most successful
money is a touchy subject [i.e. difficult to discuss in public]
Luis: this is critical - gives us more honest and perhaps less adversarial feedback
Lila: changes the frame - very interesting, might learn from this for product
Luis: in Legal team's consultation some years ago, we thought about this. But this is the first time it's implemented that way.
James: harassment survey is also designed that way
Jake: dramatically different tone
Siko: see also appendix slide on resulting changes, and linked blog post etc.
Siko: interesting to see requests from new user groups
Geoff: quality of requests?
Siko: quite good. want to make it easier for [them]
--> Lila: can we start tracking amount of [grant/support] requests per category, see how this evolves?
my guess is that the slice going to smaller entities/GS goes up, ratio of big entities from GN goes
see how will this look 12 months or 3 years from now
Siko: OK, already track $ (GS/GN), but can do this too - can see your point that it might be interesting to see user groups vs chaptes
we are about 80% on this goal
FDC proposal on track
C-team still needs to discuss
Lila: I am aware, goes into Q2
Siko: our core workflows take almost 100% of prog officers time
need to find way to resource this
Lila: see as blueprint
Siko: yes, early experiment
Siko: these disccussions take time, decided to not press in Q1
how to bring in ideas
want to run idea campaigns throughout year
Lila: example of ideas coming out of Idealab that were really impactful?
Siko: Wikiproject X
Idealab is incubator, a large number of our grants come from it
---> Lila: find some IdeaLab examples that were really successful, so that people can learn from them
Siko: Inspire campaign ran entirely in idealab, but these will need 6-12 months to show success
but point taken
Reference : https://meta.wikimedia.org/wiki/Category:IdeaLab/Ideas/Proposal/Selected
this q, rather than like 2 affiliates seeking additional support/guidance in transitions, about 12
Lila: why do we think it rose that much?
Siko: coincidence rather than fundamental reasons. it's a reactive workflow
Luis: triggered by site visits perhaps?
Siko: had some visits, but many beyond that scope
Maggie: both KPIs are about response speed
--> Lila: make sure slides are screen-shared on Hangout
Terry: why was this so successful (compared to other goal)?
Maggie: it's year-long and top importance, can't afford to fall off schedule or deprioritize
Luis: in the transition after Philippe left, talked about prioritization, made sure this got priority
(Global ban enforcement)
put together service level agreement (SLA)
easy channel where everyone can get to us
Luis: here not deprioritized, but we researched, planned and realized it was wrong plan
lack of time, hope you will read full slide deck, but call out: smooth transition; team rose to occasion and James stepped up
Learning and Evaluation
we love KPIs ;)
just as we at WMF don't create content, we also don't create learning - community does: Learning patterns are way up
Disney process: film does not get made unless there is a [concrete artefact]
brutal "shark tank" approach to deciding what goes into MPL (master project list) ;)
Rosemary: e.g. not teach conferences unless we teach community members who can spread this ("train the trainers")
Geoff: example for priority research project?
Rose: use data more, e.g. Edward focusing on surveys now - if you continue just reponding on talk pages, continue to run into resistance
fix pain point of tracking data by ....(?)
example for train the trainers : Estonia
partner a lot with Comm Resources on their survey work
Edward focusing on avoiding survey fatigue, building survey support center
Terry: overview of survey tools, I guess we have like 5-7?
and hope data can be used cross-functional
Toby: should talk about Quicksurveys tool that Reading is building
Rose: Edward did assessment of all surveys done throughout the year, central repo
Luis: only building this now, so not surprising your team didn't yet know about this support option
Rose: e.g. 50 hours spent on Wikimania survey
Lila: how many surveys did we run over the year?
Rose: over 40
tools for planning e.g. editathon, for quick just in time learning
Geoff: could highlight examples where you received data, and saw people act on that data
--> Rose: will include that next q
Lila: KPI on # learning patterns created, but more is not necessarily better - have a lot of info out there
Rose: yes, [TMI] can also be an issue internally
Luis: had IdeaLab idea that resulted in searchable index for patterns library
Other successes and misses
e.g. gap between affilates and community
Lila: really interesting to me; I think that for being an online community we don't do enough online events
make events (e.g. Israel hackathon) more accessible for remote attendees
this is a question for Quim too, for the future
Floor: interactions KPI - many of these happened at/due to Wikimania and the CEE meeting, could be lower next q
Education extension lacks tech support
Lila: need to run flag up the pole for this
right now not clear who owns this - external/internal, etc
--> Lila: resolve [Education extension's lack of tech support] this q
2. # interactions is not a success metric
I can interact with 1000 people [and not achieve things]
--> Lila: retention rate might be better indicator than # interactions
made edu collab a much more open and transparent group
Jake: this task flow system is really exciting
Lila: (question about who forms the group)
Floor: collab are community members willing to help
this is about working with them on sharing their knowledge and experiences.
Take a look at the linked Phabricator project
Luis: it's more task management than ticket management
Floor: e.g. "translate this brochure into Spanish"
Lila; exit criteria for members?
Floor: membership basically based on activity / positively contributing
Egypt: local members stepped up, forming hub
--> Lila: they [Arab community] should work with content translation team, so many synergies there
Floor: I think they are already connected, using CX tool
Lila: also about strategy
Palestina: look at content acquisition project (wiktionary)
Refine data gathering
--> Lila: these numbers (slide 47) look great, start tracking them over time
but pick what are the important metrics
e.g. teachers engaged and running *active* programs
should know how many people have dropped out (first derivative)
then others indicators will be secondary
will skip next few slides due to time limitations in this meeting
yoy growth less than expected
EBSCO is big, larger than our other donations
largest q so far
Lila: did we lose partners over time? no
LilaL: need to think how this is integrated into our editing tools, e.g. VE
--> talk to Trevor
Jake: will get to that
one gap there: don't have good KPI
can count citations, but don't know who added them
want to reduce time from signing up to getting access (now: 27 days)
tons of excitement at Wikimania, but then everyone went into vacations, pick up in Sept / Oct [i.e. not yet reflected here]
small part of our work, but hopefully seed for big things
talk to orgs, e.g. invite librarians to curate bibliographies on WP articles
--> Lila: would love to learn more offline about this [collaboration with organizations like libraries]
Jake: where are we likely to succeed?
Lila: also, need more structure - partnerships that are about content acquisition vs. content curation
find where can we scale
Jake: Develop tools: library card platform
Lila: hiring developers ? yes
Lila: talked to DPLA guys? they might have the tech we need
Jake: .... talked but might pull in larger platforms later(?)
Lila: we will become a de facto interoperability standard between these resources
Luis: scope this small
--> Lila: take it offline, talk to Trevor and Wes about this
Rachel: KPIs: contributors on product-specific pages- may mean someone thinks that page is somehow useful
Pageviews on major product pages
Defining community/product process
takes longer than expected, cross-functional
now a goal for Q2, coming along
Luis: learning for me: should have engaged more on cross-team issues as executive
Lila: KPIs don't get at what we want to achieve
support VE launch to new editors on enwiki
red because only two people engaged on-wiki - but success otherwise
a lot of the concerns community members brought up were about lack of clarity on process
received feedback that roundtable format was appreciated
hard to count reach/pageviews because of distribution to widely read channels (incl. en:VPT and Signpost)
Lila: posted to mailing lists?
Rachel: only some
Luis: what sold me on this is that 15 volunteers spend their time translating it
--> Lila: widen the audience
--> Lila: mandate for this team:
1. as few as possible of conversations result in (a red status) - i.e. design things that community finds useful
Rachel: from community perspective?
Lila: from WMF's too (we are part of the community...)
2. allow Engineering to accelerate velocity, ship many small things
in a nutshell: build the right thing and make it fast