Talk:Strategy/Wikimedia movement/2018-20/2019 Community Conversations/Survey/Concept and Methodology

Selection bias

Can you elaborate on how the selection bias from "existing liaison networks as well as social media channels" will be corrected/handled in the analysing the input? Nemo 19:34, 15 July 2019 (UTC)

Also, can we please have this cross-posted at other Wikis? We are discussing this here. Megalibrarygirl (talk) 22:44, 15 July 2019 (UTC)

Dear Nemo_bis and Megalibrarygirl, thank you so much for proactively highlighting these points. We had some quality issues regarding the translations of the survey and as a result a wider-spread rollout has been postponed. Language liaisons have been so far working with languages where the translation level was fine. Even though the survey was created in English and as a result translation issues do not affect the survey in English, distribution in it was part of the wider distribution plan and with the timeline pushback also wider distribution in English has been pushed back. Last translation was fixed on Monday and we will hopefully follow-up later this week with distribution of survey link also on wikis. The survey will remain open until at least end of July and please feel free to distribute it where you find it relevant.

As to analysis, the context of the input will be provided to the working groups. This means both highlighting how and where the answers were collected and also highlighting the gaps in the data. As the survey is of qualitative nature, the results will not be quantified and answers will be treated as rich data to inform the recommendations. For the richest data, however, the selection bias needs to be mitigated with the distribution of the survey to diverse range of stakeholders. In framing of distribution plan additional channels to the usual ones were highlighted. Pushback on the timeline however raises valid concerns regarding distribution methods of the survey. Point taken, language of the page changed. And as stated before, survey will remain open until at least end of July. --KVaidla (WMF) (talk) 03:57, 17 July 2019 (UTC)

Timing and distribution methods

For a "community conversation", I have to say it's pretty hard to find which talk page to post on in this warren of subpages! :) This seems to have only recently made its way to enwiki, as Megalibrarygirl indicated above. The timeline on this page suggests the survey period ended June 30. Rosiestep helpfully ducked in from her travels to let us know that the Community Health survey, at least, will be open till next Friday. Can we confirm that's the case for the others as well? I'd like to get this much more broadly distributed, but also don't want people to end up spending time on responses that are past the deadline for consideration. Pinging SGrabarczuk (WMF) as the creator of these pages - sorry if you're the wrong person, but maybe you know who the right person is :)

Separately, I'd like to ask about the distribution plan this page describes: "The chosen approach for distribution is using existing liaison networks as well as social media channels". How did that approach get chosen, and by whom? Was a method of distributing announcements directly to Wikimedia communities on their respective wikis considered? I have some pretty serious reservations about using social media for this sort of thing. Commercial sites like Twitter or Facebook are not the places I expect to get news about my free-knowledge hobby. (This is especially ironic considering the stated strategic directions motivating this whole exercise!) As in the section above, it raises concerns about selection bias if responses generated by social-media contacts are different in some way from responses generated by outreach methods that meet project communities where they already are, on wikis. This is apparently a multi-year project, intended to set a decade's worth of direction - I think it's worth taking the time to make sure you hear from current volunteers who invest their efforts primarily in their chosen local communities. Opabinia regalis (talk) 08:20, 16 July 2019 (UTC)

I must concur with Opabinia regalis on this. The best way to get the widest possible set of responses from people who actually edit wikipedia (any of them) is on the top of the history/watchlist pages as we open them for editing. This is done for publicizing Wikimania and for things of critical importance to the entire community, such as Requests for Adminship. If the results provided do not reflect responses from actual people trying to edit their respective wikipedias, how can it be responsive to the actual editing community, i.e. the people actually building the encyclopedia? Montanabw (talk) 19:04, 16 July 2019 (UTC)

@Opabinia regalis: roger that. I asked your question to those who are likely to know the answer, because frankly, I've got no idea regarding this timeline :) SGrabarczuk (WMF) (talk) 21:49, 16 July 2019 (UTC)

SGrabarczuk (WMF) thanks for bringing this discussion to my attention. Opabinia regalis and Montanabw, thanks for your comments and raising concerns. As I have already responded in my comment above we were mitigating some issues regarding the quality of some translations, which has caused pushback on general timeline and wider distribution of the survey. Last translation was just recently fixed and we plan to follow-up with the distribution on our end. In the meantime, I can reassure that the survey is open until at least end of July and please feel free to distribute the link where appropriate.

The distribution plan is actually not limited to liaison networks and social media. This was meant as an additional communications strategy to reach people and communities who have wider and big picture discussions off-wiki in addition to usual distribution methods. Thank you for surfacing this, language has been changed and more usual distribution channels have been added (at least in English). You are more than welcome to distribute the link to provide the working groups richer data for analysis. --KVaidla (WMF) (talk) 03:57, 17 July 2019 (UTC)

@KVaidla (WMF): Thank you, good to know on the updated timeline!

As for the distribution, that makes more sense, thanks. To be clear, the plan now that the translations are sorted out is to distribute notifications to individual wikis? Do you have a sense of when that would happen? Opabinia regalis (talk) 08:14, 17 July 2019 (UTC)

Opabinia regalis, thank you for the specification! I hope that this will happen later this week. We need to coordinate with the team. If you see any further notable gaps next week in communication, please let us know! Proactivity in clarifying things is really appreciated. --KVaidla (WMF) (talk) 08:17, 17 July 2019 (UTC)