Talk:Survey Support Desk

Want your survey reviewed? Need help with strategy and your survey? Post your questions here!

Title

The title seems too generic... Nemo 09:11, 28 October 2015 (UTC)

Hey Nemo - do you mean "Survey Support Desk" or "Surveys" for the link? Survey Support Desk is too long, especially if there will be child pages. I figure Surveys would be a good option since its for all surveys, whether WMF or not WMF. Ideas? --EGalvez (WMF) (talk) 17:53, 30 October 2015 (UTC)--

Hi Nemo - just pinging you about my question. Thanks.--EGalvez (WMF) (talk) 00:57, 4 February 2016 (UTC)

I think "Survey help" or similar would be a better title. This should be a disambiguation page for surveys. I keep getting caught by this page and I'm forced to click the category, go to the upper category and then down again to what I'm really interested in. --Nemo 09:22, 24 June 2017 (UTC)

Posted a move proposal at Meta:Proposed page moves. --Yair rand (talk) 01:08, 14 September 2018 (UTC)

Survey stats

WMF Metrics and activities meetings/Quarterly reviews/Community Engagement, October 2015:

• Lila: how many surveys did we run over the year?
• Rose: over 40

I have a few questions about that:

• 1) What were these over 40 surveys? I'd like to see a list with subject, links, date, number of participants and returns, responsible owner of the respective survey.
• 2) What results and what benefits did these surveys have? Anything tangible for Wikipedia?
• 3) How much volunteer time was spent on these surveys? (A 20 minute surveys on 150 volunteers would be 50 hours of volunteer time cost.)
• 4) How much staff time was spent on these surveys?
• 5) How do you prevent survey fatigue? How do you notice survey fatigue setting in and biasing results?

Thank you! --Atlasowa (talk) 14:31, 8 December 2015 (UTC)

Hi Atlasowa -- Thanks for your questions. About number 1, this was a misspoken. I did a short project to map out all the surveys and consultations that the WMF has done in the past and there were over 40 (of course!), so this comment in the minutes was confusion. About survey fatigue, the WMF is actively thinking about this. I'm testing out a calendar on meta at the moment for staff to use at the Community Engagement/Calendar, to try to make sure our work doesn't overlap. Since its very new, as you can see, december has quite a few projects happening. But thankfully, WMF teams are starting to use it more. This is an experiment, so we'll see how it goes. If its successful, I will see how we can share it more broadly and even translate in the next couple of months. Also - out of curiosity - why are you interested in surveys? What are your thoughts about surveys that the foundation does? Any thoughts comments on what the WMF does well or what it doesn't do well with surveys? Ideas for what we can do? Thanks! --EGalvez (WMF) (talk) 07:24, 10 December 2015 (UTC)

HI Atlasowa - was I able to answer you question from december? Thanks --EGalvez (WMF) (talk) 00:56, 4 February 2016 (UTC)

Recommendations for future surveys

In light of numerous survey results posted by the Wikimedia Foundation, where the survey data seem to suggest out-of-whack "findings" that appear to be projectable to a larger population, but are not thus projectable, because the sample was not set up as a random probability selection, there are numerous observers who would like to request that the Wikimedia Foundation make more effort to design surveys that are based on random probability samples of readers and editors, rather than an "open call" for anyone interested in responding. - Thekohser (talk) 00:03, 4 February 2016 (UTC)

Thanks Thekohser for the post. This is something we are working to learn and figure out how to address. For example, some teams have started to use talk page messages, rather than banners to create a more representative sample, and with a high response rate. See the Research:Tech_support_satisfaction_poll as an example. Do you conduct surveys? I'm wondering also if you are a member of an affiliate or a functionary on a project? Thanks! --EGalvez (WMF) (talk) 17:26, 2 March 2016 (UTC)

Yes, EGalvez (WMF), I conduct surveys. That is my profession, and over the past 25 years I have authored and fielded at least 1,500 different survey instruments against both probability and non-probability samples. I apologize, but I am unclear on what you mean by asking if I am "a member of an affiliate or a functionary on a project". - Thekohser (talk) 13:37, 22 March 2016 (UTC)

Thanks Thekohser (talk) - Wow - that is a lot of surveys. Are you a researcher? I think I was more broadly wondering what you do on the wikis - are you an admin? another role? Or if you are part of a (chapter, user group, etc.)? Thanks! --EGalvez (WMF) (talk)

I am a market research practitioner for a Fortune 50 company. I am an occasional volunteer editor on Meta, Commons, Outreach, Wikibooks, Wikiversity, Wikivoyage, Wikiquote, and other Wikimedia wikis. I own and maintain a publicly-edited wiki with over 70,000 pages. I am indefinitely blocked on English Wikipedia because of a policy dispute that I had with a Wikimedia Foundation board member and a few of his most avid supporters, even though he eventually apologized for how he initially engaged with me. However, this block has not prevented me from conducting periodic experimental content analyses on English Wikipedia, such as my exercise that found that with simple techniques, in 63% of cases of phony information being inserted into Wikipedia, it will persist for at least a week (link). I sense that you are potentially asking me to volunteer my professional acumen to aid the effort here at WMF to create more credible market research surveys. Is that a correct assumption, User:EGalvez (WMF)? - Thekohser (talk) 20:56, 22 March 2016 (UTC)

Please see how carefully Pew tries to obtain a representative sample, and how many adjustments they make. There are all sorts of separate issues involved with the Wikipedia userbase, and nobody is expecting the WMF to show the same level of care as Pew, but there is no reason to not at least be aware of the many issues involved and make an effort to address them. In particular, if indeed the problem is as bad, there is no reason to not use a reputable and/or experienced survey designer to get a good handle on the problem. Kingsindian (talk) 05:43, 4 February 2016 (UTC)

This is very useful. Thanks Kingsindian - I've added this to a learning pattern. I think it is hard to adjust or weight the results of surveys when we have gaps in knowledge of demographic proportions of users. The best data we have about demographics are surveys that use poor techniques, so its a major challenge for us. Do you use surveys? Wondering if you are involved in any affiliate or are a functionary on a project? Thanks! --EGalvez (WMF) (talk) 17:29, 2 March 2016 (UTC)

Is this page only about "Qualtrics"?

Hello everybody :)

In order to avoid to rely only on proprietary solutions, I'm contributing to this initiative: Wikimedia Italia/LimeSurvey.

My questions:

• is the Survey Support Desk only about Qualtrics?
• how do you suggest to promote that initiative and similar ones? maybe converting this page as a kind of index of tools

I would be glad to invest my spare time to free as much people as possible, trying to be pro-active and give credible alternatives.

Thank you so much for your help :) --Valerio Bozzolan (talk) 18:17, 23 February 2021 (UTC)

I've edited the page linking LimeSurvey initiatives :) --Valerio Bozzolan (talk) 07:37, 9 March 2021 (UTC)