Grants:Learning patterns/Asking the right questions

From Meta, a Wikimedia project coordination wiki
Jump to: navigation, search


MechaDuck.png
Asking the right questions
Wartime Social Survey in Britain, 1944 TR1543.jpg
problemYou need to make sure that you don't accidentally leave important questions out of your survey.
solutionUse different types of survey questions to ask for different kinds of information. To make sure you cover all of your bases, frame your survey around three or four types of question: background questions, activity questions, experience questions, and opinion questions.
creatorJtmorgan
endorse


What problem does this solve?[edit]

The most important step in performing a survey is figuring out what it is you want to learn. This sounds obvious, but many researchers realize after they have already deployed their survey that they failed to ask an important question. If you want to know how many people on different Wikimedia projects subscribe to public mailing lists, you will need to ask people which project they currently contribute to most right now. If you want to know how many people migrate from one project to another during their editing careers, you need to ask which project they joined first and which one they contribute to most right now.

What is the solution?[edit]

To make sure you cover all of your bases, frame your survey around three or four types of question:

  • background questions
  • activity questions
  • experience questions
  • opinion questions

You don't need to have an equal number of questions in these four categories, and you might not need them all, but many surveys will be composed of a mix of all four types.


Background questions[edit]

Questions about the participant themselves: who they are, where they're from, what they do off Wikipedia, etc.

Example
Teahouse is intended to be a friendly, supportive environment for all new Wikipedians, but we make a special effort to welcome female editors, who are underrepresented within the Wikipedia community. If you're comfortable telling us, what is your gender? (Teahouse Pilot project guest survey)

Activity questions[edit]

Questions about what they do now, or have done in the past.

Example
Have you ever sought help on any of the following topics on Wikipedia? (Help Project survey)

Experience questions[edit]

Questions about how they feel (or felt), or what they think, about the specific activities they participated in.

Example
Was there anything in particular you liked about the review process or the feedback? Tell us about it. (IEG submitter survey)

Opinion questions[edit]

Questions about what they think in general about the subject you're interested in, beyond their own specific experiences.

Example
What ideas do you have to make dispute resolution overall more effective and efficient? (Dispute resolution survey)

Considerations[edit]

When deciding which questions to ask, and how many, it is also helpful to keep these considerations in mind:

  • Keep your survey as short as possible. People have short attention spans. If you ask them to fill out a 50 question survey, many of them will get bored or frustrated and quit partway through, leaving you with gaps in your data and lingering ill will. Don't ask every possible question; figure out which questions are necessary and only ask those. Realistically, keep the length of time it takes to fill out your survey (in the amount of detail you desire) under 10 minutes.
  • Make multiple choice answers required and free text answers optional. No one likes being forced to write. Making basic questions multiple choice allows respondents to give you critical information quickly and easily. Making answering multiple choice questions required to finish the survey (and most online survey tools will allow this) gets you good data, as long as your questions are clear and the choices you provide make sense. Free response questions can be very useful, and most surveys will contain at least a few. If people have strong opinions about something, you can be sure that they will take advantage of this—no need to force them.
  • Use opinion questions sparingly. Opinion questions are useful for some surveys, but in many cases it is better to ask primarily Experience questions and Activity instead--people's opinions don't always reflect their behaviors![1]

Examples[edit]

Endorsements[edit]

  • J-mo
  • Rastus Vernon
  • Libcub
  • Reke
  • Liang(WMTW)
  • We conducted a survey in South Africa, which was the first of it's kind, and we felt it very important to ask the right questions so that we can find out more about our community and how we can support them. Humetheresa (talk) 09:29, 23 July 2015 (UTC)

See also[edit]

Related patterns[edit]

External links[edit]

References[edit]

  1. Dentaleconomics.com, The Art of Interviewing. [1]