Learning patterns/Getting a sample of users

From Meta, a Wikimedia project coordination wiki
A learning pattern forSurvey
Getting a sample of users
problemIt is often impossible or very expensive to get information from an entire population
solutionUse sampling to reduce cost and time
creatorEGalvez (WMF)
created on17 November, 2015

What problem does this solve?[edit]

Typically with surveys, we do not want to ask everyone in a group a series of questions. This both takes a lot of resources and takes a lot of time.

What is the solution?[edit]

Sampling a population is a better way to collect data. First, sampling reduces time and resources for everyone involved. Surveys take a lot time to develop, send out, fill out, and anlayze. Second, you have a more manageable set of data, and it will be easier for you to explore questions from a larger group of people.

Key things with Sampling[edit]

  • Make sure you have a balanced sample. If you have a specific research question or problem you are trying to solve, make sure you include people in a fair way and in a systematic way
  • Reminders. It is often best to send reminders to users. With some tools, like CentralNotice, it can be hard to send reminders.
  • Response rates are important; its not just about who you reach, but who actually answers your survey.
  • Use this graph to help you decide how large your sample should be.

Create a sampling frame[edit]

  • As with any project, drafting a document that will clearly state who you are going to sample and how is very important, not only for your own planning, but also to be prepared with people as you questions
  • Sampling frames can be very complex. This table shows a simple one to help guide your work:
Population 1 Population 2 Population 3
Target Population Active editors on Spanish Wikipedia Active editors on Arabic Wikipedia
Population size 4,000 active editors each month
Target sample size 500 active editors
Sample method Mass message

Post-Survey Weighting[edit]

  • As this pew article demonstrates, weighting samples can be a great way to make sure that the data you have are truly representative of the population.

Related patterns[edit]