Learning and Evaluation/Archive/Learning modules/3Reliability & Validity

From Meta, a Wikimedia project coordination wiki
Jump to page

Part 1: Introduction

Why Survey?
Why Surveys Are Useful
Survey instruments
Types of information
Attributes - a special case
Survey Objective and Planning

Part 2: Reliability & Validity

Reliability & Validity
Face Validity
Content Validity
Criterion Validity
Construct Validity

Part 3: Question Construction

Writing Good Questions
Questions from Existing Surveys
Constructing your own Questions
Be Specific
Be Concise
Avoid Double Negatives
Minimize Social Desirability Bias
Avoid Double-barreled questions
Avoid abbreviations, jargon, technical terms, or slang
Avoid leading questions
Avoid loaded questions
Use appropriate wording
Ask useful questions
Rely on second-hand data sparsely
Use caution when asking personal questions

Part 4: Response Options

Question types
Dichotomous pairs
Multiple choice
Check all that apply
Choosing response options

Part 5: Questionnaire structure

Important considerations
Questions order
Additional Resources

  Wikimedia Training Designing Effective Questions Menu

Part 2: Reliability & Validity

Reliability and validity are terms frequently used to describe the accuracy in which proxy measures target a phenomena.

Reliability tells us how well a survey is able to give us the same information if we were to survey the same respondents over and over.

Validity is how well a survey is able to target the phenomena of interest.

Not reliable, not valid
The survey is not measuring what it is aiming for and respondents are giving different answers each time the survey is administered.
---- -

Low reliability, low validity
The survey questions are not close enough to the target and respondents are still giving different answers each time they respond to the survey.

Reliable, but not valid
The questions are more reliable in that participants are answering similarly each time they are asked, however, the measurement is not yet targeting the phenomena.
---- ..

Valid and reliable
Questions are written so that respondents always respond the same way and the questions are giving the information needed.
---- .