Jump to content

Learning and Evaluation/Introduction

From Meta, a Wikimedia project coordination wiki
Learning and Evaluation

In 2013, the Wikimedia Foundation started talking about evaluation. We have worked to help volunteers and organizations to begin evaluating their projects and program activities, to help program leaders and evaluators to:

  • more clearly identify goals and target outcomes for projects and programs,
  • measure against those goals and share learning about projects and programs, and
  • design and learn to implement effective programs to reach goals.

Wait, wait, what is...

[edit]

What is program evaluation?

[edit]

A program evaluation is the systematic collection of information about program activities, characteristics, and outcomes. Its purpose is to answer questions like:

  • Is a program achieving its goals?
  • What makes a program effective? How can we improve a program’s effectiveness?
  • What are the reasonable targets we might expect to reach with a particular program or strategy?
  • What are the needed processes and resources to implement a program successfully?
  • What are common challenges and work around strategies for adapting a program to a different context?

Why do we evaluate?

[edit]
  • To have information for making decisions, and for improving or strengthening a program
  • To provide accountability to funders and other stakeholders
  • For transparency of efforts and outcomes, and to promote learning within an organization
  • To measure progress in meeting goals
  • To improve program design

When do we evaluate?

[edit]
  • Before the project or program is implemented
  • During the project or program
  • At several points or regular intervals during the implementation of the project or program
  • After the end of the project or program

What is a program?

[edit]

A program is a group of related projects and activities that:

  1. share the same objectives and goal(s);
  2. are repeated on a regular basis and involve a commitment, and
  3. are based on a similar theory of change while using similar processes and interventions to make that change happen.

Let’s use Editathons as an example.

  1. Editathons often have the same objectives: to increase editors and content on Wikipedia.
  2. They are repeated and replicated around the world, and have existed for several years.
  3. An example theory of change for all Editathons: people gathered at one location or at one time increases motivation, content, and the number of editors to Wikimedia.
Examples of Wikimedia programs
[edit]
  • Wiki Loves Monuments
  • Wikipedia Education Program
  • Editathons
  • Conferences
  • Hackathons

Who is a program leader?

[edit]

A program leader is anyone in the Wikimedia movement who is involved in funding, planning, organizing, or implementing Wikimedia projects and programs.

What is evaluation capacity development?

[edit]

Evaluation Capacity Development is a process of continuously increasing a shared understanding and resources (tools, knowledge, and human connections) for a community to efficiently and effectively use in order to learn, build skills, and achieve a shared goal for evaluation.

Aims of the initiative

[edit]

For this evaluation capacity building initiative, we are all in it together – this effort will be a continuous process of learning for everybody involved. We will work collaboratively and we understand that context matters.

The aims of the current initiative are to support evaluation practice that is:

Empowering
  • Foster program leader choice
  • Focus on building program leaders capacity to help themselves
  • Support and enhance desired outcomes
Participatory
  • Develop program leaders evaluation logic and skills
  • Involve program leaders as key participants. They make the major focus and design decisions and select and commit to process and outcomes
  • Program leaders draw and apply conclusions to the design of their programming.
Utilization-focused
  • Focus on intended uses and users
  • Actively involve users in all aspects of evaluation
  • Measure the extent to which goals and objectives are met
  • Provide comparable data. This helps to inform decisions to continue, expand, or reduce funding, based on costs and impact.
  • Lead to ongoing commitment to using evaluation logic and building a community culture of learning

Feedback

[edit]
Visit the Evaluation portal talk page and let us know how it helps you. Post other feedback about this site or if you have questions about program evaluation. You may also email the Program Evaluation & Design team directly at eval(_AT_)wikimedia(_DOT_)org.

Additional resources

[edit]