Apprentissage et évaluation / Fixation d'objectifs et de mesures
Partie du Centre de ressources Wikimedia
Apprentissage et évaluation
ou consultez ces présentations sur la définition des objectifs SMART :
-
Concevoir des propositions SMART - Wikimania Learning Day 2015 -
Concevoir des propositions et des plans annuels SMART - Wikimedia Conference Learning Day 2015
Buts, objectifs et chaîne de résultats dans la théorie de votre programme
La théorie du changement est une voie théorique qui articule ces étapes d'actions :
- Lier votre mission et vos activités de programmation au changement par des relations logiques de cause à effet
- Permettre de spécifier les résultats du programme et les résultats des participants sur la voie de l'impact que vous essayez d'obtenir.
- Se concentre sur des résultats-clés qui sont spécifiques, mesurables, atteignables, réalistes et limités dans le temps
Un cadre modèle logique peut vous aider à examiner et à définir la théorie du changement de votre programme.
À partir de là, il vous appartient d'établir un ordre de priorité entre les objectifs à atteindre, les résultats pertinents et les mesures et objectifs qui seront les plus importants pour raconter l'histoire.
Objectifs du programme
- Déclaration(s) générale(s) et orientée(s) vers l'avenir de la/des condition(s) souhaitée(s), déclaration(s) d'objectifs :
- Identifier le(s) groupe(s) cible(s) spécifique(s) de l'objectif,
- Fournir le « quoi » et non le « comment », et
- Ciblez le moment où l'objectif sera atteint ou se concrétisera de manière importante et mesurable.
Les exemples d'objectifs communs du programme Wikimedia comprennent :
- Construire et faire participer la communauté
- Améliorer la précision des informations
- Améliorer la qualité des contributions
- Accroître la notoriété des projets Wikimedia
- Accroître l'adhésion aux mouvements de la connaissance libre, de la connaissance ouverte et de la culture
- Augmenter les contributions aux projets
- Accroître la diversité des contributions et des contenus
- Accroître la diversité des contributeurs et contributrices
- Améliorer la perception des projets Wikimedia
- Améliorer la satisfaction des lecteurs et lectrices
- Accroître l'utilité, la convivialité et l'utilisation des contributions
- Accroître l'utilisation et l'accès aux projets
- Augmenter les compétences des personnes en matière d'édition/contribution
- Augmenter la motivation et l'engagement des bénévoles
- Respect accru des projets (c'est-à-dire acceptation par l'enseignement supérieur)
- Faire de la contribution un plaisir
- Faciliter la contribution
- Recrutement de nouveaux rédacteurs/contributeurs
- Conserver les rédacteurs/contributeurs existants
À titre d'exemple, les activités du programme, les intrants et les produits directs (extrants) pour l'objectif « Recruter de nouveaux rédacteurs/contributeurs » pourraient inclure :
| Inputs and Activities | Direct products |
|---|---|
| Invested time and money | Number of new user accounts created |
| Program event(s) | Number of new editors trained |
| Participants |
Activities and direct products of your program are important to track in order to evaluate what you did in order to assess the extent to which you did what you set out to do in terms of program implementation. However, measurable results are important for understanding your program outcomes and evaluating those results in terms of how they fit with your theory of change.
Measurable goals can vary a lot and may focus at different levels of impact based on the expected reach of your program impacts.
Program Objectives
Objectives provide guidelines for the accomplishment of goals:
- Identifies the target group (Who and/or what will change?)
- States the expected result (How will it change?)
- Specifies the degree of change in measurable terms (How much it will change? How will the change be measured?)
- Identifies the timeline for the change (When will the change have happened?)
Why Write Objectives
- Objectives are the building blocks or steps towards achieving a program's goals. (Also can be aligned/mapped as chain of outcomes)
- Objectives are specific and concise statements that state who will make what change, by how much, where and by when.
Examples of measurable result objectives (outcomes) toward the goal: “Recruiting new editors/contributors” could include results in participants skills and editing behaviors such as:
- 90% of program participants will demonstrate editing skills at the end of the program event.
- 70% of program participants will make at least one edit to Wikimedia projects within one month of their program participation.
- Increase the number of new accounts created on [Wikimedia Project Name] who make their first edit by 5% by July 2014
- 30% of program participants who created new accounts will “survive” as editors three months after their program participation.
- 10% of program participants who created new accounts will be “active” editors for three months following their program participation.
How to Write Objectives
Three broad categories:
- process (outputs/immediate outcomes)
- outcome (short and intermediate outcomes)
- impact (long-term outcomes)
The SMART Process to Developing Measurable Objectives
When writing goals and objectives, keep them SMART:
- Using the SMART Process ~ S
- Specific. Use specific rather than generalized language: Clearly state the issue, the target group, the time and place of the program.
- Using the SMART Process ~ M
- Measurable. Be clear in the objective about what will be changed and by how much. Setting this clearly at the start makes it easier to evaluate:
- Using The SMART Process ~ A
- Achievable. Be realistic about what the program can achieve in terms of the scale/scope of what is being done, the time and resources available:
- Using the SMART Process ~ R
- Relevant. Sometimes called “reasonable” or “realistic.” Objectives should reasonably relate to, and be relevant to, the goal and your program activities. Remember objectives represent the building blocks toward meeting goals:
- Using the SMART Process ~ T
- Timebound. Be clear in the objectives about the timeframe in which the program/activities, as well as expected changes, will take place:
Importantly, outcome objectives relate to various stages of outcomes and not processes.
Walk through an example [Note this is a hypothetical program]:
A program leader is planning a new program “Wiki Loves Notable Women” for improving coverage of notable women through a contest involving a month-long photo upload drive and series of online edit-a-thons to coordinate the creation of new, and improvement of existing, articles on notable women in terms of quality of coverage and images.
Process objectives would measure the activity of the program, for example:
To recruit new and existing users to participate in a contest to contribute photos of, and to articles on, notable women.
Applying SMART criteria, possible measures of Process/Activity Outcomes (a.k.a.”Outputs”):
- X participants will contribute images and/or article content during the contest month.
- X new contributors to Wikimedia.
- Participation will include contributors to X or more wikimedia projects.
- X notable women will be identified for article creation or improvement for the contest.
Outcome objectives measure the outputs and short-term, intermediate, and long-term outcomes.
Example outcome objectives include:
- Short-term objectives which may be met immediately or soon after a program implementation and focus on changes in terms of direct products generated by the program participation and learning that takes place as part of a program (i.e., changes in awareness, knowledge, attitudes, skills, interest, motivations, or intentions).
Typically, outcome objectives are based on short-term outcome objectives, or the direct products that may be output as part of the program activity. An example of objectives for short-term outcomes related to the Wiki Loves Notable Women program might include:
To increase the availability of images of notable women on Wikimedia projects.
Applying SMART criteria, possible measures of immediate or “short-term” Outcomes:
- X new images of notable women are added to the commons category during the month of the contest.
- X articles created on identified notable women during the month of the contest.
- X articles on notable women improved during the month of the contest.
- X% of identified articles are created on at least one language project during the month of the contest.
- X% of identified articles are improved during the month of the contest.
- X new notable women articles on the projects/increased representation of women on wikimedia projects.
- [If population total is known:] Increase representation of notable women by X%.
- X% of identified articles created or improved are marked for translation within a month of the contest.
Sometimes you may also create specific objectives around intermediate-term outcomes which may be met within a few months following a program implementation and focus on changes in actions in terms of behaviors, practices, decision-making, policies, or social action.
Examples of intermediate outcome objectives include:
- To increase the use of images of notable women on Wikimedia projects;
- To increase the representation of women on wikimedia projects;
- To increase the diversity of content on Wikimedia projects.
Applying SMART criteria, possible measures of intermediate outcomes:
- X participants continue improving the content created as part of the contest after the contest ends.
- X unique images of notable women added are used in Wikimedia projects within X months of the contest.
- X images of notable women added are used across projects within X months of the contest.
- X projects use the added images of notable women in their content within X months of the contest.
- X articles created or improved as part of the contest are rated as Good Articles within X months of the contest.
- X articles created or improved as part of the contest become Featured Articles within X months of the contest.
- X new notable women articles are translated into X number of language projects within X months of the contest.
Long-term outcomes generally do not have specific objectives around them as they will not happen until a long time after a program ends; they are the impacts which take several months or even years to observe.
Long-term impact objectives focus on changes in the to the social, economic, civic, or environmental conditions originally targeted and evolve as objectives are achieved along the way.
Examples of Long-term Outcomes include:
- To increase the accessibility and use of commons images of notable women in other media (e.g., other internet use, use in other publications);
- To increase the diverse representativeness of women on Wikimedia projects;
- Really (REALLY) long-term outcomes get right to your “mission”] To share in the sum of all knowledge.