Jump to content

Learning and Evaluation/Plan your program metrics/Retain community

From Meta, a Wikimedia project coordination wiki
Learning and Evaluation

Retain Community

Goal: Increase the number of active editors
Example Programs, Projects, or Events
  • Invite long-time editors to come back, like On-wiki Editing Contests
  • Invite existing editors to contribute in a new way, like Wiki Loves Monuments or through Hackathons
  • Invite existing editors to collaborate and continue editing, like WikiProjects



Measurement Tool

Inputs


Number of organizing and supporting staff and volunteers (ambassadors, mentors etc.)

Count of supporting staff and volunteers

Meeting Records, Logs, Reported estimates


Hours invested by staff and volunteers organizing and executing

Estimate of hours invested

Calendar Records, Logs, Reported estimates


In-kind donations (prizes, etc),

Estimated value in currency (comparable cost)


Funds or money used

Budget accounting

Budget spreadsheet


Participant pool to draw existing editors from

Target usernames

Project or event page or other username list



Measurement Tool

Outputs


Number of participants involved in the project or program event

Count of total number of participants (Organizers and attendees)

Project or event page or other participant tracking


Number of users

Count of participants with usernames

Project or event page or other username list


Number of new account users participating

Number of participant usernames with less than 10 edits

Wikimetrics, Threshold (from account registration forward) or Edits (from event date forward)


Number of existing editors participating

Count of participants with 1 or more edits, under their username, in their lifetime

Wikimetrics, Threshold (from account registration forward) or Edits (from event date forward)


Amount of text improved during event

Number of bytes added or removed

Wikimetrics - Bytes Added



Measurement Tool

Short term outcomes


Participant motivation to contribute

Participant self-report

Poll or questionnaire


Participant skills (editing skills, policy knowledge, process awareness)

Participant self-report AND/OR direct observation

Poll or questionnaire AND/OR quizzes/tasks with monitoring (i.e., human observer or mechanical observation via Wikimetrics/Mediawiki)


Number of new institutions/partnerships

Count of new institutions and partners working to implement Wikimedia programs

Direct observation/Program documentation



Measurement Tool

Intermediate outcomes


( Productivity )


Number of participants who became contributors
(10+ edits over lifetime)

Number of Contributors (10+ edits over lifetime)

Wikimetrics, Threshold (from account registration date forward) or Edits (from event date forward)


Amount of text improved since the program event

Number of bytes added or removed

Wikimetrics, Bytes Added


( Retention )


New Editor retention

Number of new editors (less than 10 edits) survived (i.e., 1+ edit in past 30-days) at 30 days and 3, 6, and 12 months

Wikimetrics, Survival (from account registration date) or Edits (from event date)


New Editor Conversion to Active Editors

Number of new editors (participants who had only 1-9 lifetime edits) became active editors (i.e., 5+ edits in past 30-days) at 30 days and 3, 6, and/or 12 months

Wikimetrics, Rolling New Active Editors (from account registration date forward) Rolling Active Editors (from event date forward), or Edits (from event date forward)


Number of Survived Editors

Number of existing editors (participants who had 10 or more lifetime edits) which survived (i.e., 1+ edits in past 30-days) at 30 days and 3, 6, and/or 12 months

Wikimetrics, Rolling Active Editors (from event date forward), or Edits (from event date forward)


Number of Active Editors

Number of existing editors (participants who had 10 or more lifetime edits) became active editors (i.e., 5+ edits in past 30-days) at 30 days and 3, 6, and/or 12 months

Wikimetrics, Rolling New Active Editors (from account registration date forward) Rolling Active Editors (from event date forward), or Edits (from event date forward)


Number of Very Active Editors

Number of existing editors (participants who had 10 or more lifetime edits) became very active editors (i.e., 100+ edits in past 30-days) at 30 days and 3, 6, and/or 12 months

Wikimetrics, Rolling Active Editors (from event date forward), or Edits (from event date forward)


Number of repeat participants

Number of individuals (on or off-wiki) who are returnees

Project or event page or other participant logs


Number of repeat contributors

Usernames with 1 edit during a program or event who are returnees

Wikimetrics, Edits (from event date forward)


Number of participants who increase participation in administrative roles, committees, chapters, etc.

Participant self-report, online username tracking

Questionnaire and/or manual tracking