Learning patterns/Calculating global metrics

From Meta, a Wikimedia project coordination wiki
Jump to: navigation, search
Calculating global metrics
Vitruvian Man Measurements he.JPG
problemYou need to measure your project's output according to the global metrics specified by the Wikimedia Foundation.
solutionBegin documenting your process and planning data collection before you begin your project or host your event. Use the resources made available by WMF, and reach out early if you have questions.
creatorEGalvez (WMF)
endorse
created on27 August, 2014


Global Metrics
This is the overview for seven global metrics learning patterns.
Learn what's behind Global metrics - read the overview page.

What problem does this solve?[edit]

This learning pattern describes how to successfully gather the 7 global metrics. Grant-funded projects and programs will need to include these metrics in their project reports beginning in late 2014.

Some project leaders have experience reporting these metrics, while others are encountering these reporting requirements for the first time. These metrics can be calculated accurately and relatively painlessly for all projects, as long as you understand the metrics and plan ahead.

What is the solution?[edit]

The steps below are intended to help make sure that you are able to report your project's outcomes in terms of the 7 global metrics. You should make sure that you know how you are going to collect and calculate the data you need to report before you start. If you have questions about global metrics, you can ask your grants officer, post to the global metrics talk page, or email eval@wikimedia.org.


Before you start your event, project, or program[edit]

Create event or project pages[edit]

Project or event pages can help you organize your project or event. They provide a public place for you to record the relevant information and it serves as a central location for participants to sign up, ask questions, and begin participating. Some things you may list in a project page include:

  • Project or event name
  • Project duration or event dates and times
  • Project or event details (description, focus, target audience)
  • Opt-out information for getting user data (more on this below)
  • Sign-up list for participants
  • Any survey information
  • Pages edited
  • Follow-up information about other upcoming events or who to go for questions.

Plan to collect usernames[edit]

Online
e.g. Wikipedia Adventure, International Hackathon
In-person
e.g. Editing workshop, Local Editathon
1. Obtain a list of usernames
For most online projects, simply create a project page on wiki. If you are running an event, make sure to create your event page at least two-weeks prior. Invite new users who are participating to create accounts beforehand. For some online experimental projects, like the Wikipedia Adventure or the Teahouse, you can gather usernames by building it into your code and/or using MySQL. If you need assistance with this, email eval-at-wikimedia.org. Set up an event page on wiki two weeks before, and invite new users to create accounts. If your event is in one building with one Wifi connection, only 6 accounts can be created that day. If you are unable or do not need to create an event page on wiki, have a sign-up sheet prepared at the event date with a column for username and a column for email address in case they do not have a username yet or their username is illegible.
2. Ask a user's permission to get data
Post an Opt-out paragraph, which is text that tells participants that they can request to not have their data collected Ask participants to Opt-in when you ask for their username, that is, We must ask their explicit permission to use their usernames to get data about them.

For projects that involve images or media, create a category in Commons...[edit]

To be able to track which photos are uploaded for a specific event, project, or program, you must create a unique category tag that uploaders can use when they submit a photo to commons.

... and promote the USE of images on Wikimedia projects[edit]

  • If your program is based on images and media, be sure to tell participants to add images to Wikimedia projects!
  • Be sure to use the correct event name and spelling when publicizing your event.

Learning Question: Determine how you will gather this information[edit]

Use the Learning Question Learning pattern to review the different ways you can collect this information. How you collect this is up to you!! We hope this metric will give you interesting insights about your event, project, or program.

During or After Your Project or Event[edit]

Take or Review the Wikimetrics Learning Module[edit]

The Wikimetrics learning module will help guide you through the process of getting information from your list of usernames. Be sure to take the learning module well before your reports are due in order to give you time to ask questions. The Wikimetrics learning module can be found at Grants:Evaluation/Learning modules

Follow the learning patterns to report your Global Metrics[edit]

The learning patterns associated with the Global Metrics are not meant to be exhaustive. However, these resources are being developed to provide support to a wide range of projects, programs, and events. Consult the patterns below for guidance on how to collect and report global metrics for your project. Feel free to improve the pattern to make it more relevant to others to may perform similar activities in the future. Post questions or comments on the pattern talk page.

  • Not all programs will include all 7 global metrics - for some programs a metric may not be applicable. That is, while all programs have participants, not all programs focus to add or remove text or code (bytes added and/or removed) or to upload media files for instances. Just explain why it does apply and that is perfectly okay.
  • Global metrics are not the only measures - in fact, exploring and reporting other metrics beyond the 7 global metrics is encouraged.


  1. Number of newly registered users
  2. Number of active editors involved
  3. Number of individuals involved
  4. Number of images or media used in Wikimedia projects
  5. Number of articles created or improved in Wikimedia projects
  6. Number of bytes added and removed from Wikimedia projects
  7. Learning question: Did your work increase the motivation of contributors, and how do you know?


Example report form[edit]

This is an example of how to report global metrics. Your data might change depending on your project. Some of these number might even be zero - this is perfectly ok. It is important to provide clear explanations for how you arrived your numbers in the explanation column, and any other notes or information that you feel is relevant.

Metric Achieved outcome Explanation
1. Number of active editors involved 15 All the users who had at least 5 or more edits from July 1 to July 30 (the event was July 31)
2. Number of new editors 3 These were editors who created usernames two weeks before the event, up to the end of the event.
3. Number of individuals involved 21 All new and active editors who attended, plus volunteers who organized the event.
4a. Number of new media added to Wikimedia articles. 7 7 images out of the 100 images uploaded were used in wikimedia projects.
4b. Number of new media uploaded to Wikimedia Commons. (Optional) 100 100 images uploaded were uploaded to Wikimedia Commons.
5. Number of articles added or improved on Wikimedia projects 3 1 new article + 2 stubs expanded
6. Absolute value of bytes added to or deleted from Wikimedia projects 1,200 (absolute value) {1,000 Bytes Added} + {200 Bytes Removed} = 1,200
Learning question
Did your work increase the motivation of contributors, and how do you know?
  • Yes, we sent out a survey after the event and asked "To what extent did this event increase your motivation to contribute?". 16 participants (76%) said "a lot" or "some".


You can use this template to create a table for reporting global metrics as above.


General considerations[edit]

While global metrics are required for all grantees, they may be collected differently depending on the project. For example, while many will use Wikimetrics to obtain data, some may need to use MySQL because their project requires different technical skills. Also, some metrics might be zero and that is ok. If you are a grantee, speak with your grants officer if you have questions. If you are a volunteer interested in collecting this data, but need additional help, email eval-at-wikimedia.org.

When to use[edit]

Endorsements[edit]

See also[edit]

Education Toolkit Learning Pattern
This learning pattern is part of the Education Program Toolkit.
Go to the toolkit.

Related patterns[edit]

  1. Number of newly registered users
  2. Number of active editors involved
  3. Number of individuals involved
  4. Number of articles created or improved in Wikimedia projects
  5. Number of images or media used in Wikimedia projects
  6. Number of bytes added and removed from Wikimedia projects
  7. Learning question: Did your work increase the motivation of contributors, and how do you know?

External links[edit]

References[edit]