Grants:Project/Eurecat/Community Health Metrics: Understanding Editor Drop-off/Final
This project is funded by a Project Grant
proposal | people | timeline & progress | finances | midpoint report | final report |
Work in progress
Welcome to this project's final report! This report shares the outcomes, impact and learnings from the grantee's project.
Part 1: The Project
[edit]Summary
[edit]We worked in several directions to address our overall goal of increasing the awareness of each community about their health status. In order to understand community dynamics, and to characterize editor lifecycle in relation to their interactions, we computed several metrics related to editor interactions and lifecycle over time. Computational analysis was complemented with qualitative research on the causes for drop-off based on questionnaires circulated in the community. Beyond community outreach at community events, we validated our metrics and findings with conversations and focus groups with Wikimedia affiliates. We developed interactive dashboards to expose results for all existing language editions; although still under improvement, the dashboards are available at: https://vitalsigns.wmcloud.org/ We hope that in the future the communities will routinely use this tool to monitor and improve their health.
Project Goals
[edit]Project Impact
[edit]Important: The Wikimedia Foundation is no longer collecting Global Metrics for Project Grants. We are currently updating our pages to remove legacy references, but please ignore any that you encounter until we finish.
Targets
[edit]- In the first column of the table below, please copy and paste the measures you selected to help you evaluate your project's success (see the Project Impact section of your proposal). Please use one row for each measure. If you set a numeric target for the measure, please include the number.
- In the second column, describe your project's actual results. If you set a numeric target for the measure, please report numerically in this column. Otherwise, write a brief sentence summarizing your output or outcome for this measure.
- In the third column, you have the option to provide further explanation as needed. You may also add additional explanation below this table.
Planned measure of success (include numeric target, if applicable) |
Actual result | Explanation |
Outreach at community events | Presentations at 5 community events:
|
|
Focus groups | Discussion with community members or affiliates about the design of community health metrics:
|
|
In-depth report for affiliates | In-depth analysis of community health metrics requested by 3 Wikimedia affiliates:
|
|
Survey responses | 32 responses to a survey on community health and drop-off causes (preliminary report) | |
Dashboards | Interactive dashboards to explore metrics and indicators for all existing language editions | |
Academic publications | Submitted 3 academic publications (1 published + 2 currently under review)
|
|
Source code | We created 23 GitHub repositories under the GitHub org: github.com/WikiCommunityHealth |
Story
[edit]Looking back over your whole project, what did you achieve? Tell us the story of your achievements, your results, your outcomes. Focus on inspiring moments, tough challenges, interesting anecdotes or anything that highlights the outcomes of your project. Imagine that you are sharing with a friend about the achievements that matter most to you in your project.
- This should not be a list of what you did. You will be asked to provide that later in the Methods and Activities section.
- Consider your original goals as you write your project's story, but don't let them limit you. Your project may have important outcomes you weren't expecting. Please focus on the impact that you believe matters most.
Survey(s)
[edit]If you used surveys to evaluate the success of your project, please provide a link(s) in this section, then briefly summarize your survey results in your own words. Include three interesting outputs or outcomes that the survey revealed.
Other
[edit]Is there another way you would prefer to communicate the actual results of your project, as you understand them? You can do that here!
Methods and activities
[edit]Please provide a list of the main methods and activities through which you completed your project.
Project resources
[edit]Please provide links to all public, online documents and other artifacts that you created during the course of this project. Even if you have linked to them elsewhere in this report, this section serves as a centralized archive for everything you created during your project. Examples include: meeting notes, participant lists, photos or graphics uploaded to Wikimedia Commons, template messages sent to participants, wiki pages, social media (Facebook groups, Twitter accounts), datasets, surveys, questionnaires, code repositories... If possible, include a brief summary with each link.
- Meta page of the project
- Source code
All the source code we have written in the scope of this project is available through this Github organization: github.com/WikiCommunityHealth
- Datasets
The Vital Signs metrics data can be downloaded from: vitalsigns.wmcloud.org/datasets/
- Dashboard
The Vital Signs dashboards can be accessed at: vitalsigns.wmcloud.org
- Academic publications
- Marc Miquel-Ribé, Cristian Consonni and David Laniado. "Wikipedia Editor Drop-Off: A Framework to Characterize Editors' Inactivity." Wikiworkshop 2021 (PDF)
- Marc Miquel-Ribé, Cristian Consonni and David Laniado. "Community Vital Signs: Measuring Wikipedia Communities’ Sustainable Growth and Renewal." Under review in Special Issue "Sustainability of Online Communities and Online Communities for Sustainability" of the MDPI journal "Sustainability".
- Marc Miquel-Ribé, Cristian Consonni and David Laniado. "Wikipedia, Elder or Teen? A Look at Growth, Stagnation and Decline Patterns Across 50 Language Communities." Under review at WikiWorkshop 2022.
- Presentations and Reports
- 24/11/2021 | Online presentation for the Italian Wikipedia community hosted by Wikimedia Italia | “Community Health Metric” Misuriamo lo stato di salute delle comunità di Wikipedia (slides PDF|notes PDF)
- 20/11/2021 | Viquitrobada (Catalan Wikipedia Annual Gathering) 2021 talk (program page) | Session: Measuring Catalan Wikipedia Community Health: Are We “Open” to Community Growth and Renewal?. (slides and notes PDF (català)).
- 7/11/2021 | Wikimedia CEE Meeting 2021 talk (program page) | Session: Measuring Central and Eastern Europe Wikipedias Growth and Renewal. (slides and notes PDF | video recording).
- 5/11/2021 | Wikiindaba 2021 talk (program page) | Session: African language Wikipedias - indicators for development, growth and renewal. (slides and notes PDF | video recording).
- 15/10/2021 | WikiArabia 2021 talk (program page) | Session: Measuring Arabic Wikipedia Community Health: Are We “Open” to Community Growth and Renewal?. (slides and notes PDF | video recording).
- 15/08/2021 | Wikimania | Session: Presentation and Open Discussion. (slides PDF | video recording).
- 14/04/2021 | WikiWorkWorkshop @ The Web Conference 2021 | Academic Paper/Presentation: Miquel-Ribé, M., Consonni, C., & Laniado, D. Wikipedia Editor Drop-Off A Framework to Characterize Editors' Inactivity (PDF).
- 24/10/2020 | ItWikiCon | Presentation: CristianCantoro Community Health Metrics: presentazione del progetto).
Learning
[edit]The best thing about trying something new is that you learn from it. We want to follow in your footsteps and learn along with you, and we want to know that you took enough risks in your project to have learned something really interesting! Think about what recommendations you have for others who may follow in your footsteps, and use the below sections to describe what worked and what didn’t.
What worked well
[edit]What did you try that was successful and you'd recommend others do? To help spread successful strategies so that they can be of use to others in the movement, rather than writing lots of text here, we'd like you to share your finding in the form of a link to a learning pattern.
- Your learning pattern link goes here
What didn’t work
[edit]What did you try that you learned didn't work? What would you think about doing differently in the future? Please list these as short bullet points.
Other recommendations
[edit]If you have additional recommendations or reflections that don’t fit into the above sections, please list them here.
Next steps and opportunities
[edit]Are there opportunities for future growth of this project, or new areas you have uncovered in the course of this grant that could be fruitful for more exploration (either by yourself, or others)? What ideas or suggestions do you have for future projects based on the work you’ve completed? Please list these as short bullet points.
Part 2: The Grant
[edit]Finances
[edit]Actual spending
[edit]Please copy and paste the completed table from your project finances page. Check that you’ve listed the actual expenditures compared with what was originally planned. If there are differences between the planned and actual use of funds, please use the column provided to explain them.
Expense | Approved amount | Actual funds spent | Difference |
Personnel cost for research and deelopment | 77,400 € | 83,400 € | +6,000 € |
Travel costs | 6,000 € | 0 € | -6,000 € |
Total | 83,400 € | 83,400 € | 0 € |
- Changes
Due to COVID-19, traveling was not possible; on the other hand, further work was needed to finalize the metrics and dashboards and the project duration was extended, so the funds for travel expenses were redirected to subcontracting Marc Miquel for this additional effort.
Remaining funds
[edit]Do you have any unspent funds from the grant?
Please answer yes or no. If yes, list the amount you did not use and explain why.
- No
Documentation
[edit]Did you send documentation of all expenses paid with grant funds to grantsadminwikimedia.org, according to the guidelines here?
Please answer yes or no. If no, include an explanation.
Confirmation of project status
[edit]Did you comply with the requirements specified by WMF in the grant agreement?
We have produced several results, including dashboards and academic publications; we are finalizing the analysis of data about reverts and warning templates in multiple languages.
Is your project completed?
No, we are still working on it, albeit with a reduced time dedication. We are producing the report with the latest results of our analysis.
On behalf of the team. --CristianCantoro (talk) 13:23, 27 January 2023 (UTC)
Grantee reflection
[edit]We’d love to hear any thoughts you have on what this project has meant to you, or how the experience of being a grantee has gone overall. Is there something that surprised you, or that you particularly enjoyed, or that you’ll do differently going forward as a result of the Project Grant experience? Please share it here!