Grants:Programs/Wikimedia Community Fund/Rapid Fund/Wikidata Ontology Course (ID: 22977619)/Final Report
Application type: Standard application
Part 1: Project and impact
1. Describe the implemented activities and results achieved. Additionally, share which approaches were most effective in supporting you to achieve the results. (required)
We successfully gave our course, with six two-hour lecture sessions including a one-hour guest lecture and two two-hour question-and-answer sessions. We covered all the topics in our proposed outline and added a couple of extra topics based on participant interest. The course sessions were well received.
We assigned 5 exercises, the ones for week 2-3-4 being mandatory, and gave detailed feedback on the mandatory exercises. We created a Telegram group, which currently has 19 people, and we have been keeping it active. We ended up with 9 projects from the participants. We were very pleased with the scope of the projects, although we were planning on more group projects.
In the end, the projects were the part of the course that is fostering continuing activities.
2. Documentation of your impact. Please use space below to share links that help tell your story, impact, and evaluation. (required)
Share links to:
- Project page on Meta-Wiki or any other Wikimedia project
- Dashboards and tools that you used to track contributions
- Some photos or videos from your event. Remember to share access.
You can also share links to:
- Important social media posts
- Surveys and their results
- Infographics and sound files
- Examples of content edited on Wikimedia projects
We registered 25 participants in the course, around 16 showed up to at least one session. 11 participants completed all the requirements of the course and 2 participants completed a significant amount.
Many other people were indirectly involved in the projects.
The participants set up nine projects to update or improve or use the Wikidata Ontology. Most of these projects are continuing, with some of them quite long-term.
We set up a Wikidata wiki page https://www.wikidata.org/wiki/Wikidata:WikiProject_Ontology/Ontology_Course under the Wikidata Project Ontology to serve as a central place for information about the course. This page links to the slides for all the sessions and to pages for each of the projects.
We recorded all the sessions of the course. We shared these recordings with all participants so that they could catch up with any sessions that they missed and review parts of the sessions for later.
Additionally, share the materials and resources that you used in the implementation of your project. (required)
For example:
- Training materials and guides
- Presentations and slides
- Work processes and plans
- Any other materials your team has created or adapted and can be shared with others
See above. The recordings are still unedited - we planned on working on them for a future version of the course. Interested parties can request access from us.
3. To what extent do you agree with the following statements regarding the work carried out with this Rapid Fund? You can choose “not applicable” if your work does not relate to these goals. Required. Select one option per question. (required)
| A. Bring in participants from underrepresented groups | Not applicable |
| B. Create a more inclusive and connected culture in our community | Not applicable |
| C. Develop content about underrepresented topics/groups | Not applicable |
| D. Develop content from underrepresented perspectives | Not applicable |
| E. Encourage the retention of editors | Strongly agree |
| F. Encourage the retention of organizers | Strongly agree |
| G. Increased participants' feelings of belonging and connection to the movement | Strongly agree |
| F. Other (optional) |
Part 2: Learning
4. In your application, you outlined some learning questions. What did you learn from these learning questions when you implemented your project? How do you hope to use this learnings in the future? You can recall these learning questions below. (required)
You can recall these learning questions below: How best to present information about the Wikidata ontology. How complicated can ontology content become before we start confusing people too much. How easy it is to get people interested in ontology and Wikidata maintenance work. How motivating are incentives in developing activities for User Groups. Are people that are our target demographic only interested in technical work, or are they interested in community building as well. What’s the ratio of people who would continue doing this community building work. The participant projects will explore areas of the Wikidata ontology and we expect to learn both by facilitating the projects and seeing the final results.
Our slides, lectures, and exercises provided content with a range of expected technical difficulty, from tutorial material at the start of the course to some quite sophisticated material in the latter two sessions. As well, the guest lecture was at a moderate-to-advanced level of technical sophistication. We found that participants were able to understand the material based on the questions they voiced and their answers to the exercises. We did have to provide some supplementary interaction to help some participants understand some of the more sophisticated material. So we feel that with some initial tutorial material Wikidata editors interested in the Wikidata Ontology can understand most of the technical material required to understand the Wikidata Ontology and effectively use and update it.
We found, unexpectedly, that most participants came into the course with something that they wanted to do. This meant that we did not use any incentives to increase interest but it appears that most (our estimate is about 3/4) of the participants will continue to interact with the Wikidata Ontology to maintain and improve it.
Some of our participants have already engaged existing projects and processes related to the Wikidata Ontology so they are not just doing technical work.
We have already learned interesting aspects of the Wikidata Ontology that we did not already know, for example in the books domain. We adjusted some of our course material to match the interests of participants. We saw that the structure of this course allows us and others to learn from the participants as well.
In any future version of this course we will adjust initial information about the project to better match what happened this time, but still recommend group participation in projects.
5. Did anything unexpected or surprising happen when implementing your activities? This can include both positive and negative situations. What did you learn from those experiences? (required)
We underestimated the attrition from expressing interest to registering for the course to actually participating in the course. Because the number of people that expressed interest was well above the number of participants that we felt we could support we tried to limit registration which resulted in slightly fewer participants than we wanted, even when we allowed for more registrants that we had expected to support.
We overestimated the ability and desire for participants to create group projects. Most course participants had something that they were interested in before the course and created a project proposal for their interest. Instead of requiring group projects we allowed some individual projects and undertook the added effort to evaluate and assist in more projects than we had expected. This did increase the amount of effort required to run the course.
We underestimated the expertise of participants. Most course participants had expertise in one or more areas related to Wikidata. This resulted in a number of projects that were more ambitious than we had expected. This is a positive outcome as far as we are concerned and we will support projects that continue after the end of the course. Because the projects are more ambitious than expected their effect on Wikidata will be felt after the end of the course so our metrics on effects are not useful as of the end of the course.
We expected that most of the course projects would look at a part of the Wikidata ontology and produce fixes to point problems. Instead most of the projects involve additions to the Wikidata ontology, such as a new property or a revision of how a domain is set up. The actual projects are thus more involved and have greater reach, but are harder to quantize, particularly as most of the changes will be at the end of the projects and after the end of the course. We view this as an unexpected success and believe that this will produce greater ongoing interest in the Wikidata ontology.
We learned a lot about managing participation and attendance, as things didn’t go as smoothly as we had hoped. In future iterations, we plan to collect applications through a form rather than by email. This form will include questions about consent for recording and other logistical matters. Handling these separately this time led to unnecessary time spent on our side and caused inefficiencies in communication with participants.
We realised that it would probably have been better to have more conversation hours between participants. We had an office hour in the week between the last two lessons, and the conversation was really fun for the people involved. In hindsight, we probably should have had more of these sorts of activities.
One thing we learned is the importance of being clearer about our expectations, for example, around deadlines. At times, we assigned exercises with a soft deadline of 3-4 days, hoping to receive at least a few submissions in time to provide useful feedback. However, we didn’t communicate this being a soft deadline clearly enough, which led to some participants expressing concern that the timeline was too tight. While the issue was eventually resolved through communication, it did cause unnecessary stress. In the future, we’ll make sure to explain these timelines and their purpose more clearly to avoid confusion.
6. What is your plan to share your project learnings and results with other community members? If you have already done it, describe how. (required)
Our Wikidata wiki page is the central place for the community to see information about the course and its projects. We have publicized this page in the Wikidata Weekly Status Updates and in several Wikidata Telegram groups and in the Wikidata Project Ontology page. We have set up a final publicity blurb for the course and placed it in the Wikdata Project Ontology page and several Wikidata Telegram groups and submitted it to the Wikidata Weekly Status Updates.
We will keep publicising (or encouraging others to publicise) any significant development within the projects, in the Wikiproject Ontology Telegram group, or the Wikidata Telegram group.
Part 3: Metrics
7. Wikimedia Metrics results. (required)
In your application, you set some Wikimedia targets in numbers (Wikimedia metrics). In this section, you will describe the achieved results and provide links to the tools used.
| Target | Results | Comments and tools used | |
|---|---|---|---|
| Number of participants | 15 | 17 | Telegram group membership
Participation in course sessions |
| Number of editors | 15 | 13 | Participation in course exercises |
| Number of organizers | 2 | 3 | The two course organizers and the guest lecturer |
| Wikimedia project | Target | Result - Number of created pages | Result - Number of improved pages |
|---|---|---|---|
| Wikipedia | |||
| Wikimedia Commons | 6 | 6 | 0 |
| Wikidata | 1000 | 2 | 500 |
| Wiktionary | |||
| Wikisource | |||
| Wikimedia Incubator | |||
| Translatewiki | |||
| MediaWiki | |||
| Wikiquote | |||
| Wikivoyage | |||
| Wikibooks | |||
| Wikiversity | |||
| Wikinews | |||
| Wikispecies | |||
| Wikifunctions or Abstract Wikipedia |
8. Other Metrics results.
In your proposal, you could also set Other Metrics targets. Please describe the achieved results and provide links to the tools used if you set Other Metrics in your application.
| Other Metrics name | Metrics Description | Target | Result | Tools and comments |
|---|---|---|---|---|
| New Project Ontology Members | How many people join the Wikidata Project Ontology or Wikidata ontology Telegraph group. | 7 | 10 | New members of Wikidata Telegram Ontology group (estimate) |
| Wikidata Items Affected Indirectly | How many Wikidata items are affected indirectly by the changes made in projects. This can be much higher than the number of change because the changes are in the Wikidata ontology and will likely affect all of the instances of the class. This number can vary very wikely based on the activities in the project, but we expect a large number of indirectly affected items. | 100000 | Most projects are ongoing and have not yet performed updates to Wikidata. There are several projects that have many proposed updates to Wikidata, totaling about 46,000 updates, which will likely be performed using QuickStatements. | |
9. Did you have any difficulties collecting data to measure your results? (required)
Yes
9.1. Please state what difficulties you had. How do you hope to overcome these challenges in the future? Do you have any recommendations for the Foundation to support you in addressing these challenges? (required)
As joining Telegram groups is only shown in the logs of the group it was a bit hard to measure participation.
As most of the projects are ongoing, their major contributions are in the future. We are submitting the final report now as the parts of the course supported by this grant are over. Allowing for followup reports would be useful.
We had trouble with keeping track of some things that normally should have been easy to track. This is mostly because we forgot to explicitly ask for some things. One example is that, in one of the exercises, we forgot to collect usernames. We took some notes on what to do next time(s) so we won’t forget simple things like this in the future.
Part 4: Financial reporting
[edit]10. Please state the total amount spent in your local currency. (required)
3269.98
11. Please state the total amount spent in US dollars. (required)
3269.98
12. Report the funds spent in the currency of your fund. (required)
Provide the link to the financial report https://docs.google.com/spreadsheets/d/1lDKTvRn418VpVVPXZCs5sarCk8IMLnGR4pdoEDI5dtk
12.2. If you have not already done so in your financial spending report, please provide information on changes in the budget in relation to your original proposal. (optional)
We had asked for US$5000 and were funded for US$3750. To account for this difference we decided to eliminate incentives for participants, which also cut down on bank fees, and reduced course preparation support. Our guest speaker could not accept an honorarium so we moved this expense to the course preparation support to bring it back to the original amount. The course projects were not of the sort that required financial support so we used half of this expense for more staff time to better support the larger number of projects. We had a minor extra cost to support recording of sessions and supporting sessions longer than one hour.
13. Do you have any unspent funds from the Fund?
Yes
13.1. Please list the amount and currency you did not use and explain why.
The projects did not need any direct support and we only used half (US$500) of the project support for extra staff expenses, leaving US$480.02 unspent as we had to spend US$19.98 for services from Google. We plan on sending the excess back to WMF when the final report is approved.
13.2. What are you planning to do with the underspent funds?
C. I am planning to send them back to the WMF
13.3. Please provide details of hope to spend these funds.
N/A
14.1. Are you in compliance with the terms outlined in the fund agreement?
Yes
14.2. Are you in compliance with all applicable laws and regulations as outlined in the grant agreement?
Yes
14.3. Are you in compliance with provisions of the United States Internal Revenue Code (“Code”), and with relevant tax laws and regulations restricting the use of the Funds as outlined in the grant agreement? In summary, this is to confirm that the funds were used in alignment with the WMF mission and for charitable/nonprofit/educational purposes.
Yes
15. If you have additional recommendations or reflections that don’t fit into the above sections, please write them here. (optional)
Review notes
[edit]Review notes from Program Officer:
N/A
Applicant's response to the review feedback.
N/A