Grants:Programs/Wikimedia Alliances Fund/The missing link:Incorporating policy reports into the the free knowledge ecosystem/Final Report
Report Status: Accepted
Due date: 2023-07-31T00:00:00Z
Funding program: Wikimedia Alliances Fund
Report type: Final
This is an automatically generated Meta-Wiki page. The page was copied from Fluxx, the grantmaking web service of Wikimedia Foundation where the user has submitted their midpoint report. Please do not make any changes to this page because all changes will be removed after the next update. Use the discussion page for your feedback. The page was created by CR-FluxxBot.
General information[edit]
This form is for organizations, groups, or individuals receiving Wikimedia Community Funds or Wikimedia Alliances Funds to report on their final results.
- Name of Organization: Analysis & Policy Observatory (APO)
- Title of Proposal: The missing link: Incorporating policy reports into the the free knowledge ecosystem
- Amount awarded: 41234 USD, 58214.83 AUD
- Amount spent: 58214.83 AUD
Part 1 Understanding your work[edit]
1. Briefly describe how your proposed activities and strategies were implemented.
In Step A we established an advisory group. In May, we recruited a student First Nations Editor.
Step B involved determining which publishing organisations would be considered a reliable source and selecting their reports to include in Wikidata. Drawing on discussions in Wikipedia and the variety of policies and guidelines on reliable sources, we developed a two-stage process for selecting reports to be uploaded.
A total of 64 publishers were identified for inclusion, however, selected reports from 36 publishers were uploaded into Wikidata due to the large number of reports on APO for each identified publisher. A lot of cleaning of the metadata and fixing broken links was required and took more time than had been allocated.
We created a collection on APO, called The MIssing Link Repository. This was not part of the original project plan but it was important to give Wikipedia editors the ability to search and browse the reports available to cite in Wikipedia.
Step C involved using the spreadsheet to develop a file of metadata for all selected publishers and reports to upload into Wikidata. We relied on the Advisory Group for their advice and guidance on the approach and tools to use. We modified existing tools to import Wikidata QIDs into APO and then to create metadata files to import back into Wikidata.
In Step D the aim was to build awareness of the project and create or edit content using the policy reports in Wikidata. We promoted the project to our audience and held two editing workshops at the end of June.
It was at this stage that we brought in the First Nations Editor to create or edit content on First Nations public policy issues. They found some of their engagements with Wikimedians disrespectful and culturally unsafe, and withdrew from the project prematurely. Select members of the project team and advisory group met to discuss what actions could be taken to improve the safety for First Nations editors on Wikipedia.
2. Were there any strategies or approaches that you felt were effective in achieving your goals?
We believe changing the name of the two ‘edit-a-thons’ to ‘Editing workshops’ was effective in eliciting new participants. An ‘edit-a-thon’ may indicate that the workshops are more like marathons – only for participants who are well-trained in editing and they are long in duration.
Promoting the work to our audience was an effective approach for engaging new participants. Ten people responded to our invitation to the Editing workshops and they came from a range of areas including government, academia and journalism. There were a number of people who had previously thought about getting involved in Wikipedia, or had dabbled in editing, but were not really sure how to get involved.
We believe that the Editing workshops were effective in building confidence in relatively new editors. APO’s Wikimedians in Residence had already undertaken the training and had done some edits before the workshops but they found the opportunity to ask questions and go through real edits with more experienced Wikimedians valuable.
Although not part of the original project plan, establishing the Missing Link Repository (https://apo.org.au/collection/321547/missing-link-repository) was a good way of communicating what we were doing and engaging APO’s audience. The Repository provides a concrete and visible output from the project that supports use of the reports, whereas the Wikidata entries do not enable easy searching to see what reports are in the database.
3. Would you say that your project had any innovations? Are there things that you did very differently than you have seen them done by others?
The main innovation in our project was repurposing existing tools (in Tool Forge) to design a system to do bulk uploads of metadata from the APO platform (Drupal website) into Wikidata.
We used the DOI tool for uploading documents to Wikidata (https://sourcemd.toolforge.org/index_old.php), which uses DOIs to retrieve metadata and then converts it into a TAB separated list.
We started by mapping the connected fields of selected APO reports: the primary field being publisher and the secondary connection of subjects.To do this we added the QIDs of selected publishers and subjects to the APO Drupal platform. The second stage of the DOI tool was adapted to create a TAB separated list of APO report metadata (https://sourcemd.toolforge.org/index_old.php?id=&doit=Check+source). Once we ran that in the tool it opens in quick statements for the importing into Wikidata.
To avoid any chance of duplication we then used the Beacon tool (https://wikidata-todo.toolforge.org/beacon.php?prop=7869&source=0&site=) to help import the new QIDs of imported reports into our system so that these items cant be exported again.
We then created an exporter for both mass and single reports to export the metadata in a TAB separated format as per the tool above.We now have an established method on our site to export single items to Wikidata using this approach. The TAB separated text is derived by adding /wiki to end of resource for example: https://apo.org.au/node/323196/wiki. Another tool is used for mass uploading, which is done by getting the same text file of all items by a single publisher.
4. Please describe how different communities participated and/or were informed about your work.
We published two blogs to inform our audience about this. The first blog introduced readers to the project and the second invited them to edit Wikipedia and participate in two editing workshops.
We also created The Missing Repository that presented all the reports that have been uploaded into Wikidata for users to explore and find out more about the project.
We held two Editing Workshops - these were hybrid workshops enabling participants from Australia and New Zealand to attend. The blogs, repository, and editing workshops were promoted to APO’s 15,000 newsletter subscribers who primarily work in government and community organisations, and across our two social media channels on Twitter and LinkedIn. We also advertised the workshops in our newsletters and on our website.
We also sent direct emails to publishing organisations whose reports had been uploaded to Wikidata which informed them of the project and invited them to participate in the workshops. 14 emails were sent to ten mainstream publishing organisations and the First Nations Editor personally sent 13 emails to First Nations-led organisations.
Wikimedia Australia also informed their community of the workshops and promoted these and the blogs on their social media channels.
Ten people registered to attend the workshops, with eight of them attending in the first workshop and three attending the second workshop (which focussed on First Nations policy topics). Some participants came from a range of government, community and education organisations, while others were retired or had a general interest in Wikimedia. One participant was from a First Nations community. Most participants registered with Wikipedia during the workshops.
Two follow up emails were sent to the ten people who registered for the workshops to encourage their participation beyond the workshops.
5. Documentation of your impact. Please use the two spaces below to share files and links that help tell your story and impact. This can be documentation that shows your results through testimonies, videos, sound files, images (photos and infographics, etc.) social media posts, dashboards, etc.
- Upload Documents and Files
- Here is an additional field to type in URLs.
https://blog.apo.org.au/2023/06/13/join-us-in-the-free-knowledge-ecosystem/
https://apo.org.au/collection/321547/missing-link-repository
https://lookerstudio.google.com/u/0/reporting/cfd5ef7a-13cf-416f-839d-34f5e5de303f/page/L5Rj
https://www.wikidata.org/wiki/Wikidata:WikiProject_Policy_Reports
6. To what extent do you agree with the following statements regarding the work carried out with the support of this Fund? You can choose “not applicable” if your work does not relate to these goals.
| A. Bring in participants from underrepresented groups | Disagree |
| B. Create a more inclusive and connected culture in our community | Agree |
| C. Develop content about underrepresented topics/groups | Agree |
| D. Develop content from underrepresented perspectives | Agree |
| E. Encourage the retention of editors | Agree |
| F. Encourage the retention of organizers | Disagree |
| G. Increased participants' feelings of belonging and connection to the movement. | Agree |
7. Is there anything else you would like to share about how your efforts helped to bring in participants and/or build out content, particularly for underrepresented groups?
The First Nations Editor withdrew from the project prematurely due to Wikipedia being a culturally unsafe space where they encountered racism. They wrote a report on their experiences that can be found here: https://docs.google.com/document/d/1sHGMOSzKJiDtxDoRn3X3VY1Rqto-3VhLqUBFsqyk82A/edit?pli=1 (Please contact us before distributing any further).
I was not well equipped to support our First Nations Editor because I was unaware of the issues that were present in this space. It was only after I had brought these issues to the Wikimedia community that I was informed there was work underway to document these issues to support action to address them.
The advice given to new Wikipedians – just get in and start editing, be bold – does not apply to FIrst Nations Editors. We discovered there are a lot more sensitivities around editing First Nations content. Extra support was needed and we were unable to recruit a Facilitator at the outset. This role may have been helpful to provide more one-on-one support and tailored advice and guidance to the First Nations Editor.
In my opinion, to bring in new participants from – and build content of – unrepresented communities there needs to be:
- a framework for Wikimedians respecting cultural practices
- respect for lived experience in determining what is a reliable source
- more preparation and support for FIrst Nations Wikimedians as they embark on the beginning of their Wikipedia journey.
Part 2: Your main learning[edit]
8. In your application, you outlined your learning priorities. What did you learn about these areas during this period?
Material published by organisations can be an important component of Wikimedia and the free knowledge ecosystem. Once we determined that reports published by organisations needed to be classified as “no consensus” and may be reliable depending on context, it became clear that policy reports could be reliablly sourced.
Incorporating material published by organisations presents a valuable opportunity to amplify the voices of underrepresented communities. Much First Nations knowledge is not written down in books or journal articles so it is important that the sources are widened.
There are more opportunities for further collaboration around the sharing of metadata and creating better linkages. This has the benefit of making both APO and Wikidata metadata Findable, Accessible, Interoperable and Reusable.
Previously, APO has imported and displayed information from Wikipedia. This could be established again for publishing organisations and subjects e.g. Built environment: https://apo.org.au/subject/667a5da4-6964-41f5-b1a9-f94ac8533abe https://en.wikipedia.org/wiki/Built_environment It’s still not clear how greater linkages between Wikimedia and APO could improve APO’s future sustainability. This project has broadened the Wikimedia community but it is not obvious how the APO community can be expanded. Feedback we received from members of the Wikimedia community was they were already aware of APO. More exploration of APO user needs would need to be undertaken.
9. Did anything unexpected or surprising happen when implementing your activities?
The most unexpected aspect was the feedback our First Nations Editor received from her edits in Wikipedia. It was a wake up call for me in terms of just how far we need to go to in getting widespread acceptance of First Peoples knowledge and culture.
10. How do you hope to use this learning? For instance, do you have any new priorities, ideas for activities, or goals for the future?
I am committed to continuing to work with the Wikimedia community to implement initiatives to create a culturally safe environment for First Nations Wikimedians.
Unfortunately, APO’s future is still uncertain at this stage so we do not have capacity to explore or initiative new activities or goals for the future.
11. If you were sitting with a friend to tell them one thing about your work during this fund, what would it be (think of inspiring or fascinating moments, tough challenges, interesting anecdotes, or anything that feels important to you)?
I feel that the Wikimedia community has inner and outer circles, and a lot of assumed knowledge is needed to become a part of the inner circle.
There were a couple of times I was embarking on or had started an activity, when someone would say: Have you done this? Do you need this? Without any prior explanation of what those things were or why they were needed. For example, someone asked me if I needed a dashboard but hadn’t explained to me what the dashboard would do. I made the mistake of assuming what a dashboard would be based on my experience and thought I would be able to do it myself.
12. Please share resources that would be useful to share with other Wikimedia organizations so that they can learn from, adapt or build upon your work. For instance, guides, training material, presentations, work processes, or any other material the team has created to document and transfer knowledge about your work and can be useful for others. Please share any specific resources that you are creating, adapting/contextualizing in ways that are unique to your context (i.e. training material).
- Upload Documents and Files
- Here is an additional field to type in URLs.
- N/A
Part 3: Metrics[edit]
13a. Open and additional metrics data
| Open Metrics | Description | Target | Results | Comments | Methodology |
|---|---|---|---|---|---|
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| Additional Metrics | Description | Target | Results | Comments | Methodology |
|---|---|---|---|---|---|
| Number of editors that continue to participate/retained after activities | N/A | N/A | N/A | N/A | N/A |
| Number of organizers that continue to participate/retained after activities | N/A | N/A | N/A | N/A | N/A |
| Number of strategic partnerships that contribute to longer term growth, diversity and sustainability | N/A | N/A | N/A | N/A | N/A |
| Feedback from participants on effective strategies for attracting and retaining contributors | N/A | N/A | N/A | N/A | N/A |
| Diversity of participants brought in by grantees | N/A | N/A | N/A | N/A | N/A |
| Number of people reached through social media publications | N/A | N/A | N/A | N/A | N/A |
| Number of activities developed | N/A | N/A | N/A | N/A | N/A |
| Number of volunteer hours | N/A | N/A | N/A | N/A | N/A |
13b. Additional core metrics data.
| Core metrics | Description | Target | Results | Comments | Methodology |
|---|---|---|---|---|---|
| Number of participants | Stage B: We are hoping to engage approximately 20 returning participants through the Reliable Sources Noticeboard and the volunteer advisory group.
Stage C: We are hoping to engage approximately 10 returning participants who have expertise in Wikidata including the volunteer advisory group. Stage D: We are hoping to engage approximately 10 new and returning participants in the edit-a-thon. In addition to the Wikimedians in residence (2), the Facilitator (1), and the lived experience expert (1). |
44 | 28 | Volunteer Advisor Group: (5)
Editors commenting on reliable sources query: (9) Wikidata assistance: (1) Wikimedians in Residence: (2) First Nations Editor (1) APO facilitator and participant (2) New participants in workshops (8) The areas where we overestimated participation were the participants in the reliable sources forums and those providing support with Wikidata. Members of the Advisory Group had several other roles including support with Wikidata and facilitating the editing workshops. |
|
| Number of editors | We are proposing to hold an edit-a-thon for which we expect to attract 5 returning editors and 6 Newly Registered Users (new editors) in total. This includes the lived experience expert who may or may not already be a registered user.
There will be an additional three editors who attend the edit-a-thon: the Wikimedia Facilitator (existing users) and the two APO Editors who will be trained as Wikimedians in Residence (Newly Registered Users). In total this is 6 returning editors and 8 new editors. |
14 | 17 | New participants at workshops: 8
Returning participants at workshops: 9 |
|
| Number of organizers | The organisers include the Project Lead, Wikimedia Facilitator and the two Wikimedians in Residence.
It is expected that there will be 4 members of the advisory group. The Wikimedia Facilitator will train the lived experience expert so that they can recruit new editors from their respective communities to participate in the edit-a-thon. |
9 | 6 | Project lead, 2 Wikimedians in Residence, 3 Advisory Group members. |
| Wikimedia Project | Description | Target | Results | Comments | Methodology |
|---|---|---|---|---|---|
| Wikidata | It’s expected that approximately 100 organisations and 1,000 reports will be added to Wikidata. | 1100 | 3079 | Reports in Wikidata: 3,041
Publishers in Wikidata: 38 We are now conducting further work to import all publishing organisations from APO into Wikidata. |
Data collected in this spreadsheet: https://docs.google.com/spreadsheets/d/1BD-Xh43OgOocun1v7rv2kmg2Uuzn-roK0fqlq7b2zgo/edit#gid=0 |
| Wikipedia | Approximately 6 Wikipedia pages will be created and 12 Wikipedia pages will be improved in the edit-a-thon. | 18 | 11 | We created one Wikipedia page: Royal Commission in Family Violence,
We created one Wikidata item: Coalition of Peaks Pages where sections were added: Second Stolen Generations Abolish Australia Day Indigenous self-determination Pages where references were added: Closing the Gap Koori Court Pages where edits were made: The death of Kumanjayi Wlaker The death of Ms Dhu Ganulu Gangulu |
This dashboard was created just to cover the period of the workshops (it doesn't include all edits made by First Nations Editor and Wikimedians in Residence): https://outreachdashboard.wmflabs.org/courses/Analysis_and_Policy_Observatory/APO_First_Nation_policy_workshop_(21_June_2023)
This document details the work that was planned or undertaken by the First Nations Editor: https://docs.google.com/document/d/1c4L8VtgzYFCOYy6Kzc6bBbgRzvTR0eNU3pejJbfddNc/edit?pli=1#heading=h.ajqys57z3osk |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
14. Were there any metrics in your proposal that you could not collect or that you had to change?
No
15. If you have any difficulties collecting data to measure your results, please describe and add any recommendations on how to address them in the future.
We didn't capture all edits made throughout the project. It would probably be good at the outset (i.e. proposal writing) to have information / guide / tutorial about what can be collected by creating Wiki dashboards. I was unaware of this available functionality.
16. Use this space to link or upload any additional documents that would be useful to understand your data collection (e.g., dashboards, surveys you have carried out, communications material, training material, etc).
- Upload Documents and Files
- Here is an additional field to type in URLs.
- N/A
Part 4: Organizational capacities & partnerships[edit]
17. Organizational Capacity
| A. Financial capacity and management | This has grown over the last year, the capacity is high |
| B. Conflict management or transformation | This has grown over the last year, the capacity is high |
| C. Leadership (i.e growing in potential leaders, leadership that fit organizational needs and values) | This has grown over the last year, the capacity is high |
| D. Partnership building | This has grown over the last year, the capacity is high |
| E. Strategic planning | This has grown over the last year, the capacity is high |
| F. Program design, implementation, and management | This has grown over the last year, the capacity is high |
| G. Scoping and testing new approaches, innovation | This has grown over the last year, the capacity is high |
| H. Recruiting new contributors (volunteer) | This capacity has grown but it should be further developed |
| I. Support and growth path for different types of contributors (volunteers) | This capacity has grown but it should be further developed |
| J. Governance | This has grown over the last year, the capacity is high |
| K. Communications, marketing, and social media | This has grown over the last year, the capacity is high |
| L. Staffing - hiring, monitoring, supporting in the areas needed for program implementation and sustainability | This has grown over the last year, the capacity is high |
| M. On-wiki technical skills | This capacity has grown but it should be further developed |
| N. Accessing and using data | This has grown over the last year, the capacity is high |
| O. Evaluating and learning from our work | This has grown over the last year, the capacity is high |
| P. Communicating and sharing what we learn with our peers and other stakeholders | This has grown over the last year, the capacity is high |
| N/A | |
| N/A |
17a. Which of the following factors most helped you to build capacities? Please pick a MAXIMUM of the three most relevant factors.
Peer to peer learning with other community members (but that is not continuous or structured), Using capacity building/training resources online from sources WITHIN the Wikimedia Movement
17b. Which of the following factors hindered your ability to build capacities? Please pick a MAXIMUM of the three most relevant factors.
Lack of volunteer time to participate in capacity building/training, Lack of training that fits contextual needs and interests
18. Is there anything else you would like to share about how your organizational capacity has grown, and areas where you require support?
N/A
19. Partnerships over the funding period.
| A. We built strategic partnerships with other institutions or groups that will help us grow in the medium term (3 year time frame) | Disagree |
| B. The partnerships we built with other institutions or groups helped to bring in more contributors from underrepresented groups | Disagree |
| C. The partnerships we built with other institutions or groups helped to build out more content on underrepresented topics/groups | Agree |
19a. Which of the following factors most helped you to build partnerships? Please pick a MAXIMUM of the three most relevant factors.
Institutional support from the Wikimedia Foundation, Volunteers from our communities
19b. Which of the following factors hindered your ability to build partnerships? Please pick a MAXIMUM of the three most relevant factors.
Lack of interest from partners, Lack of staff to conduct outreach to new strategic partners, Other
20. Please share your learning about strategies to build partnerships with other institutions and groups and any other learning about working with partners?
I believe that strategies to build partnerships with First Nations communities would be enhanced by effective and visible strategies to improve the cultural safety for First Nations editors in Wikipedia. Wikipedia is a very 'white space' that elevates written knowledges, where First Nations communities have always relied on oral histories and knowledges. This incompatibility is obvious to First Nations communities and creates an immediate barrier. So if obvious efforts are not made to reconcile this, the underrepresentation of FIrst Nations communities will continue.
Part 5: Sense of belonging and collaboration[edit]
21. What would it mean for your organization to feel a sense of belonging to the Wikimedia or free knowledge movement?
Ongoing contact and support including the identification of opportunities where we could be resourced to continue to contribute to Wikimedia and opportunities to attend or participate in occasional events that aren't resource intensive and would be beneficial to us. We'd be happy to reciprocate by promoting these events to our wider audience.
We already feel that APO is a central part of the free knowledge movement in Australia, along with Wikimedia, because we have a loyal subscriber and user base and we occasionally receive recognition for this.
22. How has your (for individual grantees) or your group/organization’s (for organizational grantees) sense of belonging to the Wikimedia or free knowledge movement changed over the fund period?
Somewhat increased
23. If you would like to, please share why it has changed in this way.
We received feedback from members of the Australian Wikimedia community that they were waiting for opportunities to build greater linkages between APO and Wikimedia - so that was encouraging to hear.
24. How has your group/organization’s sense of personal investment in the Wikimedia or free knowledge movement changed over the fund period?
Somewhat increased
25. If you would like to, please share why it has changed in this way.
The APO team have appreciated the opportunity to learn more about Wikimedia and how they can contribute to it.
26. Are there other movements besides the Wikimedia or free knowledge movement that play a central role in your motivation to contribute to Wikimedia projects? (for example, Black Lives Matter, Feminist movement, Climate Justice, or other activism spaces) If so, please describe it below.
Tackling misinformation. More and more we can see the need to provide open access to reliable information as misinformation and disinformation continues to spread.
This is captured in this APO blog published in 2021: https://blog.apo.org.au/2021/10/22/combating-misinformation/
Supporting Peer Learning and Collaboration[edit]
We are interested in better supporting peer learning and collaboration in the movement.
27. Have you shared these results with Wikimedia affiliates or community members?
Partially
27a. Please describe how you have already shared them. Would you like to do more sharing, and if so how?
We are still yet to publish blogs in Wikimedia forums e.g. Diff blog.
28. How often do you currently share what you have learned with other Wikimedia Foundation grantees, and learn from them?
We do not do this at all
29. How does your organization currently share mutual learning with other grantees?
N/A
Part 6: Financial reporting and compliance[edit]
30. Please state the total amount spent in your local currency.
58214.83
31. Local currency type
AUD
32. Please report the funds received and spending in the currency of your fund.
- Upload Documents, Templates, and Files.
- Wikimedia Signed Acquittal_37443.pdf
- Report funds received and spent, if template not used.
33. If you have not already done so in your budget report, please provide information on changes in the budget in relation to your original proposal.
N/A
34. Do you have any unspent funds from the Fund?
34a. Please list the amount and currency you did not use and explain why.
N/A
34b. What are you planning to do with the underspent funds?
N/A
34c. Please provide details of hope to spend these funds.
N/A
35. Are you in compliance with the terms outlined in the fund agreement?
As required in the fund agreement, please report any deviations from your fund proposal here. Note that, among other things, any changes must be consistent with our WMF mission, must be for charitable purposes as defined in the grant agreement, and must otherwise comply with the grant agreement.
36. Are you in compliance with all applicable laws and regulations as outlined in the grant agreement?
Yes
37. Are you in compliance with provisions of the United States Internal Revenue Code (“Code”), and with relevant tax laws and regulations restricting the use of the Funds as outlined in the grant agreement? In summary, this is to confirm that the funds were used in alignment with the WMF mission and for charitable/nonprofit/educational purposes.
Yes
38. If you have additional recommendations or reflections that don’t fit into the above sections, please write them here.