Grants talk:Project/Building up performing arts data in Wikidata
Hello Frederic Julien,
Thanks for beginning to draft your proposal for the Project Grants open call! I wanted to make sure you are aware that we are hosting proposal clinics for applicants to ask questions and get feedback. If you would like to attend, you can find the dates, times, and videoconference links posted on this page meta.wikimedia.org/wiki/Grants:Project. These are optional opportunities to get support improving your proposal. Let me know if you have any questions! Good luck with finishing your proposal for the February 10 deadline!
Reminder: Change status to proposed to submit
IMPORTANT: Please note that you must change your proposal status from "draft" to "proposed" by the submission deadline in order for your proposal to be reviewed in the current round. When your proposal has been successfully submitted, it will show up in the "Open proposals" list (it may take several minutes for the list to update after you submit it). Applications that are not completely filled out and correctly submitted by the deadline will not be reviewed. To submit your proposal, you must complete all fields of the application and then:
- 1. Click on "edit source"
- 2. Change "|status=DRAFT" to "|status=PROPOSED"
- 3. Click the "Publish changes" button.
Outcome of 1st project
What was the outcome of Grants:Project/Fjjulien/Modelling and Populating Performing Arts Data in Wikidata? What happened with the $33,500 CAD ? Jura1 (talk) 07:49, 12 February 2021 (UTC)
- Jura1 The project isn't quite over yet, but we already reached almost all our targets (you can read our monthly reports and midpoint report for full details). Evaluation of the workshop series is underway. The translation of the WikiProject to French remains an outstanding task (we'll be getting there shortly). The only activity we will not be able complete is the upload of works. There were simply not enough person items yet in Wikidatat to state all contributor roles. We are currently in discussion with unions and with other associations to upload their members directory. The response is so far very good and we are confident to achieve significant progress with the next project. Fjjulien (talk) 20:53, 12 February 2021 (UTC)
Getting editors who only edit on one day isn't a good goal
When it comes to the goals of a workshop, I think the goal should be that the person taking the workshop will engage with Wikidata after taking the workshop and not that they edit the maximum amount in a workshop. I would want a delievery according to which at least X users make edits after they leave the workshop. ChristianKl ❪✉❫ 16:22, 12 February 2021 (UTC)
- @ChristianKl: I agree on the fact that learning activities should strive to build long-term engagement with Wikidata. This is what we meant to accomplish in the last year with the workshop series and we were quite successful in this regard (judging by our outreach dashboard). This being said, no matter how active this small group of Wikimedians is, our level of penetration across the performing arts workforce was not as good as we would have liked. We think a clinic approach may offer a stronger value proposition ("what's in it for me") for cultural workers and for their associations than workshops: we think it may provide us with a better carrot to get the discussion started (in addition to populating organization data, which is otherwise difficult to find). By getting associations on board, we can also engage a dialogue about what data they hold and whether or not it is suited for upload to Wikidata. We would like to try this strategy out, evaluate, and continue/adjust/discontinue.
- In terms of performance measurement, we will collect clinic participants' usernames and we will track their edits during and after clinics in outreach dashboards. We will also monitor new users who join the monthly workshop/club gatherings. We just haven't set specific targets for these indicators, because we have no historic data to benchmark against. Fjjulien (talk) 21:26, 12 February 2021 (UTC)
- @ChristianKl: During the first project, one in two workshop participants (49%, 84 in total) made edits to Wikidata. With the clinic strategy, every single participant would normally make edits during the clinic. It would be possible to track edits during and post-clinic. Based on the edit rate in the first project, we could hope that 40% or more of clinic participants would do edits after the clinic. Fjjulien (talk) 20:09, 28 March 2021 (UTC)
What Wikidata modeling discussion happened?
As far as I understand "The LODEPA Wikipedia/Wikidata working group provides a forum for discussing use cases, modelling issues and prototyping activities around Wikidata." was also a goal of the last grant. On Wikidata onwiki consensus is primary for making modeling decisions that are actually in a nature that other people orient themselves based on them. While I see no issue with having off-Wiki discussions when it helps for the process, I do worry that not enough of the modelling ends up being written up in a Wikiproject. Can you tell me more about how this went with your last grant? ChristianKl ❪✉❫ 15:28, 15 February 2021 (UTC)
- @ChristianKl: As we pointed out in our midpoint report, we found it challenging to engage domain experts in pointed modelling discussions, and doing so in talk pages would have represented one additional obstacle to their participation. We held many modelling discussions in videoconference meetings as well as over Google Docs and Slack. Then, when consensus was reached one of us documented the recommended modelling strategy in the appropriate WikiProject (with lists of properties and notes about their usage). This method worked fairly well, but it had the disadvantage of keeping discussions within a closed advisory group. For the next project, we would like to rely on a broader community (the LODEPA Wikipedia/Wikidata working group - and possibly the wikiclub). We also want to hold more discussions directly in talk pages, but we need other means than pings to notify the community (pings haven't been an effective notification and engagement method) and get the discussion started. Fjjulien (talk) 17:43, 16 February 2021 (UTC)
Eligibility confirmed, Round 1 2021 - Community Organizing proposal
We've confirmed your proposal is eligible for review in Round 1 2021 for Community Organizing projects.
This decision is contingent upon compliance with our COVID-19 guidelines. If your proposal includes travel and/or offline events, you must ensure that all of the following are true:
- You have reviewed and can comply with the guidelines linked above.
- If necessary because of COVID-19 safety risks, you can complete the core components of your proposed work plan _without_ offline events or travel.
- You are able to postpone any planned offline events or travel until the Wikimedia Foundation’s guidelines allow for them, without significant harm to the goals of your project.
- You include a COVID-19 planning section in your activities plan. In this section, you should provide a brief summary of how your project plan will meet COVID-19 guidelines, and how it would impact your project if travel and offline events prove unfeasible throughout the entire life of your project. If you have not already included this in your proposal, you have until February 28 to add it.
The Community review period is now underway, from February 20-March 4. We encourage you to make sure that stakeholders, volunteers, and/or communities impacted by your proposed project are aware of your proposal and invite them to give feedback on your talkpage. This is a great way to make sure that you are meeting the needs of the people you plan to work with and it can help you improve your project.
- If you are applying for funds in a region where there is a Wikimedia Affiliate working, we encourage you to let them know about your project, too.
- If you _are_ a Wikimedia Affiliate applying for a Project Grant: A special reminder that our guidelines and criteria require you to announce your Project Grant requests on your official user group page on Meta and a local language forum that is recognized by your group, to allow adequate space for objections and support to be voiced).
Please feel free to ask questions and make changes to this proposal as discussions continue during the community review period. By March 4, make sure that your proposal has incorporated any revisions you want to make and complies with all of our guidelines. If you have not already done so, you can make use of our project planning resources to improve your proposal further, too.
The Project Grant committee's formal review for round 1 2020 will occur March 5 through March 20, 2021. We ask that you refrain from making any further changes to your proposal during the committee review period, so we can be sure that all committee members are scoring the same version of the proposal.
Grantees will be announced Friday, April 22, 2021. Sometimes we have to make some changes to the round schedule. If that happens, it will be reflected on the round schedule on the Project Grants start page. We look forward to engaging with you in this Round!Questions? Contact us at projectgrants wikimedia · org.
- @Mjohnson (WMF): Happy to learn that the project is now under review! The project does not involve any offline activities, so no COVID-19 planning is necessary. We will continue to seek community feedback up until March 4. Fjjulien (talk) 22:07, 23 February 2021 (UTC)
Comment from Wikimedia Foundation GLAM & Culture team
Hello! We’re happy to see this project continue and welcome your reflections on how to evolve your tactics for participation. We’re also pleased to see the range of funding sources, which demonstrates a high level of sectoral support for this initiative.
- @GFontenelle (WMF): We were very pleased to see another project grant proposal for the performing arts, and I immediately signed up as a volunteer. I have yet to connect with Pakoire, but I already had the opportunity to interact with DrThneed over the WikiProject Performing arts talk page. Here are a few thoughts about coordination.
- Because of the time zone difference, synchronous collaboration may be a little difficult - especially for LODEPA meetings that also involve European partners. But we could absolutely hold a few coordination meetings at the beginning of the projects and then rely on talk pages for ongoing asynchronous collaboration.
- There may also be opportunities for our New Zealand colleagues to reuse our workshop decks, and we could further collaborate in the design of data clinics presentation decks.
- Finally, there is much we could learn from one another around the engagement of Indigenous artists in Wikidata. For ethical reasons, we have deliberately chosen not to include these activities in our project grant but members of our Wikidata advisory committee have undertaken consultative activities with Indigenous artists about the representation of Indigenous identities and cultural practices in Wikidata and other open knowledge bases. I would personally love to know how Māori artists feel about Wikidata, and to learn from our New Zealand colleagues about successful engagement strategies.
- Fjjulien (talk) 19:10, 6 March 2021 (UTC)
- Hi Fjjulien! Thank you so much for taking the time to answer our comments in such an attentive way.
- We are really glad to learn there is a willingness to collaborate between the two projects.
- The time zone difference is an important issue to highlight. We understand if it is difficult to accomplish a collaboration because of it. However, if you manage to find a way, we would be really glad to see an association.
- That would be a good collaborative effort/opportunity.
- This is really important and exciting to learn. We are glad to know you are undertaking consultative activities with indigenous artists. To enhance the representation of indigenous identities and practices on Wikidata, in a respectful way, is essential. We hope you can find a way to collaborate and learn with the Performing Arts Aotearoa in that sense.
- Again, thank you so much for your feedback. Best, GFontenelle (WMF) (talk) 23:51, 15 March 2021 (UTC).
Aggregated feedback from the committee for Building up performing arts data in Wikidata
|(A) Impact potential
|(B) Community engagement
|(C) Ability to execute
|(D) Measures of success
|Additional comments from the Committee:
Opportunity to respond to committee comments in the next week
The Project Grants Committee has conducted a preliminary assessment of your proposal. Based on their initial review, a majority of committee reviewers have not recommended your proposal for funding. You can read more about their reasons for this decision in their comments above. Before the committee finalizes this decision, they would like to provide you with an opportunity to respond to their comments.
Responses to aggregated feedback from the committee
Deliverables of the first project
- “The set of items of the last year have not been uploaded as promised.”
- “Slightly inclined to "no" because the deliverables of last year seem to be less impactful as expected. Considering the planning to apply for a second round, I would have preferred to push more the work package of tests for uploading than the training sessions.”
In quantitative and qualitative terms, we reached and exceeded all our targets:
- We said we would upload 500 items. We uploaded 1637.
- We said we would deliver 4 presentation decks in English and French. We did. And we also delivered 37 video tutorials.
- We said we would have 190 participants in total. We had 391.
- In terms of enrichment of items, our workshop participants made 12,700 edits.
- We also conducted activities that went beyond the scope of the original project: we facilitated working group meetings with the Linked Open Data Ecosystem for the Performing Arts community and we initiated a consultative process with Indigenous arts and culture practitioners.
These are just a few indicators of impact. More information can be found in our monthly updates and in our draft final report.
Were we as happy with attendance at the English workshops compared to attendance at the French workshops? No. But that doesn’t mean they were not impactful.
Did the modelling and upload of works go as anticipated? No. First of all, IFLA undertook consolidation of LRMER with CIDOC-CRM and FRBRoo, and their draft documentation announced the deprecation of FRBRoo F20 Performance Work and F25 Performance plan – two fundamental classes around which all our modelling of works was supposed to happen. Considering we had promised to deliver modelling outputs harmonized with RDF ontologies, this presented a problem. We have gotten in touch with the Consolidation Editorial Group for more information, but their initial answers have not provided any indication of how they intended to model the relationship between core WEMI works and performance works. Second, a W3C community group dedicated to Performing Arts Information representation started their own modelling activities. It seemed wise not to duplicate efforts, but rather to work along with this PAIR W3C community group to model works. Third, even though we had previously uploaded person items, many cast members and contributors in the CQT work dataset were not present in Wikidata, so we had to create more person items. In the end, it required more effort than anticipated and we do not know yet if our modelling for representing works will be harmonized with the upcoming LRMoo or the PAIR-CG core performing arts ontology, but we did upload 379 works (more than the 300 we had promised).
Our group has displayed an aptitude at quickly identifying challenges and at adapting to overcome these challenges and to deliver impact. We have also reported in a transparent manner on both our successes and challenges. We think these are qualities of strong project management. Fjjulien (talk) 16:18, 28 March 2021 (UTC)
- “Following the comments from WMF GLAM & Culture Team, it would have been great to see a meaningful demonstration of efforts to address indigenous arts institutions in the project delivery team or project governance that go beyond a public call for a one-off consultation.”
- “I gave this a low score on supporting diversity and would suggest the project team addresses how they will approach diversity and accessibility. Particularly in the delivery of activities, recruitment of participants, project team, and governance.”
- “it is difficult to see how this project will build on previous work and addresses the specific diversity and accessibility needs of their region/community.”
Project advisors in the first project grant included Anju Christofferson Singh and Denise Bolduc. Anju is a developer with considerable experience working with the BIPOC community. Denise is a respected curator and producer of Indigenous performing arts events. Both are committed to continue with us during the proposed project grant.
Along with CAPACOA’s coordinator of Indigenous programs, Brit Johnston, Anju and Denise initiated a consultative process with Indigenous arts and culture practitioners.
This process was never intended to just be a “one-off consultation”. This is not how meaningful and respectful relationships with Indigenous Peoples are built.
Prior to the first event, members of the committee reached out and consulted with potential advisors and partners, including authors of the CARE Principles for Indigenous Data Governance and the First Nations, Metis and Inuit – Indigenous Ontologies, as well as with the organizer of an initiative to Indigenize Wikipedia.
Then, at the time of submitting the project grant, the initial gathering had not taken place yet. It was therefore not possible to say, if participants would want to continue the process or if they would consider Wikidata an appropriate knowledge base. To this date, we still do not know if the Indigenous arts and culture practitioners from the territory known today as Canada will want information about their identities and practices to be populated in Wikidata. For this reason, it would have been presumptuous (and extractive) to seek funding from the Wikimedia Foundation for these activities.
But the fact we did not include these activities in our project grant scope does not mean we don’t intend to do them.
So here are activities we will conduct in parallel with this project grant:
- We will continue the consultative process with Indigenous arts and culture practitioners. We intend to start discussing the representation of Indigenous information in Wikidata at our next meeting, which will take place in late April (date not confirmed yet). But we want to do this carefully because we know many Indigenous wikimedians are quite uncomfortable with wdt:P172 ethnic group to denote Indigenous identities.
- We will undertake prototyping activities with the Indigenous Performing Arts Alliance (IPAA). Our memorandum of understanding isn’t signed yet, so I cannot share the project documentation at this point. But we intend to put in place automated data production and data consumption processes between the IPAA member directory, the Artsdata knowledge graph and the IPAA Indigenous Now events listings, with possible data population and queries with Wikidata (pending on our consultative process confirming that Indigenous Arts and Culture practitioners feel comfortable and safe having their information in Wikidata). Our colleagues at Culture Creates and at the Bern University of Applied Sciences already have a proof of concept of an events listing website populated by federated queries from multiple databases (Culture in Time). We hope to be able to do the same with the IPAA listings. However, the decision does not rest with our project team. It rests with the Indigenous arts and culture practitioners served by IPAA.
We are committed to undertaking these activities in ways that respect the rights of Indigenous peoples for self-determination and data sovereignty. This implies not imposing predetermined solutions and timelines to our Indigenous colleagues. Which, in turn, implies not making any promise to the Wikimedia Foundation in this project grant. Fjjulien (talk) 21:36, 28 March 2021 (UTC)
- “The application has limited evidence of an evaluation plan.”
Throughout the first project, we demonstrated a solid capacity for performance measurement. We have monitored total and unique participation at our events. We also monitored edits by workshop participants in the outreach dashboard. And we surveyed workshop participants (results will soon be published in our final report). We intend to use the same measurement methods in the next project. Will also keep monitoring statistics in the WikiProject Performing arts.
- I provided more information on our evaluation plan for clinics in a discussion thread above.
- We haven’t produced a final set of statistics yet for the current project grant. These statistics will be generated soon and they will attest of the impact of the current project.
Wider Wikimedia community
- “Has the potential to have an impact on the wider Wikimedia movement but there seems to be a limited focus on communicating with or advocating within the wider Wikimedia community.
Since the comments about the previous project suggest that it has been challenging to retain the workshop participants, would it be useful to create pathways for the existing editing community to become more aware of the issues around performing arts data?”
- We are already in touch with many Wikimedia affiliates: Wikimedia Sweden, Wikimedia UK, Wikimedia Belgium, Wikimedia Malta and, of course, Wikimedia Canada. Our current Wikidata experts who assist us with workshops, complex queries and batch uploads – Simon Villeneuve and Dirac – are from the wider Wikimedia movement and they were recommended by Wikimedia Canada.
- We have been increasingly using talk pages to facilitate and to document discussions (see this talk page for Q105815710. We have also been using the project chat and the French bistro to notify the community and to seek input. We will continue to do so in the next project.
- We welcome suggestions from advisors on means to communicate with the wider Wikimedia community.
Rationale for the proposed project
- “The blocking problem is not clear. This year the project is based mainly on the uploading of items.”
- “It is unclear how this will be implemented to the extent that was proposed. If this can be clarified, it will be helpful.”
The project is indeed based mainly on uploading of items, because one of our blocking problems is precisely the absence of a critical mass of person items. In order to populate information about performing arts works and productions, we first need a comprehensive base of person and organization items that serve as values in role statements: choreographer, stage director, scenographer, costume designer, etc. At present, in spite of population efforts undertaken in the last year, the data is still not complete enough to populate production items without having to constantly create person and organization items.
In order to reach this critical mass of items, we will reach out to unions and associations who maintain large catalogs about artists and creators. We will also reach out to associations who maintain catalogs about organizations. Depending on the quality of the information, the frequency at which it is updated, the existence or not of a web directory (with or without persistent URIs, (with or without structured data), the existence or not of an API to query directly the catalog, we will chose one or several of the following population strategies:
- One-time batch upload;
- Automated query and upload;
- Data clinic (information populated manually by users);
- Clean-up and enrichment workshops (after an upload);
Each upload or clinic will require considerable outreach efforts. We will be reaching out to leaders of unions and associations who may not be familiar at all with Wikidata and/or who may not understand the benefit of having their members’ information present in Wikidata. Conseil québécois du théâtre will be taking the lead with outreach in the province of Quebec and with French-speaking associations outside of Quebec. Mass Culture will be taking the lead on outreach with the rest of Canada (note: the partnership with Mass Culture was not yet agreed upon at the time of submitting the project grant). We believe Mass Culture will be an outstanding partner. This organization has a mandate focused on research and their constituency is made up of arts service organizations and associations - precisely the kind of organizations likely to hold datasets about persons and organizations.
The technical assessment of datasets for suitability for upload will be done by our Wikidata experts and by Culture Creates, with support from the project director (myself). One-time uploads will be performed by our Wikidata experts. Automated query and upload processes would be put in place by Culture Creates. Fjjulien (talk) 14:24, 29 March 2021 (UTC)
- “The budget is realistic but not sufficiently detailed.”
- “A question about the budget - is the associated costs only for the person-hours to achieve those activities outlined in the budget or do they cover other expenses?”
- “It is unclear if the other funding sources are proposed or already awarded. I would request this be clarified, along with more specific explanations for what the participants will do over the proposed time period.”
- “the budget has to provide more details about the activities to be funded.”
Based on our initial project, we assessed the costs of batch uploads thusly:
- Analysis and extraction of dataset: $1,300
- Data cleaning: $1 per item
- Mapping and synchronizing of the dataset with Wikidata: $800
- Upload and post-upload quality control: $600
Based on this formula, the cost for uploading the entire directory of Union des artistes members (more than 12,000) would be around $15,000. This particular dataset would therefore represent the bulk of our costs, but it would bring us to 90%+ completion among French-speaking actors, singers and other performers (other than musicians).
The costs for automated extraction and upload would be greater. But these additional costs would be entirely paid for by CAPACOA, thanks to our funding from the Government of Canada and the Canada Council for the Arts. These additional costs were not taken into account into the project grant’s budget.
The costs for the data clinics were evaluated thusly:
- Design of presentation decks with step-by-step instructions and screenshots, in English and French, for persons and for organizations: $2,800 per presentation deck and translation x 2 = $5,200
- Coordination and delivery of clinics: $2,400 per clinic x 8 clinics = $19,200
- 15% project management: $3,720
The uploads and the clinics are two complementary strategies. It is possible that certain associations may prefer to do a batch upload rather than organizing a clinic – or that they might want to do both: an upload followed by a post-upload clinic focused on data enrichment. But we estimated the global effort under these two activity streams at $56,300. In both cases, the costs are made up of person-hours.
The costs for community organizing activities are also made up of person-hours.
The costs for infoboxes activities are also made up of person-hours. They include:
- Outreach to Wikimedia national chapters: $2,400
- Update of infobox templates to reflect latest modelling practices: $600
- Organizing and delivering events to promote the use of infoboxes: $1,800 for one event (most events are expected to take place in conjunction with WikiClub gatherings)
- Documentation on Wikiprojects: $1,000
- 15% project management: $1,170
The communication and outreach efforts are likely grossly underestimated. For a single association, outreach efforts could easily require 10 hours by CQT or Mass Culture and as many by LaCogency. CQT is ready to offer most of its outreach efforts as an in-kind contribution.
The project coordination efforts are also person-hours and they will be assumed by CAPACOA. They were estimated at 10% of project costs, which is very little considering the complexities associated with the iterative approach. Procurement of services will have to be negotiated for each upload. This will require a lot of coordination and financial oversight.
Funding from the Government of Canada is already confirmed for the fiscal year starting April 1st and for the next. Multi-year funding from the Canada Council for the Arts is still pending, but we have no reason to believe that we wouldn’t want to continue to support these activities.
The funding from the Wikimedia Foundation is an important part of our revenues, and it also has a strategic and a symbolic value. It is strategic in that our public funders require us to find non-governmental funding sources as matching funds. It is symbolic in that it may be a determining factor in building trust with associations who may have little knowledge of Wikidata, but who would recognize and trust the brand of the Foundation.
Post-deadline edit: If advisors deem that the risk level is too high at CAD $40,000 or if funding is too limited to fund both the Performing Arts Aoteara project and this project, please note that we have a capacity to move forward even if the WMF awarded less than the full CAD $40,000. We can revise the budget and the requested amount if needed. Fjjulien (talk) 22:58, 29 March 2021 (UTC)
Impact and scalability
- “The potential impact is probably modest but risks are likely low taking into account the results of the previous grant.”
- “Not relevant impact except the creation of models in Wikidata. This is a second step while in my opinion the biggest one (training) has been already released.”
- “The project fits with Wikimedia's strategic priorities through its sustainability and scalability are less clear.”
As we expressed in previous responses, there are many other activities led by project partners that were deliberately not included in this project grant. Yet each of these various pieces in our linked open data ecosystem is interdependent with activities in this project grant (and with the Performing Arts Aotearoa - Wiki Project, for that matter). Multi-prong strategies are needed to effect lasting changes.
We’re already seeing spillover effects from the first project:
- The Performing Arts Aoteara group wants to undertake similar work.
- Dance+Words, a Canadian initiative dedicated to enhancing dance-related articles in Wikipedia now plans to integrate Wikidata in their activities.
- In addition to French, the WikiProject Performing arts is now being translated in Turkish, Polish, and Bahasa Indonesia.
- Associations in other sub-sectors of the arts are undertaking equivalent activities – in Québec, Productions Rhizome recently received a provincial grant to enhance the presence of writers and their works across Wikimedia sister projects.
- Discussions are being held with Statistics Canada and other stakeholders about uploading the Open Database of Cultural and Art Facilities to Wikidata.
We feel we’re on the verge of reaching tipping points, both in terms of the amount of information (and, with it, the possibility of serving appealing data consumption use cases) and in terms of engagement. It would be unfortunate if we stopped before the performing arts community truly makes Wikidata its own and long-term sustainability is achieved. We are committed to continue the work undertaken during the first project, with or without Wikimedia Foundation support. Whether or not we can scale it and how far we can scale it will however depend on there being Wikimedia Foundation support.