Grants:Knowledge Sharing/Reports/Feedback Report: Wikimedia Community Fund Round 1
Why this report?
This Executive Summary is also available as a Google Doc.
The new Wikimedia Foundation Funds Strategy launched in July 2021 had built into its design a continuous process of learning and iteration aligned with the principles of the movement strategy. Following the first round of Wikimedia Community Fund application review, the Community Resources team went through a feedback process of both the seven newly established Regional Fund Committees and the applicants in all regions. This executive summary of the information captured in the feedback process carried out with Regional Funds Committees and applicants, following the first round of funding within the new Wikimedia Funds strategy launched in July 2021.
The Community Resources theory of change is the basis for the learning and evaluation framework. The framework established the changes we expect with the new funds' strategy and, based on this, defines metrics to measure if we are moving in the right direction or not, to bring about this change. The aim was to capture and learn from meaningful data, emphasising the importance of qualitative data that will help give a deeper understanding of the results and adjustments that should be made. For more information about this, please review this section of the report.
We created a six-step feedback process that started with a survey for all regional fund committee members and applicants. The response rate was 37(71%) for committee members and 28 (48%) for applicants. This was followed by semi-structured interviews with 14 (25%) committee members and 12 (20%) applicants from across all regions. Step 3 was holding a cross-regional reflection and learning session with committee members (70% participated). Based on the findings we shared a draft report for feedback and prioritised adjustments for the next round of funding which begins in March 2022 and those that are proritised for future rounds. Some of these adjustments, such as the application form, were reviewed asynchronously by a focus group of committee members and round 1 applicants.
The spirit of this report: The funds' strategy is based on a notion of thought partnership and trust between applicants, regional fund committees, and the Foundation. This report is not about evaluating each other - but building dialogue, empathy, and common understanding to move forward in the direction we all want.
The main audiences for this report are:
- Regional Fund Committee members and Wikimedia Funds applicants to provide an overview of what was learned from the round, their partner's perspectives, and recommended adjustments and mindset shifts.
- Wider Wikimedia community: to see how the Wikimedia Funds strategy is moving in alignment with Movement Strategy and in response to the grant relaunch consultations that aimed to create greater equity and decentralization in resource allocation and support to reduce barriers to access funds.
- Foundation Staff: to share knowledge about the learning and evaluation process the Community Resources team is developing and to coordinate adjustments and support efforts for community members.
The full report is divided into four parts. 1. General data on the first round of funding, 2. The feedback process methodology and response rate, 3. Regional Committee feedback with conclusions per region as well as general recommendations. 4. Applicants' feedback.
In this executive summary, we have tried to capture the main conclusions and recommendations of the report.
Overall conclusions about the new process and model given the feedback
- In round 1 funds were distributed in line with initial goals towards more equitable distribution, although there is still a way to go.
- With only Round 1 we have already distributed more funds than the entire previous fiscal year 20/21.
- First time able to have aggregated information to understand what applicants are working on and the programs they are requesting to fund.
New Process and model
- Overall people feel this is the right direction, even if some changes are needed in the process and tools.
- Fostering a partnership mindset between the Foundation-Committees-Applicants and better alignment is critical to the success of the new strategy moving forward.
- Empathy with clear and open communication, as well as, improved processes and tools will be fundamental to developing partnership and alignment.
- Regional Fund Committees and Applicants have ideas on how to work on improvements and are open to do it together.
- There are regional differences, in both Committees’ and applicants’ perceptions with MEA and LAC regions tending to evaluate processes more positively.
- Overall Regional Committee members felt autonomous, empowered, satisfied with their role, and worked well as teams.
- The support and role of the Community Resources team were appreciated and feedback was given on how this can be expanded.
Funding analysis for the first round of the Community Fund
- This analysis is for the Community Fund (rapid funds, general support, and conference). It does not include the Alliances Fund and Research Fund.
- In round 1 funds were distributed in line with initial goals towards more equitable distribution, although there is still a way to go. It is positive to see an increase in some regions that have been less represented, such as MEA and CEE.
- Regional funds allocations allow for a growth analysis and yearly estimates per region based on capacity and opportunities. It does not promote a logic of competing for funds between regions or types of affiliates. There was growth in the amount of funds for all regions, except for SAARC and ESEAP where we expect to see greater representation in round 2 starting in April 2022.
- With only Round 1 we have already distributed more funds than the entire previous fiscal year 20/21. In just around 1 half the fiscal year) it was $7.3 million, with an additional of 4.7 for multi-year funds covering 2023 and 2024. The expected overall increase in funding in 2022 is 82%. Round 2 expected more applicants from all regions, particularly SAARC and ESEAP and Conference Grants.
- 92% of funds were invested in General Support proposals, averaging approximately $120.000. This average varies widely amongst regions, with a higher average in North-Western Europe (NWE) and the United States and Canada (USCA), and lower in ESEAP and SAARC region. This is largely due to affiliate size and grant experience. However, it is interesting to note these regional differences have decreased, with more individuals and different types of affiliates accessing larger funds through the new funding system. 49 countries were funded, 13 with multi-year funds in all regions, except for SAARC and ESEAP in this first round.
- 7% of funds went to Rapid funds averaging $2.000 USD per fund. The average amount has not changed, despite the fact that the new strategy increased the limit to $5.000 USD. In just this round we have reached 49 countries. All of 2021 was 54.
- 80% of rapid funds went to the MEA region, with the majority being concentrated in Nigeria. This has led to a participatory process to reflect on how to promote greater coordination and efficiency to transition from multiple rapid funds to joint general support annual funds.
- For the first time, our new Funds Portal and application form enables us to aggregate information to understand some trends in grantees' intentions and proposed work. For instance, 61% of applicants say they intend to contribute towards the Skills Development goals of movement strategy.The top three knowledge content gaps that grantee partners want to address are Gender, Geographic, and Linguistic and the gaps least addressed are related to socioeconomic status. More than half of grantees intend to work in education, culture and GLAM, and diversity, but less than 30% in areas around climate change, advocacy, and human rights. More analysis around these trends will come in the following reports, as well as analysis around the results and impact of their work.
Regional Fund Committees perspectives and recommendations
Foundation: orientation & support:
- Average rating 4.1 (on from 1 to 5 scale): Highest SAARC and LAC, lowest NWE and USCA.
- Whilst there are regional differences, over 80% of committee members felt that the 8 sessions for the learning and orientation program were useful to understand grantmaking, regional challenges, and thinking about diversity and committee culture. However, they would have liked to receive more practical exercises reviewing proposals, have technical guidelines (benchmarks), and more clarity on the time dedication the role implies.
- They appreciated Program Officer support, facilitation, and translation services.
- Committees feel they need to build more capacities to analyse budgets +staff analysis, growth paths/ sustainability, exploring new programmatic approaches. They feel this could best be done with a combination of formal training and peer support, and with some specialised roles within the committee in different areas of expertise. More cross-regional connections between committees were valued.
- Whilst 60% were satisfied with the documentation provided by Community Resources, Committees requested improving staff review and analysis and to provide more examples and good practices in guidelines as well as clarity on behalf of the Foundation around some legal/political issues that committees do not determine.
Perception of autonomy and empowerment and committee culture
- Average rating from 4.2 (Scale from 1 to 5) Highest LAC, lowest MEA, CEE, and ESEAP.
- Despite regional differences, overall 81% felt they had the necessary autonomy and that decisions were made by the committee and with consensus. They highlighted the respect amongst peers, allowing diverse voices to be heard in a safe environment. Deliberations allowed for extensive dialogue and analysing with different perspectives.
- Committee members would like to collaboratively build tools and have more input into application forms, review guidelines, and funding round timelines.
- 86 % felt they worked well as a team (Highest positive evaluation), however, some committees felt they still needed more time to get to know each other, work on trust amongst peers and improve their internal dynamics. Committees with a good gender balance valued this, and those that don’t are concerned about seeking a great representation of women and cultural/ethnic minorities as well as combining different levels of expertise and experience in the movement.
Satisfaction about the process:
- Average rating 4.0 (Scale from 1 to 5): Highest LAC, lowest NWE, and USCA.
- Most committees felt that the procedures and tools worked well. Particularly the scaled process of individual, peer, to general deliberations. They valued Program Officer support in documentation, deliberation facilitation, consolidating proposal feedback, and translation services.
- The short time frame and overlap with holidays caused dissatisfaction and the Committees’ hope for collective agreements in future rounds. Members want more time for deliberations and feedback processes.
- As with applicants, there were several observations with the application's structure and several adjustments proposed in terms of character limits and additional questions to have a better understanding of the proposed work and differentiated questions depending on affiliate experience and size.
- As with applicants, committees would also like more time for live interaction with applicants and direct feedback and more time to become familiar with their work between rounds.
- Committees want to work with the Foundation to have a strategic picture of funding over the next few years and alignment around multiyear funds.
- There was a general feeling of the need for more alignment between committees, the Foundation, and applications around review expectations and mind-set, and this could be done through improving guidelines, processes, and general communications. This also involves working on a mind-set shift to be more empathetic, supportive of applicants throughout the round, and prioritise the proposal over personal biases or opinions.
- Whilst financial support in the form of stipends was important, it is necessary to continue to support members with language or connectivity issues which are still a barrier to full participation.
Satisfaction with the regional model and their role:
- Average rating 4.3 (Scale from 1 to 5): Highest CEE, NWE, and MEA, lowest USCA, and ESEAP.
- Some of the positives highlighted were a better understanding of the nuances in each context and a vision that is close to the ground. The autonomy of committees was also valued. Some members were concerned about finding ways to also keep global perspectives and avoid “regionalism”, such as having more cross-regional knowledge sharing and even cross-regional committee sessions for global/thematic proposals.
- A few members questioned if there was a need to go further in decentralizing power, giving committees more say about regional budget allocation and general funding criteria and that there should be spaces for collectively building strategic and long-term thinking around grants.
- Committees felt the need to have a greater understanding and shared vision of different organisation’s growth paths within the movement and more partnership with grantees to support learning during implementation and evaluation.
- 61% of members feel very satisfied with their role, and 34% moderately satisfied, only 5 expressed not being satisfied. Reasons for satisfaction included learning skills, being able to make grant decisions without the top-down approach, working as a team, and being able to make decisions that impact their local communities. Some members expressed that they would be more satisfied with additional training, adjustments in the procedures, and better time management.
- 83% feel that the time commitment fits in with their schedule, although more than half think it was more than they initially expected.
General views on applicants’ proposals:
- Committee members recognised the immense work and effort put into proposals and the variation in organizational size and experience.
- In future rounds, committees feel they should work on offering more support to newcomers, enable more interaction with applicants throughout the application and grant cycle, and promote the active collaboration with affiliates to develop and share best practices (ie. budgets, programs, evaluation).
- Committee members felt that many applicants needed more support to write clearer proposals. We are all working on a trust-based model, there should be technical/financial clarity to some degree. Two areas that need a lot of improvement are the budget and evaluation sections. Applicants need to be able to justify investments, particularly when they are high values, and show the relationship between budget and expected outcomes (results). Greater clarity is needed about how they hope to learn from their work and measure the impact beyond numbers. More qualitative metrics and methods should be encouraged to measure and learn from results. Quantitative metrics such as content and contribution should have clearer benchmarks and be based on learning from past work (where relevant).
- Committee members also felt that innovation should be encouraged through the application form, and also by referencing some interesting and good practices. They recognised their own need to be more open to new ideas and experiments when evaluating proposals and to support applicants to work through new ideas and experiments.
Applicants perspectives and recommendations
Foundation: orientation & support:
- Average rating 4.0 (Scale from 1 to 5): Highest: LAC, MEA, and SAARC. Lowest: NWE.
- Applicants found 1:1 conversations with Program Officers the most useful, as they gave room for in-depth-contextualised support, as well as being more “human” and “binding”.
- Conversation hours or group workshops are better for general orientation and should include interactive exercises. They should be organized in groups for different levels of proposal development and recorded for asynchronous orientation. Direct correspondence over email to answer questions or give guidance was also valued.
- Dedicated Regional Program Officers were seen as having a positive impact, particularly in regions of LAC, MEA & SAARC (57% stated they were very satisfied with having regional PO’s closer to their context and would like to see stability in these officers). In some regions, more Program Officer participation in the early stages of planning the proposal would be valued.
- 75% say that direct communication from Program Officers is the main source for receiving information). Other forms of communication (Meta, community members) are only more common in USCA/NWE regions.
- Applicants recognised the importance of translating all application forms and guides, as well as holding conversations in regional languages where possible.
- However, perceptions across regions around tools and services to support applicants varied. Generally with higher levels of satisfaction in LAC and MEA, with other regions expressing less satisfaction or knowledge of these (NWE/USCA/SAARC). This shows the need for better communication around these tools and more periodic 1:1 conversations early on in the grant cycle whilst applicants are developing their initial drafts. Translation support for applicants writing their proposals in other languages should be offered (this was highlighted for Korean and Hebrew) and there should be more active communication around translated guides and forms.
- Whilst the application guide and template were useful, these can be improved with the necessary adjustments in the form and more examples of different types of projects and good practices. Video tutorials were also considered useful, but they need to be sent to applicants in time and directly (50% did not know about these).
- 95% said they did not receive support from other Foundation teams, but many considered this important.
- Peer-to-peer support during proposal writing was also considered important and the Community Resources team should support more spaces for this. This can be done within regions but also with cross-regional exchanges around projects working with similar themes or approaches. Key areas to work on are developing theories of change, defining metrics and evaluation plans, more clarity on budgets, and explaining the relationship between investments and the desired outcomes (results).
- More applicants in LAC + CEE think the new strategy is most definitely enabling a closer, partner, and trust-based relationship with the Community Resources team. However, some applicants in NWE feel there has not been a substantive change and only two applicants said they felt it had “gone backward” (LAC/NWE).
Satisfaction about the process:
- They would like to see more timely communication around the round and each step. In future rounds, there should be more synchronization between the timing of the new process and the timing of chapters preparing their annual plan and presenting it to the board, partners, etc. They requested more time for proposal writing (more than a month) and at least 3 weeks for feedback and adjustments. As with Regional Fund Committees in NWE, SAARC, and ESEAP regions, there was discomfort over the overlap with holidays.
- Applicants would like to have more information on Regional Fund Committees, their review framework, and expectations.
- As with Regional Funds Committees, there was a feeling that more alignment was needed between what Program Officers were recommending and Committees’ feedback, for example, on the detail of information needed around multi-year proposals.
- Several applicants support the change to using the Fluxx portal, even if it does not necessarily reduce time barriers. Some found the portal easier than using Meta (mostly newer grantees) and appreciated the technical support provided whilst applying.
- For some, the application form felt easier and the Fluxx translation to a meta page was a good time-saving factor. Some applicants asked for greater clarity on how the Fluxx - Meta interface works and what information will be published.
- 54% said the portal did manage to reduce tech and language barriers - this was higher in the MEA and LAC region. The ability to apply in any language was also valued.
- The application form in itself was a great point of discussion. There were some regional variations in this analysis: More applicants in LAC and MEA felt the objectives were met (over 50%), whilst less so in NWE, CEE, and USCA (less than 20%). On average, only 30% felt that it met 3 of the key objectives of: a. having common categories for analysis and aggregate information, b. Capturing the change the organizations wants to bring about with their work and their overall strategy and c. reduce time. Many felt it added more work and that character restrictions and questions structures were limiting. However, a majority felt that the objectives could be reached with minor adjustments. A summary of these proposed adjustments and those that were made can be found in the Next Steps section.
- Learning and evaluation is an area that requires more support and capacity-building. Whilst many applicants felt comfortable with this section in the form, many expressed the need for more clarity in the guidelines about the flexibility to define metrics that are relevant for each proposal, and really emphasise qualitative metrics for deeper learning from results. In this sense, some felt that it was better not to recommend additional metrics in the form as applicants may feel restricted by these. Applicants asked for more support to develop evaluation plans based on theories of change, with good examples of evaluation for different types of projects and 1:1 support during implementation. In general, more formal training in this area and investment in expertise is needed within proposals. Some applicants highlighted that tools for measuring quantitative metrics are still deficient (despite community efforts made to improve tools such as the dashboard, among others) and that there needs to be a Foundation and Movement-wide plan to address this. Finally, some applicants (particularly in the LAC region) proposed regional sessions to think of metrics that make sense for regional strategies/changes.
- There were no structured conversations with committee members during application writing, those that happened were more ad hoc. Committee members could organise or participate in spaces for proactive support during this phase, as well as more live interaction during the feedback process.
- 50% of applicants felt that they did exclude some aspects in their proposals because they felt it would not be approved given funding restrictions or because it may seem too ambitious or innovative. Things applicants “felt were left out” varied across regions, for instance, more ambitious investments for volunteer incentives beyond the common forms of support in transport, food, etc. (MEA), regional leadership exchange programs (LAC), more ambitious plans to connect with government educational policies (SAARC), as well as more general things like more staff, applying for multi-year when it was the first annual fund, international travel costs and costs of carrying out activities in remote areas.
- It is interesting to note a perception in NWE and CEE about funding restrictions as a major limitation, given that there was growth in funding for all regions. It evidences the need for clearer communication between all partners to mitigate misperceptions about restrictions within the new strategy. This could also be helped with discussions about long-term funding strategies and perspectives that are of interest to committee members and applicants alike.
General views on Regional Funds Committee feedback and funding decision:
- Average rating 3.6 (Scale from 1 to 5): Highest: MEA and CEE. Lowest: SAARC, LAC, NWE.
- Applicants recognised the immense amount of work and commitment Committees put into evaluating proposals.
- Some applicants stated that the feedback was valuable for offering them new perspectives to improve proposals, particularly very specific reflections on staffing conditions, metric evaluation, etc. Where there were live interactions, they were valued as they allowed for clearer feedback and productive discussions and adjustments.
- However, many felt that the feedback could be improved. It should go beyond asking questions or clarifications and should be more of a strategic review with analysis that can bring new perspectives and help applicants better develop sections of their proposal. It should be communicated in a way that is constructive and not based on value-judgments or individual interests, but on making clear and specific suggestions of how the proposal can be improved or clarified, using examples if possible.
- As with Regional Committee members, applicants also recognised a misalignment between what Committees were requiring and what was asked in the form or recommended by Program Officers. They also hope to see common understanding within and across committees about these guidelines and criteria to make evaluations seem less arbitrary or personalized.
- Applicants also expect committee members to do more research to understand their past work, with the help of Program Officers. Also, to have a better understanding of specific organizational aspects, such as how civil society organizations work and governance issues (this was particularly highlighted in the LAC region). Also, applicants highlighted the need for committees to be more familiar with who their work is responding to Movement Strategy recommendations and piloting some approaches in alignment with this.
- They would also like to see Committee members develop more knowledge around specific issues: such as programmatic approaches, budgets, evaluation.
- They would also like to see greater diversity in committees with a better balance with more experienced members in chapter/user groups but also external members and more openness to newcomers, new ideas, and innovation.
- 50% of applicants fully agreed with the funding decision, 35% somewhat agreed, and 15% did not. Those that did agree with the funding decision were more inclined to have found committee feedback and interaction productive. Those that somewhat agreed or did not agree, highlighted many of the ways committee capacities, interactions, and feedback can improve, as mentioned in the above points.
Satisfaction with the regional model:
- 31% of applicants think that the regional model definitely contributes to descentralised decision-making and 61% think that it does but needs some changes, and 7% (2 people) said that it does not contribute at all. There is some regional variation with more applicants in MEA and CEE regions thinking it definitely contributes.
- Applicants would like to see some of the changes previously mentioned, particularly about training and capacity building for committees, more aligned review criteria that are shared with applicants, greater partnership with regional committees, and expanded support from the Community Resources team and other foundation teams.
- A few applicants stated that the criteria and procedures for selecting committee members should be reviewed and in the future members could be elected by community members and include current board members. It is important to note that board members are currently not excluded to participate. Committee members have the agency to propose new members and board members can be proposed. Applicants also valued including more experienced members leading affiliates or Wikimedia projects, as well as newcomers that can offer different perspectives.
- In the long term, a few applicants would like to see how the work of the regional fund committees will be integrated into developments around the Global Council and Regional Hubs. This could also mean looking at how committee members could be elected by communities and membership criteria.
- A few applicants mentioned that they would expect more alignment with the principle of “subsidiarity” and that it still feels a bit top-down or Fondation led. There were also some questions about regional definitions ie. Brazil in LAC.
- They would also like to see more data to understand how resources are being allocated by language, region, project, etc.
Next steps and adjustment plan:
We are on a learning journey together. We want to thank everyone who contributed to this report. Based on this report the Community Resources team is working to implement immediate adjustments and a road map for longer-term adjustments.
Below is a summary of some of the adjustments already underway for round 2 which we will continue to test, iterate and adapt for the next fiscal year. Not all changes could be made before round 2 given the short timeframe, considerations for fairness and equity between round 1 and round 2 applicants, as well as adjustments that need more time and partnership to develop between all parties involved.
- The application form: see the annexed table below for more details on what was done and what will be considered for future rounds and why. We know that these were very close to round 2 and this may have caused some disruption for applicants working on their draft, but we hope that the changes and guidance will have an overall positive impact on their work and we greatly appreciate their support and adaptability.
- Additional guidelines, tools, and spaces to support applicants:
- Adjusted application form with quick tips and guide with some examples and links to round 1 proposal. Clearer guidance on expectations for multi-year proposals.
- More structured 1:1 conversations with applicants as they are drafting their proposals, focused on supporting thinking about the overall change (results) and effective strategies (theories of change), learning and evaluation, and budgets.
- Asynchronous proposal review and feedback from Program Officers, where requested.
- Let’s Connect Peer learning sessions for proposal writing and follow-up peer support where possible.
- Committee review framework public on Meta.
- More active communication of tools for applicants and FAQ document on meta to guide them.
- Automated Fluxx-Meta process and clarity on what documents will be published.
- Improvements in tools and procedures for Regional Fund Committee:
- Improved staff analysis and review framework to be tested by Committees in Round 2. Clearer guidance on expectations for multi-year proposals.
- Currently working on documenting all procedures and templates and agreeing on these with Regional Funds Committees as they move into round 2.
- Optional pre-review preparation session to carry out test reviews to align expectations and resolve any questions.
- Guidelines on cultivating a review mindset that builds trust and welcomes new ideas (this includes aspects such as empathy, communication, constructive feedback, being awareness of bias, openness to innovation, etc).
- Fostering alignment and greater partnership between all partners involved:
- All of the above adjustments to procedures, templates, and guidelines are focused on creating better alignment and thought partnership between Community Resources-applicants-Regional Funds Committees.
- Safe space expectations:
- Guidance for Program Officers to better monitor this and orientate applicants, particularly new applicants, around talk-page expectations and how to seek support if incidences occur.
Some adjustments that we will start to work on for next fiscal year
- Improved guidelines with more examples and in different formats (ie. video tutorials)
- More peer-learning spaces.
- More benchmarks on HR and finance.
- More time and training reviewing proposals
- Analysis of reporting to learn from results as a basis for future decisions
- Analysis of global challenges in each region
- More formal training opportunities for Regional Fund Committee members and applicants
- More cross-team collaboration within the Foundation to support Regional Fund Committees and applicants and build capacities.
Annex 1: Summary of application form adjustments proposed and implemented.
|Recommended changes (both applicants and regional committee members)||Changes made for round 2|
||1. Done: adjustments made to the review framework and form. We are working with committees to establish a shared understanding of the depth of review and guidelines on feedback.
2 &3. Done: Character limits for many open questions increased. Three questions were included to provide a timeline, risk mitigation plans, and to describe general plans for multi-year funding.
4. Done: The first three questions in the proposal section were reworded so applicants can better describe their vision and strategy.
5. Partial: We have included some questions that will be more relevant to larger affiliates and are optional for smaller ones. We will continue to evaluate this need for different forms after the adjusted application form has been tested. Creating multiple application forms would not be ideal.
6. Done: Reduced to 24 questions.
7. Done:The learning and evaluation section was reorganised so that it is clear to applicants that they can openly define qualitative and quantitative metrics that really help them learn from their work and show their overall results (outcomes).
8. No: We have not created these templates yet, but will evaluate this need for next fiscal year.
9. Done: Clearer budget questions and template. For future rounds we can include examples. The budget section was adjusted so that applicants can simply upload their budget using the template provided or their own format.
10. Partial: We received feedback from a focus group of round 1 applicants and committee members, however, we will continue testing throughout round 2.
11. Done: We now have an automated tool and an open procedure of how and what is public on Meta.
- East South East Asia Pacific → ESEAP
- South Asia Association for Regional Cooperation → SAARC
- Latin America and the Caribbean → LAC
- USA/Canada → US/CA
- Central Eastern Europe/Central Asia → CEE/CA
- North West Europe → NWE
- Program Officer → PO
- Community Resources → CR
You can review the full report in English, French, and Spanish: