Grants:APG/Proposals/2016-2017 round 1/Amical Wikimedia/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$75,796	$54,568	-27.94%
Budget	$83,041	$75,239	-9.4%
Staff (FTE)	1	1	0%

Overview of strengths and concerns

This section summarizes the strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths

• WMCAT’s volunteer-focused approach includes leadership development.
• WMCAT has a scalable approach to GLAM partnerships.
• WMCAT’s effective use of funding consistently leads to results

Concerns

• WMCAT’s plans lack relevant targets, year over year.
• WMCAT’s strategy is underdeveloped, for an organization receiving FDC funding.
• WMCAT’s plan does not have a strong focus on gender diversity.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

• Amical’s community of contributors is highly motivated build more resources in the Catalan language. Catalan Wikipedia, with more than 500,000 articles, is unusually active for a language with nine million speakers.
• Amical Wikimedia has a long-standing good relationship with its core community, which enables effective coordination among association-supported activities and community-led activities. Many Catalan contributors are involved with the association, and Amical assesses this when measuring their own performance.
• Amical continues to have productive and lasting relationships with institutional partners in GLAM and Education, which is a strong foundation for the continuity and sustainability of its programs. Amical also maintains good relationships with partners in the Wikimedia movement.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

• Amical Wikimedia is a high achieving organization, especially with respect to the amount of funding received in past years. At the same time, Amical does not set targets for its work, which makes it impossible to understand how well Amical is performing against their own goals and expectations. This is an important aspect of understanding an organization’s ability to plan and implement programs.
• In terms of past performance, Amical’s greatest achievement is their success in connecting with networks of institutional partners to create self-sustaining partnerships, a strategy that maximizes the scale of their work. They have successfully shifted their focus from library networks to education networks, where they are applying this same principle.
• Amical’s education program has been implemented in 55 classrooms as of September 2016, and in partnership with 14 of the 21 Catalan universities. 200 of 370 public libraries in the network of Catalan libraries are participating in their Bibliowikis project, which trains librarians to facilitate access to Wikipedia and organize editathons and events. After Amical’s interventions, these partners are self-organizing events.
• Amical’s midpoint achievement of 12,857 articles in 2016 puts them on track to meet their 2015 achievement of 25,113 articles; on the other hand, Amical may not be on track to exceed their 2015 achievement of 6,642 images in use, having achieved 1,669 images in use by the middle of 2016.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

• Amical leverages their single staff person by making their activities self-sustaining, and this is possible because this staff person has strong ties with and credibility within their communities, and because Amical has a community of core volunteers that they actively cultivate as leaders. Amical’s approach to leadership development is explicit in their plan, and is included in the design of every program.
• Amical has a working board that is involved in day to day activities, rather than a governance board focused solely on the big picture. This may explain Amical’s focus on implementation rather than strategy, which makes sense for an organization of their size. Amical does not have any female board members.
• Amical does not employ the services of a paid accountant to support its volunteers, which is a recommended best practice for an organization with this size budget. This has been a concern for several years.
• For an organization its size, WMCAT has been active in establishing good practices from their experience, notably on outreach wiki, documenting strengths and weaknesses in progress reports and showing how they can adapt their programs to integrate lessons learned.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

• Amical has a strategic plan in place covering 2014-2018, with five priorities including, cohesion (refers to volunteer engagement and community health), discourse, content, territory, readers. This strategic plan includes some long term projects.
• Amical’s strategic plan is minimal among organizations in the FDC process. It does not include an assessment of risks and opportunities, or provide a long term plan for the resources they will need. There is also no mechanism in place for assessing progress against or revising this plan, although a five-year strategy period is long for an organization of Amical’s size.
• Some programs in the current annual plan are well-aligned thematically with the five strategic areas outlined, especially for the priorities of content and cohesion; however, it is not clear how this annual plan puts Amical on a clear path toward achieving their strategic vision or long term targets. For example, views were not tracked in the 2015 impact report, the 2016 progress report, or anywhere in this proposal although number of views are a long term target. Catalan Wikipedia is thriving at more than 500,000 articles, but it is unclear how Amical’s programs will lead to 750,000 articles by 2018.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

• We appreciate that Amical has included thoughtful metrics for their programs, but a lack of targets year over year makes it impossible to assess their potential impact or their ability to plan and execute programs, beyond evidence of past success.
• Amical has selected two grantee-defined metrics that seem integral to their organization’s DNA: members engagement and percentage of repetition. These metrics may allow Amical to understand their progress in their work with contributors as well as the self-sustaining nature of their partnerships. However, these metrics are not currently well-defined enough to give a clear idea of what will be measured and how. For example, there is no time period offered for the repetition metric.

• Amical’s program plan builds on their expertise in several areas. They are continuing their education work, with a focus on leveraging educator expertise to fill content gaps. Reflecting their community-based approach, they use a personalized mentorship model to motivate active contributors as well as new student contributors. Amical Wikimedia does not include SMART objectives for this program or targets related to outcomes, although they include a target of 75 classrooms.
• Amical is continuing their focus on libraries, but reducing the amount of staff time committed to this area. Through Amical’s “kick off and follow system”, staff is able to help volunteers jump start activities that are continued by partners. Beyond creating self-sustaining projects, volunteers are also empowered through this approach.
• Amical’s community program focuses on writing contests, where they have sound expertise in organizing innovative online events, such as Ephemerides, a writing challenge designed to fill content gaps that match a calendar (for example Roald Dahl related articles on the anniversary of his birth). Another interesting community-led initiative is the WikiArs project (collaboration with design students), where Amical will continue to offer low key support.
• Amical’s community program includes training for volunteers to build skills in specific areas. It is unfortunate that Amical only targets outputs (such as number of workshops), as it would be interesting to have a clear idea of how Amical expects volunteers to benefit from this important work.
• Amical has been supporting a few projects related to bridging the gender gap, but does not engage in pro-active gender-related work.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

• Amical’s budget and grant request is decreasing in the current year. Spending and income patterns are not consistent from year to year, although this has sometimes been due to unexpected revenue.
• Amical’s $75,000 budget is consistently focused on impact. Staff is devoted solely to programs and operating expenses are kept to minimal necessary levels. Non-staff expenses are divided relatively equitably among programs, with slightly more funding going to community support. Community support may not be Amical’s most impactful program in terms of quantitative results, but community support work is clearly an enabler for success in its other two programs.
• Amical’s staffing and funding needs do not require that they remain in the FDC process, so Amical could consider other funding options that have less resource-intensive planning and reporting requirements, especially if they continue to want to scale back their budget and staffing needs. The Simple APG funding stream may be a good option for Amical, since basic strategic planning and working boards are common among organizations using this funding stream.
• Amical is building up significant reserves, due to excess income over several years. Their reserves are currently around $45,000, which would cover 10 months of staffing expenses in case of an emergency.
• We did not find evidence that Amical is tracking in-kind donations.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Neither	Amical Wikimedia (WMCAT) has a brief strategic plan in place that contains long term targets, but there is no clear plan in place for tracking progress against these targets. Five years is a long strategic timeline for an organization of Amical's size and dynamism, and there is no mechanism in place for reviewing and adjusting this plan. While program activities are linked to this strategy in spirit, it is unclear how they will result in achieving the long term goals outlined. As an organization with a working board and a small budget, long term strategy may not be Amical's emphasis.
P2. Potential for impact at scale	Major strength	WMCAT's approach to leveraging networks and developing self-sustaining partnerships while empowering volunteer leaders is scalable, and serves multiple organizational goals simultaneously. Their Bibliowiki project involves more than half of all Catalan public libraries for example. This approach is employed in their GLAM and education work, to empower partners to devise their own projects without ongoing intervention from WMCAT staff.
P3. Objectives and evaluation methods	Concern	We appreciate that WMCAT is measuring its results year over year. However, WMCAT's lack of targets has been a concern year over year, and is a barrier to understanding the full depth of WMCAT's past performance and effectiveness.
P4. Diversity	Neither	WMCAT has been partnering with Wikimujeres and integrating more gender approaches into its programs (Viquidones, Viquiprojecte 20x100 Dones in Wikiquote). It is, however, less clear in this year's proposal how they intend to address diversity questions. While they mention diversity as one of their core focus (tackling gender, geographical, linguistic and historical diversity), there are no specific targets associated with this topic.
Organizational effectiveness
O1. Past results	Strength	Relative to their size, WMCAT is a consistently high performer in terms of articles improved, and in terms of qualitative achievements around empowering partners and volunteers at the center of its work.
O2. Learning	Strength	WMCAT documents learning in their reports, highlighting successes and areas for improvement. They act on those learnings to adapt and change their programs according to feedback and experience. They also work to share their learnings in the Education collab to devise best practices with other entities in the movement. Amical's approach to learning is appropriately commensurate with budget size.
O3. Improving movement practices	Strength	WMCAT has a track record of sharing good practices, particularly in the realm of GLAM and Education. WMCAT has developed several international partnerships with other affiliates (Wikimedia Spain, Wikimedia Mexico, Wikimedia France, Wikimujeres, the Basque Wikimedians and Wikimedia UK). Amical's movement contributions are appropriately commensurate with their budget size.
O4. Community engagement	Major strength	WMCAT engages a high ratio of Catalan contributors in its work. Community is core to this organization's DNA and volunteer leadership is a core component of every single program. Amical exceeds expectations in this area, and models good practices for other organizations.
O5. Capacity	Strength	WMCAT has a unique organizational model that builds capacity in its community at large by designing programs that aim to be self-sustaining and promote volunteer leadership. Amical is constantly making efforts to bring in new contributors, volunteers, and supporters, even with limited resources. Amical's sole program staff takes on a lot of responsibility, as does their volunteer board that manages operations. At the same time, staff and board are highly competent in program implementation. With this model, it may be difficult for Amical to meet FDC-level standards in operational and strategic areas and receive the support they need, but it is clear they can achieve results.
Budget
B1. Past budgeting and spending	Strength	Amical appears to use funds effectively year over year, with program results that are commensurate with grant amounts. Amical is not always accurate at forecasting revenue or expenses, which may be more typical for a smaller and more dynamic organization.
B2. Budget is focused on impact	Major strength	The budget is focused on areas that are likely to lead to strong program results, especially investment in Amical's sole program staff, with minimal operating expenses.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 21:34, 8 November 2016 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.