Grants:APG/Proposals/2016-2017 round 1/Wikimedia Nederland/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$378,981	$378,981	0%
Budget	$429,420	$471,720	+9.85%
Staff (FTE)	4	4	0%

Overview of strengths and concerns

This section summarizes the strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths

• Programs have potential to scale beyond the borders of the Netherlands.
• WMNL has a high quality strategy in place, and a track record of strategic planning.
• WMNL is thoughtfully addressing community health and globally relevant content.

Concerns

• Past results, especially quantitative results, are not commensurate with funding.
• WMNL continues its ongoing dependence on APG funding.
• WMNL has had difficulty meeting its fundraising targets.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

• Despite working with a community with a large proportion of active Wikipedians among speakers of their language, the community dynamics on Dutch Wikipedia have been identified (through WMNL’s community surveys) as particularly challenging. This means that community health work may be especially difficult and also especially relevant. We are impressed that WMNL is not shying away from this challenge, but is addressing it directly.
• WMNL is now taking advantage of opportunities in its environment to collaborate with institutions in the Netherlands that hold large amounts of content relevant to emerging communities.
• WMNL is making some progress on non-APG financial resources compared with previous years, projecting $42,000 in 2016 compared with $10,000 in 2015, but they are still heavily reliant on APG funding for an organization with local fundraising opportunities.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

• WMNL reports that in mid-2016, editor decline on the Dutch Wikipedia is stabilizing compared with declines in 2014 and 2015, and that 411 people are participating in WikiSaturdays.
• WMNL is on track to deliver the education toolkit by Q3 of 2016.
• In the first half of 2016, WMNL set up the infrastructure for the Nederland and the World program, including participation from 3 GLAM partners and 2 Wikimedia chapters, as well as 4,586 files donated.
• Despite generating 87,637 image uploads in 2015, use rates for these images were low in 2015; 2016 achievements for images in use are also low at 132.
• Despite well-designed programs and approaches, Wikimedia Nederland’s performance in quantitative metrics has been disappointing, year over year. Despite achieving 3,275 pages (includes uploads to Commons) by the midpoint of 2016, WMNL is targeting 66,800 pages (includes uploads to Commons) in 2017. It is not clear how WMNL will achieve this based on past performance.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

• Wikimedia Nederland is an organization with a thoughtful learning culture, and there is evidence that Wikimedia Nederland applies learning to improve its program design and its approaches. For example, we see Wikimedia Nederland refining the Wikipedian-in-Residence model and the Wiki Loves Monuments model, and documenting learning in important areas like community health.
• We have recognized the thoughtfulness of WMNL’s work over the years, but have expressed concerns that this thoughtfulness sometimes leads to tentative or cautious plans; however, we note a difference in WMNL’s approach this year, when we see that WMNL is boldly tackling challenging and important questions that touch the entire movement.
• WMNL is actively working to build the capacity of the volunteers it works with, in areas like community health and GLAM, which has the potential for long-term capacity benefits. As part of this work they are measuring skill development and attitudinal shift. WMNL recognizes the importance of developing offline contributors and improving the environment in which online contributors operate, to the success of its programs.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

• Wikimedia Nederland has been a consistent performer in strategic design over several years. This year they have unveiled a new strategic plan, which is rooted in reflective analysis of their past performance. This includes detailed cost analysis of different program approaches.
• We see encouraging strategic directions outlined in this bold plan, as well as a clear analysis of risks, opportunities, and priorities. They address risk analysis from the point of view of the organization and their project of focus, Dutch Wikipedia.
• Wikimedia Nederland’s plan is well-aligned with its strategy. In each of the programs included in its annual plan, WMNL includes an analysis of risks and opportunities, which highlights the strategic alignment of these programs. A thoughtful approach shines through even in the implementation of bold initiatives.
• WMNL’s strategy also includes an analysis of the resources needed to complete their plan, including projections of their budget size unto 2020.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

• The program plan is easy to follow and understand, yet includes nuanced explanations and relevant metrics. This program plan is a clear reflection of their high quality strategy. Despite this, it is difficult to understand how projects listed in each program are connected through common objectives or a similar theory of change.
• Wikimedia Nederland’s programs are likely to have impact that can ripple through the movement, given their focus on innovative approaches to community health and GLAM work that has impact extending beyond their borders.
• We appreciate WMNL’s focus on community health, and in particular their attempts to measure aspects of community health including editor retention with a diversity lens. This is a challenging metric, and yet highly relevant to what WMNL would like to achieve. We acknowledge that this challenging area for WMNL may not yield outstanding results immediately, but at the same time is critical to WMNL’s success.
• In its GLAM partnerships work, WMNL has ambitious targets for images in use and articles created or improved. Among this collection of projects, Nederland and the World and Project Nature stand out. Project Nature is rooted in an established approach that WMNL pioneered around thematic content, while Nederland and the World is a new idea that is being explored in 2017.
• WMNL has chosen to include communications activities as a program in this proposal, which may distract from less operational work that is included in this program area, such as providing critical support to the cross-chapter FKAGEU, which may result in substantive long term change.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

• With a budget of about $470,000 is 2017, WMNL projects an 80% reliance on APG funding. This will only be achieved if fundraising targets are met, which has been a challenge in 2016 and past years. WMNL plans to cap growth by 2020, according to their strategic plan.
• Past grants have not resulted in commensurate impact, although WMNL is experimenting with approaches that may yield long term results. The 2017 budget is weighted toward impactful programs like Content ($134,000, staff and non-staff).
• WMNL has a history of producing quality budgets that include staff time allocations and a useful level of detail. In 2015 and 2016, WMNL’s expenses exceeded their revenues.
• At the end of 2017, WMNL projects that its reserves will be $132,000, or equal to staff expenses for about 6 months, in the event of an emergency, which is in line with the FDC recommendation about decreasing their reserves.
• WMNL does not appear to be tracking or reporting in-kind donations.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Major strength	WMNL has a new and very high quality strategic plan in place, issued from a thorough and detailed process, which has been effectively implemented in this year's annual plan. WMNL has been consistent in this area.
P2. Potential for impact at scale	Strength	WMNL has struggled to achieve results in areas like articles improved and images in use. However, this year's plan works to scale beyond their borders with their Netherlands and the World focus. Their Community health program also has potential for impact across the Wikimedia projects. They have been playing an important role in FKAGEU, which works on a European scale.
P3. Objectives and evaluation methods	Strength	WMNL has included relevant metrics that focus on outcomes, and their plan is clear enough that they will have the capacity to implement it. WMNL is thoughtful about its evaluation methods, and has been improving in this area year over year.
P4. Diversity	Major strength	WMNL's has a program devoted specifically to gender work, and includes relevant metrics. In addition, they are now working on releasing GLAM content that is relevant for emerging communities.
Organizational effectiveness
O1. Past results	Concern	WMNL is beginning to achieve some traction in new program areas, but results in terms of content added and new contributors are still low, especially for an organization that has received large grants year over year.
O2. Learning	Major strength	WMNL is testing bold approaches in areas like community health and in its content focused work. They are experimenting with new ideas (community health, Nederland and the world) and refining and adapting established approaches (e.g. WiR, thematic content programs, WLM). They are documenting and sharing these results widely. It is clear from their strategic plan that they are reflecting on their work and applying what they have learned.
O3. Improving movement practices	Strength	WMNL has hosted international events, such as the GLAM Wiki conference and introduced international initiatives such as WLM. Their innovative approaches may be adapted by others in the movement. WMNL has been actively participating in FKAGEU.
O4. Community engagement	Neither	WMNL has challenges in the area of community engagement, which they have acknowledged, but they are actively working to address these challenges. And seem to be getting some traction, such as for example the Arbcom on the Dutch Wikipedia referring to one of their Community Health program.
O5. Capacity	Strength	WMNL has created a good quality plan and has the resources to complete it. WMNL has long term plans to increase their volunteer capacity.
Budget
B1. Past budgeting and spending	Concern	WMNL has relied on APG funding for most of its revenue since joining the FDC process. In the past year they have considerably improved their performance against fundraising targets, and have managed to help secure external funding for a Wikipedian in Residence (not managed by them). However, their external fundraising achievements are low and they have been running a deficit for several years. While this helped them reduce the high amount of reserves, it does not constitute a good trend.
B2. Budget is focused on impact	Neither	Despite a fine plan, based on past results, it seems unlikely that the budget will lead to programmatic impact commensurate with funds requested. WMNL has nevertheless shown that they could adapt their programs and work to tackle challenging and important issues (such their qualitative involvement in community health) that will be important in achieving long term impact. Their thoughtful plan also addresses the decline of notoriously high impact projects (such as Wiki Loves Monuments) to give them new directions aimed at keeping motivation and results high.

This staff proposal assessment is the work of FDC staff and is submitted by: Winifred Olliff (WMF Program Officer) talk 21:28, 8 November 2016 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.