Grants:APG/Proposals/2016-2017 round 1/Wikimedia Sverige/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$306,072	$345,150 (2017, 2018)	+12.77% (2017), 0% (2018)
Budget	$805,674	$859,935 (2017), $895,479 (2018)	+6.73% (2017), +4.13% (2018)
Staff (FTE)	7.75	8.0 (2017), 8.5 (2018)	+3.2% (2017), +6.25% (2018)

Overview of strengths and concerns

This section summarizes the strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths

• WMSE’s staff and board capacity enabled a successful ED transition.
• WMSE is doing innovative work that others can learn from.
• WMSE is achieving recognition of expertise on a national level, in GLAM and advocacy.

Concerns

• 2015 and 2016 global metrics achievements are lower than expected.
• WMSE’s strategic plan is underdeveloped.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

• WMSE’s expertise in the field of open knowledge is recognized on a national scale, through their successful work with GLAM partners, and by the Swedish government, which has included them on a list of official consultees on related legislation.
• Institutional partners in Sweden are technically savvy and curious about open knowledge. They are resource-rich, and often have good incentives to participate in innovative programs such as collaboration with WMSE offers. WMSE has worked with institutions to identify their most pressing needs, and has developed expertise accordingly, resulting both in fruitful partnerships and in a proved track record of expertise that can be leveraged to further partnership opportunities.
• Swedish Wikipedia is a large and active Wikipedia for a language with 9 million speakers worldwide. The community Swedish-language Wikipedia contributors is close knit, and most people in the Swedish-language community have met at least one other Swedish Wikipedian. Many get together on a regular basis to discuss their work. WMSE’s strong relationship with its community enables its quality program work. They have regular channels for learning about what the community needs and wants. A large number of online contributors in Sweden are also members of the chapter.
• WMSE has opportunities to bring additional funding to the movement, and they have leveraged their general support funding from the FDC in order to develop strong expertise in this area. Combined with the relevance of WMSE’s externally funded initiatives, this has resulted in more funds available for Wikimedia work worldwide.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

• In 2015 we saw growth in Wikidata-focused work, and more content generated by self-sustaining partnerships.
• By mid-2016, WMSE was performing well against many of their program-specific goals. WMSE is on track to have a WikiSpeech prototype developed by the end of 2016 and by mid-2016, WMSE had 110 subject experts participate in article review. WMSE has not seen any progress with increasing pageviews to the Swedish Wikipedia projects (measured as part of its Use program), and they have not been able to measure retention as part of their community program. WMSE has achieved 219 productive edits from unique individuals in underrepresented groups, toward its annual goal of 365 edits.
• Results reported in the progress report for the first half of 2016 show that WMSE is not always on track to meet their global metrics targets. For example, they have reported 15,055 new or improved articles against a target of 580,350. They have explained that Wikidata work, which is driving this high target for articles, may be delayed until 2017.
• Some activities like Wiki Loves contests will take place later this year; WMSE is also shifting focus towards external communications instead of events, and therefore anticipate falling short of their targets as a result of this change. In 2015, WMSE also underachieved against their targets, reporting 3,535 pages of 694,175 targeted.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

• WMSE has successfully leveraged APG funding to build their fundraising capacity and technical expertise. WMSE has 7.75 FTE, which is a large staff for a Wikimedia Chapter, and WMSE’s staff has important technical and community-based expertise in relevant areas such as technology, communications, and financial planning.
• WMSE’s long-time ED left the organization in 2015 and WMSE brought on Anna Troberg as ED in January 2015. This transition was enabled by WMSE’s stable board and committed staff team. Through this transition, WMSE has shown that staff responsibilities are well-distributed and their staff can withstand a period of transition without interrupting their duties.
• Over the past year, WMSE has improved in the area of sharing movement practices, participating actively in international learning activities like the Wikimedia Conference. They are planning to host the international Diversity Conference in Sweden this year.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

• WMSE completed the strategic process it started in 2015 and produced a brief strategic plan with four strategic foci: Access, Use, Community, and Enabling. WMSE’s strategic plan covers 2016-2020. This has enabled WMSE to participate in the multiyear funding pilot, creating program plans for 2017 and 2018.
• WMSE’s strategic plan does not provide an analysis of risks and opportunities, nor are specific long-term strategic goals provided in the form of qualitative goals or quantitative targets. This may make it difficult for WMSE to track progress against its strategic priorities. Estimates of resources needed and plans for budget and fundraising development are also not addressed over the long-term strategic period.
• Despite the lack of information contained in the strategic plan, WMSE has done a good job of implementing its strategy in its annual plans for 2017 and 2018, which strike a good balance of providing detailed information and a sense of priority.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

• WMSE’s strategy and plan have a strong emphasis on diversity, including specific activities that target Wikipedia’s gender imbalance, the inclusion of minority languages in its education work, and its focus on Wikispeech. WMSE has been focusing on and measuring gender-related work for several years, and is building some expertise in this area. WMSE’s gender-focused work includes online and offline approaches.
• WMSE’s program plan reflects a cohesive strategy, providing a coherent set of relevant objectives for each program, even where programs include a range of activities and projects. Evaluation for each program is well-defined, targets are clear and may be attainable. WMSE has selected a diversity-related metric as one of its grantee-defined metrics, which is relevant to its strategic focus; yet tracking the number of blog posts is unlikely to provide insight into how effective WMSE’s outreach is.
• WMSE continues to pilot and develop innovative programs and approaches that the wider movement can learn from. For example, WMSE has been a pioneer among chapters in defining a reader-focused program. In community support and GLAM, they focus on providing volunteers and partners with technical expertise. WMSE has been interested in achieving long-term changes in attitudes about free content for many years, and continues that approach in the current plan. WMSE is developing metrics in the area of usage and trust, to be used when evaluating its 2018 plan.
• WMSE has been building their expertise in advocacy, and may have opportunities to begin to influence policy at the national and EU levels.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

• In 2015 and 2016, WMSE set ambitious fundraising goals, and adjusted their budgets when goals were not achieved. WMSE has a goal of minimizing dependency on any one funder, which is considered good practice in the nonprofit world. In 2017, their requested APG of $345,000 would be 39% of their total revenue, and 38% in 2018.
• WMSE has unusually low overhead expenses for an organization with a permanent office and a large staff, even when expenses for the Enabling program which are more properly classified as operational are included as operational expenses. Spending in 2017 is focused on the Access program (353,000 including staff). This area includes expenses for impactful programs like GLAM, education, and advocacy. Programs are similarly weighted in 2018.
• Wikimedia Sverige’s reserves at the end of 2016 will be $128,000, and Wikimedia Sverige projects that they will grow to about $150,000 by the end of 2017. This is a prudent move, since increased 2017 reserves would cover about 3 months of staffing expenses in the event of an emergency.
• Wikimedia Sverige does not mention tracking in-kind donations in this proposal form.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Strength	WMSE has a clear strategic plan in place, developed collaboratively by the board and members, but it is minimal and does not include long term goals or risk analysis. This strategy forms the backbone of their annual plans and has effectively driven consistency year over year. It is also well aligned with Wikimedia strategic priorities.
P2. Potential for impact at scale	Neither	WMSE is developing innovative programs and approaches that could scale. Some projects, such as WikiSpeech have the potential to scale beyond Sweden's borders. Performance in online impact has been lower than expected in 2015 and 2016, which leads to question how well their proposed programs really will scale.
P3. Objectives and evaluation methods	Neither	WMSE has been struggling with setting targets, and has been setting low targets. This could mean that their evaluation methods are not reliable enough to plan and track accurately. While their diversity metric is a valuable and interesting one, it is difficult to understand how measuring the number of blog posts will significantly track any kind of impact.
P4. Diversity	Major strength	WMSE has been working steadily in gender issues in their programs. They have selected diversity as one of their grantee-defined metric and gender diversity is core for their Community program. Seeking diversity on the board is an organizational goal. They also tackle diversity with minority language work, or with their work on WikiSpeech.
Organizational effectiveness
O1. Past results	Neither	WMSE is not on track to meet global metrics targets in 2016, and has reported low achievements against its targets in 2015. On the other hand, WMSE is measuring up well against many of its program-specific targets. WMSE is on track to launch their WikiSpeech prototype, are making headway in encouraging edits from underrepresented groups, and have exceeded expectations in the area of expert review.
O2. Learning	Strength	WMSE continues to effectively document and apply learning.
O3. Improving movement practices	Strength	WMSE has improved in the area of sharing movement practices. WMSE's program-level staff have been active in sharing with the international movement, and have a lot to offer in terms of skills and experience. WMSE is also testing new approaches that may be adapted in other parts of the movement, and are hosting the upcoming diversity conference.
O4. Community engagement	Strength	WMSE continues to effectively engage communities and volunteers in the planning and implementation of its work.
O5. Capacity	Major strength	WMSE has undergone a successful ED transition, which was enabled through the competence and commitment of its board and staff. Staffing structure works well and duties are effectively distributed. Staff has expertise in key technical areas, and WMSE prioritizes and includes volunteers.
Budget
B1. Past budgeting and spending	Neither	WMSE has been receiving large grants from the APG process since their inception. WMSE has successfully leveraged APG funding to build capacity in fundraising and key program areas, which is enabling them to deliver programs with the potential to scale. However, despite an overall good quality of its budgets, WMSE has slightly underachieved on their fundraising targets and has had to adjust their budget accordingly in 2015 and 2016.
B2. Budget is focused on impact	Strength	WMSE has created a clear budget that is weighted toward impact. Results in 2015 and 2016 have not been impressive in terms of online content, considering the amount of funding received. This raises some concerns about potential for impact that is commensurate with the funding requested, but this 2017 plan has the potential to succeed.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 21:47, 8 November 2016 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.