Grants:APG/Proposals/2017-2018 round 1/Wiki Education Foundation/Staff proposal assessment

Wiki Education Foundation The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$101,996	$750,000	635.32%
Budget	$2,427,800	$2,767,007	13.97%
Staff (FTE)	11.5	11.5	0%

Overview

This section summarizes the themes that emerged from the proposal, which are explained in the detailed narrative section of this assessment.

Themes

• Lack of strategy
• Capacity to implement
• Historical alignment
• Large funding request
• Question about diversity

Wiki Ed has experience and expertise which gives them capacity to implement this plan, especially since it includes some clear goals and targets. However, the lack of strategic plan fails to give a coherent backbone to their programs. Wiki Ed is an effective organization, based on learning, which process relies on paid staff. Wiki Ed's classroom program has made significant quality contributions to the English Wikipedia but its focus on exclusively North America is somewhat difficult to reconcile with the new Wikimedia strategic direction.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

Wiki Ed is focused on English Wikipedia, and the regions of the United States and Canada. Their program is designed to fit within existing educational systems and institutions in the US and Canada, and emerged out of the Public Policy Initiative hosted within the Wikimedia Foundation. This early grant allowed Wiki Ed to launch their education program, and other education program models have since been adopted in many parts of the world.

Since then, the Wiki Education Foundation was formed as an independent nonprofit organization relying on other funding sources. They recently received funding from the Wikimedia Foundation in the form of a Simple APG for almost $100,000. Wiki Ed has explained that they have returned to the FDC process due to challenges competing with the Wikimedia Foundation in their fundraising environment.

Working within the higher education systems in the US and Canada gives Wiki Ed access to a more gender-balanced cohort of content contributors than is typically found on English Wikipedia, and this is a significant opportunity. Despite this, working within these systems may limit their impact to affect diversity in other areas, since bias present within these higher education systems may preclude working with a more racially and socio-economically diverse group of constituents within the US. Furthermore, a focus only on the US and Canada is not likely to increase linguistic or geographic diversity on a large scale.

Working within higher education also creates opportunities to connect with experts, which Wiki Ed is seeking to take more advantage of through its non-classroom programs.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

Wiki Ed has had success implementing their classroom program for many years. Over the years, they have refined their goals and targets for this work, and are consistently meeting expectations in terms of content contributions generated by this program. They have refined their approaches in order to target specific content gaps and have had success doing so. For example, their gender-focused partnerships have led to the creation of more than 3,500 relevant articles in this area, which is an impressive achievement.

We appreciate Wiki Ed’s emphasis on article quality, and also the ways in which they have managed to implement their program in the English Wikipedia environment, which can be a challenge to operate in.

Wiki Ed made some invaluable contributions to movement learning when team members were operating under the Wikimedia Foundation umbrella, which has led to many impactful education programs operating throughout the world. Even today, many people who are starting out new education initiatives rely on the resources created by members of that team.

Based on what this organization has achieved in the past, with respect to the classroom program, we think it is likely that they will be able to achieve what they have set out to achieve.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

Wiki Ed is an experienced and effective organization with a clear and logical staffing model. Their staff team is both experienced and capable. Beyond this, they have their systems and processes well-mapped and documented, and they have years of resources and work to build on in implementing their classroom program. The board includes people with various experience relevant to the nonprofit sector and the higher education systems in the regions target, and board and staff both include people recruited from the contributor communities.

Wiki Ed has had challenges involving volunteers in their work, and their approach shifted away from relying on volunteer campus ambassadors. While this process is staff-heavy, it is good to see that Wiki Ed is able to learn from challenges and change their approaches when they are not working. This more staff-intensive approach may be difficult to scale without infinitely expanding the organization and its budget.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

Wiki Ed does not have a strategic plan. While they have made some attempts to connect the content of their application with WMF’s new strategic vision, a lack of a strategic plan makes it difficult to determine how Wiki Ed sees their work developing in the long term. This is especially concerning given the large amount of funding requested.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

Wiki Ed has included relevant targets for each of its programs, as well as logic models that effectively explain how they expect each of their programs to work. Extensive background is offered to explain the classroom program, including an explanation of how Wiki Ed defines quality content in the context of this work. Wiki Ed explains in detail how the program works, and it seems clear that they have the expertise to achieve the results they are targeting.

Wiki Ed’s understanding of how their work is leading to more quality content on English Wikipedia is sophisticated, and builds on years of work in this area.

While all three programs have high potential for impact, the targets proposed do not correspond to the amount of funding requested.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

Despite Wiki Ed’s capacity and achievements, we are concerned that the amount of funding requested cannot lead to commensurate impact, based on the targets provided. Furthermore, Wiki Ed’s emphasis on English Wikipedia means that its contributions to geographic and linguistic diversity of the Wikimedia movement may be limited.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Concern	Wiki Ed is requesting 750,000 US dollars and does not have a strategic plan in place. On a greater time scale, we lack an understanding of how their approaches each year fit into a longer term vision.
P2. Potential for impact at scale	Neither	Wiki Ed does have a great potential for impact in North American higher education. However, their narrowed geographical focus as well as the lack of focused and deliberate action to widen its impact across the knowledge divide is concerning for an organization of this size. It will be interesting to see how Wiki Ed decides to expand their scale in the next few years.
P3. Objectives and evaluation methods	Strength	Wiki Ed has developed interesting criteria for evaluating article quality, which are both innovative and easy to use, including their use of the dashboard software to track contributions, which allows for clear reporting and tracking and for a good overview of their program results. They are thinking about how to better track retention of teachers, which will enhance their ability to evaluate the success of their programs.
P4. Diversity	Neither	We appreciate the work that Wiki Ed has done with partners that are targeting content related to gender and other aspects of diversity, such as the National Women's Studies Association, which have resulted in large numbers of articles in relevant content areas. Wiki Ed reports 60% of editors participating in their programs are women, but this could easily be explained by the fact that this reflects the gender ratio in US higher education.
Organizational effectiveness
O1. Past results	Strength	Wiki Ed has a lot of experience doing similar programs in the past, and has achieved their goals for their classroom program work.
O2. Learning	Strength	Wiki Ed has done a lot of research on their own impact, and have tried to understand what impact their work informs. Whether on the quality of articles added to Wikipedia, or their research on learning outcomes, they are keen on understanding their environment to adapting to it. The dashboard development also stemmed from their observation of what worked and what didn't in a previous approach, showing a commitment to applying findings to adapt and change.
O3. Improving movement practices	Strength	Wiki Ed has made good contributions to movement practices on a historical scale, launching the first education program through the Public Policy Initiative housed within WMF, which has led to a rich array of education work worldwide. They have also produced resources that have been adapted and used by many.
O4. Community engagement	Neither	Wiki Ed has identified difficulties working with volunteers as campus ambassadors and has provided technical and staffing solutions to effectively address these challenges. There is community representation on their board, and they appear to work effectively with communities on the projects affected by their work. Wiki Ed is not a community-based organization like most other organizations in the FDC process.
O5. Capacity	Neither	Wiki Ed has a staff team that includes relevant expertise and many years of experience. Some staff have been implementing similar programs for as long as eight years! They have set up a functional nonprofit organization with strong capacity in a number of areas. At the same time, they have themselves identified challenges with engaging volunteers effectively in their work.
Budget
B1. Past budgeting and spending	Concern	Wiki Ed is a developed organization with accounting and budgeting capacity, and with fundraising staff. Nevertheless, their revenues and expenses have recently been unstable as they are entering a period of uncertain funding.
B2. Budget is focused on impact	Concern	The amount requested is too high with respect to what this organization is aiming to achieve, and results seem to be dependent on the ongoing financing of their staffing model.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 01:28, 9 November 2017 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.