Grants:APG/Proposals/2017-2018 round 1/Wiki Education Foundation/Staff proposal assessment

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search

Wiki Education Foundation The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview[edit]

Summary[edit]

Current (projected)

Upcoming (proposed)

Proposed change (as a +/- percentage)

FDC or other relevant funding

$101,996 $750,000 635.32%

Budget

$2,427,800 $2,767,007 13.97%

Staff (FTE)

11.5 11.5 0%

Overview[edit]

This section summarizes the themes that emerged from the proposal, which are explained in the detailed narrative section of this assessment.

Themes[edit]

  • Lack of strategy
  • Capacity to implement
  • Historical alignment
  • Large funding request
  • Question about diversity

Wiki Ed has experience and expertise which gives them capacity to implement this plan, especially since it includes some clear goals and targets. However, the lack of strategic plan fails to give a coherent backbone to their programs. Wiki Ed is an effective organization, based on learning, which process relies on paid staff. Wiki Ed's classroom program has made significant quality contributions to the English Wikipedia but its focus on exclusively North America is somewhat difficult to reconcile with the new Wikimedia strategic direction.

Staff proposal assessment narrative[edit]

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness[edit]

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment[edit]

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

Wiki Ed is focused on English Wikipedia, and the regions of the United States and Canada. Their program is designed to fit within existing educational systems and institutions in the US and Canada, and emerged out of the Public Policy Initiative hosted within the Wikimedia Foundation. This early grant allowed Wiki Ed to launch their education program, and other education program models have since been adopted in many parts of the world.

Since then, the Wiki Education Foundation was formed as an independent nonprofit organization relying on other funding sources. They recently received funding from the Wikimedia Foundation in the form of a Simple APG for almost $100,000. Wiki Ed has explained that they have returned to the FDC process due to challenges competing with the Wikimedia Foundation in their fundraising environment.

Working within the higher education systems in the US and Canada gives Wiki Ed access to a more gender-balanced cohort of content contributors than is typically found on English Wikipedia, and this is a significant opportunity. Despite this, working within these systems may limit their impact to affect diversity in other areas, since bias present within these higher education systems may preclude working with a more racially and socio-economically diverse group of constituents within the US. Furthermore, a focus only on the US and Canada is not likely to increase linguistic or geographic diversity on a large scale.

Working within higher education also creates opportunities to connect with experts, which Wiki Ed is seeking to take more advantage of through its non-classroom programs.

Past performance[edit]

Will this organization's past performance with program implementation enable this plan to succeed?

Wiki Ed has had success implementing their classroom program for many years. Over the years, they have refined their goals and targets for this work, and are consistently meeting expectations in terms of content contributions generated by this program. They have refined their approaches in order to target specific content gaps and have had success doing so. For example, their gender-focused partnerships have led to the creation of more than 3,500 relevant articles in this area, which is an impressive achievement.

We appreciate Wiki Ed’s emphasis on article quality, and also the ways in which they have managed to implement their program in the English Wikipedia environment, which can be a challenge to operate in.

Wiki Ed made some invaluable contributions to movement learning when team members were operating under the Wikimedia Foundation umbrella, which has led to many impactful education programs operating throughout the world. Even today, many people who are starting out new education initiatives rely on the resources created by members of that team.

Based on what this organization has achieved in the past, with respect to the classroom program, we think it is likely that they will be able to achieve what they have set out to achieve.

Organizational effectiveness[edit]

Will this organization's overall effectiveness enable this plan to succeed?

Wiki Ed is an experienced and effective organization with a clear and logical staffing model. Their staff team is both experienced and capable. Beyond this, they have their systems and processes well-mapped and documented, and they have years of resources and work to build on in implementing their classroom program. The board includes people with various experience relevant to the nonprofit sector and the higher education systems in the regions target, and board and staff both include people recruited from the contributor communities.

Wiki Ed has had challenges involving volunteers in their work, and their approach shifted away from relying on volunteer campus ambassadors. While this process is staff-heavy, it is good to see that Wiki Ed is able to learn from challenges and change their approaches when they are not working. This more staff-intensive approach may be difficult to scale without infinitely expanding the organization and its budget.

Strategy and programs[edit]

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy[edit]

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

Wiki Ed does not have a strategic plan. While they have made some attempts to connect the content of their application with WMF’s new strategic vision, a lack of a strategic plan makes it difficult to determine how Wiki Ed sees their work developing in the long term. This is especially concerning given the large amount of funding requested.

Programs[edit]

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

Wiki Ed has included relevant targets for each of its programs, as well as logic models that effectively explain how they expect each of their programs to work. Extensive background is offered to explain the classroom program, including an explanation of how Wiki Ed defines quality content in the context of this work. Wiki Ed explains in detail how the program works, and it seems clear that they have the expertise to achieve the results they are targeting.

Wiki Ed’s understanding of how their work is leading to more quality content on English Wikipedia is sophisticated, and builds on years of work in this area.

While all three programs have high potential for impact, the targets proposed do not correspond to the amount of funding requested.

Budget[edit]

Is this plan and budget focused on the programs with the highest potential for online impact?

Despite Wiki Ed’s capacity and achievements, we are concerned that the amount of funding requested cannot lead to commensurate impact, based on the targets provided. Furthermore, Wiki Ed’s emphasis on English Wikipedia means that its contributions to geographic and linguistic diversity of the Wikimedia movement may be limited.

Summary of expert opinions (if applicable)[edit]

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework[edit]

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

  • Major strength
  • Strength
  • Neither a strength nor a concern
  • Concern
  • Major concern

Criterion

Assessment

Description

Program design

P1. Strategy

Concern Wiki Ed is requesting 750,000 US dollars and does not have a strategic plan in place. On a greater time scale, we lack an understanding of how their approaches each year fit into a longer term vision.

P2. Potential for impact at scale

Neither Wiki Ed does have a great potential for impact in North American higher education. However, their narrowed geographical focus as well as the lack of focused and deliberate action to widen its impact across the knowledge divide is concerning for an organization of this size. It will be interesting to see how Wiki Ed decides to expand their scale in the next few years.

P3. Objectives and evaluation methods

Strength Wiki Ed has developed interesting criteria for evaluating article quality, which are both innovative and easy to use, including their use of the dashboard software to track contributions, which allows for clear reporting and tracking and for a good overview of their program results. They are thinking about how to better track retention of teachers, which will enhance their ability to evaluate the success of their programs.

P4. Diversity

Neither We appreciate the work that Wiki Ed has done with partners that are targeting content related to gender and other aspects of diversity, such as the National Women's Studies Association, which have resulted in large numbers of articles in relevant content areas. Wiki Ed reports 60% of editors participating in their programs are women, but this could easily be explained by the fact that this reflects the gender ratio in US higher education.

Organizational effectiveness

O1. Past results

Strength Wiki Ed has a lot of experience doing similar programs in the past, and has achieved their goals for their classroom program work.

O2. Learning

Strength Wiki Ed has done a lot of research on their own impact, and have tried to understand what impact their work informs. Whether on the quality of articles added to Wikipedia, or their research on learning outcomes, they are keen on understanding their environment to adapting to it. The dashboard development also stemmed from their observation of what worked and what didn't in a previous approach, showing a commitment to applying findings to adapt and change.

O3. Improving movement practices

Strength Wiki Ed has made good contributions to movement practices on a historical scale, launching the first education program through the Public Policy Initiative housed within WMF, which has led to a rich array of education work worldwide. They have also produced resources that have been adapted and used by many.

O4. Community engagement

Neither Wiki Ed has identified difficulties working with volunteers as campus ambassadors and has provided technical and staffing solutions to effectively address these challenges. There is community representation on their board, and they appear to work effectively with communities on the projects affected by their work. Wiki Ed is not a community-based organization like most other organizations in the FDC process.

O5. Capacity

Neither Wiki Ed has a staff team that includes relevant expertise and many years of experience. Some staff have been implementing similar programs for as long as eight years! They have set up a functional nonprofit organization with strong capacity in a number of areas. At the same time, they have themselves identified challenges with engaging volunteers effectively in their work.

Budget

B1. Past budgeting and spending

Concern Wiki Ed is a developed organization with accounting and budgeting capacity, and with fundraising staff. Nevertheless, their revenues and expenses have recently been unstable as they are entering a period of uncertain funding.

B2. Budget is focused on impact

Concern The amount requested is too high with respect to what this organization is aiming to achieve, and results seem to be dependent on the ongoing financing of their staffing model.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 01:28, 9 November 2017 (UTC)

Staff proposal assessment framework[edit]

  • Major strength. This is something the organization does very well, and this is a strong indicator of future success.
  • Strength. This is something that the organization does well, and this could indicate future success.
  • Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
  • Concern. This is something that the organization does not do well, and this could make funding the application a risk.
  • Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion

Description

Program design

P1. Strategy

The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.

P2. Potential for impact at scale

Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested

P3. Evaluation methods

Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.

P4. Diversity

Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.

Organizational effectiveness

O1. Past results

This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.

O2. Learning

This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.

O3. Improving movement practices

This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.

O4. Community engagement

This organization effectively engages communities and volunteers in the planning and implementation of its work.

O5. Capacity

This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.

Budget

B1. Past budgeting and spending

This organization has a history of budgeting realistically and managing funds effectively in the past.

B2. Budget is focused on programmatic impact

Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.