Grants:APG/Staff proposal assessment form

From Meta, a Wikimedia project coordination wiki
Jump to: navigation, search

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) deliberations process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal. Members of the FDC also review each proposal in its entirety, including relevant background documents, inputs from WMF grantmaking and finance staff, and the detailed discussions surrounding each proposal that include questions and comments from community members. FDC staff do not advise the FDC about whether or not a proposal should be funded, or about how much funding a proposal should receive, but rather seek to identify key strengths and concerns that the FDC may consider while making funding recommendations.

This staff proposal assessment consists of three parts: (1) Overview, including a financial summary and a summary of strengths and concerns; (2) Staff proposal assessment narrative, including a nuanced analysis of the programs, context, and the feasibility of each plan; (3) Staff proposal assessment rubric, assessing each proposal across three dimensions by identifying areas of strength and concern. Read more about the methodology behind the Staff proposal assessment rubric on this page.

Staff proposal assessments do not represent the views on any one staff person, since all FDC staff (unless one of them recuses for some reason) work together to complete each staff proposal assessment, and also may consult with WMF staff and other experts as needed. Each staff proposal assessment is signed and published by one staff person, but the assessment is formed by consensus among the FDC staff team. While assessments will not be changed after they are published except to correct significant factual errors, organizations are welcome to share responses on the discussion page of each staff proposal assessment if they would like to provide the FDC with additional information or perspectives on the assessments.

Overview[edit]

Overall assessment and eligibility[edit]

Organization
Eligibility
Overall assessment

Financial summary[edit]

Current (projected) Upcoming (proposed) Proposed change (as a +/- percentage)
FDC or PEG funding
Budget
Staff

Overview of strengths and concerns[edit]

This section summarizes the key strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths[edit]

Concerns[edit]

Staff proposal assessment narrative[edit]

This section takes an in-depth look at the strategy behind this proposal, this organization's broader context, and the feasibility and risks of this plan.

Context and potential for impact[edit]

This section takes a close look at this organization's context, since there will be unique factors specific to this organization or to its environment that enable this plan to have impact or that make this plan less likely to have impact. Here are some questions we will consider:

  • How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact? Are there risks and opportunities presented by this organization's environment?
  • Will this organization's overall effectiveness enable this plan to succeed? Does it raise any concerns?
  • Will this organization's past performance with program implementation enable this plan to succeed? Does it raise any concerns?
  • Are there extraordinary events or other factors that need to be considered as part of this organization's context?


Strategy, program design, and program implementation[edit]

This section takes a close look at this organization's plans for its programs, including its past performance with program implementation. In this section, we may also identify specific programs that seem to have particularly high or low potential for impact. Here are some questions we will consider:

  • Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?
  • Is this organization proposing significant changes to its strategy, or are the strategic plan and annual plan consistent with past strategies?
  • Is this organization's strategy and are this organization's programs aligned with the Strategic Priorities of Increasing participation, Improving quality, or Increasing reach?
  • Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked?
  • Based on proposed plans and past work, which programs may have the highest potential for impact corresponding to the amount of funding requested, and which programs may not have impact corresponding to the funds requested? Is this plan and budget focused on the programs with the highest potential for online impact?


Feasibility and risks[edit]

How likely is it that this organization can implement this plan successfully if funded? Is this plan likely to achieve impact corresponding to the funds requested?

Feasibility[edit]

What enables this plan?


Risks[edit]

What are the risks?


Summary of expert opinions and community commentary[edit]

This section will summarize any expert opinions or community commentary offered through the APG proposal process, or through additional interviews and research.


Staff proposal assessment rubric[edit]

FDC staff have assessed this proposal across three dimensions and the criterion listed below according to a non-linear scale: Major strength, Major concern, Strength, Concern, Neither a strength nor a concern. Read more about the methodology behind the Staff proposal assessment rubric on this page.

Criterion Assessment Description
Program design and strategic alignment
P1. Strategic alignment
P2. Targets and logic models
P3. Needs assessments
P4. Potential for impact at scale
P5. Evaluation methods
P6. New ideas and approaches, or thoughtful adaptations
P7. Diversity {{{P7}}} {{{P7 description}}}
Organizational capacity and effectiveness
O1. Past results
O2. Stability and leadership
O3. Learning
O4. Movement leadership
O5. Community engagement
O6. Capacity
O7. Extraordinary events {{{O7}}} {{{O7 description}}}
Budget
B1. Past budgeting and spending
B2. Proposed budget is realistic
B3. Budget is focused on programmatic impact

This staff proposal assessment is the work of FDC staff and is submitted by:

Staff proposal assessment rubric description[edit]

This rubric assesses annual plan grant proposals across the three dimensions of (1) Program design and strategic alignment, (2) Organizational capacity and effectiveness, and (3) Budgeting by identifying strengths and concerns in each of these areas.

In each assessment, an organization’s context is taken into account and so criteria may be applied differently in different contexts. For example, a smaller organization with less experience will not be expected to have a formal learning and evaluation program that a larger more experienced organization might have, and so what may constitute a major strength for a smaller organization in the same category may not constitute a major strength for a larger organization. With this approach, the criteria listed here are not weighted equally, and so an “overall assessment” cannot be determined simply by adding up the numbers of strengths and concerns identified for each dimension. Also note that the scale here is not linear and so we will not assign any numerical values to the assessments given for each criterion. More information about each criterion is shared in the assessment. We still believe a rubric like this is helpful, as it ensures that staff analysis of each proposal is based on the same criteria, even as they are applied in a nuanced way.

The dimension “Program design and strategic alignment” addresses many high-level elements, focusing on how likely programs are to achieve progress toward some of the Wikimedia movement's strategic priorities. Programs need to show clear links between strategic priorities and program objectives, program objectives and targets, and targets and program activities. In addition, programs should focus on areas that will impact participation, reach, and quality on the Wikimedia projects, with an emphasis on approaches and programs that increase diversity.

Meanwhile, the dimension “Organizational capacity and effectiveness” looks at the ability of an organization to achieve its plan based on both past experience (as an indicator of future success) and current practice. An effective organization both learns from past experience and contributes to movement learning in the process. This dimension also includes effective and stable leadership that models good practices, and a rational approach to growth that is supported by the organization’s strategy and context.

Finally, we analyze the organization’s past experience with and current plan for budgeting. Here, we look at how the organization has performed against its plans and budgets in the past.

To complete the assessment, we assess each of the criterion in each category according to the following scale, which is not necessarily linear:

  • Major strength
  • Strength
  • Neither a strength nor a concern
  • Concern
  • Major concern
Criterion Description
Program design and strategic alignment
P1. Strategic alignment Programs objectives are strongly aligned with the Wikimedia strategic priorities of increasing reach, improving quality, or increasing participation.
P2. Targets and logic models Programs have specific, measurable, time-bound objectives with clear targets, and programs present clear links between planned activities and desired results.
P3. Needs assessments Program objectives are based on needs assessed in consultation with stakeholders, including the communities and volunteers that are working together with this organization to achieve its program objectives.
P4. Potential for impact at scale Programs could lead to significant impact on the Wikimedia strategic priorities of increasing reach, improving quality, or increasing participation, at scale and corresponding to the amount of funds requested.
P5. Evaluation methods Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems.
P6. New ideas and approaches, or thoughtful adaptations Programs will test new ideas and approaches or will thoughtfully adapt ideas and approaches that may work in this organization’s context.
P7. Diversity Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational capacity and effectiveness
O1. Past results This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Stability and leadership Stable, effective leadership and good governance will enable this plan to succeed. Any leadership transitions are managed effectively.
O3. Learning from the past This organization is addressing risks and challenges effectively, is learning from and documenting past experiences, and is applying learning to improve its current and planned programs.
O4. Movement leadership This organization effectively shares learning about its work with the broader movement and beyond. This organization is a leader that helps the movement to grow.
O5. Community engagement This organization effectively engages key communities and volunteers in the planning and implementation of its work.
O6. Capacity This organization has the resources and ability (for example, expertise, staff, experience managing funds) to do the plan proposed.
O7. Extraordinary events Extraordinary events must be taken into account since they affect the overall assessment of this organization or plan.
Budget
B1. Past budgeting and spending This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Proposed budget is realistic This proposal includes a budget that is comprehensive and clear, and that corresponds to the activities proposed.
B3. Budget is focused on programmatic impact Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.