Template:FDC staff proposal assessment/round2 2013-2014/sandbox

Please do not edit the FDC staff proposal assessment template. Instead, if you have comments or feedback, we welcome that on the talk page. Previous discussion of this form is here.

Methodology behind the 2013-2014 Round 2 FDC staff assessments

The FDC staff assessments provide a snapshot of the staff review of Annual Plan Grant (FDC) proposals, and identify both strengths and concerns with the work proposed.

• We did this through both portfolio analysis - looking across proposals and calibrating across different organizations - as well as assessing the context of each individual organization and its proposal.
• Inputs into the staff assessment process this round include:
  • The proposals themselves, with the attached documents (strategic plans, annual plans, detailed budgets, supplementary documents)
  • Past and current reports (quarterly progress reports on annual plan grants, other grant reports, annual financial and narrative reports)
  • A portfolio view of financials across proposals for both rounds this year Grants:APG/Proposals/2013-2014_round2/Financial_overview and Grants:APG/Proposals/2013-2014_round1/Financial_overview.
  • Internal financial, program evaluation, and grants compliance inputs from the WMF Finance, Programs, Legal, and Grantmaking teams.
  • Detailed comments and questions collected during the Community Review period, which are also summarized in these staff assessments.
• We used a few key principles for the assessment, across all proposals. These are not new, and are detailed in the narrative and scoring sections of the assessment. They include:
  • Impact on Wikimedia projects and the global movement, commensurate with funds requested. We looked for the rationale behind the work organizations do, aligning them to the Wikimedia strategic priorities and goals. We looked particularly for direct and indirect impact of this work on Wikimedia projects; for example, growth in contributors and content donation. Most importantly, we looked at whether the funds requested were justified in the context of current and potential impact of the proposed work.
  • Organizational strategy, leadership, and governance. We looked for evidence that an organization had a good understanding of its own context, and the needs, opportunities and challenges of the communities it seeks to serve or partner with. We looked for board, staff and volunteer leadership that is committed, effective and engaged with these communities and practising movement values of openness and transparency.
  • Rates of growth and financial sustainability. In this round, no applicants asked for funds within the recommended maximum growth rate in FDC funding of 20%, which is itself a significant rate in many contexts. The minimum growth rate in the FDC/movement funding proposed is 22.7% and the maximum proposed is 216%. Given the challenges faced by some organizations with raising funds externally, there are significant concerns about the long term financial sustainability of this level of dependence on movement resources in areas where other funding may be available.
  • Program design, learning, and evaluation. We recognize and appreciate that organizations in this round aim to link their work directly to outcomes on the Wikimedia Projects they are targeting and to work toward aligning their high-level goals across programs; at the same time, this work does not always coincide with clear links between these high-level targets and the direct outcomes of program activities, which may make it difficult to understand the effectiveness of some program activities. We appreciate the good work that organizations have done in this area so far, and urge organizations to continue to work toward making the links between program activities, objectives, and high-level targets clear.
  • Inclusion of exceptional organizations. Two organizations were included in this round that required exceptions. The Centre for Internet and Society is the first non-Wikimedia organization to participate in the FDC process, and was made eligible through a resolution by the WMF Board of Trustees. As a result, detailed attention was paid to the financial data presented by CIS to understand the work specific to the Wikimedia movement that will be funded by the FDC, in addition to understanding their broader budget context, and only this work was described in detail in the CIS proposal and analyzed in the staff assessment. Other than this, the process for assessing the CIS proposal has been the same as that for other proposals. The Wikimedia Foundation also participated in this round of the FDC process; at the FDC’s request we did not complete a staff assessment for the Wikimedia Foundation.

Note: We shared our nearly final draft assessments with all FDC applicants a day ahead of publishing them, so that organizations had some lead time for sharing these with their Board and staff. We then took all responses into consideration where possible, making factual changes and clarifying our statements where necessary. We encourage organizations to note all substantive comments and concerns on the discussion pages of these assessments.

Proposal overview

Name of eligible organization
Summary staff assessment [substantial concerns: none/some/many]
Total amount proposed to the FDC for the upcoming year annual plan in currency requested
Estimate of total amount proposed to the FDC for the upcoming year annual plan in US dollars
Exchange rate used to calculate estimate in US dollars and date accessed (mm/dd/yy)
Date of proposal submission
Adherence to proposal process [yes/no]
Adherence to legal, regulatory, and policy regulations [yes/no]
Adherence to US anti-terrorism laws [yes/no]

Financial summary

A detailed summary of the financial information from all proposals is here: financial overview document.

	Projected 2013	Proposed 2014	Proposed change (as a +/− %)
FDC or WMF funding
Overall budget
Staff costs

Annual plan

Overview of staff assessment

Strengths

What are the top strengths of the plan?

Concerns

What are the top concerns with the plan?

Strategic alignment

To what extent does the plan address the strategic priorities of the movement?

Context of the organization and community

To what extent does the context of the organization (e.g., community participation, fundraising context, sociopolitical environment) enable or inhibit the organization's potential for impact? To what extent does the organization's experience enable or inhibit the organization's potential for impact?

Feasibility and risks for the annual plan

How feasible is the annual plan? Specifically:

• To what extent are the budget, staffing, availability of volunteers, and other resources realistic for achieving the anticipated outcomes?
• Are the timelines realistic for the proposed initiatives and associated activities?
• Are there acknowledged and/or unacknowledged risks in the proposal?

Summary of expert opinions and/or community commentary

Summary of the opinions of subject-matter experts on the plan

No expert commentary or opinion was requested for this plan.

Scores, comments and feedback

Scores provide only a preliminary assessment for consideration by the FDC. The rubric for scoring is below.

Dimensions	Criteria	Score (1–5)	Comments and feedback
Potential for impact against strategic priorities	(A) The plan addresses the Wikimedia movement strategic priorities of the Wikimedia movement (infrastructure, participation, quality, reach, and innovation)
	(B) The initiatives in the plan have the potential to lead to movement strategic priorities, if executed well
	(C) The initiatives in the plan have the potential to lead to significant impact, given context and amount of funds proposed
Ability to execute	(D) The organization has the resources, skills, and capacity (staff, volunteer, non-staff) for the plan to succeed
	(E) The organization has a record of success in similar initiatives
	(F) The organization's leadership is effective, committed, and relatively stable
Efficient use of funds	(G) The funding requested is reasonable given the proposed initiatives
	(H) The organization has a record of using funds efficiently and staying on budget
Quality of proposed measures of success	(I) Clear indicators of success are outlined in the plan
	(J) A plan is in place to track the proposed metrics
	(K) The organization has a feasible plan in place to track the proposed metrics
Potential to add knowledge and/or other benefits to the movement	(L) The initiatives in the plan, if successful, could be helpful and/or productively adapted elsewhere in the movement
	(M) The organization is ready to share with the movement the lessons learned from the initiatives

Scoring rubric

Dimensions	Evaluation criteria	1 = Weak or no alignment with criterion	3 = Moderate alignment with criterion	5 = Strong alignment with criterion
Potential for impact against strategic priorities	(A) The plan addresses the Wikimedia movement strategic priorities	The plan will not directly address any strategic priorities	The plan will partially address strategic priorities or other goals that are closely related to the strategic priorities	The plan will directly address several strategic priorities
	(B) The initiatives have the potential to lead to the strategic priorities, if executed well	The initiatives have no clear connection to strategic priorities	Some or all of the initiatives have an unclear relationship to strategic priorities	The initiatives have been shown (by the organization or the movement) to lead to strategic priorities
	(C) The initiatives have the potential to lead to significant impact, given context and amount of funds proposed	The initiatives will have marginal impact in the local community and the broader Wikimedia community, given context and amount of funds proposed	The initiatives proposed may have a significant impact in the local community, given context and amount of funds proposed	The initiatives could have significant impact at scale, both in the organization's local community and in the broader movement, given context and amount of funds proposed
Ability to execute	(D) The organization has the resources, skills, and capacity (staff, volunteer, non-staff) for the plan to succeed	The organization has neither the number nor the amount of resources, skills, and capacity needed for success.	The organization has some relevant staff and volunteer capacity, but may be under-resourced for some initiatives	The organization is well-resourced with the relevant skills and capacity for success
	(E) The organization has a record of success in similar initiatives	The organization has had trouble succeeding in similar initiatives	The organization has not engaged in similar initiatives in the past	The organization has conducted similar initiatives in the past and has been successful in achieving desired results
	(F) The organization's leadership is effective, committed, and relatively stable	Leadership has been unstable (e.g., changing on an annual basis)	Current leadership is committed to the organization; some concerns exist around stability or effectiveness of leadership	Current and past leaders have demonstrated the ability to develop and execute on strategic plans; leadership team has been stable and transitions have been smooth
Efficient use of funds	(G) The funding requested is reasonable given the proposed initiatives	Based on FDC experience and relative to other proposals, the plan contains significant over- or under-budgeting, and/or has not thoroughly accounted for costs	Based on FDC experience and relative to other proposals, the budget is generally consistent with the initiatives; some initiatives are likely to be over- or under-budgeted	Based on FDC experience and relative to other proposals, the budget is reasonable, and rigorous cost projections have been presented
	(H) The organization has a record of using funds efficiently and staying on budget	The organization has a poor record of budgetary management: e.g., consistently and significantly below budget (such as 50% underspending), and drawing on organizational reserves due to overspending	The organization has a middling record of budgetary management; e.g., generally on-budget for its biggest initiatives and cost items, properly recording any over- and under-spending, and with plans in place for improvements	The organization has consistently strong record of financial management; e.g., consistently staying on or below budget (within ~5% of budget), and putting surplus funds towards the organization's and/or movement's goals
Quality of proposed measures of success	(I) Clear indicators of success are outlined in the plan	Initiatives have no associated indicators of success, or the proposed indicators are unrealistic and/or unmeasurable	Each initiative has indicators of success, but at least some indicators are unrealistic or are not clearly connected to the initiative	Each initiative has a realistic and carefully framed set of indicators of success associated with them, which are rigorous and time-bound
	(J) A system is in place to track the proposed metrics	The plan does not include methodology or timing for tracking and collecting metrics to measure progress against proposed indicators of success	A proposal to track progress against indicators exists, but has flawed methodology or unrealistic timelines and initiatives	The plan includes timing and tracking of proposed metrics, and additional initiatives (e.g., surveys, research) to track indicators
	(K) The organization has a feasible system to track the proposed metrics	No systems are in place to track metrics, or staff or volunteer capacity to manage metrics tracking	There is some overall capacity (e.g., staff, volunteers) to track and monitor metrics, but the organization does not clearly have the relevant skills to do this	High-quality systems, and the necessary skills and capacity to manage these systems, are in place
Potential to add knowledge and/or other benefits to the movement	(L) The initiatives, if successful, could be helpful and/or productively adapted elsewhere in the movement	The initiatives would not be relevant to other movement groups	Some initiatives may be relevant to other movement groups	Many initiatives proposed are new or have seldom been tried elsewhere in the movement; if successful, other movement entities would benefit from doing similar initiatives
	(M) The entity is ready to share with the movement the lessons learned from the initiatives	The entity is isolated from other movement groups and rarely shares information; the entity's leaders and members seldom participate in movement-wide functions	The entity is somewhat active within the movement, and is developing more relationships with other movement entities with which it could share lessons learned	The entity has regular communication with other movement entities, is a regular participant in Wikimedia events, and has a track record of sharing with others

This form was created by: