Grants:APG/Proposals/2013-2014 round2/The Centre for Internet and Society/Staff proposal assessment
Applicants should read this document to familiarise themselves with the assessment criteria; however, it should be edited only by FDC-related staff. The staff will use the document to set out their objective assessment of each funding request from eligible entities – specifically, to determine the extent to which plans align with the mission goals of the Wikimedia movement, and the extent to which the organization is well-placed to execute those plans effectively. The staff will base this report on proposals from entities and input from the community and other stakeholders (including, where required, deeper due-diligence on the organization's previous track record within the Wikimedia movement). The staff will provide completed reports to the FDC, which will then make a recommendation to the Board of Trustees on funding allocations.
- 1 Methodology behind the 2013-2014 Round 2 FDC staff assessments
- 2 Proposal overview
- 3 Financial summary
- 4 Annual plan
- 4.1 Overview of staff assessment
- 4.2 Strategic alignment
- 4.3 Context of the organization and community
- 4.4 Feasibility and risks for the annual plan
- 4.5 Summary of expert opinions and/or community commentary
- 5 Scores, comments and feedback
- 6 Scoring rubric
Methodology behind the 2013-2014 Round 2 FDC staff assessments
The FDC staff assessments provide a snapshot of the staff review of Annual Plan Grant (FDC) proposals, and identify both strengths and concerns with the work proposed.
- We did this through both portfolio analysis - looking across proposals and calibrating across different organizations - as well as assessing the context of each individual organization and its proposal.
- Inputs into the staff assessment process this round include:
- The proposals themselves, with the attached documents (strategic plans, annual plans, detailed budgets, supplementary documents)
- Past and current reports (quarterly progress reports on annual plan grants, other grant reports, annual financial and narrative reports)
- A portfolio view of financials across proposals for both rounds this year Grants:APG/Proposals/2013-2014_round2/Financial_overview and Grants:APG/Proposals/2013-2014_round1/Financial_overview.
- Internal financial, program evaluation, and grants compliance inputs from the WMF Finance, Programs, Legal, and Grantmaking teams.
- Detailed comments and questions collected during the Community Review period, which are also summarized in these staff assessments.
- We used a few key principles for the assessment, across all proposals. These are not new, and are detailed in the narrative and scoring sections of the assessment. They include:
- Impact on Wikimedia projects and the global movement, commensurate with funds requested. We looked for the rationale behind the work organizations do, aligning them to the Wikimedia strategic priorities and goals. We looked particularly for direct and indirect impact of this work on Wikimedia projects; for example, growth in contributors and content donation. Most importantly, we looked at whether the funds requested were justified in the context of current and potential impact of the proposed work.
- Organizational strategy, leadership, and governance. We looked for evidence that an organization had a good understanding of its own context, and the needs, opportunities and challenges of the communities it seeks to serve or partner with. We looked for board, staff and volunteer leadership that is committed, effective and engaged with these communities and practising movement values of openness and transparency.
- Rates of growth and financial sustainability. In this round, no applicants asked for funds within the recommended maximum growth rate in FDC funding of 20%, which is itself a significant rate in many contexts. The minimum growth rate in the FDC/movement funding proposed is 22.7% and the maximum proposed is 216%. Given the challenges faced by some organizations with raising funds externally, there are significant concerns about the long term financial sustainability of this level of dependence on movement resources in areas where other funding may be available.
- Program design, learning, and evaluation. We recognize and appreciate that organizations in this round aim to link their work directly to outcomes on the Wikimedia Projects they are targeting and to work toward aligning their high-level goals across programs; at the same time, this work does not always coincide with clear links between these high-level targets and the direct outcomes of program activities, which may make it difficult to understand the effectiveness of some program activities. We appreciate the good work that organizations have done in this area so far, and urge organizations to continue to work toward making the links between program activities, objectives, and high-level targets clear.
- Inclusion of exceptional organizations. Two organizations were included in this round that required exceptions. The Centre for Internet and Society is the first non-Wikimedia organization to participate in the FDC process, and was made eligible through a resolution by the WMF Board of Trustees. As a result, detailed attention was paid to the financial data presented by CIS to understand the work specific to the Wikimedia movement that will be funded by the FDC, in addition to understanding their broader budget context, and only this work was described in detail in the CIS proposal and analyzed in the staff assessment. Other than this, the process for assessing the CIS proposal has been the same as that for other proposals. The Wikimedia Foundation also participated in this round of the FDC process; at the FDC’s request we did not complete a staff assessment for the Wikimedia Foundation.
Note: We shared our nearly final draft assessments with all FDC applicants a day ahead of publishing them, so that organizations had some lead time for sharing these with their Board and staff. We then took all responses into consideration where possible, making factual changes and clarifying our statements where necessary. We encourage organizations to note all substantive comments and concerns on the discussion pages of these assessments.
|Name of eligible organization||The Centre for Internet and Society|
|Summary staff assessment [substantial concerns: none/some/many]||Some concerns|
|Total amount proposed to the FDC for the upcoming year annual plan in currency requested||18,406,454 INR|
|Estimate of total amount proposed to the FDC for the upcoming year annual plan in US dollars||297,264.23 USD|
|Exchange rate used to calculate estimate in US dollars and date accessed (mm/dd/yy)||0.01615, 1 INR to USD (2014-03-01)|
|Date of proposal submission||2014-04-01|
|Adherence to proposal process [yes/no]||yes|
|Adherence to legal, regulatory, and policy regulations [yes/no]||yes|
|Adherence to US anti-terrorism laws [yes/no]||yes|
A detailed summary of the financial information from all proposals is here: financial overview document.
This financial information is only for CIS's budget for Wikimedia-related work (India A2K), and does not include information on the organization's total budget, which you may find in the financial overview.
|Projected 2013||Proposed 2014||Proposed change (as a +/− %)|
|FDC or WMF funding||US$242,250.00||US$297,264.23||22.7%|
Overview of staff assessment
The Centre for Internet and Society (CIS) is deemed eligible to participate in 2013-2014 Round 2 of the FDC process, but CIS is not a Wikimedia organization, and runs many programs other than its Wikimedia-related work. While CIS has provided numbers describing its overall budget as well as its budget for its Wikimedia activities (India Access to Knowledge (India A2K)), we have focused this assessment on the plans and budget for its Wikimedia activities (India A2K). The scoring rubric, however, looks at some organizational effectiveness criteria which are for the organization as a whole. We reached out to other funders of CIS to understand its track record on performance and reporting in other program areas. CIS is requesting to use FDC funding only for its Wikimedia-related work (India A2K).
What are the top strengths of the plan?
- CIS is the first non-Wikimedia organization participating in the FDC process, and brings valuable expertise and perspective, in research, policy, and other areas, to its Wikimedia work. It has the capacity and experience to execute programs, and to measure and report on results.
- CIS is working in India, a strategic priority for Wikimedia work with high potential for impact and learning on a global scale.
- CIS has provided a detailed and comprehensive plan with clear rationale that is designed to have impact directly on the online Wikimedia Projects. CIS’s plan is supported by needs assessments and individual language community strategies, with links between program objectives and activities.
What are the top concerns with the plan?
- The amount proposed is significant in CIS’s local context.
- CIS has created a complex and ambitious plan that might be a challenge for any organization to execute. Furthermore, CIS’s potential for impact and its ability to engage volunteers may be limited by the sometimes challenging working environment in India’s Wikimedia movement, and the contentious history of WMF’s initiatives in India.
- The long-term impact of CIS’s programs on the priority language communities is still unknown. It is not clear to what extent the communities support CIS’s strategy and whether they will continue to grow as a result of these interventions. CIS’s implementation strategies currently rely intensively on staff time and effort, and may not be scalable in other contexts or sustainable over longer periods of time without significant growth in community leadership and volunteer support.
To what extent does the plan address the strategic priorities of the movement?
Alignment with the strategic priorities
- CIS’s India A2K program is designed to have a targeted impact on Wikimedia Projects and the online communities that contribute to these projects by working specifically with seven language communities to increase both the number of contributors and the amount of quality content donated as well as strengthen these communities for long-term sustainability. These focus areas were identified based on clearly defined rationale, and needs assessments were conducted for each focus area.
- For each program, the alignment between targets, program activities, and strategic priorities is clearly stated with good rationale.
Program objectives and measurable outcomes
- Program groupings in this proposal are logical and share common objectives. Links between targets, objectives, and proposed program activities are mostly clear, and program activities are well-placed in the context of broader program objectives.
- There is an implied link between metrics related to contributions on the Wikimedia Projects and CIS’s activities in its language area work plans, but it may be difficult to isolate CIS-driven results from other factors. A stronger contribution analysis would be possible if results are demonstrated consistently across several of the language area plans; baseline data over past years may be helpful in showing a significant difference between each Wikipedia before and after CIS’s interventions.
- CIS has not provided a strategic plan for its organizational work with this proposal, although it has provided a detailed annual plan for its India A2K (or Wikimedia-related) work.
- CIS has conducted needs assessments for each language plan to define its strategies for work in each area. CIS has also worked with an expert advisor to map the context and potential partnerships of each priority language.
Context of the organization and community
To what extent does the context of the organization (e.g., community participation, fundraising context, sociopolitical environment) enable or inhibit the organization's potential for impact? To what extent does the organization's experience enable or inhibit the organization's potential for impact?
Online focus and potential for global impact commensurate with requested resources
- CIS’s efforts are clearly focused on Indic language Wikipedias and Wikisources and capacity-building in these communities. CIS’s language area strategy with expert support, training programs, and focus on Wikisource as an offline open educational resource, are useful approaches that may contribute to movement learning.
- CIS is operating in a high-potential region where the number of active editors is increasing, and which has been identified as a priority area for movement work. If successful, its programs have the potential to achieve impact on a global scale and to be adapted by others in the movement doing similar work with specific language groups.
- The long-term results of CIS’s work with specific language communities are still unknown; it has been less than a year since A2K adopted the language focus strategy. In this sense, funding these activities may be a risk. On the other hand, CIS’s strategy of building capacity in communities based on the results of needs assessments may result in positive long-term results for the Wikimedia communities it seeks to serve. It is likely that CIS will be able to effectively measure and report on the outcomes of its work; this will be a significant contribution to movement learning.
- CIS’s strategy for its stand-alone projects may not be the most effective for the language communities each project is targeting, given that projects other than Wikipedia (for example, Wikisource or Wiktionary) may be more effective entry-points for working with language communities like Tulu or Santhali.
Financial sustainability and proposed growth
- CIS is proposing 18,406,454 INR (~US$297,264) in FDC funding, which is a growth rate of 22.7% from the previous year’s grant funding from WMF of 15,000,000 INR (~US$242,250) . At the same time, CIS is growing the total budget for India A2K program more significantly from 15,000,000 INR (~US$242,250) total in the current year to 28,565,454 INR total (~US$461,332), or a growth rate of 90%. CIS’s total organization budget is growing significantly as well to 64,392,036 INR (~US$$1,039,931) in the proposed year from 48,630,323 INR (~US$785,380) in the current year, or a growth rate in total organization budget of 32%. In other words, CIS is growing significantly overall, is equally significantly expanding its Wikimedia-related work, and is requesting a growth in Wikimedia movement funding a little over the guardrails recommendation of 20%.
- The amount of funding proposed is high given CIS’s local context; this concern has been echoed by many community members. As CIS has explained, the higher-than-usual salary structure is inherited from the former WMF India Program.
- CIS has strong governance and financial systems, and is requesting support for relatively low overhead costs. CIS does offer the movement some learning about Wikimedia work with allied organizations, and the potential savings in using the existing infrastructure of well-established organizations.
- CIS has been successful in raising funds externally and engaging funding partners globally in its work. CIS plans to raise funds from other partners to support its non-Wikimedia work and also plans to mobilize in-kind support and funds from other sources amounting to 10,159,000 INR (~US$164,067.85) of the India A2K budget. Funds requested from the FDC will go directly to supporting Wikimedia work.
- The proposed use of funds is consistent with the plan, with the highest-cost program being the language area work plans at 9,566,440 INR (~154,498), with significant funds within this program going to staff (3,819,840 INR, or ~US$61,690.42), travel (1,850,000 INR, or ~US$29,878), and events (2,200,000 INR, or ~US$35,530). Of total funds marked for India A2K, about 8,118,540 INR (~US$$131,114) or 28% go to staff costs. Total staff travel for this India A2K is also significant, with 2,970,000 INR (~US$47,966) marked for travel expenses (funded from both FDC and from other sources). At 2,320,000 INR (~US$37,468), regranting is also significant.
Feasibility and risks for the annual plan
How feasible is the annual plan? Specifically:
- To what extent are the budget, staffing, availability of volunteers, and other resources realistic for achieving the anticipated outcomes?
- Are the timelines realistic for the proposed initiatives and associated activities?
- Are there acknowledged and/or unacknowledged risks in the proposal?
- CIS is a respected global leader in its field of expertise (Internet and Society), which is closely related to Wikimedia’s work. CIS has high credibility with other funders in successfully executing complex projects and reporting effectively on its work, and has also been communicating well with WMF during the course of its current grant by providing regular detailed information about its funded activities. CIS has the experience and systems in place to handle working in a complex regulatory environment like India.
- CIS has a plan in place to measure the results of these programs and, based on its expertise, it is likely that CIS will provide quality evaluation of its Wikimedia work. CIS is motivated by the potential for learning that its Wikimedia work offers, even beyond the Wikimedia movement.
- As a non-Wikimedia organization, CIS does pursue a broader mission with a scope broader than most Wikimedia organizations, and this presents an opportunity for the organization to leverage its competencies and resources to achieve Wikimedia work at a lower cost to the movement, as well as offer a useful external perspective.
- CIS has invited the community to engage in its work and its planning processes (for example, by sharing its language area work plans with the priority editing communities), and has provided prompt and informative replies during the proposal review process. While CIS has struggled to keep its processes on wiki in the past, it is actively improving in this area. CIS has been forthright about sharing and discussing its mistakes.
- CIS has a history of actively supporting other movement groups in India, including by incubating the WMIN chapter and by serving as a fiscal sponsor to other WMF grantees in India.
- CIS is proposing to grow its Wikimedia program activity (India A2K) by 90%, which is an ambitious plan that includes many activities and could be a challenge to execute effectively; however, concerns are somewhat mitigated by past performance on similar complex initiatives as reported by CIS’s other funders. The increase in FDC funding requested for this program (India A2K) is 23%, since CIS also plans to mobilize in-kind support and funds from other sources to support this growth; although this is an ambitious fundraising goal, CIS does have a track record of past success raising funds.
- CIS’s programs rely on intense and sustained efforts from staff, and programs may not be easily scalable without significant community participation and leadership.
- CIS faces a sometimes contentious working environment in the Wikimedia movement in India, which may inhibit its potential to achieve impact. While CIS supported the establishment of WMIN well before it sought funds from the Wikimedia movement and the Indian context has the potential and scope to absorb the efforts of multiple complementing groups and organizations, the coordination and complementarity of CIS’s work with Wikimedia India (WMIN) is unclear despite ongoing efforts to establish better coordination protocols. CIS, WMIN, and the language communities, may need to develop reasonable coordination strategies in such a diverse, complex, and challenging context.
- CIS has made great efforts to increase its community engagement, but adapting to movement culture as a non-Wikimedia organization will be an ongoing challenge. The value of CIS’s work in particular, and more broadly the value of supporting a non-Wikimedia organization in India, has been challenged by WMIN and other members of the Wikimedia community in India. It is not clear if CIS will be able to effectively engage the volunteer community in India, which CIS will need to do in order for the plan to succeed.
Summary of expert opinions and/or community commentary
Summary of the opinions of subject-matter experts on the plan
No expert commentary or opinion was requested for this plan.
Summary of community commentary on the plan
For detailed comments from the community, the FDC and FDC staff, as well as responses from the organizations themselves, please visit the proposal form discussion page. The summary offers a brief overview of the questions or comments from community members, and is not an endorsement of the views expressed on the proposal form discussion page.
Discussion about CIS’s proposal to the FDC is been very active, and CIS is responsive to questions and comments throughout the community review period. Several themes emerge through the questions and comments on the discussion page:
- Several express concerns about the high cost of this proposal in a local (India) context, and particularly, concerns about high administrative costs (including rent), travel costs, and foremost, high salary costs inherited through the transition from the WMF India Program. Beyond staff costs, some express concerns about using paid staff to perform work that could be performed by volunteers, and about the sustainability behind strategies like using anchor staff to support language communities. In response, CIS provided details about staff salaries and descriptions of the roles of anchor staff.
- Many community members and members of the EC of WMIN express concerns about the role of CIS in the Wikimedia movement in India, and particularly, about the relationship between CIS and WMIN, which some perceive as competitive. Some community members have concerns about funding any external organization to do catalyst work on any Wikipedia, while some community members have specific concerns about the process for selecting CIS as a partner and, particularly, about a lack of transparency in that project and in CIS’s current work. Both WMF and CIS respond to concerns in detail with information about the transition of the WMF India Program’s work to CIS, about WMF’s current strategy in India, and about CIS’s current work on its grant from WMF. Some community members have concerns that CIS’s relationship with WMIN could inhibit its ability to do its work effectively, and some express concerns about CIS’s ability to mobilize volunteers in India.
- Several community members discuss CIS’s long term strategies for continuing its work in India, or for handing over its work to the WMIN chapter or future language-based user groups in India. Questions are raised about the sustainability of a high-touch approach in working with language communities. Several community members ask questions about the size of the language communities and Wikipedias involved with the work plans.
- Community members express concerns about the metrics used by CIS in this proposal to understand the results of this work; in some cases it could be difficult to show the difference between the results of an intervention and a community’s natural trends, or to attribute results directly to the work of CIS.
- Community members request clarification about the proposed 10% institutional development fee (a portion of the budget to be spent freely by the organization on charitable activities not specified in the grant); while a normal practice in the grantmaking world, it is unusual in the Wikimedia world.
- In response to a request, FDC staff clarifies the eligibility status of CIS: CIS is deemed eligible as a result of a resolution voted on by the WMF Board of Trustees.
- Community members raise specific concerns and questions about specific programs, including the Wikidata India Marathon, Gujarati Wikisource, work on Odia Wikipedia, Nurturing Mediawiki and Tech Talent in Indic Communities, Urdu WEP at MANUU, and the strategies for selecting particular language work areas and forming plans for those work areas.
- Community members, some likely sockpuppets, raise several concerns about potential conflicts of interest within CIS or WMF, which are addressed.
Scores, comments and feedback
Scores provide only a preliminary assessment for consideration by the FDC. The rubric for scoring is below.
|Dimensions||Criteria||Score (1–5)||Comments and feedback|
|Potential for impact against strategic priorities||(A) The plan addresses the Wikimedia movement strategic priorities of the Wikimedia movement (infrastructure, participation, quality, reach, and innovation)||4.00||The plan directly addresses several movement priorities, including Increasing participation and Encouraging innovation. The rationale presented in this proposal is clear.|
|(B) The initiatives in the plan have the potential to lead to movement strategic priorities, if executed well||4.00||Given the potential for impact in this region, initiatives if successful will contribute to success on a global scale; initiatives can also be adapted or learning applied to work in other areas of the movement.|
|(C) The initiatives in the plan have the potential to lead to significant impact, given context and amount of funds proposed||3.50||The initiatives could have significant impact at scale, for both local communities and the broader movement. At the same time, the amount of funding proposed is relatively significant in the local context, and a challenging working environment may inhibit this plan's potential for impact.|
|Ability to execute||(D) The organization has the resources, skills, and capacity (staff, volunteer, non-staff) for the plan to succeed||3.00||CIS has demonstrated capacity in key competency areas through its track record with past work, including with other funders, but serious concerns exist regarding its ability to effectively engage volunteers. This ambitious plan may also be a challenge to execute even for an organization with demonstrated capacity.|
|(E) The organization has a record of success in similar initiatives||3.50||CIS has an excellent track record in its past work with other funders, but it is still new to working within the Wikimedia context and has not yet demonstrated a definitive track record for success in this context.|
|(F) The organization's leadership is effective, committed, and relatively stable||4.00||This organization has capable and visionary leadership, though the transition from the WMF India Program to the CIS partnership grant has been a significant challenge.|
|Efficient use of funds||(G) The funding requested is reasonable given the proposed initiatives||3.50||While this proposal could have impact, the funding requested is significant in a local context, even while some of the costs (salaries, in particular) are a legacy from the WMF India program.|
|(H) The organization has a record of using funds efficiently and staying on budget||4.00||CIS has a solid track record in capable budgeting and has presented financial information in this proposal clearly; at the same time, this program is expensive.|
|Quality of proposed measures of success||(I) Clear indicators of success are outlined in the plan||4.00||Based on the information in this proposal, it will be possible to understand and analyse if CIS's activities are successful or not; it will be possible to draw connections between this organization's activities and impact on the online projects.|
|(J) A plan is in place to track the proposed metrics||4.00||A clear plan is in place to track the direct outcomes of these activities and the impact of these activities on the online projects.|
|(K) The organization has a feasible plan in place to track the proposed metrics||4.00||This organization clearly has the capacity to track indicators, based on its track record of success in understanding the impact of its programs.|
|Potential to add knowledge and/or other benefits to the movement||(L) The initiatives in the plan, if successful, could be helpful and/or productively adapted elsewhere in the movement||4.50||If successful, the proposed initiatives could be adapted to other contexts and lead to significant impact on the online projects.|
|(M) The organization is ready to share with the movement the lessons learned from the initiatives||4.00||CIS has the capacity to understand the results of its work and to document it in a way that will make it possible for other organizations to adapt its work to their contexts. While it has struggled with doing work on wiki in the past, it is working actively to be more communicative and is motivated by sharing its learnings with the broader community.|
|Dimensions||Evaluation criteria||1 = Weak or no alignment with criterion||3 = Moderate alignment with criterion||5 = Strong alignment with criterion|
|Potential for impact against strategic priorities||(A) The plan addresses the Wikimedia movement strategic priorities||The plan will not directly address any strategic priorities||The plan will partially address strategic priorities or other goals that are closely related to the strategic priorities||The plan will directly address several strategic priorities|
|(B) The initiatives have the potential to lead to the strategic priorities, if executed well||The initiatives have no clear connection to strategic priorities||Some or all of the initiatives have an unclear relationship to strategic priorities||The initiatives have been shown (by the organization or the movement) to lead to strategic priorities|
|(C) The initiatives have the potential to lead to significant impact, given context and amount of funds proposed||The initiatives will have marginal impact in the local community and the broader Wikimedia community, given context and amount of funds proposed||The initiatives proposed may have a significant impact in the local community, given context and amount of funds proposed||The initiatives could have significant impact at scale, both in the organization's local community and in the broader movement, given context and amount of funds proposed|
|Ability to execute||(D) The organization has the resources, skills, and capacity (staff, volunteer, non-staff) for the plan to succeed||The organization has neither the number nor the amount of resources, skills, and capacity needed for success.||The organization has some relevant staff and volunteer capacity, but may be under-resourced for some initiatives||The organization is well-resourced with the relevant skills and capacity for success|
|(E) The organization has a record of success in similar initiatives||The organization has had trouble succeeding in similar initiatives||The organization has not engaged in similar initiatives in the past||The organization has conducted similar initiatives in the past and has been successful in achieving desired results|
|(F) The organization's leadership is effective, committed, and relatively stable||Leadership has been unstable (e.g., changing on an annual basis)||Current leadership is committed to the organization; some concerns exist around stability or effectiveness of leadership||Current and past leaders have demonstrated the ability to develop and execute on strategic plans; leadership team has been stable and transitions have been smooth|
|Efficient use of funds||(G) The funding requested is reasonable given the proposed initiatives||Based on FDC experience and relative to other proposals, the plan contains significant over- or under-budgeting, and/or has not thoroughly accounted for costs||Based on FDC experience and relative to other proposals, the budget is generally consistent with the initiatives; some initiatives are likely to be over- or under-budgeted||Based on FDC experience and relative to other proposals, the budget is reasonable, and rigorous cost projections have been presented|
|(H) The organization has a record of using funds efficiently and staying on budget||The organization has a poor record of budgetary management: e.g., consistently and significantly below budget (such as 50% underspending), and drawing on organizational reserves due to overspending||The organization has a middling record of budgetary management; e.g., generally on-budget for its biggest initiatives and cost items, properly recording any over- and under-spending, and with plans in place for improvements||The organization has consistently strong record of financial management; e.g., consistently staying on or below budget (within ~5% of budget), and putting surplus funds towards the organization's and/or movement's goals|
|Quality of proposed measures of success||(I) Clear indicators of success are outlined in the plan||Initiatives have no associated indicators of success, or the proposed indicators are unrealistic and/or unmeasurable||Each initiative has indicators of success, but at least some indicators are unrealistic or are not clearly connected to the initiative||Each initiative has a realistic and carefully framed set of indicators of success associated with them, which are rigorous and time-bound|
|(J) A system is in place to track the proposed metrics||The plan does not include methodology or timing for tracking and collecting metrics to measure progress against proposed indicators of success||A proposal to track progress against indicators exists, but has flawed methodology or unrealistic timelines and initiatives||The plan includes timing and tracking of proposed metrics, and additional initiatives (e.g., surveys, research) to track indicators|
|(K) The organization has a feasible system to track the proposed metrics||No systems are in place to track metrics, or staff or volunteer capacity to manage metrics tracking||There is some overall capacity (e.g., staff, volunteers) to track and monitor metrics, but the organization does not clearly have the relevant skills to do this||High-quality systems, and the necessary skills and capacity to manage these systems, are in place|
|Potential to add knowledge and/or other benefits to the movement||(L) The initiatives, if successful, could be helpful and/or productively adapted elsewhere in the movement||The initiatives would not be relevant to other movement groups||Some initiatives may be relevant to other movement groups||Many initiatives proposed are new or have seldom been tried elsewhere in the movement; if successful, other movement entities would benefit from doing similar initiatives|
|(M) The entity is ready to share with the movement the lessons learned from the initiatives||The entity is isolated from other movement groups and rarely shares information; the entity's leaders and members seldom participate in movement-wide functions||The entity is somewhat active within the movement, and is developing more relationships with other movement entities with which it could share lessons learned||The entity has regular communication with other movement entities, is a regular participant in Wikimedia events, and has a track record of sharing with others|