Grants:APG/Proposals/2015-2016 round1/Amical Wikimedia/Staff proposal assessment/fr

L’évaluation de la demande par l’équipe est l’un des éléments utilisés dans le cadre du processus de revue des demandes du Comité de distribution des Fonds (FDC), et est étudiée par par le FDC en préparation des délibérations en présentiel à chaque tour. Le but de cette évaluation est de donner au FDC une vue d’ensemble des avis d’experts de l’équipe FDC sur les demandes de subvention de plan annuel, et inclut 1/ une vue d’ensemble de toutes les demandes de ce tour ; 2/ une évaluation narrative ; 3/ une évaluation qui classe chaque demandeur selon des critères spécifiques sur la conception de programmes, l’efficacité organisationnelle et le budget.

Vue d’ensemble des évaluations

High quality of proposals and annual plans

Les demandes de ce tour sont de bien meilleure qualité. Elles sont accompagnées de plans stratégiques détaillés, et certains de ces plans incluent des méthodes de suivi de progrès sur le long terme. À quelques exception près, les dmeandes ont été davantage ciblées et organisées autour de 2-3 programmes aux objectifs communs. La plupart des demandes sont centrées plus clairement sur les aspects programmatiques, incluant moins de détails opérationnels. Les organisations ont inclus des modèles logiques et d’autres explications claires (eg des études de besoin et des enquêtes communautaires) pour démontrer en quoi leur travail est nécessaire et en quoi les activités proposées sont susceptibles de mener à des résultats.

Nous apprécions les efforts consacrés à ces demandes, et sommes convaincus que la grande qualité de ces demandes vont aider le comité de distribution des fonds (FDC) à comprendre le travail prévu par chaque organisation. De plus, des demandes de qualité avec des objectifs clairs donneront une base solide à l’évaluation des résultats de chaque organisation dans l’année à venir.

Meilleures cibles et narration efficace

L’inclusion des indicateurs globaux (global metrics) dans chaque demande a permis une meilleure compréhension de l’ensemble des demandes du premier tour de subvention de plan annuel, et devrait aider à montrer l’impact des organisations bénéficiant des subventions de plan annuel dans quelques domaines clé.

We know global metrics don’t tell the entire story of an organization’s impact. Organizations have included program-specific metrics that measure results that are specific to their work. We see organizations thinking about ways to understand the quality of the content contributed through their work, as well as ways the content is used by contributors (e.g. use rates for media), or readers (e.g. pageviews). Organizations are doing important work that is not yet reflected in global metrics, such as work in the areas of policy advocacy, building the MediaWiki community, and attempts to address community health. Organizations are striving to find good ways to show their achievements in these areas, and to link these achievements to online impact. We appreciate the different ways organizations are working to show the impact of their programs through metrics that are specific to their contexts, and we know this work will continue to be important moving forward.

Beyond the metrics, organizations are improving in their abilities to effectively tell stories and document lessons learned, both in terms of organizational learning and program learning. We have seen significant improvement from last year, with several organizations becoming systematic in the way they document and share their stories. We appreciate where organizations have highlighted important aspects of their work through effective storytelling.

Santé de la communauté

This year, several organizations are using the lens of community health to structure their work with online contributors. Wikimedia Nederland is doing innovative work around training Arbitration Committee members and administrators. Amical Wikimedia, Wikimedia Nederland, and Wikimedia Israel are addressing online conflicts in their communities. We encourage them to share what they learn about this developing approach with the broader movement and Wikimedia Foundation, as we all stand to gain from these experiences.

Beyond this work, we have seen many organizations implement community surveys to gain a better understanding of the communities they work with and their specific characteristics and challenges. We see movement organizations as clear leaders in this area, since most have direct relationships with their communities and are well-equipped to address challenges that are specific to the geographies and languages they work with. We appreciate that learning how to address challenges in specific Wikimedia communities may be used to improve our understanding of community health in the broader Wikimedia context, which is one reason we see this work as so important.

Many organizations are looking beyond Wikipedia and Commons communities, to focus on other projects. Wikimedia Sverige and Wikimedia CH are both focusing on Wikidata this year, and Wikimedia Israel and Wikimedia Serbia are continuing their strong work on Wiktionary. A number of organizations are focusing on other projects like Wikisource and Wikivoyage, which may offer significant opportunities for improving content and increasing participation and reach.

Travail juridique et politique

Organizations are focusing on policy areas like copyright reform in a consistent and coordinated way. Wikimedia organizations already have a track record of participating in policy discussions. Some organizations, like Wikimedia Sverige, are seen by others as policy experts in their regions.

However, organizations do not yet have a strong track record with influencing policy, as it remains a nascent area of work. Plans focus on processes and mechanics of policy works rather than identifying specific and achievable policy goals.

Outside of policy work, organizations are also working on supporting contributors with legal expertise (e.g. Wikimedia CH and its work on copyright guidance for users on Commons).

In general, we see this policy work as well-aligned with Wikimedia Foundation’s strategies in these areas. We encourage organizations to continue to articulate how these policy goals benefit Wikimedia projects. We acknowledge that it is challenging to measure the outcomes of this work, and encourage further discussion.

Technique

Organizations are continuing to request funding for technical work, with a focus on technology tools. In general, we have seen an improvement in plans for technology work and have seen these projects proposed on a smaller scale. Some organizations already have expertise in these areas, while others are working to build more expertise. Beyond Wikimedia Deutschland’s software development plans, Wikimedia Sverige is working with its community to address bug reporting, and Wikimedia Sverige and Wikimedia CH are building relevant tools to expand the use of Wikipedia (text-to-speech, Kiwix and Wikimini). Wikimedia UK is also continuing work on VLE (Virtual Learning Environment) and QRpedia. Wikimedia Israel is working on ways to support technical contributors to MediaWiki, and Wikimedia Ukraine is planning to support the development of gadgets relevant to its community.

Coordination entre organisations

Les organisations continuent à se coordonner entre elles pour améliorer les pratiques du mouvement et accomplir des buts communs. Wikimedia Nederland, Wikimedia Österreich, Wikimedia Ukraine, et Amical se distinguent par la documentation conséquente des leçons et l’amélioration des pratiques du mouvement, et Wikimedia Israel, Wikimedia Argentina et Wikimedia Sverige contribuent grandement également.

Wikimedia Argentina et Wikimedia Ukraine mènent des collaborations régionales comme Iberocoop et le groupe CEE. Les organisations collaborent tant au delà des langues (eg Amical et Wikimedia Sverige) que au sein d’une langue (eg DACH). Plusieurs organisations soutiennent des groupes et activités internationaux, comme Wikimedia Ukraine (Wiki Loves Earth), Wikimedia Nederland (conférence GLAM), Wikimedia Österreich (conférence Wikisource), Wikimedia Israel (Hackathon). Les organisation en Europe se coordonnent sur les question de lobbying européen.

Au delà des outils technologiques, des organisation comme Wikimedia Österreich, Wikimedia Nederland, Wikimedia Sverige et Wikimedia Israel travaillent à des outils pour le travail de plaidoyer et d’éducation qui peuvent bénéficier au mouvement dans son ensemble. Les organisations sont également en dialogue actif avec l’équipe Community Engagement de la Wikimedia Foundation pour se coordonner avec la Wikimedia Foundation sur l’apprentissage dans le mouvement.

Qualité des budgets

Budgets in this round varied in quality. Some organizations are still struggling to include an appropriate amount of detail in their budget. We believe more conversation is necessary to agree on the level of detail needed in these budgets, and to establish shared norms for good practices in budgeting and financial reporting. WMF plans to provide more in-depth guidance for budgets in the future.

Wikimedia Sverige and Wikimedia Nederland have performed consistently in the area of making quality budgets and offering transparent financial reporting, and we’ve also seen significant improvement from smaller organizations like Amical and Wikimedia Serbia. Several organizations included staff time and costs for each program. This is a good practice, which can help organizations demonstrate how staff are contributing to different program areas, and tracking this closely will also help organizations understand how their resources are being allocated.

In some cases, however, we received budgets that included line large line items without sufficient detail for understanding the costs. This means that we could not draw strong conclusions about how budgets were weighted toward impact in many cases.

Finally, we noticed large amounts allocated to travel expenses, printing and promotional materials, and it is sometimes difficult to link these significant expenses to commensurate impact. Where expenses are significant, we would like to see clearer links to impact.

Pilote : une demande de subvention limitée

The FDC makes general support grants to support programmatic and operational costs. For the first time, the FDC has received a request for a restricted grant, from Wikimedia Deutschland. We are piloting this approach to specifically support Wikimedia Deutschland’s software development focus. In a restricted grant, funds are allocated to specific line items in an organization’s budget.

Vue d’ensemble

Résumé

	Actuel (prévu)	À venir (proposé)	Changement proposé (en pourcentage +/-)
Allocation FDC ou PEG	$92,100.08	$76,281.72	-17%
Budget	$97,845.50	$84,695.15	-13%
Équipe salariée	1	1	0.00%

Vue d’ensemble des forces et inquiétudes

Cette section résume les forces et points d’inquiétudes identifiés par l’équipe, expliqués dans la section narrative de cette évaluation.

Forces

• Focused plan leverages proven organizational expertise (partnerships, community health)
• Strong community and volunteer engagement
• Lean and effective organization, decreasing reliance on Annual Plan Grant funding

Inquiétudes

• Lacks SMART objectives and clear targets for each program

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Contexte and efficacité

This section takes a close look at this organization's context. Here are some questions we will consider:

Environnement

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

• Catalan language projects are growing in content and editors, as is readership. Amical Wikimedia engages a high number of Catalan contributors in the organization as members and volunteers.
• While Amical’s work focuses on Catalan language projects, which are relatively small, language pride in Catalonia and government support may continue to motivate Catalan speakers to engage.
• Amical has impressive opportunities to collaborate with institutions in the cultural sector, which also enables more support from organizations outside of Wikimedia, and expands the reach of the projects.

Performance passée

Will this organization's past performance with program implementation enable this plan to succeed?

• Amical Wikimedia’s proposed plan builds on their past successful work and leverages the organization’s existing expertise in community health and partnership building.
• Amical continues to be a learning organization that uses their experiences to deepen their program work and make their work more effective year over year.
• Amical Wikimedia engages active contributors effectively in their education work as mentors, which provides benefits to long term contributors as well as encouraging contributions of content from students.
• Amical has been doing impressive work with libraries, which is leading to self-sustaining work on the part of many institutions, and which is continuing as part of their education program and as part of their core work in the plans for 2016.
• Amical is one of the highest performers in terms of global metrics so far in 2015, with impressive achievements in the area of content as well as in participation.

Efficacité organisationnelle

Will this organization's overall effectiveness enable this plan to succeed?

• Amical Wikimedia is a lean organization with only 1 FTE, and staff resources are focused on programs. Their unique model, where staff focus on programs and the board takes on administrative work, continues to be very effective in their context.
• Amical Wikimedia documents learning effectively for an organization of their size, and is growing in their ability to measure their work and set targets. Furthermore, Amical Wikimedia excels at applying learning to deepen their program work and improve outcomes. We hope to see continued work in this area as Amical builds their expertise as a learning organization.
• Amical’s leadership (board and staff) are stable, effective, and dedicated, and Amical has a large number of volunteers to implement their programs rather than relying on just a few people.
• Amical has developed strong expertise in partnership-building, and their consistent record of success continues to inspire confidence in Amical’s partners.

Stratégie et programmes

This section takes a close look at this organization's programs. Here are some questions we will consider:

Stratégie

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

• Amical continues to plan work in a way that is consistent with their strategic plan, which has five strategic goals (cohesion, discourse, content, territory, and readers).
• Amical’s plan builds on past work toward Amical’s larger strategic goals.
• Amical’s programs are strongly aligned with Wikimedia movement priorities.

Programmes

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

• In this proposal, Amical Wikimedia clearly articulates the goals and importance of their community work.
• Amical is planning to explore community health by looking at conflicts on Catalan projects.
• This year, Amical Wikimedia included more targets than it has in the past for programs. However, programs lack SMART objectives and targets, making it difficult to understand and articulate what Amical Wikimedia will achieve against what they have achieved this year.
• Targets for the coming year include social media, so the number of individuals involved is high.
• Core projects will add content (images and articles). Amical Wikimedia anticipates that an impressive number of articles (10,000) will be created or improved through their work in 2016.
• Building on their successful work in GLAM, in the year ahead the organization is turning their focus to education. Amical is well-qualified to work in education, because of the organization’s close connections with community members and the commitment to ensuring articles are reviewed for quality.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

• Amical Wikimedia is requesting a grant of $76,281, which is 17% less than last year’s request of $92,100. Amical Wikimedia plans to reduce their overall budget by 13%, thereby reducing their Annual Plan Grant funding overall and also the percentage of APG funding supporting their total budget.
• Amical is predicting to underspend on their budget in the current year, and has adjusted their budget in the planned year to reflect lower expenses. This may indicate that Amical Wikimedia is working to improve their budgeting accuracy.
• With their lean staffing plan, Amical Wikimedia continues to allocate funds to impactful work.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Critère	Évaluation	Description
Conception de programmes
P1. Stratégie	Force	Amical has a sufficient strategic plan in place for an organization of their size, and each of their programs are strongly aligned with this strategy and with movement priorities. Rationale for strategic alignment is clearly explained.
P2. Potential for impact at scale	Force	Amical's planned programs build on their strong track record of programmatic and organizational effectiveness, and is pursuing a self-sustaining partnerships strategy with the potential for scalability within their region. Targets set are ambitious and yet realistic based on past performance. Targets correspond to the amount of funding requested.
P3. Objectives and evaluation methods	Inquiétude	Amical has not provided SMART objectives for their programs, or baselines for understanding its work year over year. Plans for evaluation are not always clear.
P4. Diversité	Neither	Other than including programs that have target audiences that include many female participants, Amical does not appear to be specifically addressing gender or reaching out to currently diverse communities.
Efficacité organisationnelle
O1. Résultats passés	Force majeure	Despite its small size and lean staffing model, Amical is one of the highest-performing APG organizations. While we have expressed concerns around capacity in previous years, Amical has overachieved consistently year over year. Amical's community of contributors and readers continues to grow and thrive under their stewardship.
O2. Learning	Force	Amical is a learning organization that has greatly improved its ability to effectively document and share its work. There is evidence in this proposal that Amical is applying learning to improve its programs.
O3. Improving movement practices	Force	Amical has improved since last year in documenting and sharing. We commend Amical for their case study and encourage them to continue to share their learning more broadly.
O4. Community engagement	Force majeure	Amical has close connections with the communities it partners with, with a high percentage of active editors engaged directly in the chapter's work as members and volunteers. Community members are deeply involved with the organization's planning processes, and community work is prioritized as part of Amical's core approach.
O5. Capacité	Force	Amical has performed well in the past, despite concerns expressed by FDC staff. The organization is now shifting its staff focus to education, which is a newer program that will require more attention, and this approach makes sense.
Budget
B1. Past budgeting and spending	Force	Amical is likely to underspend in this current grant period, but has requested a smaller grant for 2016.
B2. Budget is focused on impact	Force majeure	Amical is a frugal and efficient organization with one full time staff member and a Board who handles administration. Budget aligns to impactful work.

This staff proposal assessment is the work of FDC staff and is submitted by: KLove (WMF) (talk) 20:07, 9 November 2015 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Critère	Description
Conception de programmes
P1. Stratégie	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested.
P3. Méthodes d’évaluation	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversité	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Efficacité organisationnelle
O1. Résultats passés	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O3. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O4. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O5. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O6. Capacité	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.