Grants:APG/Proposals/2017-2018 round 1/Wiki Education Foundation/Progress report form
Purpose of the report
This form is for organizations receiving Annual Plan Grants to report on their progress after completing the first 6 months of their grants. The time period covered in this form will be the first 6 months of each grant (e.g. 1 January - 30 June of the current year). This form includes four sections, addressing grant metrics, program stories, financial information, and compliance. Please contact APG/FDC staff if you have questions about this form, or concerns submitting it by the deadline. After submitting the form, organizations will also meet with APG staff to discuss their progress.
Metrics and results overview - all programs
We are trying to understand the overall outcomes of the work being funded across our grantees' programs. Please use the table below to let us know how your programs contributed to the Grant Metrics. We understand not all Grant or grantee-defined Metrics will be relevant for all programs, so feel free to put "0" where necessary. For each program include the following table and
- Next to each required metric, list the outcome/results achieved for all of your programs included in your proposal.
- Where necessary, explain the context behind your outcome.
- In addition to the Global Metrics as measures of success for your programs, there is another table format in which you may report on any OTHER relevant measures of your programs success
For more information and a sample, see Grant Metrics.
- Classroom Program
|1. number of total participants||8,295||We are already halfway to our annual goal, putting us on track to meet it if not exceed it.|
|2. number of newly registered users||7,998||We are more than halfway to our annual goal and anticipate easily reaching it.|
|3. number of content pages created or improved, across all Wikimedia projects||8,920||Due to our focus on quality courses, we've accepted fewer small projects classes that made small edits to a lot of articles, so we're a bit behind on this goal.|
|4. number of words added to the article namespace||6.7 million||We are more than half way to the quantity goal, so we should be able to meet this if not exceed it.|
|5. number of articles that have at least a 10-point improvement ORES-based quality prediction score||2,231||We're about 40% of the way to meeting this goal, and it's possible that we will still reach it through our Fall 2018 courses, but we also may fall just short of it.|
- Visiting Scholars Program
|1. number of total participants||15||We're supporting the anticipated number of Visiting Scholars.|
|2. number of newly registered users||0||All program participants for this program are existing editors.|
|3. number of content pages created or improved, across all Wikimedia projects||344||We're halfway to the pages edited goal, thanks mostly to User:Rosiestep's work.|
|4. number of words added to the article namespace||224,000||We expect to fall short of our overall goal, given the level of productivity we've seen so far from Visiting Scholars.|
|5. number of articles that have at least a 10-point improvement ORES-based quality prediction score||93||We expect to fall well short of this goal, especially given we're reducing emphasis on this program.|
- Wikipedia Fellows Program
|1. number of total participants||42||We exceeded our goal for the pilot, and given we are putting significant emphasis on scaling this program up, expect to well exceed our overall goal. 9 fellows participated in the pilot; the others just got started in new cohorts in June.|
|2. number of newly registered users||36||We exceeded our goal for the pilot, and given we are putting significant emphasis on scaling this program up, expect to well exceed our overall goal.|
|3. number of content pages created or improved, across all Wikimedia projects||142||We exceeded our goal for the pilot, and given we are putting significant emphasis on scaling this program up, expect to well exceed our overall goal.|
|4. number of words added to the article namespace||76,000||We've already exceeded our annual goal and expect to grow this number even further as we scale this program up.|
|5. number of articles that have at least a 10-point improvement ORES-based quality prediction score||21||We fell short on this goal for the pilot, which accounted for 8 of the 21, as editing patterns were different than anticipated. See the section on this program for a longer explanation.|
- Total of all programs
|1. number of total participants||8,352||We're more than halfway to our annual goal and anticipate meeting it.|
|2. number of newly registered users||8,034||We're more than halfway to our annual goal and anticipate meeting it.|
|3. number of content pages created or improved, across all Wikimedia projects||9,406||We're a little under halfway to our goal, and we expect to come in right around our goal.|
|4. number of words added to the article namespace||7.0 million||We're more than halfway to our annual goal and anticipate meeting it.|
|5. number of articles that have at least a 10-point improvement ORES-based quality prediction score||2,345||We've made progress but are a little behind on this goal; we still think it's possible to reach but it will be a challenge.|
Telling your program stories - all programs
The first half of 2018 has been a period of great impact for Wiki Education. As the academic term within the U.S. and Canada education system changes at the end of the calendar year, January 2018 marked the beginning of the spring 2018 term for our Classroom Program, where we supported the largest number of courses we've ever supported. Our Visiting Scholars program continued as expected. We also kicked off our pilot of our new Wikipedia Fellows program, in which we taught academic experts how to edit Wikipedia in their disciplines. In this report, we will detail our activities and progress toward goals for each program.
On the fundraising front, the beginning of the year proved exceptional. In addition to the FDC Annual Plan Grant, we received the pledged matching gift from the Stanton Foundation. We also received an unrestricted $500,000 gift from the Pineapple Fund in January. Finally, we began offering what we term "topic sponsorships" in which an individual or organization interested in a particular topic area can sponsor 10 classes of students working to improve that discipline each year. Craig Newmark Philanthropies provided the first sponsorship, in the topic area of Women's Studies, in February.
In late January, our board met in San Francisco, and a major outcome of the meeting was a confirmation of Wiki Education's new strategic direction for 2018–21. Our full strategic plan focuses on the three strategic goals of Equity, Quality, and Reach. More about our new strategy is outlined below.
In February, our staff met for our biannual All Staff meeting in San Francisco. Our February All Staff is traditionally when we begin our annual planning process for next fiscal year, and this year was no different, but the backdrop of the new strategic plan created extra enthusiasm among staff. The many discussions we had around the future strategic direction of Wiki Education provided us with enough ideas and food for thought with regard to next year's plan. Also, as part of the All Staff meeting, we engaged in mapping out our organization's capacity and robustness in different areas. Our discussions around these topics weren't just an informative exercise for everybody on staff, the results also added to a global capacity map of the Wikimedia universe, and we found the process very useful for our staff.
Since our proposal was submitted, we had a few planned staffing changes. Classroom Program Manager Helaine Blumenthal went on maternity leave between December 2017 and April 2018, and Community Engagement Manager Ryan McGrady (who runs our Visiting Scholars and Wikipedia Fellows programs) went on an educational leave from April to October 2018. To fill in for these two positions during their scheduled leaves, we hired Will Kent as an interim program manager. Will adeptly covered for Helaine while she was out, and is now focused on growing the Fellows program in Ryan's absence. In an additional scheduled leave, Director of Programs and Deputy Director LiAnna Davis will be out on maternity leave between July and October 2018.
Changes we've made based on FDC feedback
In the FDC's funding allocation recommendation, the group offered us some feedback that we took into account in the first six months of our grant term. We'd like to take this opportunity to highlight some specific work we've done and plans we've made to address comments from the FDC, in a few key areas.
- Strategy, including content gaps and knowledge equity
As we outlined in our proposal, our last strategic direction had expired in 2017, and we had decided to postpone the strategic planning process to the end of 2017 so that we could best align it with the Wikimedia Movement Strategy process. As mentioned above, we have now published our new three-year strategy, focusing on three key areas: Equity, Quality and Reach. Our strategic goals are: (1) Increase knowledge equity by focusing on content and communities that are under-represented on Wikipedia and Wikidata; (2) Provide people who seek knowledge online with accurate information in topic areas that are underdeveloped; and (3) Reach large audiences with free knowledge by making Wikipedia and Wikidata more complete.
The Wikimedia Movement Strategic Direction also focuses on knowledge equity, and we are proud to be able to reflect that in our own strategy, not only in the area of equity but also in focusing our quality and reach efforts on underdeveloped topic areas. (More on this later.) While we had explicitly excluded our work on the Program & Events Dashboard from our funding request in 2018 at the request of the Wikimedia Foundation, we have included it as a strategic objective in our new strategy. We see this as working both toward the Wikimedia Movement Strategy's knowledge as service and knowledge equity — providing tools and resources for other Wikimedia movement organizations and individuals to be able to run and track their own programs and events, especially those related to improving knowledge equity.
With this strategy, we are also explicitly committing to moving beyond Wikipedia to include programs to improve the representation, accuracy, and completeness of Wikidata as well. In our context of the United States and Canada, more and more people are relying on devices like Alexa, Siri, or Google Home to discover information, and much of this information is coming from Wikidata. While the bulk of our work will still be on improving Wikipedia, we want to begin programs addressing the knowledge equity issues and content development opportunities on Wikidata as well.
The FDC's assessment of our work called out a perceived lack of intentionality in our work on identifying content gaps and knowledge equity. We agree that to date, our work on identifying content gaps has mostly relied on the expertise of our on-staff Wikipedia editors — certainly an anecdotal and not rigorous approach. We are eager to fix this, as we think better identification of content gaps on the English Wikipedia could benefit not only Wiki Education in our programs, but also other organizations working on the English Wikipedia as well as individual editors who are seeking to fill content gaps. In our new strategy, objective 2.1 calls this out: "Develop and implement a method for assessing Wikipedia content quality on a regular basis, so that we and others can easily identify and target areas that need improvement." While we're eager to get started on this, we are trying to carefully assess our own capacity and not overcommit ourselves such that we wouldn't be able to do a good job with this objective; thus, we intend to begin this work in 2019.
In terms of knowledge equity, Wiki Education is limited in the sense that unlike other organizations funded through the FDC process who work within a geographic region only, we work within a specific subset — higher education — of a large geographic region. Despite the wide variety of languages spoken by U.S. residents, the language of instruction within higher education in the U.S. is English, making it a challenge for us to do significant programmatic work on other language Wikipedias. However, we firmly believe there are significant intersectional knowledge equity gaps on the English Wikipedia, and that we as a movement should spend significant attention on those, since the English Wikipedia gets almost half the pageviews of all language Wikipedias and is often used as the starting texts for programs that translate into other language Wikipedias. In particular, the English Wikiepdia's coverage of topics related to race, gender, and sexuality, and especially the intersectionality of those three, leaves much be to be improved upon. Our new strategic objective 1.1 calls this out: "Target content areas that are underdeveloped in order to reduce systemic bias on Wikipedia and Wikidata." Wiki Education's Classroom Program recruitment work specifically targets courses that examine the intersections of race, gender, and sexuality, and strive to improve Wikipedia's coverage of these topics. We also see significant potential for this work in the Wikipedia Fellows Program as well, and we specifically call out our intentionality with regard to knowledge equity in the sections on those programs in this report.
Improving the diversity of contributors of that knowledge is also important. The FDC identifies our work in gender, which we highlighted because we have demographic data from the Wikipedia editing community to which we can compare it. But we also ask whether students identify as "non-white"; in spring 2018, our results showed 30% of participants in our program identify as non-white. We are unaware of any such study of the English Wikipedia editing community to which we can compare this number, but we think this is likely a higher percentage of non-white editors than the general editing population. As outlined in strategic objective 1.2 of our new strategy ("Attract content contributors from a wide variety of backgrounds, such as gender identity, ethnicity, sexual orientation, age, income, religion, etc."), we are committed to intentionally bringing a diverse group of contributors to Wikimedia projects. In the remainder of this year, we are working with an experienced consultant to create outreach materials specifically aimed at improving our approach to equity.
- Financial costs
The FDC called out our "high financial cost model", which we understand from conversations was centered around the cost of the Visiting Scholars Program as well as the desire to offset some of our Classroom Program costs by asking universities to pay for our support, in a fee-for-service model.
Due to the FDC comments about its high costs, our own challenges in scaling the Visiting Scholars Program, and the exciting potential of the Wikipedia Fellows Program based on the pilot we ran this spring, we have decided to significantly cut the Visiting Scholars Program for the remainder of this year. We have moved the program into maintenance mode, and will spend significantly less on it than planned in our proposal. More information on this is available in the section on the Visiting Scholars Program, below.
As referenced in our strategic plan, we are actively moving forward with developing a fee-for-service model to support sustainability of our Classroom Program. We have nearly secured funding for a consulting firm who will help us explore various potential fee-for-service models during the remainder of 2018. In particular, these consultants will help us analyze the market potential for Wiki Education services, conduct a customer voice analysis to help us determine specific customer needs and opportunities, and work with us to develop a robust business plan that we can pilot in 2019. As staff at Wiki Education comes from academic and nonprofit backgrounds, we believe we do not have the expertise in-house to develop a business plan for a fee-for-service model. We have identified a consulting firm to work with that has supported other nonprofits in the open education space in the United States to successfully create their fee-for-service models, and we believe they are best positioned to guide us in this respect as well.
The first phase of the consultation, the market analysis, will include a high level overview of the target market and a synthesis of systemic and cyclical factors impacting the market. This analysis will result in the development of a competitive landscape and positioning analysis, or market map. The market map will help us identify potential gaps or weaknesses in the market among competitors and also deliver insights into the sales cycles and go-to-market models of potentially competing products. This phase of the work will also provide information about preliminary market entry opportunities, including implications and hypotheses for the evolution of our current free offerings toward subscription-based offerings.
The second phase of the consultation, the voice of the customer analysis, will help verify the findings of the first phase through large-scale primary research. This phase will define specific questions and offering concepts to be further tested through interview-based and large-scale quantitative research with faculty and administrators. This research will result in further validation of customer-level tailwinds and headwinds facing our services and go-to-market efforts. It will also result in forecasted revenue growth models and investment assumptions within the market based on customer insights. Most importantly, this second phase will determine the overall viability of a fee-for-service business model for Wiki Education.
If the first two phases of this consultation reveal a viable market and potential strategy, we will move into the third phase: business plan development. In this phase, data from the first two phases will be synthesized and applied to a set of recommended product and go-to-market strategies. We will also determine what gaps may exist for us in the market and the level of effort and resources necessary to address them. The key result of this phase will be a business plan that includes revenue scenarios, assumptions, investment risks, notable product requirements, pricing models, required resources for product development, and an anticipated go-to-market process.
We expect to complete all three phases of this project in the next six months, and will report on our progress in our final report to the FDC.
|Measure of success||Goal (2018)||Progress to date||% completed||Notes|
|New discipline-specific handouts developed||4||3||75%||We published discipline specific handouts on science communications, cultural anthropology, and LGBT studies. We expect to publish at least two more in 2018, exceeding our goal.|
|New academic association partnership agreements signed||2||1||50%||We have made progress in discussions with associations, anticipate completing this within the year.|
|Academic association conferences attended to recruit instructors||15||7||47%||We attended conferences for the Linguistic Society of America, American Astronomical Society, Ocean Sciences Meeting, African American Studies Association, National Women's Studies Association, Midwest Political Science Association, and Association for Psychological Sciences. We have an additional five confirmed conferences scheduled in 2018, and anticipate confirming several additional conferences in the coming months.|
|Number of students||16,000||8295||51.8%||We are more than halfway to our goal, and anticipate reaching it by the end of the year.|
|Percentage of students who identify as women||60%||51%||n/a||This is a lower proportion of women than expected based on previous terms. We have not yet identified the root cause(s).|
|Number of articles edited||19,200||8920||46%||We anticipate falling a bit short on this goal because of our decision to focus on quality of courses (see below).|
|Amount of content added||11.2 million words||6.73 million||60%||We expect to meet this goal easily; our focus on quality enabled us to add more content to Wikipedia.|
|Number of articles edited with at least 10 ORES point improvement||5,000||2231||45%||We're a little behind on this goal; we think it's possible to make it up in the Fall 2018 academic term, but we may fall just short of it.|
|Instructor satisfaction rate (percentage who indicate interest in teaching with Wikipedia again)||98%||97%||n/a|
|New student learning outcome study completed||1||0||0%||We sadly had to cut this project due to the reduction in funding from our initial request.|
What we've done
The academic calendar in the United States and Canada education system matches up well to the annual reporting cycle; the "spring term" runs from January to June, and the "fall term" runs from September to December. That means at this halfway point in our FDC grant, we're able to report on outcomes from our spring 2018 term.
Spring 2018 was our best term to date, in terms of impact to Wikipedia, meeting our goals, and success in scaling our program. We're easily on track to meet or exceed most of our Classroom Program goals for the year, thanks to the success this spring. Spring terms traditionally have been better for us, in part because universities on the quarter system actually have two quarters (one from January to March and one from April to June) encompassed within what we define as the "spring term", meaning there are more potential courses for us to work with each spring than there are each fall.
In the onboarding process for courses, we set internal goals for both "returning" courses — instructors who have taught with us in a past term and are retained to teach with Wikipedia again — and "new" courses — those taught by first-time instructors in our program. We exceeded our internal goals for both new and returning courses this term, supporting 180 new courses and 220 returning instructors. Our new courses showed the importance of our ongoing work, both on general visibility for teaching with Wikipedia (which accounted for 39% of all of our new courses in spring 2018) and through our targeted recruitment through our outreach (which accounted for 28% of new courses), encouraging existing participants to refer colleagues (which accounted for 18% of new courses), and partnerships with academic associations (which accounted for 15% of our new courses).
Maintaining the ongoing stream of new instructors in our program requires engagement at academic conferences, and in 2018 so far, we've focused our recruitment efforts on our two core initiatives: Future of Facts, where we target politically relevant topic areas, and Communicating Science, to further the impact of our 2016 Year of Science initiative. In support of these two initiatives, we attended seven academic association conferences to recruit new faculty into the Classroom Program. We have additional confirmations from five other conferences in 2018, and we expect to add a handful of additional conferences to our list before the end of the year. These conferences are crucial for us to maintain a steady stream of new faculty interested in teaching with Wikipedia who help us grow our impact.
We also specifically targeted areas this spring related to knowledge equity, to align with the Wikimedia Movement Strategic Direction. Staff attended both the National Association of African American Studies & Affiliates Joint National Conference and the National Women's Studies Association Regional Meeting to recruit professors teaching topic areas at the intersectionality of race, gender, and sexuality. The new classes that will join our program from these conferences will supplement our existing work improving Wikipedia's coverage about topics like African-American history, indigenous people in the United States, women in STEM, South American colonial history, African-American women, African-American theatre, and antisemitism, just to name a few.
It's not just recruiting these courses, though, that's important — we also need to support the students in their editing work. Our existing women's studies and biographies handouts have been helpful in supporting equity-related student work to date; in 2018, we added a LGBT+ subject-specific brochure to our offerings. In response to student needs, we also published new discipline-specific brochures in science communication and cultural anthropology.
By most metrics, our spring 2018 term was a success. We broke new records in the number of courses and students we supported, the number of articles created, and the number of articles improved, while maintaining a very high level of instructor satisfaction. We've maintained a good relationship with the English Wikipedia community, and we've continued refining our processes and training materials to ensure that student editors put minimal cleanup burden on the community. Based on survey results so far, we saw an unexpected drop in the proportion of women in the courses we supported compared (51.3%) with previous terms; we will investigate possible causes for this once the spring 2018 survey results are all in, if that result holds.
What's worked well
- Quality improvement
In spring 2018, we focused our efforts for retention on our highest-quality courses. In our ongoing efforts to scale our impact, we sought out ways to give each course participating in our program a numerical ranking factoring in the amount of content added per student, the general quality of student work as assessed by our Wikipedia Content Experts on staff, and the staff time that went into that particular course.
Courses that produced a high volume of high quality work with little staff intervention score the highest. We focused our retention efforts for spring 2018 on these high-rated courses. We encouraged mid-rated courses to participate again as well. For courses that scored low on our scale, we evaluated them on a case-by-case basis: Was the problem that led to their low rating an anomaly? Is there something we could do (e.g., have a conversation with the instructor) that could help the course move to be high rated in the future? Or do we think this instructor or course simply isn't a good fit for a Wikipedia assignment? We welcomed back previously low-rated courses if we felt like they had the possibility to potentially contribute positively in the spring 2018 term, and discouraged others from teaching with Wikipedia again.
As part of this quality push, we stopped taking on new "small assignments" courses. These courses tended to have large numbers of students who each added only a sentence or two to Wikipedia. We found with these courses, many students didn't take the time to fully understand Wikipedia's sourcing requirements. Our detailed online trainings take a significant amount of time to complete, and many students who were only asked to make a minor edit didn't see the value in investing all that time, leading to poor outcomes. Additionally, our student learnings outcomes research found that students got much less out of these small assignments than the full writing assignment.
This quality push had a major impact on our spring 2018 term. Our Wikipedia Content Experts reported much more high-quality content coming into Wikipedia, much less staff time spent on challenging courses or incidents on Wikipedia, and 10% more content contributed per student than in fall 2017. This work to identify the highest quality courses and spend more effort on retaining those instructors, while discouraging unsuited classes from participating, has been very important in helping us increase our impact with more courses, more student editors, and more content, while maintaining the same level of staff support and thus overall program cost.
- Knowledge equity
We also feel particularly proud of our work in spring 2018 in the area of knowledge equity. These quotes from some of our instructors tell the story of why our Classroom Program is so empowering for students while they improve the English Wikipedia's coverage of underrepresented topics:
- "I love using Wikipedia with students of color, particularly African American students who realize the power of using Wikipedia as a form of knowledge activism contributing perspectives from non-dominant cultures to the 6th most visited site on the web. It also helps them learn to overcome barriers from gatekeepers and learn practices of resilience. Many People of Color are indifferent or in despair about their place in our society. Editing Wikipedia is a practical way to empower their actions and teach them both better academic writing skills and critical thinking."
- "The Wikipedia assignment helped me to structure my lectures and it created a more valuable context for doing research. It became more than a grade. I was able to incorporate some of the social justice issues surrounding lack of access to information which provided even more value to the assignment for the student."
- "This was a course on women in science, so many of the students would get frustrated as we learned about how women are treated in science. It was so wonderful, then, to be able to remind them that they were actually creating new knowledge about women in science, and women of color in science, to right some of the wrongs they were learning about. This in turn made the students feel like they were active participants in the solution, and not just learning about a problem."
- "I imagine Higher Education running Wikipedia assignments as a core part of undergraduate education in Arts, Humanities and Social Sciences, especially around confronting issues of inclusion and exclusion."
- Making an impact on overall numbers
In the Wikimedia Foundation Audiences team's Metrics highlights presentation in May 2018, their slides contained this text as a highlight of the quarter: "In January, global new editor retention hit its highest level since 2007", with the sub-bullet: "Apparent cause is the trend toward higher and higher January and September spikes on the English Wikipedia". This highlight immediately struck us, since that maps nearly precisely to the start of our terms, where we've had higher and higher numbers of student editors participating in our programs. We dug into the notes and asked some questions of WMF staff and learned that they're using a new definition of retained new editors for that metric: An account that has made at least one edit in the 30 days since they've registered, and then made a second edit in the next 30 days. We asked for additional data, and were told that every month, there's roughly 50,000 new editors, of whom around 3,000 are retained. In the most recent January spike, that retained number jumps to about 5,200, meaning a difference of about 2,200 new editors retained.
We ran the numbers for our January 2018 program participants to see how those numbers played out. We had 3,952 program participants register in January 2018; 2,945 of them made an edit within 30 days of registration. Of those, 2,134 made an edit within both the first 30 days and the second 30 days, fitting the Wikimedia Foundation's new definition of a retained new editor. From what we can tell, our program is responsible for 2,134 of the 2,200 (or 97% of the) "spike" in retained new editors that Wikimedia highlighted in this metrics report.
As the Audiences team acknowledges in the notes where they describe the new metric, the WMF's Learning and Evaluation team uses a different metric to measure retention — 3 and 6 months after the conclusion of a program — and it would be ideal to standardize this across the WMF teams. We agree with this assessment, particularly because we don't think this "spike" measures retained new editors in the way it's meant to. Our student editors are making edits multiple months in a row because of the way we've structured their assignments for their courses; they make initial edits to get used to the editing interface, then do research and write and revise their articles over the course of many weeks. In fact, we set the minimum amount of time for a Wikipedia assignment as 6 weeks, but we recommend 10–12. Given this, it's not at all surprising that out students are largely responsible for the spike seen in the Audiences metrics slides, but it's also not an indication that those students will actually continue editing after the duration of their course assignment. As we mentioned in our proposal, we focus our retention efforts on instructors, who will then bring back another batch of students to improve that subject area on Wikipedia, term after term.
But given the Wikimedia Foundation's trumpeting of this spike as something to be celebrated as a highlight, we are happy to see that Wiki Education's work is primarily responsible for this boost, and that the growth of our programs, year over year, also shows up in these spikes. We see the fact that one organization's programs work can create such a meaningful boost in global numbers as to be highlighted by WMF as being an indication of our programmatic successes so far this year.
What hasn't worked
- Student Learning Outcomes Research Study
In our FDC APG proposal, we highlighted our desire to conduct a longitudinal study of student learning outcomes from Wikipedia writing assignments. We had completed a one-off study in fall 2016 by hiring a full-time researcher who saw the process through IRB approval, conducted a pre- and post-study of all student editors in our program, conducted focus groups with some courses, and analyzed the data into a final research report. While this research was critically important in quantifying the student learning outcomes from Wikipedia assignments, we felt like we should only engage in a longitudinal study if we were able to commit the funding for a multi-year effort to track student learning outcomes over time. Given the reduction in funding of our proposal from the FDC vs our initial ask, we were not fully confident in our ability to keep this research funded for the years we would need to in order for it to be an effective next step in research. We have put this project on hold. While we still believe a multi-year study would offer additional insights, we do not feel like this is the highest importance of projects to commit multi-year funding to, especially in light of our new strategy, which focuses exclusively on impact to Wikimedia projects, not student learning.
- Guided Editing
In our proposal, we mentioned we were anticipating receiving funding for our Guided Editing system, a responsive technical help support system for student editors. While we have had interest from a number of funders, the funding has not yet come in at a level where we're confident in our ability to complete the project. We would strongly prefer to not start the project until we have secured enough funding that we are sure we can complete it, so we have not started this project. We are still in talks with several funders about starting the project, but spending for this project is on hold until we have a more firm commitment. We believe this is the fiscally responsible approach to the project, even though we still believe Guided Editing will enable us to scale our Classroom Program impact even further.
The next six months
Each summer term, we support a handful of courses — somewhere between 20 and 30, usually. We don't actively recruit for summer term as many fewer courses are taught during the summer term in the U.S. and Canada education systems, but we support courses who proactively approach us and are interested in teaching with Wikipedia during the summer term. Instead, we focus our efforts on recruiting for the fall term.
We are confirmed to speak at the American Psychological Association annual meeting in San Francisco in August. We also have a workshop scheduled at the American Sociological Association annual meeting, as well as hosting a booth to recruit more participants. We also have a booth and panel at the National Communication Association conference. We're finalizing details for booths and presentations at the American Studies Association and American Anthropological Association conferences. Emory University has also secured funding to bring Wiki Education staff to campus in September for a full-day workshop on teaching with Wikipedia. Through these events and a handful of other conferences still in the works, we anticipate recruiting additional faculty for our Future of Facts and Communicating Science initiatives, as well as identifying courses on topics related to equity that we can bring into the program in future terms. We also expect to sign two additional partnership agreements with other associations.
This fall, we expect to maintain a similar number of courses as we supported in spring 2018 — around 400. As we learned from our work to focus on high-quality courses, we'll spend our retention efforts on bringing back the highest performing courses from fall 2017 and spring 2018. We'll also carefully evaluate which courses received low ratings, identifying which ones could use an intervention to change something and bring their scores up and which courses we should instead discourage from participating in our program again. We are looking forward to working with a volunteer engagement coach to determine the most effective ways of retaining the instructors we most want to retain.
We anticipate meeting our goals for content, number of editors, and other impact targets through the support of courses in the fall 2018 term. We are not entirely certain if we will meet our goal for number of articles improved by at least 10 points in ORES; given we are expecting to support about the same number of courses in fall 2018 as we did in spring 2018 because of the variances in the academic calendar, we anticipate reaching around the same number of articles improved by at least 10 points. We are hopeful that our ongoing quality course work will lead to more courses that have higher outcomes in this quality metric, although given our experimentation with it as a metric is still in its infancy, we can't confidently predict this. At the end of 2018, we are eager to look at a year's worth of using this ORES metric as a stand-in for quality, and determine what its strengths and weaknesses are as a metric.
Visiting Scholars Program
|Measure of success||Goal (2018)||Progress to date||% completed||Notes|
|Number of Scholars||15||15||100%||We've supported the number of scholars as anticipated.|
|Number of blog posts about Scholar work||15||7||47%||We are on track to meet this goal.|
|Find sponsors for qualified Scholar applicants within 6 months||60%||33%||n/a||During the last six months, we have struggled to find sponsors. So far we've secured new one sponsor for an applicant and paired one applicant with an open position, and we have several leads we hope will pan out soon.|
|Number of articles edited||680||334||49%||Thanks mostly to User:Rosiestep's work on creating short biographies of women, we're halfway to meeting this goal.|
|Amount of content added||810,000 words||224,000 words||28%||We expect to fall well short of this goal.|
|Number of articles edited with at least 10 ORES point improvement||435||93||21%||We expect to fall well short of this goal.|
What we've done
For the last year, we've been trying to scale the Visiting Scholars Program with limited success. We've been pitching it to potential Sponsors at conferences we attend and talks we give. We've been encouraging existing editors to fill positions we have open. And we've been working with existing Scholars to renew their positions as needed.
We've had some successes in this: In March, we were able to pair User:Textaural at the University of Windsor, who offered access to sources to improve Wikipedia's coverage of the history of southwestern Ontario. In May, we paired User:Amqui with Rutgers University, and this collaboration is planned to increase the availability of information about endangered languages on the English Wikipedia. We also opened two new positions, one at the University of New Mexico and one at North Dakota State University. Our existing Visiting Scholars continued producing great content.
But thanks in part to FDC feedback about how expensive this program was for what we got from it, we took a hard look at whether the Visiting Scholars program was really worth the staff time we've been devoting to trying to scale it. We looked at challenges like the positions at the University of North Carolina and the University of San Francisco that have both been open since 2016. We are still trying to match a handful of qualified applicants with narrowly focused resources that we haven't been able to find access to for them yet. Each of these Sponsor-Scholar relationships takes a significant amount of time to foster, and our attempts to scale it more easily weren't showing the results we'd like. Given this, we've decided to move the Visiting Scholars program into maintenance mode, decrease our staff attention to it, and spend that time instead on the more promising Wikipedia Fellows program, as described below.
What's worked well
Our existing Visiting Scholars continued to do great work so far in 2018, including:
- User:Gen. Quon used sources from his University of Pennsylvania Visiting Scholar position to bring the article on Orientius's 5th-century poem, the Commonitorium, to Good Article status.
- User:Wehwalt, Visiting Scholar at George Mason University, brought the article on U.S. President James K. Polk and the Connecticut Tercentenary half dollar to Featured Article status.
- User:Rosiestep has been prolific in her creation and expansion of biographies of women authors, including uploading images of them from Northeastern University's collection to Wikimedia Commons.
We had also experimented with offering $350 honoraria to Visiting Scholars this year, to see if that helped us with attracting and retaining Visiting Scholars. Our initial results showed this was successful; we found placements faster for the Deep Carbon Observatory, University of Windsor, and Rutgers University positions we think in part because of the offer of an honorarium. We also heard from two existing Scholars that the presence of the honoraria encouraged them to remain active in the program. These data points align with what we know anecdotally from talking with Wikipedians during Visiting Scholars recruitment in the past — that even a small honorarium separates this program from all of the others which seek to spur on volunteer editors to do more volunteer editing. Given our plans to decrease attention to this program, however, we will not be offering the honoraria in the future.
What hasn't worked
As mentioned above, attempting to scale this program didn't work well at all. One thing we attempted this year was to try to move the Visiting Scholars Program into more of a cohort system, which would decrease our staff time spent on the management and tracking end of things. Ultimately, there's just too much that's out of our control: when our primary contacts get back to us, when they hear back from their supervisors, etc. Sometimes it's a matter of when they have available time, sometimes they want it to coincide with the start of a semester, and sometimes they want it to start during the less busy parts of a term. Even those that seemed inclined to work within those parameters (or who liked the idea of the cohort in general), could not (or, at least, did not) follow through. It might be possible if we were willing to delay taking someone on for several months, and then hope that we could get a Visiting Scholar in a short period of time (or, likewise, if we were willing to wait to have a good Visiting Scholar start for a long period of time, and risk either them or the sponsor being disinterested). But ultimately, we decided this was not actually a staff time savings.
Ultimately, we determined the staff time we are spending to grow this program is not worth the growth we're getting from it.
The next six months
We are spending significantly less staff time — and thus less budget — on this program in the coming half-year, and we are shifting that attention to the Wikipedia Fellows program, which, as we describe below, we think shows more promise to scale. We are not canceling this program, however, just moving it into maintenance mode for the next year. What this specifically means:
- We will continue to support existing Sponsor-Scholar relationships, and foster renewals if both parties are interested in continuing the relationship.
- We will continue to attempt to find Visiting Scholars for the open positions we currently have.
- We will continue to use our networks of college and university collections to find Sponsors for both existing and new qualified Scholar applicants. We believe this is an important value-add that Wiki Education can provide to the English Wikipedia community; an individual editor is unlikely to get access to a university's collection if they ask as an individual volunteer, but we as a nonprofit with an established program can ask on their behalf a lot more successfully. We will continue to provide this service for qualified editors who ask us to.
We anticipate that because we are decreasing emphasis on this program and spending less money on it, we will not meet our goals for it as set out in the proposal. Instead, we anticipate making up ground, impact-wise, with the Wikipedia Fellows program as described below.
Wikipedia Fellows Pilot
|Measure of success||Pilot Goal||Pilot Actual||% completed||Notes|
|Number of academic associations signed on to participate||3||3||100%||The American Sociological Association, the Midwest Political Science Association, and the National Women's Studies Association participated in the pilot.|
|Number of Fellows||6||9||150%||Each association provided 3 Fellows rather than our goal of 2.|
|Number of articles edited||24||65||271%||Overall, Fellows were dramatically more active than anticipated, editing more articles than expected.|
|Amount of content added||7,200 words||29,100 words||404%||Fellows added four times the amount of content we expected them to in the pilot.|
|Number of articles edited with at least 10 ORES point improvement||12||8||67%||While we expected each Fellow would significantly expand one article, we found that most instead chose to make smaller edits to more articles. For the eight articles that Fellows did add at least 10 points to on the ORES scale, however, they increased the ORES rating from an average of 41.7 before to 67.5 after, a nearly 26-point jump that indicates the hard work that went into their articles.|
What we've done
In May, we published an extensive overview and evaluation of the Wikipedia Fellows pilot on Meta. At more than 10,000 words, it offers a comprehensive view of everything we did during the pilot program. In this mid-term report, we'll summarize our activities, but we encourage anyone who is interested to consult the full report.
Last fall, Wiki Education prepared for the Wikipedia Fellows pilot by engaging three of our association partners — American Sociological Association, the Midwest Political Science Association, and the National Women's Studies Association — in committing to participating. These three associations distributed a call to their members (subject matter experts in sociology, political science, and women's studies, respectively) to apply to become a Wikipedia Fellow. We had overwhelming interest: For what had been allocated at 6 Fellow positions, we had 87 applicants. Since we had so much enthusiasm, we expanded the pilot to selecting 3 Fellows per association. Wiki Education staff, in consultation with our partner associations, selected the candidates based on perceived interest in editing Wikipedia, interest in editing high-traffic articles, understanding of POV rules on Wikipedia, and ability to commit time in the first three months of 2018.
In January, we kicked of weekly meetings with the Fellows, using Zoom videoconferencing software. Using our Dashboard platform, we created a project page for the Fellows, assigned them training tasks and milestones along the way, and tracked their work. We guided Fellows in how to use their subject matter expertise in the most fruitful ways for Wikipedia. We answered questions both in the weekly videoconference meetings and through a Slack channel for the participants. At the end of the Fellows pilot, we asked Fellows to each fill out a survey — which we had a 100% response rate on — and guest write a post for Wiki Education's blog about their experiences.
What's worked well
- Quality and quantity of contributions to Wikipedia
We had hypothesized based on past projects like this that some Fellows would make either minor or no edits; it's rare that an entire cohort of people stays active enough through a three-month program to significantly improve at least one article. We were pleasantly surprised that all of the Fellows added at least several hundred words to at least one article! Having 100% of our participants actually make meaningful contributions to Wikipedia was a major accomplishment, and one we're quite proud of.
At the end of the pilot, Fellows had contributed to 64 articles, including 2 new articles. They added about 29,100 words. The articles they made significant improvements to received about 1 million pageviews, and all of the articles they improved together received 3 million pageviews. Individual numbers of words added ranged from 464 to 7,946. The mean number of words added per Fellow was 3,233, and the median was 2,233. The difference between mean and median reflects the fact that the two most active contributors contributed 54.3% of all of the words. ('Words added' does measure the impact of some important activities, such as removing problematic content and rewriting existing content.) Some Fellows made a small number of large edits, while others made a large number of small-to-medium edits. These all exceeded our content goals for the pilot.
As subject-matter experts, Fellows were able to easily recognize gaps in Wikipedia's coverage of topics, had the knowledge to fill those gaps, and had knowledge of the relevant literature that let them support their additions with high-quality sources that represented mainstream academic thinking in their discipline. For example, when Sine Anahita read through the sociology article, she was able to identify the lack of information about the sociology of gender, and was able to fill this gap. Similarly, Anahita was able to draw upon her knowledge of the sociology of race to add an overview of the sociological view of race in the race (human categorization) article and Michael Ramirez was able to introduce the "social construction of masculinity" into the masculinity article. The ability to identify important, but non-obvious gaps is a skill that allows subject-matter experts to improve the quality of of articles that seemed otherwise complete.
Articles that span an entire field of study can be especially difficult to write well because they require both breadth and depth of knowledge of the field. Far too often, articles like this end up giving disproportionate attention to many small details and controversies that catch the attention of Wikipedia, while still lacking a broad overview of the topic. Since she was able to approach it from an expert perspective, Jenn Brandt was able to make substantial improvements to women's studies by expanding the article to add missing details, trim or remove areas that had excessive detail, and rework it in general to more appropriately define and contextualize the field of study. When Nicole Kalaf-Hughes looked at the Procedures of the United States House of Representatives she found an article that lacked a crucial component of the what goes on in the House, and was able to add a section on speaking from the Floor of the House. In a similar vein, Maria Velazquez's additions to Black science fiction, R.G. Cravens' additions to LGBT conservatism in the United States, and Bradley Zopf's additions to Arab immigration to the United States, Definitions of whiteness in the United States, and Race and ethnicity in the United States were all examples of areas where Fellows were able to identify gaps and fill them using high-quality sources.
As academics, the Fellows were skilled at using the scholarly literature even when topics fell outside their primary areas of interest. When Sine Anahita discovered that there was no article on interruption, she was able to draw on the scholarly literature to create Interruption (speech), while Michelle Gohr was able to do something similar with body horror. Jenn Brandt's improvements to the Margaret Atwood article were sufficient to warrant a Good Article nomination. After the conclusion of the pilot, she shepherded the article through a lengthy and detailed review and promotion. For many Wikipedians, the back and forth of the Good Article and Featured Article review processes require mastery of a fairly steep learning curve, but academics experienced in the peer review process come to Wikipedia with that skill in place.
Overwhelmingly, program participants were enthusiastic about their participation, and walked away with a good sense of Wikipedia. Some comments we received in the survey demonstrate this:
- "I feel a billion times better about editing Wikipedia."
- "Exceeded my expectations. A wonderful uplifting experience. Excellent opportunities for professional development. And I feel like I made some good contributions and am excited for my future work on Wikipedia."
- "It was an amazing experience, and my only regret is that it's ending. I want to continue to be involved with Wikipedia, even beyond just an editor, but I'll certainly continue editing as well."
- "I absolutely loved this program. I didn't know what to expect from the program, but I gained so much in the academic and personal sense. I feel I am a better scholar and can be a better instructor because of the program. More than that, I feel that the program helps me to be a better citizen, as I am now able to contribute to public knowledge beyond the college classroom."
- "This was a fantastic opportunity for both professional and personal growth. I look forward to continuing to edit and teach with Wikipedia. Now, more than ever, partnerships like this are vital."
- Knowledge equity
Wikipedia Fellows from all three associations added their expertise in topics related to knowledge equity to Wikipedia, especially in topics related to race, gender, and sexuality. You can see the specific edits they made to each of the articles listed on the Dashboard by clicking on the "Current Version w/Authorship Highlighting" icon, the one that shows a page with highlighted text. Fellows added sections or made significant improvements related to knowledge equity to the following articles:
- Arab immigration to the United States
- Black science fiction
- Definitions of whiteness in the United States
- Interruption (speech) (new article)
- Jewelle Gomez
- LGBT conservatism in the United States
- Margaret Atwood (brought by a Fellow to Good Article status)
- Race (human categorization)
- Race and ethnicity in the United States
- Women's studies
Several Wikipedia Fellows reflected themselves on the topic of knowledge equity and the opportunity Wikipedia editing creates.
- In a blog post he wrote for our website, ASA Wikipedia Fellow Dr. Royal G. Cravens, III explained: "One of my goals for this Fellowship was to increase the reliability of information related to LGBT politics, my own research focus being LGBT political behavior. In reflecting on this experience, it is now apparent to me that Wikipedia provides a platform to amplify minority scholarship ... it is more important than ever to recognize the contributions of those scholars (and their research agendas) whom the academy has long marginalized. I must say, however, this is not the same as #promoteyoself – a popular movement to encourage marginalized scholars to promote their own work. Although I encourage scholars to promote their own work, for the purposes of Wikipedia editing, scholars should use their knowledge and resources to cite underrepresented authors and edit/create pages related to underrepresented topics. Only then can the full power of Wikipedia be brought to bear in enhancing the voice and scholarship of underrepresented people."
- In a blog post she wrote for our website, NWSA Wikipedia Fellow Michelle Gohr wrote: "By understanding the Wikipedia back end and skills needed to edit, academics and educators have the potential to make critical changes. Not only can we enrich the content within Wikipedia itself, but we can use it as a powerful tool for teaching through a learning community. Through the lens of an academic and librarian, this to me is the true power of Wikipedia and academic involvement in it. Because anyone can edit, we can disrupt knowledge gatekeeping and production which has, for a long time, privileged and primarily reflected white, wealthy, cis, hetero, male thought and histories. We can reconstruct and re-contextualize authority and decolonize knowledge systems by incorporating strong feminist and indigenous epistemologies, and we can do this by participating. While we do this, our students will be watching, reading, and maybe even participating themselves, and we’ll know that the content they’re getting is (or can be with some help and edits) well rounded, representative, and maybe even radically transforming."
What hasn't worked
The full evaluation report documents the woes we had scheduling meeting times for Wikipedia Fellows, resulting in us having to split the cohort into two groups to find meeting times that worked for everyone. A key learning for the future, especially given the interest we had in the program, is to instead set specific meeting times in the call for applications, and only accept those who can commit to making that particular meeting time for that cohort from the beginning.
While the Fellows cohort was successful as a pilot, we also devoted a lot of staff time to it: It's a very expensive program for its impact. That can be expected for piloting something, but we need to find ways to scale the program so that we get more impact out of it without significantly raising the cost. This will be a focus of our work in the coming fiscal year.
Finally, while we met our content goals, we fell short a bit on our ORES goal for measuring quality content. We have a full discussion of this in our evaluation report, but we think there are a couple of factors at play here:
- We had anticipated Fellows would make significant improvements to two articles. Instead, we found Fellows gravitated toward moderate improvements that added their subject matter expertise to multiple articles. This can be seen in the fact that we significantly overshot both our content and articles edited goals, but didn't make our ORES goal: Instead of focusing their efforts on large improvements to two articles, Fellows tended to make moderate improvements to many articles.
- Most Fellows chose to work on articles that were already mid-rated on the ORES scale; since ORES doesn't take into account the quality of sources or the expertise in coverage of a topic, the edits the Fellows made don't show up as well in the ORES ratings. For example, the average ORES score before a Fellow worked on an article was 41.7; for articles improved by our Classroom Program student editors, it was only 24.9, showing that student editors tend to pick start/stub class articles that are easier to get more improvement of on the ORES scale than the articles Fellows selected.
While we would like to see Fellows contribute more content in future cohorts, and we will test whether we can encourage this through curriculum changes in future cohorts, we will need to do future evaluations of whether this ORES measurement is a good fit for the Fellows program. We don't want to encourage Fellows away from adding a particular discipline's expertise in a short section of a longer article, for example, just because that won't show up in the ORES ratings. Edits like that are still an important value-add to Wikipedia that only subject matter experts like Wikipedia Fellows can bring to Wikipedia content, and we don't want to lose sight of that as an important outcome for the program.
The next six months
As mentioned above, we would like to determine how to scale the Fellows program; we think it has significant potential to be very impactful in the coming years. In June, we began our next round of cohorts, testing a variety of factors, including:
- Theme: two of the cohorts we are specifically theming around the Midterm Elections in the United States, asking Fellows to contribute nonpartisan, neutral content to articles specifically about candidates and issues that U.S. citizens will be voting on in November 2018. Our expectation is these cohorts will improve information on Wikipedia that voters will then consult when making their electoral decisions this fall, making it very important to have well-sourced, neutral information improved by subject matter experts. Another will be themed with our ongoing "Communicating Science" initiative, and a third will (like the pilot) have no theme at all and be multi-disciplinary.
- Size: We want to vary the number of Fellows in a cohort to see how this effects the group dynamics and outcomes.
- Curriculum design: We are making some modifications to our curriculum based on feedback from the pilot program.
- Length: We are testing expanding the length of the program to see if additional weeks will result in more content being added to more articles.
- Program manager: The program manager for the pilot, Ryan McGrady, is an experienced Wikipedian; since he's on educational leave this summer, our program manager for these cohorts, Will Kent, will have knowledge about Wikipedia but instead a background in library science; Wikipedia Content Experts Ian Ramjohn and Shalor Toncray will be providing the expertise in Wikipedia. We are interested to see if this has effects on the cohorts.
- Wikipedia Content Expert time: Since these cohorts are taking place during the summer months for us, when we have few student editors active in the Classroom Program, we'll be interested to see how the availability of Ian and Shalor affects the cohorts.
In addition to these cohorts we've already started, we anticipate adding 11 additional cohorts that will start this fall. We're still working out details with our association partners, but we want to continue testing the above variables as well as others to determine what learnings we can glean. We don't expect each cohort to succeed; we want to instead test what time-consuming parts of the Fellows program we can reduce our emphasis on and to what extent before Fellows stop editing entirely. We hope that the extensive testing of different models for the Fellows program that we are undertaking this summer and fall will lead to valuable learnings about how we can create a scalable program to extend the impact of our pilot for years to come.
Revenues received during this six-month period
Please use the exchange rate in your APG proposal.
Table 2 Please report all spending in the currency of your grant unless US$ is requested.
- Please also include any in-kind contributions or resources that you have received in this revenues table. This might include donated office space, services, prizes, food, etc. If you are to provide a monetary equivalent (e.g. $500 for food from Organization X for service Y), please include it in this table. Otherwise, please highlight the contribution, as well as the name of the partner, in the notes section.
Revenue source Currency Anticipated Q1 Q2 Q3 Q4 Cumulative Anticipated ($US)* Cumulative ($US)* Explanation of variances from plan Individuals USD $30,000 $554,000 $7,000 $561,000 $30,000 $561,000 We received a large unanticipated donation from the Pineapple Fund in January. Foundations: FDC Annual Plan Grant USD $750,000 $233,000 $0 $233,000 $750,000 $233,000 The remaining $167,000 of the $400,000 the FDC recommended to us is anticipated in Q3 after this grant report is accepted. Foundations: Stanton Foundation USD $400,000 $240,000 $0 $240,000 $400,000 $240,000 As outlined in our proposal, this is a match of FDC funds, so the remaining $167,000 is expected once the second FDC payment comes through. Foundations: Other USD $1,945,000 $0 $100,000 $100,000 $1,945,000 $100,000 The majority of this was budgeted to come in during Q3 and Q4; we have several verbal commitments and project we'll receive another $1.3 million in Q3 and Q4. TOTAL USD $3,125,000 $1,027,000 $107,000 $1,134,000 $3,125,000 $1,134,000 We had expected to have total revenues of $1.6 million in Q1 and Q2; the majority of the gap comes from the $350,000 reduction in revenues from the FDC.
* Provide estimates in US Dollars
Spending during this six-month period
Please use the exchange rate in your APG proposal.
Table 3 Please report all spending in the currency of your grant unless US$ is requested.
- (The "budgeted" amount is the total planned for the year as submitted in your proposal form or your revised plan, and the "cumulative" column refers to the total spent to date this year. The "percentage spent to date" is the ratio of the cumulative amount spent over the budgeted amount.)
Expense Currency Budgeted Q1 Q2 Q3 Q4 Cumulative Budgeted ($US)* Cumulative ($US)* Percentage spent to date Explanation of variances from plan Classroom Program USD $828,183 $199,476 $166,342 $365,818 $828,183 $365,818 44.2% Reduced spending due to us not receiving the research grant, PR contract ended earlier, and delayed printing, partially offset by higher personnel cost due to maternity leave. Visiting Scholars Program USD $225,920 $52,742 $54,002 $106,744 $225,920 $106,744 47.2% Wikipedia Fellows Pilot USD $223,382 $55,590 $60,708 $116,298 $223,382 $116,298 52.1% Guided Editing USD $583,050 $0 $0 $0 $583,050 $0 0% We did not receive the Guided Editing grant as anticipated, so we did not spend anything on the project. Operations, Fundraising, and Governance USD $906,472 $196,310 $194,622 $390,932 $906,472 $390,932 43.1% Reduced spending on fundraising travel and slightly reduced expenses for staff meetings. TOTAL USD $2,767,007 $504,119 $475,674 $979,793 $2,767,007 $979,793 35.4% Excluding the Guided Editing portion of the budget, cumulative spending is at 44.9%.
* Provide estimates in US Dollars
Is your organization compliant with the terms outlined in the grant agreement?
As required in the grant agreement, please report any deviations from your grant proposal here. Note that, among other things, any changes must be consistent with our WMF mission, must be for charitable purposes as defined in the grant agreement, and must otherwise comply with the grant agreement.
- As noted in the longer section above, we will be redirecting effort from the Visiting Scholars program to the Wikipedia Fellows program, as it shows more promise. We do not anticipate meeting the Visiting Scholars goals, but we anticipate significantly exceeding the goals of the Wikipedia Fellows program.
Are you in compliance with all applicable laws and regulations as outlined in the grant agreement? Please answer "Yes" or "No".
Are you in compliance with provisions of the United States Internal Revenue Code (“Code”), and with relevant tax laws and regulations restricting the use of the Grant funds as outlined in the grant agreement? Please answer "Yes" or "No".
- Once complete, please sign below with the usual four tildes.
- Sage (Wiki Ed) (talk) 21:01, 12 July 2018 (UTC)
Resources to plan for measurement
- Global metrics are an important starting point for grantees when it comes to measuring programmatic impact (Learning Patterns and Tutorial) but don’t stop there.
- Logic Models provide a framework for mapping your pathway to impact through the cause and effect chain from inputs to outputs to outcomes. Develop a logic model to map out your theory of change and determine the metrics and measures for your programs.
- Importantly, both qualitative and quantitative measures are important so consider both as you determine measures for your evaluation and be sure to ask the right questions to be sure to capture your program stories.
Resources for storytelling
- WMF storytelling series and toolkit (DRAFT)
- Online workshop on Storytelling. By Frameworks institute
- The origin of storytelling
- Story frames, with a focus on news-worthiness.
- Reading guide: Storytelling and Social change. By Working Narratives
- The uses of the story.
- Case studies.
- Blog: 3 Tips on telling stories that move people to action. By Paul VanDeCarr (Working Narratives), on Philanthropy.com
- Building bridges using narrative techniques. By Sparknow.net
- Differences between a report and a story
- Question guides and exercises.
- Guide: Tools for Knowledge and Learning. By Overseas Development Institute (UK).
- Developing a strategy
- Collaboration mechanisms
- Knowledge sharing and learning
- Capturing and storing knowledge.