Wiki Education Foundation/Outreach Pilot final report
The Outreach to High Achieving Students Pilot began as a response to the success of Wiki Ed's Classroom Program. Since 2010, the Classroom Program has proven that student editors will produce high-quality Wikipedia content within the classroom context. But what happens when students are asked to edit outside the classroom? Through this pilot, we sought to help students create unique projects that bridge academic and personal interests while filling Wikipedia’s critical content gaps. The main goal of this pilot was to find out if university students in the U.S. could be engaged in improving Wikipedia content as an extracurricular activity.
Specifically, the pilot project was asked to:
- target high achieving students
- evaluate the viability of extracurricular student editing and find ways to predict future success in this area
The following activities were deemed beyond the scope of the project:
- creating student clubs on campus
- making editing Wikipedia "fun for fun's sake"
- asking students and the community to come up with goals on their own
- increasing brand awareness or creating affiliation
The purpose of this document is to summarize the results of the 6-month pilot and to provide recommendations for next steps.
We understand that extracurricular editing is competing for students' time with other priorities. For this reason, our target student group audience includes those who we consider "high achieving" in their field as well as those with the potential interest in Wikipedia editing.
Originally, we defined high-achieving students as students who were invited to join honor societies in their discipline due to course completion and GPA requirements. We later expanded our outreach to include student groups studying any subject who met our criteria for selection, the details of which are covered in the next section, "Preparation." We believe students involved in honor societies and other extracurricular student clubs are meeting to discuss topics they are passionate about. We posited that passion should lead to high-quality contributions to content gaps in their fields, and may lead to students continuing to edit Wikipedia after the term ends.
We hypothesized that high-achieving students in the U.S. might provide high-quality contributions to Wikipedia during their extracurricular time and that they might be enthusiastic about the project.
In November 2014, Wiki Education Foundation hired an Outreach Manager to plan, execute, and evaluate the pilot program. During the planning phase, which concluded in January 2015, the Outreach Manager got support from the senior management and the communications teams, and consulted with experts in the field, including Pete Forsyth of Wiki Strategies.
Tasks completed during the preparation phase of the pilot included:
- Development of outreach materials for students
- Creation of workshop presentation template and script
- Research and outreach to individual student clubs as well as to academic associations and honor societies
A two step criteria for selection process was also created in order to guide the identification of potential student clubs as well as the decision making around club commitment. The first step of this process included finding clubs that met two specifications: first, were they located in California specifically or on the west coast generally, and second, were they working in disciplines that supported species, photography, and editing or proofreading assignments. Once clubs were identified that met both these specifications, we began our cold emailing and outreach. As interested student groups were identified, we then used our second criteria for selection metric. We created this tool in order to predict the success of a student club in our program. In it, we rated each student club out of a 50 point total, taking into consideration six criteria, including response time and communication, enthusiasm, mission alignment, leadership quality, geographic location, and project quality. We considered clubs that reached a point total of 41 or above to be good candidates for the term. More about this index can be found in Appendix 1.
From December 2014 to March 2015 we reached out to a total of 190 individual student clubs. These clubs were led by students, and sometimes, overseen by an advisor. Of the 190 that we contacted, 49 student chapters were affiliated with honor societies involved in the Association of College Honor Societies. The remaining 141 student chapters were subject specific but not affiliated with an honor society. Our goal this term was for at least 30% of our outreach to be honor society specific. In actuality, 25% of our student club outreach was directed to honor society chapters.
As a result of this student club outreach, we achieved a response rate of about 16% from student clubs, or 30 out of 190. This response rate includes those who replied that they were not interested, those who replied they were interested in learning more, and those who wanted to learn more but who we later decided not to work with. Of the 30 clubs that responded to our outreach, we decided to move forward and work with 6 individual chapters. This represents an overall conversion rate of 3%.
In January of 2015, we decided to expand our outreach to the actual honor society organizations. We felt that Wikipedia editing's strong compatibility with community service would be ideal for creating buy-in from the national level. Throughout the spring term, we reached out to 24 national honor society organizations and achieved a response rate of 46%. We consider this response rate to be much higher than anticipated with 11 out of 24 organizations responding.
Of those contacted, 25% or 6 out of the 24 honor societies, sent out newsletters or email blasts to their student members on our behalf. From this top down outreach, we got in touch with two individual student chapters that were interested in learning more about our program. Of the two groups we got in touch with, we moved forward and ran our program with one of them.
- Communication Materials
At part of the pilot, we created outreach email templates and informational flyers for the academic associations and for students. We also ordered promotional materials as a thank-you for students who participated in our activities.
During the execution phase of the pilot, we tried to engage students through workshops and field trips. Prior to meeting with the students, we worked with the club leader to discuss academic and personal interests of the club members. Based on this information, we identified potential content gaps where student editors could make an impact both visually with photo uploads as well as in writing and proofreading work.
During our workshops, we tried to target these club-specific content gaps to help students see the direct impact they can make. We found topical news stories that highlight current events surrounding a related Wikipedia page and presented both sources to the student. Then we asked students to think about what information from the news source could benefit the Wikipedia coverage of this topic. Once the students found a gap, we encouraged them to be bold and make the edit. During our workshops we also asked students to think about what editing qualities they think would make a good or featured article (clarity, well written, neutral, verifiable, and comprehensive coverage of topics are a few) as well as to think about what kinds of images would best represent certain article topics. Thematically, we tried to focus on the direct impact our students can have, both for their own skill sets as they develop their writing skills, media literacy, critical thinking, and community service hours, as well as from a global perspective. We want our students to know that their work can and will make a difference to one of the world's most visited websites.
Our field trips hoped to explore whether these types of activities would encourage students to upload images to Wikipedia and also increase the students' overall enthusiasm about adding content.
Between February and May 2015 we worked with the following student clubs:
- University of California, Berkeley Berkeley Water Group Idea Lab (2 workshop sessions, 1 one-day field trip planned and executed): 12 attendees total
- University of Arizona GeoClub (1 workshop session, 1 half-day field trip planned and executed): 13 attendees total
- University of California, Santa Barbara Art, Design, & Architecture Museum Club (1 workshop session, 1 half-day field trip planned, but not executed): 11 attendees total
- Portland State Lambda Pi Eta (1 workshop session, no field trip): 2 attendees total
- Oregon State Hydrophiles (1 workshop session, no field trip): 4 attendees total
- Oregon State Pi Alpha Xi Horticulture club (1 workshop session, no field trip): 4 attendees total
Content Goals & Outcomes
- Goals & Contributions
Our goal this term was to work with 16 student clubs. Assuming that a club would consist of 10 students on average, we set a target of having 4 out of those 10 students edit Wikipedia. We set a content target of 3 kB of text added to Wikipedia and 10 images uploaded to Commons per active editor.
|# active editors
|# characters added
|# images added|
|Target||160||64||12 kB (36,864 characters)||40|
|Actual||44||24||3.3 kB (3,336 characters)||21|
We did not hit either our participation or content targets.
- Total student edits this term from club related work
- 44 students
- 60,083 characters added (~20 kB of content)
- 46 articles improved and 3 articles created
- 21 images uploaded, 10 of which are in use on Wikipedia
We used two surveys this term to track progress with our students and workshops. The first was a post workshop survey handed out to 4 student clubs immediately following the end of a workshop. This survey was meant to gauge what went well and what didn't during each workshop, and to ask the students to help us improve. We used the feedback from this survey to make iterative adjustments on out workshop model, including adding more Wikipedia stories into our workshops as well as a photo uploads tutorial.
The second survey was sent out electronically to all students at the end of the term. This survey was meant to learn from students what their experiences on Wikipedia had looked like and what their editing goals were for the future.
The details from these surveys can be found in Appendix 2.
This 6-month long pilot answered several questions. First, whether students at U.S. universities would edit Wikipedia as an extracurricular activity. Second, whether they would improve Wikipedia's quality. Third, how enthusiastic would students be about editing Wikipedia in their free time?
It's clear that U.S. students can improve Wikipedia's quality in a classroom setting and in their free time. However, our pilot suggests that, without additional incentives, most of these students choose not to do so.
The greatest challenge in targeting extracurricular editing was the absence of external incentives. Our A/B tests show that field trip participants were more motivated student editors. Without field trips as an incentive to spark that motivation, however, participation in Wikipedia-related activities dropped. With a grade, field trip, or staff visit, students contribute to Wikipedia. Without those incentives, they contribute significantly less, or not at all.
Unfortunately, field trips are not a scalable way of incentivizing students. We see greater opportunities in models that rely on alternative, scaleable solutions. For example, honor societies or academic associations could incentivize Wikipedia editing by counting editing time as a community service.
From the Wiki Education Foundation's perspective, this pilot was a success. We answered the questions we set out to address. While our pilot generated less data than we had expected, we are confident that further experimentation with more participants would confirm our findings. For this reason, and with the amount of learnings we collected, we will not be extending the pilot beyond spring 2015.
- Criteria for Selection – part 1
Student clubs and honor societies had not been previously identified, so the bulk of the work during November, December, and January included researching and contacting those who met our selection criteria:
- Geographic Location: We chose to focus on student programs located on the west coast in regions such as California, Arizona, Washington, and Oregon. We made this decision because these locations cut down on staff travel time from our office in San Francisco, CA, and because the warm climates allowed for species editing and photography projects.
- Types of student groups: We chose to keep our focus on student groups studying academic disciplines that supported species, photography, or editing and proofreading projects. We chose these because we believe editing in species/classification spaces is made accessible through the use of templates, uploading images to Wikipedia and adding them to article spaces is an easy skill to learn and holds the potential to have huge impact, and because we already have the capacity through our Classroom Program to support writing assignments.
- Criteria for Selection – part 2
After we received interest from a student club and discussed the goals of the project, it was time for us to make a decision about which clubs to move forward with. In order to make this determination, we came up with an enhanced "Criteria for Selection" that would help us project the success of a student project. This projection includes an evaluation of the following criteria for each individual club. The criteria are rated out of a total score of 50. Some criteria are rated out of 5, while those criteria rated out of 10 were seen as "high priority" criteria.
- Response time & Communication ( R ) : Rated on a scale of 1–5, this is a representation of the communication skills and response time of the student leader. A score of 5 represents a leader who was quick and efficient in their communication, and a score of 1 represents a leader who was slow to respond and who exhibited poor writing or communication skills.
- Enthusiasm ( E ) : Rated on a scale of 1–10, this is a representation of the engagement and enthusiasm of the student leader. A score of 10 represents a leader who was highly engaged, asked thoughtful questions, and showed excitement about the opportunity. A score of 1 represented a leader who exhibited no visible excitement and little to no engagement during Skype or phone call discussions.
- Mission alignment ( M ) : Rated on a scale of 1–10, this is a representation of the students' excitement about Wikipedia. A score of 10 represents a student leader who seems to be a champion of Wikipedia and is excited about closing content gaps in their field. A score of 1 represents a student who is questioning of Wikipedia and its role and who does not see the benefit of student contributions for the student editors or for the community.
- Leadership quality ( L ) : Rated on a scale of 1–10, this is a representation of the overall organizational goals of the student leader and the student club. A score of 10 represents a leader who is very engaged, has a goal for the club for the term that Wikipedia can help them reach, and who is a good spokesperson for themselves and their club. A score of 1 represents a leader with little to no understanding of the goals of the club, who is not engaged with their membership, and who does not act as a spokesperson for themselves or their club.
- Geographical Location (G): Rated on a scale of 1–5, this is a representation of the ease of travel to the club location. A 5 is a club that exists within 2 hours driving distance, while a 1 is a club that requires a cross country flight.
- Project Quality (Q): Rated on a scale of 1–10, this is a judgement on the potential quality of the editing project decided upon by the student group and the outreach manager. A score of 10 represents a project that meets a content gap goal, is reproducible in type and scale for future terms, and seems achievable. A score of 1 represents a group that is unable to name a project theme.
Each club was rated on all these criteria in order to project the success of their student project. The table below is an example of how we stored this information. In the execution section, you can see how we scored the clubs we moved forward with. We believe that clubs scoring 44 and above can and should contribute high quality content.
|Club Name||Geographic Location (1–5)||Leadership (1–10)||Mission Alignment (1–10)||Response Time/ Communication (1–5)||Enthusiasm (1–10)||Project Quality (1–10)||Total Score (Out of 50)||Project Details & Notes|
|club name||city - location score||leadership score||mission alignment score||cmn score||enthusiasm score||project score||total score||notes|
- Post workshop survey
Overall, we had 20 students fill out this survey. From it, we learned that the most helpful part of the workshop was the editing exercise (17). Other responses detailed the most helpful part of the session to be the question and answer session (1), help on creating an account and a demonstration of a photo upload (1), and getting to see the Wikipedia facts around gender gap and content gap work (1). The least helpful part of the session was identified to be the background on Wikipedia by 2 of the students, while the other 18 felt that the entire session was helpful. When asked what students wished we had covered, we found that from the first workshop, students wanted more funny stories and experiences about Wikipedia, something we took into consideration and later adapted into our workshop communication. Other things students wanted to hear included how to create a page (1), getting to upload a photo themselves (1), more about the history of Wiki Ed (1), more instruction on how to find articles to edit (1), and more time to learn about formatting and practice editing (2). We began doing uploading tutorials at our later workshops with this feedback.
On a scale of 1-5, 1 being very easy and 5 being very hard, students were asked how easy to understand the session was.
|--||Score of 5 – very hard to understand||4 – hard to understand||3 – neutral, not easy or hard to understand||2 – easy to understand||Score of 1 – very easy to understand|
|Number of votes||5||2||0||4||9|
One student commented that the language of Wikipedia was new to her, which made it difficult to follow along. But overall, the majority (65% or 13/20) said the workshop was easy or very easy to understand.
- End of term survey
We had 17 students respond to our end-of term- survey as of June 4, 2015. This is a 38% response rate.
- Students were asked if they had done any editing following the workshop. 58.8% of students responded that they had edited since the workshop, (10/17 YES, 7/17 NO). The top reason for their continued editing was cited as "sharing knowledge is important to me" at 50% (5/10). Other responses included "I enjoy sharing information on a topic I'm passionate about" (20%, 2/10); "I feel I have a lot to contribute" (10%, 1/10); and "it's fun" (20%, 2/10).
- Those who had not continued editing (41.2% of students) cited being too busy (71.4%, 5/7) and editing being too difficult (14.3%, 1/7). All our questions also included the option to write in a response. One student replied "My expertise would be where I choose to edit/add content and it might be self promoting (14.3%, 1/7) as a reason for not contributing. Other response options included "I don't have anything to contribute," and "I don't see the point in editing Wikipedia." Neither of these options received any responses.
- When asked if they had uploaded any photos to Commons, 17.6% of responses said they had and 82.3% of responses said they had not added photos. Those who did upload photos reported that they did because they thought it illustrated a topic well (66%) and because they liked having a copy of their photo online to share with friends (33%).
- Many students said that they had added photos either at the workshop or after. When asked if they added their photos to a Wikipedia page, 75% of respondents who said they uploaded a photo said yes. 100% of those students reported that they did this because they liked making Wikipedia a better resource.
- Those who had not uploaded any photos reported that they hadn't because they were too busy (42.8%, 6/14), they didn't have photos to contribute (42.8%, 6/14), or that they intended to but were too busy (14.3%, 2/14). This last response was a write in response from two students.
- When asked about their editing potential in the future, 100% of respondents or 17/17 students reported that they would edit an existing article in the future; 58.8%, or 10/17, of all respondents said they would create a new article in the future; 82.3%, or 14/17, respondents said they would add a photo in the future; 58.8%, or 10/17, said they would add references to an existing article in the future; and 35.3%, or 6/17, said they would copyedit an existing article in the future.
- When asked what might prevent them from editing Wikipedia in the future, 81.25%, or 13 of 16 respondents said they don't have time to contribute; 12.5% or 2 of 16 said they can't think of anything to contribute; and 6.25% or 1 respondent said that editing Wikipedia is not fun for them.
- End of term survey results
(as of June 4, 3:20pm) - 17 total responses (out of 44 potential)
- Have you done any Wikipedia editing since the workshop? 10/17 YES (58.8%), 7/17 NO (41.2%)
- If yes, why did you continue to edit Wikipedia?
- 5/10 (50%) Sharing knowledge is important to me.
- 2/10 (20%) I enjoy sharing information on a topic I'm passionate about.
- 1/10 (10%) I feel I have a lot to contribute.
- 2/10 (20%) It's fun.
- If no, what factors prevented you from editing Wikipedia?
- 5/7 (71.4%) I'm too busy.
- 1/7 (14.3%) It's too difficult.
- 1/7 (14.3%) My expertise would be where I choose to edit/add content and it might be self promoting.
- Have you uploaded any photos to Wikimedia Commons since the workshop? 3/17 YES (17.6%), 14/17 NO (82.3%)
- If yes, Can you tell us why you uploaded photos?
- 2/3 (66%) I thought it illustrated a topic well
- 1/3 (33%) I liked having a copy of the photo online to share with friends
- If you uploaded a photo, did you add your photo to a Wikipedia article? 3/4 YES (75%), 1/4 NO (25%)
- What motivated you to add your photo(s) to Wikipedia?
- 3/3 (100%) I like making Wikipedia a better resource
- If you haven't uploaded a photo since the workshop, can you tell us why?
- 6/14 (42.8%) I'm too busy
- 6/14 (42.8%) I don't have photos to contribute
- 2/14 (14.3%) I intend to but keep putting it off because I have been busy.
- Do you think you might do any of the following activities in the future?
- 10/17 (58.8%) Create a new article
- 14/17 (82.3%) Add a photo
- 17/17 (100%) Edit an existing article
- 10/17 (58.8%) Add references to an existing article
- 6/17 (35.3%) Copyedit an existing article
- What might prevent you from editing Wikipedia in the future?
- 13/16 (81.25%) I don't have the time to contribute
- 2/16 (12.5%) I can't think of anything to contribute
- 1/16 (6.25%) It's not fun for me