Learning and Evaluation/Evaluation reports/2015/Writing Contests/Outputs
Peer Review in Process now through July 3
The Learning & Evaluation from the WMF will be actively reading and responding to comments and questions until July 3.
Please submit comments, questions, or suggestions on the talk page!
How many participants are involved in contests?
How much content is typically produced?
What are some relationships between budget and participants or hours and content?
Start and end dates were reported for 32 of the 39 implementations (82%). A broad range of contest lengths is included in these data: the shortest contest lasted one week and the longest lasted 43 weeks. As illustrated in the graph below, most events (78%) lasted less than 12 weeks. All contest lengths were rounded to the nearest whole week.
Participant counts were obtained for 39 (100%) of the events. The number of participants in each contest or event ranged from 1 to 146, with an average of 18 participants Of the total participant base (1,027 users), 114 (11%) were newly registered. Participants counts in this figure are not unique; if a participant attended more than one contest, they are counted twice.
“Some judges are people who used to participate in the contest...people change, they graduate from college and do other things. Maybe they don’t have enough time to edit substantially enough for six months to be competitive, but they have time to judge for a couple of weeks.”
Cost per Participant
Measuring cost-per is a way to help inform setting reasonable goals and targets and to compare performance. It is especially useful if one has no experience with running a writing contest and you can see what others have done. It is important to use extreme caution in applying across international contexts.
"[C]ommunity on IRC suggested that they have a contest to correct typos to improve the quality and credibility of Wikipedia. During the competition, people were encouraging each other through micro-contests on IRC, for example, seeing who could complete the most edits in 10 minutes. In the end, a lot of people wanted to donate their prizes, which were small anyway, because they don't like the idea of paid editing."
Content Production 
Articles Created and Improved
Contest participants create and improve many articles. From the 35 contests reporting this metric, the range was 12 to 3,741 articles created or improved, where the average contest produced 134 new articles.
"The community on Hebrew WP doesn't have a lot of physicists, so they can’t contribute a lot to the content. They try to help with the styling of the pages and things like that, but it is not as involved as the regular competition. If you are someone who likes to write about physics, the competition is like a celebration, a festival, because all of the sudden there are a lot of new pages about the topic."
Pages of Text
Text pages are a way to frame how much content is added to articles. Bytes added to or removed from an article are converted into language characters and then into text pages. Generally, 1 or 2 bytes equals 1 character depending on the language, and one text page is defined as 1,500 characters. From the 30 contests reporting bytes added, the average contest resulted in about 238 total text pages. By counting pages of text added and removed in the contests, we see that a total of 13,300 pages of text were added (88% of pages), while 1,814 pages of text were removed (12% of pages). The average number of pages removed was 21. These numbers are specific to articles submitted for the contest only, and do not include other articles created or improved for the project by the participants included in the cohort during the contest time period.
Budget & Content Created
Cost per article created or improved
Eight contests had data on budget and articles created or improved. The cost per article created or improved has a wide range $0.04 USD to $85.11, but a narrow distribution, where 75% of the reported cost per article created or improved is below $8.30 USD. For the average contest, the cost per article created or improved is $0.59 USD.
Cost per page of text
There were eight contests with both budget and text pages. The cost per text page has a small range and narrow distribution, where the cost per text page for any program is less than $6.80 USD and the cost per text page for 75% of the contests is less than $2.05 USD. For the average writing contest, the cost per text page is $1.14 USD.
Weeks, Participation, Text pages
The bubble graph below illustrates the length of the event in association with participation and absolute text pages added or removed. The correlation analysis shows a significant positive relationship for 30 contests.  As the length of writing contests increases, contests tend to include more participants. Likewise, as the length of contests increases, the numbers of absolute text pages also tend to increase. In examining the correlation coefficient, the length of each contest in number of weeks had a positive correlation with total participants. and text pages.
Budget, participation, and text pages (per week)
Only 10 contests reported both budget and participation and 8 reported both budget and text pages. We conducted correlation analysis with extra caution since having few data points makes it difficult to draw conclusions. We divided each measure (budget, participation, text pages) by the length of each contest in number of weeks in order to have a standard weekly unit of measurement for comparison. The graph below compares budget, number of participants, and absolute text pages, dividing each by the length of the contest in number of weeks. Budget per week does not show a significant relationship with number of participants per week. However, budget per week tends to increase as absolute text pages created or improved per week increases.
Hours, participation, and content
In terms of relationships between hours invested and numbers of participants or amount of content, there were only five contests for which both hours and numbers of participants were reported, and five for which both hours and amounts of content were reported. The correlation analysis demonstrated no relationship between hours and absolute text pages with this very small set of comparison points. or hours and participation. This means that there may or may not be a relationship and we need more and better data to find out.
Participation, articles created or improved and text pages (per week)
Knowing that there is a positive correlation between the length of the event in weeks with numbers of participants and with the amounts of text added or removed, we look here at how participation is related to how much content was added or removed while considering the lengths of the contests. The numbers of participants per week tend to increase when the numbers of articles created or improved per week increase. Similarly, we find that numbers of participants per week increase as text pages per week increase.