Learning and Evaluation/Evaluation reports/Wikimania 2019 Evaluation

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search
Patricia Costillo CC BY-SA 4.0

Wikimania 2019 was held in Stockholm, Sweden and hosted by Wikimedia Sverige. This year's theme was "Stronger Together: Wikimedia, Free Knowledge and the Sustainable Development Goals". Wikimania 2019 also piloted a new structure around "Spaces," aimed toward a more decentralized program design.

Key Findings for Consideration[edit]

  • The vast majority of attendees found the venue and conference environment safe and comfortable.
  • Wikimania attendees--especially first-time attendees--are more likely to be female than is generally seen in online contributor communities.
  • Attendees found conference content relevant and interesting, but responded less positively to the conference theme and "Space" structure.
  • A common complaint was how far venue sites and conference accommodations were from one another, impeding participation and interaction. Many also expressed a desire for clearer signs and directions.

Data Overview[edit]

This evaluation report primarily summarizes data collected through a survey of Wikimania attendees in the two weeks following the event. It also draws from aggregated data collected at attendee registration. 879 people attended, 156 of whom were Wikimedia Foundation staff. All attendees were sent a survey, though Foundation staff responses are excluded from most of the analysis. Below it is noted where Foundation staff data are included.

Survey Response Rates
Community Members Wikimedia Foundation Staff
Number of Wikimania Attendees 723 156
Survey Respondents (% of attendees responding) 409 (57%) 94 (60%)

Demographics: Who Attended Wikimania 2019?[edit]

Heat map of Wikimania 2019 Attendee Home Countries

Wikimania participants came from 83 countries. The vast majority of attendees came from Europe (68%), with large contingents from Sweden (n=88) and Germany (n=86), as well as from the United States (n=68), France (n=33), and India (n=27). 12% came from Asia and the Pacific, 11% from North America, and 2-3% each from South/Latin America, Africa, and Arab States (see this page for more information on how countries are classified into regions).

38% of attendees identified as female, 57% as male, and 1.5% identified as genderqueer or nonbinary (due to the way gender was collected at registration, transgender attendees are not counted here and this is likely an incomplete picture of gender minority attendees). While female participants are underrepresented at Wikimania compared to the general population, they are more represented at Wikimania than in estimates of the Wikimedia contributor population (ranging from 6% to 25% female, depending on data source and project). A similar gender makeup was also seen among Wikimania 2018 attendees.

We also asked attendees how many days of work it took for them to pay for registration, lodging, or travel. 49% of all attendees worked more than 5 days to pay for their attendance, and this figure was much higher for those who applied for a scholarship (61%) than those who did not (38%).

While only 62% of attendees reported that English was one of the languages they were most comfortable communicating in at registration, 95% indicated in the survey that they were "mostly" or "completely" comfortable receiving information in English. This indicates that survey respondents might be more comfortable communicating in English than those who did not respond.

Who Was New to the Event?[edit]

"Do you intend to attend Wikimania in the future?" First-time attendees, 2019

At registration, 46% of attendees indicated that 2019 marked their first Wikimania. New attendees were less likely to respond to the survey than return attendees, with only 38% of survey respondents said that this was their first event. New attendees were only slightly more likely to come from Europe (73%) than returning ones (68%). Of those who completed the survey, they were also slightly more likely to be female (43%) than return attendees (34%).

Most (69%) new attendees who completed the survey said they intended to return to another Wikimania, whereas 26% were unsure if they would. The 5% of first time attendees who did not plan to return cited reasons like next year's location of Bangkok being too far for them to travel (n=2) or that they had only intended to attend once to learn about the conference or the movement (n=2).

What Was the Experience of Scholarship Applicants?[edit]

The majority of scholarship applicants found the process easy to understand (85%), the application easy to complete (82%), and that they were informed of the result in a timely manner (81%). Applicants who were not awarded scholarships found the application similarly easy to complete (78%) as those who were (82%), but were far less positive about the ease of understanding the process (56%) and especially the time it took to be notified of their application status (41%). No major themes emerged in open-ended feedback about the application process, but several respondents indicated that they would have liked to have booked their travel earlier.

Conference Venue and Environment[edit]

This section includes responses from all attendees including WMF staff, which reported similar experiences as all other attendees.

"How much do you agree or disagree with the following statements about the conference venue and environment?", Wikimania 2019 in Stockholm

Survey respondents were generally satisfied with the conference venue, particularly its physical (92%) and language (93%) accessibility, feelings of safety (94%), and upholding of the Friendly Space Policy by other attendees (91%). Many attendees thought the location was beautiful and inspiring (n=20). Overall, 88% were satisfied with the conference venue and environment, a slightly lower proportion than the 93% reported by Wikimania in Cape Town attendees in 2018. Reports of physical and language accessibility were also slightly lower than in the previous year.

The university was a very nice space. I liked the indoor and outdoor nature of the venue and the fact it is a university (a place for knowledge).

There [were] way too few signs pointing to the various spread out session rooms. Quite frequently, I was lost and had to ask around. The area for [one] session was unbearably noisy. It would have been nice to get an entire building/hallway for the... track with lots of small breakout rooms to have discussions in a silent and safe environment.

While most indicated that they were able to participate in a way comfortable to them (90%), they were somewhat less positive about being able to fully see, hear, or participate (79%). This often related to room layout (n=25), with reports of some spaces being too small, big, or noisy for the session's size or purpose. Only 76% of respondents were able to stay in a place convenient to the conference site, and only 61% found it easy to get where they needed to go at the venue. The most frequent critical feedback was that conference rooms were confusingly labeled and difficult to find even with the maps provided (n=62), and that the distance between accommodations (n=32) and venue buildings (n=45) made it difficult to make it to sessions on time or at all. At times, this also eroded feelings of community and made food and quiet spaces less accessible (n=8). Several people (n=9) also noted that there were limited dietary options for vegetarians and vegans, or that there were poor food options generally. Another 6 mentioned that the venue was not set up to accommodate people with limited mobility, though these respondents did not indicate having such difficulties themselves.

Presenters' experiences, Wikimania in Stockholm 2019

Experience of Presenters[edit]

132 non-WMF staff respondents had presented or led a session. 9% of presenters/session leaders reported issues with technology in their session, but 85% were satisfied with the tools and technology available. In qualitative feedback many cited problems having their slides ready, not having the necessary equipment to connect their computers, and a lack of clarity regarding how to access help with these issues (n=27). 83% were satisfied with the communication from organizers, as well as their overall experience. 80% found it easy to find how their content fit within the session or "Space" it was placed in.

Conference Content[edit]

"How much do you agree or disagree with the following statements about the conference program and content?" Attendee responses, Wikimania in Stockholm, 2019

The vast majority of community attendees found the content of the conference to be relevant to them (92%), and a majority was pleased with the quality of the plenaries (79%). Attendees also indicated that they gained relevant knowledge (88%) and that they intended to apply it to their Wikimedia work (90%). Attendees were less likely to report gaining explicitly practical skills, however (64%).

"I was satisfied with the overall quality of the Space sessions." All attendees of Wikimania in Stockholm, 2019.

Fewer felt positively about the "Spaces" structure (65%) and the conference theme (63%). Many articulated the value of this year's theme as providing an important orienting value to both the conference and the movement (n=56), and others saw value in connecting the Wikimedia movement's activities and goals to the United Nation's Sustainable Development Goals (SDGs) in order to acknowledge the work that Wikimedians already do related to these goals as well as offering more partnership opportunities (n=41). More critical comments found either that SDGs were not relevant to Wikimedia (n=13) and artificially constrained the content of the conference (n=7), or that the theme was not adequately reflected in most of the content (n=16). Only 59% indicated that they were able to attend most of the sessions that interested them, which might reflect in part the difficulties getting around discussed above, as well as a high level of interest in many of the scheduled sessions.

Wikimania "Spaces"*[edit]

Sessions were organized into 19 thematic Spaces. Survey respondents were asked which three Spaces they attended the most sessions in, and the most common responses were the Technology, GLAM, Strategy, and Education Spaces. The least common responses were the Libraries and Readership Spaces, though these Spaces offered far fewer sessions.

All respondents (100%) who attended many sessions in the Readership Space were satisfied with the quality, but did not offer qualitative feedback. The GLAM (94%), Research (92%), and Environment Space (92%) attendees also responded positively to those sessions' quality. No themes emerged from the qualitative feedback provided about sessions in each of these spaces.

Respondents were the least satisfied with the quality of content in the Advocacy (71%), Strategy (72%), and Diversity (73%) Spaces. Qualitative feedback about the Advocacy Space indicated that while some sessions were of high quality, others were more problematic (n=2). Critical feedback about Strategy Space sessions highlighted issues with the size and noise level in session rooms (n=9), but also a lack of clarity in the information that was shared (n=3) and ineffective facilitation (n=2). Some respondents (n=2) took issue with the focus of the Diversity Space sessions on mostly gender and not other diversity issues.

*Note: The first 73 respondents to the post-event survey saw a question that presented them with an incomplete list of Wikimania Spaces to indicate their participation, so these responses are excluded from summaries of Space attendance. They are included in measures of participant satisfaction with specific Spaces.

What Are Attendees' Favorite Parts of Wikimania?[edit]

On the whole, attendees (WMF staff included) valued Wikimania for the chance to interact with other Wikimedians in both structured and unstructured ways (n=137), especially those whom they would have no other opportunity to connect with in-person. Some critiqued the more dispersed physical spaces of the venue for inhibiting some of the usual social interactions (n=8), but the vast majority mentioned that these valuable interactive experiences were their favorite moments at Wikimania. They also noted the value of sharing their own ideas and work, and getting feedback and recognition, through presentations (n=13).

The people's reaction to Maher's statement in closing ceremony: 'There is no place for harassment in Wikimedia'.

There were some very informative, helpful and interesting discussions with Wikimedians from other countries. These were impossible to predict but very rewarding.

Surströmming! The Swedish hosts were just amazing! I am so impressed by their hospitality & organization. They did an amazing job.

The friendly and enthusiastic reception of my poster presentation!

This year in particular, many people noted that the opening and closing ceremonies were especially moving and that they enjoyed the ensuing receptions (n=80). Many also noted tasting Surströmming (n=16) as quite memorable.

Icem4k [CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0)
Avery Jensen [CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0)
Rhododendrites [CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0)