Jump to content

Research:Supporting Commons contribution by GLAM institutions/Monitoring activity and tracking impact after upload

From Meta, a Wikimedia project coordination wiki

"There are only a few tools to measure and monitor it but still on a very basic level."

Pretty much all GLAM projects have increasing the visibility and use of their media content as a primary goal. Uploading their content to Wikimedia Commons is a step towards that goal, not an end in itself. In order to understand the impact of a content donation, you must be able to monitor and measure what happens to the content after it is uploaded.

Many GLAM institutions that partner with Wikimedia on content donation projects have organizational goals (or mandates) that relate to making their collections available to a wider audience—beyond the people who are able to visit the institution in person.

Most GLAM organizations believe that they can increase the visibility of their collections by uploading them to Commons. However, once collections are uploaded, it can be challenging to track how those media are being used, or viewed, and the extent to which the upload project has increased awareness of the GLAM organization.

Desired or anticipated impacts of GLAM projects[edit]

Participant What do you hope to accomplish by uploading these media items?
R_1rpP5qSd2Xi356z "Encouraging better awareness about these collections, and broader public knowledge about them."
R_SOxTwuflXqfrSql "open up a collection from a museum that is currently closed to the public; improve content on Wikipedia on Brazilian history and art; rely on this case for outreach"
R_12f18tF10RlXNRj "Valuing and encouraging the use of archive photos on Wikimedia projects and the spreading of contents from cultural institutions."
R_eLpYKWSXdeawV7r "That they are used on Wikimedia projects. Usually, for museums, getting a share of the potential media views generated by WP is an important reason to share their collections with WM online."
R_0IfHtWHzJspMP0R "Improve the description quality and especially identify pictures that describe monuments we have not been able to identify yet; Increase visibility of these images."
R_12f18tF10RlXNRj "Use in articles and other places, more free knowledge, visibility for the GLAM."
R_1IoSpLpNexGrn89 "Getting more quality images about a more diverse range of subjects in Commons, including about subjects we do not have an image of yet."
R_2v1G4rSChUjXhEe "Illustrate biographical articles written as part of the project."
R_2Cfx3PSRNZLpX8i "Uptake in Wikipedia, most the Dutch version, but also the English and other language versions of WP; Enable re-use outside Wikipedia-projects; Long-term storage of images"
p10 "GLAM is an activity that helps countries preserve history and culture. It is the only way to record unrecorded information online. This is especially an issue in Africa, it is important to keep record of history to help young generation learn about the culture and the past."

GLAM participants and institutions engage in donation projects for a variety of different reasons, and often have multiple impact-related goals for the same project.

  • Incorporation of media into Wikipedia articles. Getting their media incorporated into Wikipedia articles is a major motivator for many GLAM institutions, because of the visibility of Wikipedia.
  • Re-use of media outside of Wikimedia projects. Some GLAM institutions hope that uploading their collections to Commons under free licences will facilitate their re-use beyond Wikipedia as well. Currently, the primary blocker for this use-case is probably content discovery: Commons search is not very good for finding specific kinds of media (by type, topic, license, keyword, subject, etc.) even when an item has rich metadata associated with it. As such, inclusion in highly-visible Wikipedia articles is likely the best way for GLAMs to encourage re-use of their media items outside of Wikipedia as well.
  • Increased visibility of GLAM collections. GLAM organizations want to spread the word about their collections, and donating to Commons can facilitate this in several ways: facilitating the use of their content in Wikipedia articles; public archiving of their collections; and through the publicity that sometimes accompanies a large GLAM content donation (although media coverage is not guaranteed and usually requires the GLAMs themselves to be pro-active about publicizing their work, especially if they aren't the Smithsonian).
  • Increased traffic to GLAM institutions. GLAM institutions hope that with increased visibility comes increased traffic—in the form of foot traffic to their public collections, and web traffic to their institutional websites and online archives. This is facilitated through press, and especially through hyperlinks between media item pages on Commons and the institution's website—especially for GLAMs with descriptions of their collections available online, or publicly-accessible collection databases.
  • Synergistic impact with related initiatives. GLAM media upload projects are often associated with other Wikimedia movement initiatives such as edit-a-thons. In these cases, uploading media from GLAM collections is done in parallel with creating and curating Wikipedia articles where those media items can be showcased.
  • Long term public hosting. GLAMs that are smaller or which have fewer resources may want to use Wikimedia Commons as the primary long-term web host for their digital collections.
  • Media curation. GLAMs may hope that uploading collections to Commons will lead to active curation of those collections by volunteers, who can continue to create and validate item metadata.
  • Preserving cultural heritage. Many GLAMs have a mandate to preserve artifacts or records associated with particular cultures. Commons provides a way of preserving cultural heritage publicly and persistently.

Experience with monitoring and measuring impact[edit]

Participant How have you measured the impact of uploading this collection?
R_2WBiN2s3sEZtiC8 "I use GLAMorous, Glamorgan, Baglama2 and other tools to analyze file usage. I regularly update statistics in a monthly report published as a subpage of GLAM project on Wikipedia."
R_2aDpIRqOGFNbzhG "I looked at a list of what images were used on Wikipedia projects. I think it was built-in to Commons?"
R_12f18tF10RlXNRj "Global metrics for upload and edit-a-thons, planning an "executive report card""
R_UhCRgWd0FjJKwr7 "It was difficult. The collection has never been much tracked, because tools were on a different place than on Commons, and required a manual activation from someone else (BaGLAMa)."
R_31Y03TqvZTgtOaY "I have no idea how to do this although I have asked multiple people for help."
R_3Dnl9GOIOlmkV8d "I think we write some statistics after we have complete the physic experiment in the report that we gave to the local grant provider, other then that actually we don't have human resource to track the result."
R_sO67qlEBjWLViN3 "I've tried but the existing tools never seemed to work and/or I just couldn't figure out how to make them work! I would LOVE training on this if it's available."
R_2QGm3KJ7PPrGLxA "We tried to find statistics about views of the uploaded files. It was difficult to see if they actually were used/viewed."
R_2zIVoVPUa3q0hfl "BaGLAMs, GLAMORGAN, etc. The regular Magnus Manske tools! Note that the GLAMwiki toolset project also did write a report on the needs of GLAMs for metrics and statistics, much of which is not yet realised. For example, the Wikipedia page views that most GLAMs now use (Our image X is used in a Wikipedia article that was viewed Y time!) is a very weak metrics that does not actually say whether the article reader even noticed the media file much less clicked it. There are also no view or download stats from Commons itself. "
p4 "Every 6 months or so, we make a report for the museums, libraries, we work with. We give them: Amount of visits to the item, How many times it was used in Wikipedia. They’ve never asked for more."
p9 "Regarding consumption [of video], we would love to see not just pageviews, but also logging whether people clicked the play button."

GLAM organizations that have worked with the existing set of tools for monitoring what happens to media items after they are uploaded are almost universal in their praise of these tools. However, they also note important limitations and describe use-cases that are not well served by the current suite of tools.

  • Lack of awareness of monitoring tools. Many GLAM participants reported that they were not aware that tools for monitoring views/re-use/etc. of media items were available. Similar to other findings about lack of awareness of resources among GLAM project participants, and the need for centralized, comprehensive information resources and a more structured onboarding process.
  • Usability of monitoring tools. Some GLAM participants also reported that they were confused about how to use monitoring tools, and that end-user documentation was inadequate.
  • Lack of granularity of monitoring tools. Some of the available monitoring tools do not provide enough granularity of information about the impact of the content. For example, not tracking whether an image was viewed (as part of a page view).
  • Lack of easy reporting functionality. Currently, collecting metrics about item traffic and re-use is done on an ad-hoc basis: there is no way to automatically record usage data at specific intervals and track (or visualize) activity on an ongoing basis. Some GLAM institutions expect to be able to view "status reports" of the impact their donation has had at regular intervals. Currently, these reports must be manually assembled—a process which often involves gathering disparate metrics from several monitoring tools and formatting this information for easy consumption, which is a labor intensive activity. Unless the GLAM institution retains wiki- and tech-savvy staff or volunteers after the upload project is complete, these kinds of reports are seldom developed, and maintaining a manual reporting workflow like this is often unsustainable over the long term.
  • Gaps in the monitoring tool infrastructure. One interview participant noted that there was no good way to track unique plays of video content. Another expressed frustration that image downloads were not tracked. Other suggestions included tracking usage of media outside of Wikimedia projects and tracking curation activities (such as improvements to item metadata).
  • Lack of notification of important changes to collections. One implied issue faced by GLAM projects is that important things often happen to the media items they donate without their knowledge. Some of these changes can have negative consequences: a category that is used for tracking the collection as a whole may be changed, or a link to the GLAM institution's website may be removed from the item template. Items may also be deleted in batch. There is currently no straightforward way for GLAM institution representatives (who are often not active on Wikimedia Commons) to become aware of such dramatic changes without manually locating and visually checking each media item.