Research:Referrer
Referrer data, derived from the HTTP referer field, provides important insight into how people reached content on Wikimedia projects -- i.e. where was the link they clicked on. These referers break down into four main categories:
- external search engines
- external websites (like Reddit or StackOverflow)
- internal (clicks from other Wikimedia pages)
- none, a catch-all category that includes:
- direct visits via pasting the URL into address bar or setting Wikipedia as a homepage
- missing referrers as is common with apps / email clients
- scrubbed referrals as can happen with VPNs/proxies
- often-times bots that are not using web browsers
It is generally believed that the search engine (for these search engines) and internal referrer data is highly accurate. External website referrals and none referrals, however, are more complicated to analyze. task T195880 has a long discussion about the none referrers, though the data there is prior to the introduction of the automated tag (see above). The none traffic also likely contains a fair number of referrals that come from apps that link to Wikipedia -- e.g., within text messages, from websites that are frequently accessed as apps such as Reddit, Youtube, or Facebook. It is difficult to estimate how prevalent this is but Youtube offers a rare insight into one example of this. Specifically, almost all the Wikipedia links on Youtube are added by Youtube as context to videos and come with a provenance parameter that is passed along to our servers via the request URL as opposed to the referer metadata. By comparing this independent source of pageview counts that were referred from Youtube with what we can detect via inspecting the referer data, we found (as of May 2020) that about 40% of Youtube referrals were missing (logged as none in the data). The specific pattern for other apps will vary but this provides an understanding of the potential magnitude of this issue for certain external site referral data:
- Youtube (desktop) -> Wikipedia (desktop): referer = https://youtube.com (<5% of data)
- Youtube (mobile) -> Wikipedia (mobile): referer = https://m.youtube.com (<1% of data)
- Youtube (mobile) -> Wikipedia (app): referer = https://youtube.com/mobile (<<1% of data)
- Youtube (app) -> Wikipedia (mobile; Chrome+Android): referer = https://youtube.com/mobile (>50% of data)
- Youtube (app) -> Wikipedia (mobile; not Chrome+Android): referer is dropped (~40% of data)
- Youtube (app) -> Wikipedia (app): referer = https://youtube.com/mobile (<<1% of data)
Site-specific referrer behavior[edit]
| Site/App | Referrer behavior | More information | Last verified |
|---|---|---|---|
| never sets a referrer (a conscious choice resulting from privacy concerns) | Jan 2021 | ||
| TikTok | generally seems to set a referrer | task T324376 | Dec 2022 |