CentralNotice/Request/surveying readers and contributors to Wikipedia

surveying readers and contributors to Wikipedia

• Jullienn (talk · contribs · page moves · block user · block log), primary contact (Requested at 12:27, 6 February 2023 (UTC))

Central Notice Settings
What is the campaign duration? 2023-05-11 to 2023-05-26 for fr es, tr; 2023-05-17 to 2023-05-28 for pt, de ; 2023-05-20 to 2023-05-31 for uk, it, pl, ar, nl; 2023-06-01 to 2023-06-12 for en → 2 weeks for each language. Proposed calendar (compatible with the other demands, I think, providing I have not done any reading error) Which projects will you be targeting? Wikipedia What languages will you be targeting? English, French, Turkish, Italian, German, Ukrainian, Polish, Ukrainian, Spanish, Portuguese, Dutch, Arabic . Do you wish to show banners to Logged In users, Anonymous Users or Both? Do you want to target users with a specific number of edits or average monthly? Any user (both logged in and anonymous) What countries will your campaign target? Any country accessing the languages Banner/Campaign Diet: To be determined by Central Notice admin

What is the purpose of the campaign? How will you measure the success of the campaign?

Description -

The campaign aims at surveying the Wikipedia users on how they consider, use the information provided by Wikipedia, according to the context (professional, personal, etc.) It will also ask the contributors specific questions about their participation and their involvement.

The language versions of the questionnaire are available here

It is managed by a group or researchers from several public research institutions, mainly French, coordinated by the research network Marsouin with, we hope, collaboration with the different language communities.

More precisely, the survey aims at addressing the following questions (open list):

• for the contributors, what does that mean to be a contributor, how it is perceived by their relative, how this impacts their practice, their vision of Wikipedia and of its governance (see the WMF's taxonomy of "gaps", p. 16)
• for the readers, how they use Wikipedia, but also what does acceptability as a source of information means, according to the context - school, university, information, etc.), for them and for their "audience" (see WMF's taxonomy of "gaps", p. 10)
• this questionnaire will rely on and expand the one done in 2019 by the WMF (would be happy to have feedback from Isaac_(WMF) or Jmorgan_(WMF)

Again, we would like to have inputs from the community of what questions we could address together!

The list of the languages contacted, and the people volunteering to translate is maintained in the project's Research page

The survey is inspired by the survey to the French users made by the same team in 2015 and in 2011.
See the results of the former surveys (in French: 2011, 2015), and by the survey made by the WMF in 2019.

Metrics - We are expecting to reach ~16.000 answers in two weeks in each language addressed, as in 2015.

IRB approval - The research has been evaluated and approved by IMT Atlantique's IRB.

What banner(s) will you use? What will be your landing page?

Banners - The Banners will have the same text, translated in the language of the wikipedia surveyed (see table below)

The text (English version):

Please help us better understand what you do on Wikipedia! By responding to this questionnaire created by an international group of university researchers and members of the Wikipedia community.
Answer the questionnaire (Answers are collected outside the Wikimedia Foundation's servers, by the public research center Marsouin.org, in France)

Landing Page - The landing page will be the questionnaire's first page, in the targeted language. The questionnaires are all hosted on Marsouin's LimeSurvey site, in France (on OVH Cloud)

All the banners are translated in the survey's research page.

Is this project grant funded? Please provide a link

Type of grant -
No grant, several public research centers involved

However, this project is conducting while Nicolas Jullien is visiting professor at Università Ca'Foscari Venezia

Link to grant -

Discussion

Hi Jullienn, I have a quick question around this campaign. When are the exact dates you are running it? We (the WMF) has fundraising banner campaigns in both Italy and the Czech Republic scheduled for the dates I see you mention in the CN calendar. Do you think you will run the survey in either of these countries? The fundraising banners are only shown to logged out users. Thanks, JBrungs (WMF) (talk) 07:00, 7 February 2023 (UTC)

Hi JBrungs (WMF). Thank you very much for your question, it'll help me precise our needs.

• we do not have precise dates in mind, so we can be flexible. It is not a problem either if in some languages the banner appears a bit later than in other
• this said, for the time being we are able to do the questionnaire in French, Turkish, and in English (maybe arabic). Probably in Italian as well (I am currently in Italy) but we would very much like having the support/interaction with the communities to do so
• other languages and timing are pending the possibility to exchange with the community on their needs and on the translation

Jullienn (talk) 12:06, 7 February 2023 (UTC)

Thank you @Jullienn for getting back to me.

Generally, you can find what fundraising campaigns are happening for non-logged in users in the CN calendar.

In case your campaign runs at the same time as one of the fundraising campaigns, readers will still see your banner after they saw 10 fundraising banners. So the overlap would not mean that your banners do not show at all.

I will keep an eye on this page and see how your plans develop and if you need any further help or information, just ping me.

Best, JBrungs (WMF) (talk) 12:10, 7 February 2023 (UTC)

hum, do I understand you well, @JBrungs (WMF) if I say that it means that a person would have to open 10 pages before seeing out banner? If yes, that would not do, as there would be a huge bias toward the big Wikipedia users in our dataset.

That obliges us to schedule our banners out of the period of fundraising. It is a constraint but as you say we have the date and we can adapt (and it is us to be adaptable anyhow).

Best, Jullienn (talk) 14:21, 7 February 2023 (UTC)

Hi @Jullienn,

Yes, that is correct. A reader will see fundraising banners for the first 10x they visit a wiki page and after that your banner would display. Please note, that for the time you are running the survey this is only applicable to non-logged in readers in Italy (please note that we might not run banners in Italy during this time, this is still to be determined) and the Czech Republic. Logged in users will see your banner straight away as we do not display fundraising banners to them.

I always make sure that I add our fundraising campaigns to the Central Notice calendar as soon as I can, so you can always check there when and where we are having fundraising banners up.

Best, JBrungs (WMF) (talk) 07:42, 8 February 2023 (UTC)

Ok, well, providing that our demand is accepted, I think the best would be to adjust "our" banner's time slot to the final fundraising calendar; we don't mind a delay, but better to have the same protocol for all the languages targetted. Jullienn (talk) 08:26, 8 February 2023 (UTC)

hi @JBrungs (WMF)

here is the calendar for the survey I've just proposed. I think it fits with all the other demands!

Proposed calendar:

• 2023-04-21 to 2023-05-04 for fr, en, it, pt
• 2023-04-28 to 2023-05-11 for uk, pl, es, de
• 2023-05-04 to 2023-05-18 for tr, ar, nl

Jullienn (talk) 13:09, 13 April 2023 (UTC)

Thanks @Jullienn. Have a good campaign. JBrungs (WMF) (talk) 14:01, 13 April 2023 (UTC)

thanks @JBrungs (WMF) If you want to look at the Engkish questionnaire (and feedback on it/ on the quality of the questions and of the language, feel free to do so! It is here

The addresses to the questionnaires are listed here, in the project's research page. (talk) 07:21, 17 April 2023 (UTC)

Hi @DerHexer and @Martin Urbanec, regarding this demand, do you need any more information? I would be happy to talk with you about the calendar if modifications are needed, and about the settings of the banners and of the landing pages. --Jullienn (talk) 10:53, 17 April 2023 (UTC)

• Not a CNA but since this is a survey I'd like to point out to the sections CentralNotice/Usage guidelines §§ Banner requirements​ and Surveys. Since this survey seems to be conducted in a third-party website, I'd be interested to hear if privacywikimediaorg has any concerns about that. I see LimeSurvey is mentioned, which may be okay, but the text of the banner should IMHO clearly indicate that this is conducted outside Wikimedia servers so users can make an informed choice of whether to visit that website or not. —MarcoAurelio (talk) 15:51, 26 April 2023 (UTC)

  hi; thanks @MarcoAurelio for that remark. It is not a problem for us, if it has to be added on the banner Jullienn (talk) 13:14, 27 April 2023 (UTC)

• The link under Answer the questionnaire above may be broken: https://https//questionnaires.marsouin.org/index.php/978939 has two hhtps. —MarcoAurelio (talk) 15:52, 26 April 2023 (UTC)

  Thanks, corrected Jullienn (talk) 14:59, 27 April 2023 (UTC)

Central Notice admin comments

@Jullienn: I've created the campaign and banner. I think that it can be used for all languages with the language codes added or removed at the start and end date. Our standard banner works a little bit different than your proposed text plus links, in fact a fully clickable banner might even make more sense than just underlined words. Please check the previews of the first four languages: English, French, Spanish, Turkish (I've used the texts and links from here). Please check everything, ask for changes, or approve it so that I can enable the campaign. Please ping me as I will be on the road for the next days and will not check my watchlist regularly. PS: The German translation is not understandable, it would require a couple of hours by a native speaker to have it fixed. Unfortunately, I don't have the ressources to do that. Best, —DerHexer (Talk) 18:13, 9 May 2023 (UTC)

Hello @DerHexer

thank you very much for all this!

I come back to you today.

As far as the banner is concerned, what you say makes way more sense: a fully clickable banner, of course. I would just bold the "questionnaire" (in in English: "by responding to this questionnaire", and so on) => what is to be put in bold is the first link in the proposed banner texts where you took them, here.

Regarding the German, I have a German native speakers colleague correcting it (that is a reason why I did not want it to start right away).

Thanks! Jullienn (talk) 07:52, 10 May 2023 (UTC)

@Jullienn: I feared you'd say that. It's not so easy to change the markup in a translatable banner. I have to break everything into segments, which means that this one sentence will then become three parts whose middle part would get bold. I can do that but I don't know when as I'm traveling to London right now. The current version I could enable right away. Of course, I can adjust it if need be. Best, —DerHexer (Talk) 08:04, 10 May 2023 (UTC)

@DerHexer

Well, Yes I think it is important, but not enough for a no-go (if it starts slow it's not a problem).

So, I would say: let's launch the campain!

I can help do the fix if you think it can help, and if/when you had time it can be ajusted later, no? Jullienn (talk) 08:21, 10 May 2023 (UTC)

@Jullienn: Yes, it's possible to change the display later. I've enabled the banner now, boarding to London has just started. ;D —DerHexer (Talk) 08:45, 10 May 2023 (UTC)

Thanks!

it works fine.

Have a good flight and enjoy London! Jullienn (talk) 08:58, 10 May 2023 (UTC)

@DerHexer No need to fix the banner (bolding things, see below) the "test" made shows that there are enough answers like that Jullienn (talk) 10:30, 16 May 2023 (UTC)

Hi @DerHexer

As the banner are not back, and to diminish your workload, maybe the best would be to start the languages in less waves?

Like:

today (2023-05-16) to 2023-05-30 for fr (30%), es (30%), tr (50%), pt (30%); 2023-05-20 to 2023-06-03 for de (30%) uk (50%), it (30%), pl (50%), ar (30%), nl (50%); 2023-06-04 to 2023-06-17 for en (10%). What do you think? Jullienn (talk) 09:45, 16 May 2023 (UTC)

I plan to post in the de-wikipedia today or tomorrow about the survey, once it is reread by 2 native speakers Jullienn (talk) 09:46, 16 May 2023 (UTC)

@Jullienn:, someone is reporting a problem with this banner it seems: en:Wikipedia:Village_pump_(technical)#Banner_link_error (See also: phab:T336353) —TheDJ (talk • contribs) 09:19, 10 May 2023 (UTC)

@TheDJ

Yes, OVH seems to have a problem...

better unplug the banners during 1h if possible Jullienn (talk) 09:23, 10 May 2023 (UTC)

@Jullienn I disabled the banners. It would not surprise me if the amount of traffic was too much for the server. If that's the case, we can limit the banner so that it only displays on a certain % of Wikipedia pageviews, e.g. 5% or 10%. Would that be acceptable? I'm certain you can still reach your 16,000 answers. Peter Coombe (WMF) (talk) 09:34, 10 May 2023 (UTC)

@Pcoombe (WMF)

It was that!

We've changed the config and it should be ok at full charge, but you are right better being prudent.

I think the best would be to replug the banners for the Turkish Wikipedia first (100%, possible now), then the English (30% around 12:00 GMT), then the Spanish one hour later and the French one hour after the Spanish (both at 50%). What do you think? Jullienn (talk) 10:10, 10 May 2023 (UTC)

(because the goal is to reach 16000 answers per language, not for the beauty of the figure, but because to be able to statistically anayze some minorities populations/phenomenon, we need quite a big number of answers) Jullienn (talk) 10:23, 10 May 2023 (UTC)

To have different traffic levels and start times for each language I will need to split it into multiple campaigns, which will take some time. Peter Coombe (WMF) (talk) 10:39, 10 May 2023 (UTC)

Ah, I understand. Well, let's try everything at 50% then: according to our calculus the servers are able to manage 100% now, but better being prudent, and 50% will allow for collecting enough data for the "smaller" wikipedia of next week Jullienn (talk) 11:05, 10 May 2023 (UTC)

@Jullienn I'm already done splitting them and setting up. Turkish is up now at 100%, and I confirmed the link works fine now. English will go up 20% at 12 UTC (we need to be extra cautious since English is by far the highest traffic). Spanish 50% at 13 UTC. French 50% at 14 UTC. Peter Coombe (WMF) (talk) 11:12, 10 May 2023 (UTC)

@Pcoombe (WMF) That's perfect! Thank you very much indeed Jullienn (talk) 11:16, 10 May 2023 (UTC)

@Jullienn The survey is having trouble again, and I have disabled the banners. Please ensure the server is properly fixed before requesting them back again. Peter Coombe (WMF) (talk) 13:15, 10 May 2023 (UTC)

Hello @Pcoombe (WMF) From what I see from the server things were running ok (the server "seems to consider" that everything is swell; we are looking deeper Jullienn (talk) 14:34, 10 May 2023 (UTC)

Hi @Pcoombe (WMF)

Thank you for your patience!

I think that the safer way to handle the problem would be to drop the English for the time being and to launch it at the end (like 06/01 -> 16/13, no banner request for this period), because it seems working ok by itself (and with the turc), but when combined with bigger wikipedia such as French or Spanish; the combine traffic generates random errors (not that much considering the total amount for what I see but still -- around 5%). In the same time, we are looking if there is a problem with the database part (limesurvey uses one base and several tables, so we are looking at migrating to a more stable database infrastructure)

What do you think? Jullienn (talk) 14:52, 10 May 2023 (UTC)

Thanks. I'm going to leave any more decisions about restoring this campaign up to @DerHexer or one of the other community CentralNotice admins. I really only stepped in to deal with the urgent issue of having a broken link live. Peter Coombe (WMF) (talk) 15:55, 10 May 2023 (UTC)

Ok, thanks.

Well, @DerHexer, we have updated the capacities, and as written supra, I think it's best to postpose the banner opening for the en-wikipedia at the end of the campaign (I've update the calendar accordingly).

But in my opinion, better opening first the Turkey when you can (it works for sure, but let's put it at 50% if possible) and then, tomorrow (05-11) the French and the Spanish (at 20%).

It's up to you to decide and it's true that we won't know if it works without trying, but I am pretty confident this time.

Sorry again for the mess (but I think it's the first time that a questionnaire that big is played in so many languages, and I really hope that the results it will provide will pay all us for the efforts! Jullienn (talk) 19:07, 10 May 2023 (UTC)

@Jullienn has this vendor been engaged, or are we just assuming what their failures were? If they were, have they provided an explanation or remediation? — xaosflux ^Talk 15:59, 10 May 2023 (UTC)

Are you the service provider here? — xaosflux ^Talk 16:00, 10 May 2023 (UTC)

Clearly the resources allowed (http and database) were not enough; we are increasing them - we are using an cloud service provider Jullienn (talk) 16:23, 10 May 2023 (UTC)

So we think it's their mistake (they should have been clearer about the limitation) but in the end it's our, as we support and we make the community support the consequences: we should have anticipated and increased the resources beforehand, and avoid this mess. Jullienn (talk) 16:55, 10 May 2023 (UTC)

Tracked in Phabricator:
Task T336353

• @Pcoombe (WMF): got the survey to load once and started looking though it, there is an untranslated answer set in the question, "Have you ever used Wikipedia for the activities just mentioned?" (Survey language set to English, Answer set is in French). The external survey may need additional QA review before relaunching, ensuring that all languages are working properly. — xaosflux ^Talk 13:37, 10 May 2023 (UTC)

  Hello @Xaosflux, you are speaking of the questionnaire in English, right? I think I've found the problem and corrected it, thank you (even after checked and rechecked, I let that passed, sorry) Jullienn (talk) 14:40, 10 May 2023 (UTC)

  Yes, I had the English survey, but at least one of the answer prompts was in French. — xaosflux ^Talk 15:44, 10 May 2023 (UTC)

  right, my mistake, corrected Jullienn (talk) 16:46, 10 May 2023 (UTC)