Research talk:Active editor spike 2015 (July update)/Work log/2015-07-09

Thursday, July 9, 2015

Looks like I am picking this back up. My task is to update my analysis with the last few months of data. However, I think I can do a little better than that. I have some ideas.

Timeseries analsysis

First, let's look at some raw counts and extract some seasonal components. I'll focus on the same set of wikis that User:Dario (WMF) looked at earlier.

Timeseries (Commons). A timeseries component analysis of the number of Monthly Active Editors is plotted for Commons

Timeseries (German Wikipedia). A timeseries component analysis of the number of Monthly Active Editors is plotted for German Wikipedia

Timeseries (English Wikipedia). A timeseries component analysis of the number of Monthly Active Editors is plotted for English Wikipedia

Timeseries (Spanish Wikipedia). A timeseries component analysis of the number of Monthly Active Editors is plotted for Spanish Wikipedia

Timeseries (French Wikipedia). A timeseries component analysis of the number of Monthly Active Editors is plotted for French Wikipedia

Timeseries (Italian Wikipedia). A timeseries component analysis of the number of Monthly Active Editors is plotted for Italian Wikipedia

Timeseries (Dutch Wikipedia). A timeseries component analysis of the number of Monthly Active Editors is plotted for Dutch Wikipedia

Timeseries (Russian Wikipedia). A timeseries component analysis of the number of Monthly Active Editors is plotted for Russian Wikipedia

Timeseries (Wikidata). A timeseries component analysis of the number of Monthly Active Editors is plotted for Wikidata

Still growing

• Wikidata
• Commons

Growing again (maybe)

• Spanish

Flattening (maybe)

• English
• Italian
• Russian

Continuing decline

• Dutch
• German

So, it doesn't look like there's a clear indication of substantial/sustained growth here. Although the recent trend in the Spanish Wikipedia looks promising. I could put English into the "Growing again (maybe)" group, but the recent months beneath the trend have me worried that we need more time to say.

Regardless, there are only two wikis here where my eyeballing would suggest continuing decline. Flattening and maybe growth is a good thing. Let's try to find out the source by breaking down the flow of editors towards active editor status.

Thanks Aaron, super interesting (also the editor model insights below)!

However, I not seeing an answer to the question whether the earlier spike in overall active numbers has been sustained. The above per-project observations are all very important of course. But the metric which WMF has been emphasizing for years in our strategic goals and public communications is total active editors across all projects per month. (It appears in the http://reportcard.wmflabs.org/ as "Active Wikimedia Editors for all Wikimedia Projects (5+ edits per month)" and forms part of the quarterly metrics scorecard.) Considering the enormous attention that its development has received by the community and the media over the years, we owe this audience an update in case the previous decline/stagnation narrative is no longer correct, even in case WMF is moving to using other metrics for our own work. Regards, Tbayer (WMF) (talk) 16:59, 15 July 2015 (UTC)

Hi Tbayer (WMF). I'm not sure what you are asking me for. It seems that I have made my conclusions clear. If you want to declare some sort of victory or change a narrative, then I'll leave that to you. IMO, I won't see reason for celebration until we can improve the retention of our new/old active editors. Until then, we're just sending more people into the funnel and I don't see a good reason to believe that this pattern is at all sustainable. I fear that, if you declared a recovery now, you'd find yourself mistaken once the short-term gains of forcing/encouraging new users register peter out. As I can't predict the future, I don't know what to tell you except what I have documented here. --Halfak (WMF) (talk) 15:48, 18 July 2015 (UTC)

Editor model breakdown

One of the flaws of my earlier visualizations (e.g. File:Monthly_active_editors.aug_2013-feb_2015.overall.svg) is that it's hard to compare sub-segments with large numbers against sub-segments with small numbers (e.g. new active with surviving new active).

I think we can use z-score scaling to compare across metrics more effectively. If you have a vector of values x, the rescaled z-score vector would be (x-mean(x))/sd(x). This means that values at the mean are equal to zero and deviations from zero are reported in standard deviations. This allows us to compare fluctuations in counts/rates that are at very different scales.

Before I can do that, I need to break down the wikis like I did last time.

big wikis (> 5000 active editors in June, 2015)
> big_wikis [1] commonswiki dewiki enwiki eswiki frwiki

medium wikis (> 1000 and <= 5000 active editors in June, 2015)
> medium_wikis [1] arwiki fawiki hewiki itwiki jawiki kowiki nlwiki plwiki ptwiki ruwiki [11] trwiki zhwik

small wikis (<= 1000 active editors in June, 2015)

> small_wikis [1] abwiki acewiki afwiki [4] afwikibooks afwikiquote afwiktionary [7] akwiki alswiki amwiki [10] amwiktionary angwiki angwiktionary [13] anwiki anwiktionary arbcom_dewiki [16] arbcom_enwiki arbcom_nlwiki arcwiki [19] arwikibooks arwikimedia arwikinews [22] arwikiquote arwikisource arwikiversity [25] arwiktionary arzwiki astwiki [28] astwiktionary aswiki aswikisource [31] avwiki aywiki aywiktionary [34] azwiki azwikibooks azwikiquote [37] azwikisource azwiktionary barwiki [40] bat_smgwiki bawiki bclwiki [43] bdwikimedia betawikiversity bewiki [46] bewikibooks bewikimedia bewikiquote [49] bewikisource bewiktionary be_x_oldwiki [52] bgwiki bgwikibooks bgwikinews [55] bgwikiquote bgwikisource bgwiktionary [58] bhwiki biwiki bjnwiki [61] bmwiki bnwiki bnwikibooks [64] bnwikisource bnwiktionary boardwiki [67] bowiki bpywiki brwiki [70] brwikimedia brwikiquote brwikisource [73] brwiktionary bswiki bswikibooks [76] bswikinews bswikiquote bswikisource [79] bswiktionary bugwiki bxrwiki [82] cawiki cawikibooks cawikinews [85] cawikiquote cawikisource cawiktionary [88] cbk_zamwiki cdowiki cebwiki [91] cewiki chapcomwiki checkuserwiki [94] chrwiki chrwiktionary chwiki [97] chywiki ckbwiki collabwiki [100] cowiki cowikimedia cowiktionary [103] crhwiki crwiki csbwiki [106] csbwiktionary cswiki cswikibooks [109] cswikinews cswikiquote cswikisource [112] cswikiversity cswiktionary cuwiki [115] cvwiki cvwikibooks cywiki [118] cywikiquote cywikisource cywiktionary [121] dawiki dawikibooks dawikiquote [124] dawikisource dawiktionary dewikibooks [127] dewikinews dewikiquote dewikisource [130] dewikiversity dewikivoyage dewiktionary [133] diqwiki dkwikimedia donatewiki [136] dsbwiki dvwiki dvwiktionary [139] dzwiki eewiki elwiki [142] elwikibooks elwikinews elwikiquote [145] elwikisource elwikiversity elwikivoyage [148] elwiktionary emlwiki enwikibooks [151] enwikinews enwikiquote enwikisource [154] enwikiversity enwikivoyage enwiktionary [157] eowiki eowikibooks eowikinews [160] eowikiquote eowikisource eowiktionary [163] eswikibooks eswikinews eswikiquote [166] eswikisource eswikiversity eswikivoyage [169] eswiktionary etwiki etwikibooks [172] etwikimedia etwikiquote etwikisource [175] etwiktionary euwiki euwikibooks [178] euwikiquote euwiktionary extwiki [181] fawikibooks fawikinews fawikiquote [184] fawikisource fawiktionary fdcwiki [187] ffwiki fiu_vrowiki fiwiki [190] fiwikibooks fiwikimedia fiwikinews [193] fiwikiquote fiwikisource fiwikiversity [196] fiwiktionary fjwiki fjwiktionary [199] foundationwiki fowiki fowikisource [202] fowiktionary frpwiki frrwiki [205] frwikibooks frwikinews frwikiquote [208] frwikisource frwikiversity frwikivoyage [211] frwiktionary furwiki fywiki [214] fywikibooks fywiktionary gagwiki [217] ganwiki gawiki gawiktionary [220] gdwiki gdwiktionary glkwiki [223] glwiki glwikibooks glwikiquote [226] glwikisource glwiktionary gnwiki [229] gnwiktionary gotwiki guwiki [232] guwikiquote guwikisource guwiktionary [235] gvwiki gvwiktionary hakwiki [238] hawiki hawiktionary hawwiki [241] hewikibooks hewikinews hewikiquote [244] hewikisource hewikivoyage hewiktionary [247] hifwiki hiwiki hiwikibooks [250] hiwikiquote hiwiktionary hrwiki [253] hrwikibooks hrwikiquote hrwikisource [256] hrwiktionary hsbwiki hsbwiktionary [259] htwiki huwiki huwikibooks [262] huwikiquote huwikisource huwiktionary [265] hywiki hywikibooks hywikiquote [268] hywikisource hywiktionary iawiki [271] iawikibooks iawiktionary idwiki [274] idwikibooks idwikiquote idwikisource [277] idwiktionary iewiki iewiktionary [280] igwiki ikwiki ilowiki [283] incubatorwiki iowiki iowiktionary [286] iswiki iswikibooks iswikiquote [289] iswikisource iswiktionary itwikibooks [292] itwikinews itwikiquote itwikisource [295] itwikiversity itwikivoyage itwiktionary [298] iuwiki iuwiktionary jawikibooks [301] jawikinews jawikiquote jawikisource [304] jawikiversity jawiktionary jbowiki [307] jbowiktionary jvwiki jvwiktionary [310] kaawiki kabwiki kawiki [313] kawikibooks kawikiquote kawiktionary [316] kbdwiki kgwiki kiwiki [319] kkwiki kkwikibooks kkwiktionary [322] klwiki klwiktionary kmwiki [325] kmwiktionary knwiki knwikiquote [328] knwikisource knwiktionary koiwiki [331] kowikibooks kowikinews kowikiquote [334] kowikisource kowikiversity kowiktionary [337] krcwiki kshwiki kswiki [340] kswiktionary kuwiki kuwikibooks [343] kuwikiquote kuwiktionary kvwiki [346] kwwiki kwwiktionary kywiki [349] kywikibooks kywikiquote kywiktionary [352] ladwiki lawiki lawikibooks [355] lawikiquote lawikisource lawiktionary [358] lbewiki lbwiki lbwiktionary [361] lezwiki lgwiki lijwiki [364] liwiki liwikisource liwiktionary [367] lmowiki lnwiki lnwiktionary [370] lowiki lowiktionary ltgwiki [373] ltwiki ltwikibooks ltwikiquote [376] ltwikisource ltwiktionary lvwiki [379] lvwiktionary map_bmswiki mdfwiki [382] mediawikiwiki metawiki mgwiki [385] mgwiktionary mhrwiki minwiki [388] miwiki miwiktionary mkwiki [391] mkwikimedia mkwiktionary mlwiki [394] mlwikibooks mlwikiquote mlwikisource [397] mlwiktionary mnwiki mnwiktionary [400] mrjwiki mrwiki mrwikibooks [403] mrwikiquote mrwikisource mrwiktionary [406] mswiki mswikibooks mswiktionary [409] mtwiki mtwiktionary mwlwiki [412] mxwikimedia myvwiki mywiki [415] mywiktionary mznwiki nahwiki [418] nahwiktionary napwiki nawiki [421] nawiktionary nds_nlwiki ndswiki [424] ndswiktionary newiki newikibooks [427] newiktionary newwiki nlwikibooks [430] nlwikimedia nlwikinews nlwikiquote [433] nlwikisource nlwikivoyage nlwiktionary [436] nnwiki nnwikiquote nnwiktionary [439] novwiki nowiki nowikibooks [442] nowikimedia nowikinews nowikiquote [445] nowikisource nowiktionary nrmwiki [448] nsowiki nvwiki nycwikimedia [451] nywiki ocwiki ocwikibooks [454] ocwiktionary officewiki ombudsmenwiki [457] omwiki omwiktionary orwiki [460] orwiktionary oswiki otrs_wikiwiki [463] outreachwiki pagwiki pamwiki [466] papwiki pawiki pawiktionary [469] pcdwiki pdcwiki pflwiki [472] pihwiki piwiki plwikibooks [475] plwikimedia plwikinews plwikiquote [478] plwikisource plwikivoyage plwiktionary [481] pmswiki pnbwiki pnbwiktionary [484] pntwiki pswiki pswiktionary [487] ptwikibooks ptwikinews ptwikiquote [490] ptwikisource ptwikiversity ptwikivoyage [493] ptwiktionary quwiki quwiktionary [496] rmwiki rmywiki rnwiki [499] roa_rupwiki roa_rupwiktionary roa_tarawiki [502] rowiki rowikibooks rowikinews [505] rowikiquote rowikisource rowikivoyage [508] rowiktionary rswikimedia ruewiki [511] ruwikibooks ruwikimedia ruwikinews [514] ruwikiquote ruwikisource ruwikiversity [517] ruwikivoyage ruwiktionary rwwiki [520] rwwiktionary sahwiki sahwikisource [523] sawiki sawikibooks sawikiquote [526] sawikisource sawiktionary scnwiki [529] scnwiktionary scowiki scwiki [532] sdwiki sdwiktionary sewiki [535] sewikimedia sgwiki sgwiktionary [538] shwiki shwiktionary simplewiki [541] simplewiktionary siwiki siwiktionary [544] skwiki skwikibooks skwikiquote [547] skwikisource skwiktionary slwiki [550] slwikibooks slwikiquote slwikisource [553] slwikiversity slwiktionary smwiki [556] smwiktionary snwiki sourceswiki [559] sowiki sowiktionary specieswiki [562] sqwiki sqwikibooks sqwikinews [565] sqwikiquote sqwiktionary srnwiki [568] srwiki srwikibooks srwikinews [571] srwikiquote srwikisource srwiktionary [574] sswiki sswiktionary stewardwiki [577] stqwiki strategywiki stwiki [580] stwiktionary suwiki suwikiquote [583] suwiktionary svwiki svwikibooks [586] svwikinews svwikiquote svwikisource [589] svwikiversity svwikivoyage svwiktionary [592] swwiki swwiktionary szlwiki [595] tawiki tawikibooks tawikinews [598] tawikisource tawiktionary test2wiki [601] testwiki testwikidatawiki tetwiki [604] tewiki tewikiquote tewikisource [607] tewiktionary tgwiki tgwiktionary [610] thwiki thwikibooks thwikiquote [613] thwikisource thwiktionary tiwiki [616] tiwiktionary tkwiki tkwiktionary [619] tlwiki tlwikibooks tlwiktionary [622] tnwiki tnwiktionary towiki [625] tpiwiki tpiwiktionary trwikibooks [628] trwikimedia trwikinews trwikiquote [631] trwikisource trwiktionary tswiki [634] tswiktionary ttwiki ttwiktionary [637] tumwiki twwiki tyvwiki [640] tywiki uawikimedia udmwiki [643] ugwiki ugwiktionary ukwiki [646] ukwikibooks ukwikinews ukwikiquote [649] ukwikisource ukwikivoyage ukwiktionary [652] urwiki urwikiquote urwiktionary [655] uzwiki uzwiktionary vecwiki [658] vecwikisource vecwiktionary vepwiki [661] vewiki viwiki viwikibooks [664] viwikiquote viwikisource viwikivoyage [667] viwiktionary vlswiki votewiki [670] vowiki vowiktionary warwiki [673] wawiki wawiktionary wikimania2010wiki [676] wikimania2014wiki wowiki wowiktionary [679] wuuwiki xalwiki xhwiki [682] xmfwiki yiwiki yiwikisource [685] yiwiktionary yowiki zawiki [688] zeawiki zh_classicalwiki zh_min_nanwiki [691] zh_min_nanwikisource zh_min_nanwiktionary zhwikibooks [694] zhwikinews zhwikiquote zhwikisource [697] zhwikivoyage zhwiktionary zh_yuewiki [700] zuwiki zuwiktionary

Counts of users

OK. Now I want to look at counts like I did last time first, but the rates in-between the counts will be coming next.

So, when we look at counts, we're looking at a flow from R:Newly registered users to active editors. I've ordered the cells of the plot so that they roughly reflect this order, but it's hard to say where 'reactivated' belongs. I just put it at the end.

Scaled MAE breakdown (big wikis). Scaled mae counts for big wikis are plotted with a loess regression fit to the trend.

Scaled MAE breakdown (medium wikis). Scaled mae counts for medium wikis are plotted with a loess regression fit to the trend.

Scaled MAE breakdown (small wikis). Scaled mae counts for small wikis are plotted with a loess regression fit to the trend.

Scaled MAE breakdown (wikidata). Scaled mae counts for wikidata are plotted with a loess regression fit to the trend.

Woah! Check out that spike in registrations that started in July 2014.

Rates of activation

Now, we also want to get a sense of the rate at which editors "activate" (cross the 5 edits threshold or continue to cross the threshold) because that will determine R:Newly registered users transition and sustain their active status.

... and I have to run away. Looks like I'm picking this up in the morning. --Halfak (WMF) (talk) 23:50, 9 July 2015 (UTC)

---

I'm back! And working on the rates bit again.

Activation rates (big wikis). Scaled MAE activation rates for big wikis are plotted with a loess regression fit to the trend.

Activation rates (medium wikis). Scaled MAE activation rates for medium wikis are plotted with a loess regression fit to the trend.

Activation rates (small wikis). Scaled MAE activation rates for small wikis are plotted with a loess regression fit to the trend.

Activation rates (wikidata). Scaled MAE activation rates for wikidata are plotted with a loess regression fit to the trend.

So there you have it. It looks like, for all but wikidata, there's a drop in new editor activation rates that correspond to the large growth in registrations. The surviving new editor rate and old active rates hold pretty constant.

For Wikidata, it's interesting to see the rising rate of new editor activation correspond to a steady decay in the surviving new editor rate. This might mean trouble for Wikidata, but I suspect that it's a natural part of being a newish project. --Halfak (WMF) (talk) 18:59, 10 July 2015 (UTC)

Or it means that it's getting more softcore editors, e.g. driven by edit links in the infoboxes of the Spanish and Russian Wikipedia. --Nemo 12:57, 23 August 2015 (UTC)