Research talk:Anonymous editor acquisition/Signup CTA experiment/Work log/2014-06-16

From Meta, a Wikimedia project coordination wiki

Monday, June 16th[edit]

Picking up where I left off on Friday, I should have a dataset with a complete set of reverted & archived flags.

--Halfak (WMF) (talk) 14:40, 16 June 2014 (UTC)[reply]

The geometric mean number of article revisions saved is plotted by experimental condition.
Article revisions. The geometric mean number of article revisions saved is plotted by experimental condition.
The geometric mean number of productive edits is plotted by experimental condition.
Productive revisions. The geometric mean number of productive edits is plotted by experimental condition.
The geometric mean number of productive editors is plotted by experimental condition
Article revisions. The geometric mean number of productive editors is plotted by experimental condition

It looks like -- no matter how I measure it -- overall productivity went down in the pre-edit condition. --Halfak (WMF) (talk) 14:48, 16 June 2014 (UTC)[reply]

Now it's time to get some basic stats on burden. Since I just measured the revert rate, I can figure out how much burden relevant anons inflicted on Wikipedians.

The geometric mean reverted article revisions per user/token is plotted by experimental condition. Note that this method for measuring revert activity does not control for decreased article activity.
Reverted article revisions. The geometric mean reverted article revisions per user/token is plotted by experimental condition. Note that this method for measuring revert activity does not control for decreased article activity.
The proportion of overall reverted article revisions is plotted by experimental condition. Note that this method for measuring revert proportion over-weights highly active individuals.
Reverted revisions prop. The proportion of overall reverted article revisions is plotted by experimental condition. Note that this method for measuring revert proportion over-weights highly active individuals.
The proportion of reverted sampled article revisions is plotted by experimental condition. Note that this method for measuring revert activity controls for both prolific users and decreased article activity.
Reverted sampled revision prop. The proportion of reverted sampled article revisions is plotted by experimental condition. Note that this method for measuring revert activity controls for both prolific users and decreased article activity.

OK, so I measured this two naive ways and a robust way. #Reverted article revisions plots the expected number of reverted article revisions per user/token. This measurement is affected by the decreased rate of article editing. #Reverted revisions prop plots the proportion of reverted article revisions for the whole group. This measurement is affected most strongly by prolific editors and is therefor, not robust to outliers. For #Reverted sampled revision prop, I randomly sampled 1 article revision from each user/token who made at least one and included that in the proportion measurement. This method controls for both concerns.

So, it looks like editors in the pre-edit condition are consistently less burden than the control.

Here's the data:

wiki bucket n main_revisions.sum main_revisions.geo_mean main_revisions.geo_se.upper main_revisions.geo_se.lower productive_edits.geo_mean productive_edits.geo_se.upper productive_edits.geo_se.lower reverted_main_revisions.sum reverted_main_revisions.geo_mean reverted_main_revisions.geo_se.upper reverted_main_revisions.geo_se.lower productive.k main_editor.k sample_reverted.k productive.prop productive.se reverted_main_revision.prop reverted_main_revision.se sample_reverted.prop sample_reverted.se
dewiki control 42441 12314 0.08973791 0.09150446 0.08797422 0.06175928 0.06329375 0.06022702 2123 0.02960919 0.03042239 0.02879663 2383 3636 1329 0.05614854 0.00111745 0.1724054 0.003403962 0.3655116 0.007986389
dewiki post-edit 38593 8075 0.08627629 0.08798654 0.08456873 0.05689025 0.05833854 0.05544394 1942 0.0303633 0.03121313 0.02951417 2127 3356 1309 0.05511362 0.001161622 0.2404954 0.00475606 0.3900477 0.008419684
dewiki pre-edit 39331 10166 0.07274083 0.07439115 0.07109304 0.05510174 0.05657932 0.05362622 1319 0.0192839 0.01997635 0.01859193 2014 2722 776 0.05120643 0.001111426 0.1297462 0.003332693 0.2850845 0.008653068
enwiki pre-edit 198936 61737 0.107385 0.1083025 0.1064682 0.08511951 0.08593776 0.08430188 10209 0.02552958 0.02591357 0.02514574 14982 18763 4380 0.07531065 0.0005916562 0.1653627 0.001495185 0.2334381 0.003088221
enwiki control 208205 77814 0.1321544 0.1331415 0.1311681 0.09897115 0.09983017 0.09811281 14828 0.03657449 0.03701676 0.03613241 18566 24619 6873 0.08917173 0.0006245774 0.190557 0.001407916 0.2791746 0.002859024
enwiki post-edit 189530 65349 0.1311697 0.1321865 0.1301538 0.09805692 0.09894065 0.09717389 13201 0.0361366 0.03659432 0.03567909 16891 22510 6338 0.08912046 0.0006544556 0.2020077 0.001570594 0.2815637 0.002997745
frwiki pre-edit 30115 9673 0.1023034 0.1046157 0.0999959 0.08316826 0.08527892 0.08106169 1191 0.02137573 0.02223767 0.02051451 2228 2764 605 0.07398306 0.001508288 0.1231262 0.003340898 0.2188857 0.007864968
frwiki post-edit 28649 9874 0.1238606 0.126376 0.1213509 0.09654348 0.09880157 0.09429004 1435 0.02865029 0.02963839 0.02766313 2568 3324 833 0.08963664 0.001687701 0.1453312 0.00354676 0.2506017 0.007516538
frwiki control 31714 10589 0.1230304 0.1254209 0.120645 0.09200022 0.09410328 0.0899012 1894 0.03245121 0.03347216 0.03143127 2691 3631 1034 0.08485212 0.001564773 0.1788649 0.00372428 0.28477 0.00748957
itwiki pre-edit 20310 8328 0.117881 0.1210721 0.1146989 0.1004393 0.1034149 0.09747166 800 0.02071121 0.02176015 0.01966334 1669 1984 368 0.08217627 0.001927072 0.09606148 0.003229042 0.1854839 0.008726339
itwiki control 20297 9856 0.1431397 0.146633 0.139657 0.1156532 0.1188154 0.1124999 1341 0.03125235 0.03259601 0.02991044 1982 2462 549 0.0976499 0.002083567 0.1360593 0.00345347 0.2229894 0.008389027
itwiki post-edit 18359 9604 0.1442438 0.1479507 0.1405488 0.118409 0.1218074 0.1150208 1047 0.02996144 0.03128555 0.02863902 1808 2237 490 0.09848031 0.002199064 0.1090171 0.003180211 0.2190434 0.008744712

--Halfak (WMF) (talk) 15:43, 16 June 2014 (UTC)[reply]

Retrieved from "https://meta.wikimedia.org/w/index.php?title=Research_talk:Anonymous_editor_acquisition/Signup_CTA_experiment/Work_log/2014-06-16&oldid=8905448"