Template A/B testing/ImageTaggingBot Analyses
Overview [edit]
Templates invlolved in the experiment: z131 - z142 [1]
Timeframe: 2011-12-21 - 2012-02-21
Analyses Results [edit]
Counts: (Z-template - count)
131 - 107
132 - 107
133 - 114
134 - 207
135 - 188
136 - 219
137 - 42
138 - 41
139 - 43
140 - 18
141 - 23
142 - 24
Total Control: 374
Total Test: 759
Edit Activity [edit]
Case: z131 vs. z133 [edit]
Below is the measurement of all users that went on to make at least one edit in namespace 6 after receiving the ImageTagging template given that they were not blocked after seeing the template.
Test response: 9.649%
Control response: 18.69%
|
Modelling Analysis, Registered Users, z133 (test) vs z131 (control) - R Output
|
|---|
Call:
glm(formula = template ~ metric, family = binomial(link = "logit"),
data = temp_df)
Deviance Residuals:
Min 1Q Median 3Q Max
-1.250 -1.250 1.107 1.107 1.440
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.1688 0.1456 1.159 0.2463
metric -0.7667 0.4026 -1.904 0.0569 .
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 306.15 on 220 degrees of freedom
Residual deviance: 302.37 on 219 degrees of freedom
AIC: 306.37
Number of Fisher Scoring iterations: 4
|
Test Users that edited namespace 6 after the posting [edit]
[1] "Bolidist" "Chrisyasay" "Dborase" [4] "Dj Ran" "Dpsthota11" "Einie101" [7] "Erapo" "Fuse809" "HahnMahlay" [10] "Hlakungpui" "Iambonusboy" "Ilovethe2000s" [13] "Imhungry4444" "Iviscera" "Jazzbeard" [16] "JLara-cl" "Jrobin08" "Kennethcooke609" [19] "Kimmeleshkolot" "Kracoukas" "Maslusar" [22] "MasterSmcd" "Mishrasanjeew" "NEWBRAIN" [25] "NickSCFC" "Peppage" "Pg 6475" [28] "Purificationwiki" "QwerpoiXX" "RTBoughner" [31] "Salilpawar1" "Smallzc" "Swaggie73" [34] "The 5th Silver Beatle" "TheBigJagielka" "Timepricer" [37] "Wadamz" "Warriorpoet23" "WISEPHAROHPSN" [40] "Zachary.burrows"
Control Users that edited namespace 6 after the posting [edit]
[1] "1000 Volts" "Ansonjae" "Balbon32" [4] "Bloodios" "Bubertov" "D20120101" [7] "Dborase" "Dertogada" "Diango" [10] "Duncan3dc" "Emix.G.A" "Foxonline" [13] "Fuse809" "Galwaybuck" "Hafeezullah2k" [16] "Ilovethe2000s" "Iviscera" "Jazzbeard" [19] "JLara-cl" "K.yakovleva" "Kennethcooke609" [22] "KingFredrick VI" "Klr8" "Kracoukas" [25] "KristineValdheima" "Malekalameh" "MasterSmcd" [28] "MelanieBrown" "MikelZap" "Mohdwasi" [31] "Monica982" "Mthen" "Muzammil786" [34] "Naveen Ramanathan" "NEWBRAIN" "Nick Rokk" [37] "Pg 6475" "Pilch 84" "Polo phil" [40] "Prz4587ill" "Purificationwiki" "Rcoyy" [43] "Rsamahamed" "RTBoughner" "Scribblerman" [46] "Smallzc" "Sorayanwayne" "SpongePappy" [49] "Sumitkumarjha75" "Swaggie73" "The 5th Silver Beatle" [52] "TheDarkPyrano100" "Victorianny" "Voravitjessica" [55] "Wadamz" "Willtoco" "Zavila" [58] "Zywxn"
Blocking [edit]
Case: z131 vs. z132 (test) -- This result is no longer valid. For documentation purposes only. [edit]
Number of blocks after the posting was a marginally significant (p-value = .145) predictor of the template. The control template had a lower rate.
Test average blocks / user after posting: 0.06854839
Control average blocks / user after posting: 0.03212851
|
Modelling Analysis, Registered Users, z132 (test) vs z131 (control) - R Output
|
|---|
Call:
glm(formula = template ~ blocks_after, family = binomial(link = "logit"),
data = all_data)
Deviance Residuals:
Min 1Q Median 3Q Max
-1.651 -1.165 -1.165 1.190 1.190
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.03018 0.09136 -0.330 0.741
blocks_after 0.54926 0.37683 1.458 0.145
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 688.99 on 496 degrees of freedom
Residual deviance: 686.60 on 495 degrees of freedom
AIC: 690.6
Number of Fisher Scoring iterations: 4
|
Case: z134 vs. z135 (test) -- This result is no longer valid. For documentation purposes only. [edit]
Number of blocks after the posting was a marginally significant (p-value = .120) predictor of the template. The test template had a lower rate.
Test average blocks / user after posting: 0.002645503
Control average blocks / user after posting: 0.01703163
|
Modelling Analysis, Registered Users, z132 (test) vs z131 (control) - R Output
|
|---|
Call:
glm(formula = template ~ blocks_after, family = binomial(link = "logit"),
data = all_data)
Deviance Residuals:
Min 1Q Median 3Q Max
-1.147 -1.147 -1.147 1.208 1.931
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.07146 0.07156 -0.999 0.318
blocks_after -1.62419 1.04421 -1.555 0.120
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 1092.4 on 788 degrees of freedom
Residual deviance: 1088.6 on 787 degrees of freedom
AIC: 1092.6
Number of Fisher Scoring iterations: 4
|
Case: z137 vs. z139 (test) -- This result is no longer valid. For documentation purposes only. [edit]
Number of blocks after the posting was a marginally significant (p-value = .19) predictor of the template. The test template had a lower rate.
Test average blocks / user after posting: 0.03947368
Control average blocks / user after posting: 0.109589
|
Modelling Analysis, Registered Users, z132 (test) vs z131 (control) - R Output
|
|---|
Call:
glm(formula = template ~ blocks_after, family = binomial(link = "logit"),
data = all_data)
Deviance Residuals:
Min 1Q Median 3Q Max
-1.218 -1.218 1.137 1.137 1.494
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.09591 0.16882 0.568 0.57
blocks_after -0.81601 0.62203 -1.312 0.19
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 206.50 on 148 degrees of freedom
Residual deviance: 204.48 on 147 degrees of freedom
AIC: 208.48
Number of Fisher Scoring iterations: 4
|
Overall Warnings [edit]
Average warnings / user before 0.8744589 and afte 0.8556999.
Fraction of users warned before: 0.4300144
Fraction of users warned after: 0.3347763
Th latter is a significant difference.
