In this study, we will two versions of the pre-edit workflow to a control.
- A control, in which an anonymous (IP) editor see no differences -- represents the current user experience where no calls to action appear before a user registers.
- A pre-edit v1, Call to action which is triggered by an anonymous editor clicking any of the edit buttons on an article (same as "pre-edit" from study 1)
- A pre-edit v2, Call to action which is triggered by an anonymous editor clicking any of the edit buttons on an article that emphasizes the optional nature of registration and renders "continue editing" as a button.
English Wikipedia, pre-edit v1
English Wikipedia, pre-edit v2
Rationale: We suspect that the primary reason for reduced productivity in the pre-edit condition from study 1 is that some anonymous editors will refuse to register an account and the CTA did not make it clear that registering an account was optional. We specifically designed pre-edit v2 to make this optional nature clear. We suspect that by making this clear, we can recover those productive edits that were lost due to the confusion.
Hypothesis 1: Anonymous editors in the pre-edit v2 condition will be more productive than anonymous editors in the pre-edit v1 condition.
Rationale: If anonymous editors edit with some regularity, then the pool of anonymous editors may be drained through registration CTAs. If this is true, we should see lower conversion rates for a follow-up experiment.
Hypothesis 2: A smaller proportion of anonymous editors will register in pre-edit v1 than in pre-edit from study 1.
A change was deployed to English Wikipedia that enabled bucketing anonymous users into the experimental buckets at approximately 2014-07-10 00:00:00 UTC. Experimental CTAs were shown to users for approximately 7 days -- until 2014-07-17 00:00:00 UTC.
|baseline n (estimate)||4754.273|
|v1 %||1.871 (+0.504)|
|v2 %||1.59 (+0.223)|
|v1 factor %||136.9|
|v1 n estimate||6508.355|
|v2 factor %||116.4|
|v2 n estimate||5531.674|
- Analysis code repository: https://github.com/halfak/Anonymous-phenomena