Research talk:VisualEditor's effect on newly registered editors/May 2015 study/Work log/2015-06-08

From Meta, a Wikimedia project coordination wiki

Monday, June 8, 2015[edit]

Time for some preliminary stats!

  • editing == made at least one edit in 24h
  • main_editing == made at least one article edit in 24h
  • talk_editing == made at least one talk edit in 24h
  • user_editing == made at least one user/user_talk edit in 24h
  • wp_editing == made at least one wp/wt edit in 24h
  • productive == made at least one article edit that was not reverted in 24h
  • enabled == currently has "visualeditor" user property enabled.
bucket via_mobile editing.k main_editing.k talk_editing.k user_editing.k wp_editing.k productive.k enabled.k n
control 0 3227 2461 427 782 86 1682 110 9794
experimental 0 3250 2397 386 822 107 1669 9693 9728
control 1 928 795 85 126 15 414 6 3670
experimental 1 951 819 93 150 14 431 3775 3779

First, the most important test IMO, productivity:

prop.test(c(1682, 1669), c(9794, 9728))

	2-sample test for equality of proportions with continuity correction

data:  c(1682, 1669) out of c(9794, 9728)
X-squared = 2e-04, df = 1, p-value = 0.9898
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.01051037  0.01085275
sample estimates:
   prop 1    prop 2 
0.1717378 0.1715666 

Well... that's not significant. What if we make the denominator editors who make at least one edit?

> prop.test(c(1682, 1669), c(3227, 3250))

	2-sample test for equality of proportions with continuity correction

data:  c(1682, 1669) out of c(3227, 3250)
X-squared = 0.3532, df = 1, p-value = 0.5523
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.01695828  0.03233565
sample estimates:
   prop 1    prop 2 
0.5212271 0.5135385

Still insignificant. What about those who make at least one article edit?

> prop.test(c(1682, 1669), c(2461, 2397))

	2-sample test for equality of proportions with continuity correction

data:  c(1682, 1669) out of c(2461, 2397)
X-squared = 0.8744, df = 1, p-value = 0.3497
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.03924835  0.01359831
sample estimates:
  prop 1   prop 2 
0.683462 0.696287 

Still insignificant. --Halfak (WMF) (talk) 22:04, 8 June 2015 (UTC)[reply]


It looks like we might find a difference in talk page usage. Let's look at that.

> prop.test(c(427, 386), c(9794, 9728))
...
X-squared = 1.781, df = 1, p-value = 0.182

No significance, how about if we only look at editors who edit at all?

> prop.test(c(427, 386), c(3227, 3250))
....
X-squared = 2.5871, df = 1, p-value = 0.1077

Closer, but still not significant. --Halfak (WMF) (talk) 22:09, 8 June 2015 (UTC)[reply]

What are they doing in the WP/WT namespace?[edit]

I was curious what the small set of users who work in the WP namespace are doing. I suspect they are socks or legitimate alternative accounts who don't represent newcomers. Let's check what they are up to.

> sample(user_metrics[wp_editing==1 & !via_mobile,]$user_id, 10)
 [1] 25331019 25358683 25373415 25357718 25393716 25367694 25347246 25386145
 [9] 25383014 25352645

http://quarry.wmflabs.org/query/3931

Looks like only one of the set is actually a sock. It also looks like it is common for newcomers to find the WP:Sandbox and to request article creations despite the fact that they could create them. --Halfak (WMF) (talk) 22:26, 8 June 2015 (UTC)[reply]

FYI that the en.wp Sandbox now does mention options to test VE even for IP editors (it'd be interesting to see if this has any impact whatsoever on VE-enabled sandboxes. --Elitre (WMF) (talk) 16:02, 15 June 2015 (UTC)[reply]