User:Renklauf/Archive 2

Thursday, March 1st

• Regenerating postings for z60 and z62

• much of this was taken up by debugging postings.py since all of these template postings were being missed

• chat with Dario and Karyn (separately) regarding next steps with Huggle analysis

Friday, March 2nd

• Huggle 3 analysis on specific template postings - (60,62,66,76 VS. 61,63,67,77)

Data Munging / Filtering: 

Only tracking edits in the first three days after posting
Blocks after = 0 (no blocks after seeing template), 
namespace = 0, 
first_warning = TRUE

> Non-registered

3 <= deleted edits before <= Inf 
test dataponts = 214
test dataponts = 177

> Registered:

5 <= deleted edits before <= Inf 
test dataponts = 30
test dataponts = 30


Findings:

For non-registered the mean decrease in test edits exceeded the control 83.83% and 75.02% respectively.  The result is 94.59% confident.
For registered the mean decrease in control edits exceeded the test 83.20% and 70.58% respectively.  The result is 84.00% confident.

The result of the effect is swapped between registered and non-registered users.

Modelling Analysis, Non-Registered Users - R Output

Call: glm(formula = template ~ edits_decrease, family = binomial(link = "logit"), data = all_data) Deviance Residuals: Min 1Q Median 3Q Max -1.319 -1.319 1.043 1.043 1.596 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -0.1510 0.2243 -0.673 0.5007 edits_decrease 0.4769 0.2476 1.926 0.0541 . --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: 527.28 on 383 degrees of freedom Residual deviance: 523.36 on 382 degrees of freedom AIC: 527.36 Number of Fisher Scoring iterations: 4 Percentage decrease in deleted edits Test: Min. 1st Qu. Median Mean 3rd Qu. Max. -1.6670 0.8260 1.0000 0.8384 1.0000 1.0000 Percentage decrease in deleted edits Control: Min. 1st Qu. Median Mean 3rd Qu. Max. -1.6670 0.6818 1.0000 0.7502 1.0000 1.0000

Modelling Analysis, Registered Users - R Output

Call: glm(formula = template ~ edits_decrease, family = binomial(link = "logit"), data = all_data) Deviance Residuals: Min 1Q Median 3Q Max -1.5728 -1.0761 -0.1728 1.2366 1.2894 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) 0.8939 0.6968 1.283 0.200 edits_decrease -1.1533 0.8205 -1.406 0.160 (Dispersion parameter for binomial family taken to be 1) Null deviance: 83.178 on 59 degrees of freedom Residual deviance: 81.049 on 58 degrees of freedom AIC: 85.049 Number of Fisher Scoring iterations: 4 Percentage decrease in deleted edits Test: Min. 1st Qu. Median Mean 3rd Qu. Max. -0.2703 0.4309 0.9071 0.7058 1.0000 1.0000 Percentage decrease in deleted edits Control: Min. 1st Qu. Median Mean 3rd Qu. Max. 0.0000 0.7770 0.9857 0.8320 1.0000 1.0000

• Post Optimizely and Israeli data to data hub
• email Aaron!
• diff engine queries

Monday, March 5th

• Revisited Huggle analyses - began looking at strict edit counts after the posting
• sent email to Aaron Swartz

Tuesday, March 6th

• Met with Erik M. to discuss pulling metrics from db42
• Wrote scripts to generate and plot sample counts for Huggle tests

Wednesday, March 7th

• Retooled scripts to produce plots for measuring edits up to 30 days after
• Meeting with Odiago and Zack
• Gathered together all Huggle analysis into meta-page (see links section below)
• executed time-bound query on alpha (still running)

Thursday, March 8th

• Communicate with Yusuke / Diederik regarding diff DB on alpha
• Interpret Huggle results - post to meta
• Optimizely Data - data hub

Friday, March 9th

• Prepared script to extract rev urls from text diff pattern matching: http://svn.wikimedia.org/viewvc/wikimedia/trunk/community-analysis/sandbox/src/rev_urls_from_ids.py
• Worked on analyzing Huggle results

Monday, March 12th

• Look at additional experiments - what can be said about Twinkle, ImageTaggingBot, CorenSearchBot
• PT Editor List
• Better define metrics on Huggle Analysis - what do they mean?

Tuesday, March 13th

• Summarized and corrected some mistakes with Huggle Analysis [1] - this included a better definition of the metrics used to measure the performance of the new templates in the experiment
• Tried to describe the experiments - I still need to understand how the experiments fundamentally differ

Wednesday, March 14th

• Analyses Twinkle / ImageTaggingBot / CorenSearchBot - working through Twinkle results
• Meeting with Dario and Judd on micro on effects of post-transaction

Thursday, March 15th

• Continue Twinkle analysis
• Stanford Brown bag + pre-meeting
• Huggle results w/ Steven and Maryana - it turns out the Twinkle results may be tougher to suss out than first imagined. The problem lies in the for PROD templates we need to measure whether deleted revisions actually correspond to the article on which the PROD tag was places ... similarly, for AFD articles revisions in namespace 4 should be measured

Friday, March 16th

• Begin work on Shared IP
• Gather metrics for all namespaces for Huggle experiments
• Count Template instances for Steven / Maryana using postings script

Week of March 19th

• Analyses completed for all namespaces in Huggle
• Discussed with Stu the Huggle experimental method and the shared IP experiment
• Built active editors plot: http://community-analytics.wikimedia.org/active_editors

Monday, March 26th

• Determine which tested users are still editing - query running
• Twinkle / Shared IP / ImageTaggingBot - work on one of these experiments

• generated namespace 4 edit metrics for Twinkle tests

• fix editor threshold notify - done
• revision caching - scripts active cron on aluminium

Tuesday, March 27th

• Work on revision, bot, and active editor caching
• Template Report review

Wednesday, March 28th

• Work on revision, bot, and active editor caching
• Meet with Dario / Karyn / Steven / Maryana to begin the process of defining E3 projects
• Meet with Dario / Karyn / Howie to discuss analytics specifications

Thursday, March 29th

• work on active editor caching
• Data munging for SharedIP - analyzing Stu's tables
• Analytics specing and use cases with Dario on E3 backlog projects
• Review report, pick out some plots, add content

Thursday, March 29th

• Shared IP Analysis
• Meet with Howie, Dario, Diederik, Karyn to discuss plans for a free form data store for the E3 team

Monday, April 2nd

• Largely focused on writing scripts for Shared IP testing - tried to create a table for first revision by user but this proved to be too resource intensive
• Began implementing the "SlaveLoader" python class to handle data processing from enwiki slave tables

Tuesday, April 3rd

• Meeting with Steven and Maryana regarding Huggle Report
• Sketched out data collection for Twinkle PROD testing
• Finished off shared IP analysis:

• compared archive group before bot was active and after .. it seemed that the non-archive group had more editing however the different time periods added a large amount of bias
• Looked again at groups based on header templates. Mismatched sample sizes along with large differences in pre-bot edit activity among groups introduces a bias that would make it difficult to isolate the effect of archiving

Wednesday, April 4th

• PROD data collection implementation
• WikiSym Submission
• Input on report