User talk:Halfak (WMF)

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search

Missing thread?
See the /Archive


Volunteer retention research[edit]

Been learning a lot over the years about the nature of volunteering in the wiki communities, and am digging in to help improve volunteer recruitment and retention methods. Would love to connect with you for some Q&A to better understand the research you've done in the past and where you see the biggest possibilities and challenges now. Email me to figure out when we could talk? Thanks! DrMel (talk) 23:02, 8 December 2016 (UTC)

Do we have any data on the gender makeup of early editor loss, the people who never make it to 10 or 100 edits? Did the gender makeup of editor retention change in 2007-ish? Forgive me if I've overlooked this information somewhere. HLHJ (talk) 02:56, 8 April 2018 (UTC)

Regretfully, I don't know the answer to this. I've been trying to get a proper funnel analysis of early editing (and pre-editing) dynamics in place for a while. It would be great to know where the drop-offs are around gender and other demographics. I have not been able to get the work prioritized. However, I know that LZia is looking into expanding our research around diversity of contributors next year. Maybe this is something that is on her plate. --Halfak (WMF) (talk) 14:57, 12 April 2018 (UTC)
Forgot to ping HLHJ --Halfak (WMF) (talk) 14:57, 12 April 2018 (UTC)
Thanks, Halfak (WMF) for looping me in.
Hi HLHJ, Hi DrMel. A few pointers to share here:
  • We're currently working on growing contributor diversity as part of the current annual plan (the program). We're done with Objective 1 and 2, and we are working on Objective 3 this quarter (April-June). If you want to get pings on the status of Objective 3, please follow Phabricator:T190776 and Phabricator:T190775. If you want more detailed information about each of these tasks, please check the research on eliciting new editors' interests as well as designing a framework for increasing the retention of women in Wikipedia's editor pool. The parent research for both of these efforts where you can find some literate review may also be interesting for you.
  • The above is current year's work and this year ends at the end of June 2018 for us. What we have proposed to do as part of next year's annual plan (pending FDC+Board review, recommendation, approval) is:
  • Finalize the model for editor-editor matching recommendation. Think of this as something that can be helpful for both newcomers and experienced editors: what if we had a system where you could get recommendations about what editors have similar interests to you? Or, what if we could learn about the interests of newcomers fast, without having to wait for them to interact with our systems for a long time?
  • Further tune and test the framework that we hope to help us keep a more diverse editor pool on Wikipedia (and possible other Wikimedia projects).
  • Create baseline statistics that can help us understand the map of diversity at many stages of the editing funnel, something Halfak mentions above.
The pointer to the annual plan is available for your review. Please check Output 2.2 (which is focused more on readers) and Output 3.1 and 3.2 (continuation of this year's research linked above), and Output 3.3 where we want to create a map of the different stages of the editing to understand the state of the diversity better. Note that this is beyond gender diversity and I'd like for us to at least try to include ethnicity in it as well.
I hope this overview helps. And please keep in touch as you seem to be interested in this space. :) --LZia (WMF) (talk) 19:34, 12 April 2018 (UTC)
Thanks to Halfak and LZia (WMF). I'm sorry, I managed to miss this notification. There is an interesting intersection here. If, as Research:Voice and exit in a voluntary work environment says, we are short on women because women have less self-confidence and are more likely to shy away from conflict, then new editors leaving because they face hostility and are made to feel useless are more likely to be female. The research suggests that rejection and futility discourage editors. So the gender bias in editor retention should correlate with the editor retention rate, assuming the theory is at least partly right. A parallel hypothesis for non-dominant cultures would predict a parallel correlation.
I really appreciate the efforts being made on this. I'm afraid I'm going to be critical here, please take it as attempted support rather than hostility for your efforts. From personal experience, I've never felt any need of help finding things to edit; I could keep a dozen of me busy. Nor did I want to interact with people; there are better places to do that. I wanted to make useful edits. If I'd felt more insecure, I might have wanted explicit reassurance that my edits were useful, like, say, thanks notifications. Making useful edits was much easier before automated tools made it easier to revert edits than fix them. The research seems to me to suggest that we need ways to increase fixing and decrease reverting of good-faith edits by new users. While focussing on modifying the behaviour of new editors is easier politically, I suspect it will not be as effective as modifying the behaviour of established editors. HLHJ (talk) 02:40, 22 October 2018 (UTC)

Possibly investigate data regarding Single Edit Tab and Visual Editor[edit]

Some time ago Single Edit Tab was deployed on EnWiki, with a default placing all new users into VE first. The default was latter reversed, starting all new users in the Wikitext editor. This was not a controlled experiment, but it might be worth checking whether any noticeable signal jumps out at the point of initial deployment or at the point that the default was reversed. I'd love to see a simple plot of the data over time, with critical points marked, just to eyeball whether there was some visible jump. We could look at the percentage of editors who successfully make a first edit, editor retention, or anything else you think might be relevant.

Does this sound worth doing? Alsee (talk) 14:31, 5 February 2017 (UTC)

Hey Alsee. I'm not too familiar with the history of these deployments since I haven't been actively involved in the development of VE for a while. Do you know the dates of deployment? What kind of stats would you want to look at during these time periods? I wonder if Neil P. Quinn-WMF has already done some exploration. --Halfak (WMF) (talk) 15:33, 6 February 2017 (UTC)
@Alsee: No, I haven't explored that. It's an interesting question but I'm not sure looking at the graphs would be all that illuminating since there's a lot of baseline variation in that data. You're welcome to file a task requesting that analysis by following the instructions at mw:Editing/Analysis, but the reality is that I have a very long backlog and I'm not likely to be able to do it any time soon.—Neil P. Quinn-WMF (talk) 18:17, 6 February 2017 (UTC)
It's possible that we could so something basic, but useful, with a few Quarry queries. Alsee, are you familiar with Quarry? I'd be happy to give you a hand with picking up some of the basics. --Halfak (WMF) (talk) 22:57, 6 February 2017 (UTC)
@Neil P. Quinn-WMF: and Halfak. Thanx.
  • Here's the Phab link[1] for the Single Edit Tab deployment. Successful gerrit merge[2] Apr 12 2016, 11:02 AM. It was definitely live on EnWiki prior to Apr 12 2016, 4:30 PM. A 5 hour window at most.
  • Here's the Phab link[3] for changing the VE-default to a Wikitext-default. Successful gerrit merge[4] on May 19 2016, 7:22 PM. I'm not sure if that's when it went live on EnWiki, or if it was soon after that.
I know any signal will probably be lost in the noise, but if there is a signal big enough to spot then it's a signal that's worth knowing. The bonus here is that this is a two-for-one deal. One search can span the two deployment dates, and two significantly different potential signals. I'm flexible on what to look at, but my general thought was some of the same basic data that were checked in the May 2015 Visual Editor experiment. For new accounts: what percentage made at least one edit (and maybe at least 5 edits?), one week editor survival (I think 3 month survival was checked?), total edits. Whatever is easy to pull out of the database, whatever you have time for, whatever you think might turn up meaningful data.
I would love to pick up Quarry skills, and as a programmer I'm sure I could. The examples I've seen are semi-readable even with zero knowledge. I don't think I can dive into it now though. Do you have a good link on it that I can bookmark? Alsee (talk) 00:53, 7 February 2017 (UTC)
@Alsee: Hey Alsee. Sorry for the short response, but since you are a programmer, yes, I'm sure you'd be able to deal with Quarry quite easily! The documentation is located at Research:Quarry and provides a decent introduction to the basic Quarry-specific bits. Other than that, you just have to know (1) SQL, which you can learn in about a thousand places online if you're not familiar with it—I've used the Tech on the Net tutorial before and it's pretty good and (2) the way the MediaWiki database is laid out, which is documented at mw:Manual:Database layout—as it says there, the most important tables for you are page, revision, and user tables.Neil P. Quinn-WMF (talk) 02:12, 23 February 2017 (UTC)

Your feedback matters: Final reminder to take the global Wikimedia survey[edit]

(Sorry to write in Engilsh)

A barnstar for you![edit]

Barnstar standard.png The Original Barnstar
Thank you for your interesting lightning talk at the Wikimedia Conference 2017. Ijon (talk) 08:43, 4 April 2017 (UTC)

Update regarding Wikipedia and Wikimedia Commons tutorial videos[edit]

Hello,

I regret to inform you that the series of motivational and educational videos project, which had been planned introduce Wikipedia and some of its sister projects to new contributors, is being discontinued.

There are multiple factors that have led to this decision. The initial budget and time estimates were far too small for a project of this scale and complexity. Also, my simultaneous involvement in Cascadia Wikimedians User Group was problematic due to the shortage of human resources for the user group, which resulted in my spending far more time trying to help the user group than I had planned, so my time and attention were diverted from this video project to assisting the user group.

You can find more information in the final report for the grant.

I regret that this project did not fulfill the hopes that many of us had for it, and I hope that in the future someone with the necessary resources will choose to resume work on it or a similar project. If you are interested in working on this or a similar project then please contact the WMF grants team.

On a personal note, I am retiring from the Wikimedia community. Perhaps I will return someday.

Regards,

--Pine 23:20, 30 August 2017 (UTC)

Series director and screenwriter

Join ORES project[edit]

Hi!

I've just discovered New Developers page now. And I want to know if it is possible to join the project.BamLifa (talk)

Hi BamLifa! Yes. You're very welcome. The best way to work with us to to find us in IRC. See #wikimedia-aiconnect. I'm 'halfak'. 'Amir1' and 'awight' should also be able to help you get started if I'm not around. --Halfak (WMF) (talk) 18:42, 1 September 2017 (UTC)

Hey Aaron[edit]

Hello, Aaron, i thought i let you know that your userpage is a bit outdated, as you still are mentioning the volunteer project, Revision scoring as a service, which is now obviously Scoring Platform Team. Anyway, have a great week! Zppix (talk) 19:26, 7 November 2017 (UTC)

Yes check.svg Done! Thanks. --Halfak (WMF) (talk) 11:18, 13 November 2017 (UTC)

Oauth Consumer proposal[edit]

Hello Halfak! Sometime ago, you approved my oauth application Chlambase.org with your EpochFail account. I have created another model organism database for a new pathogen, Myxococcus xanthus and I could use your help again to get my oauth application approved! It should be identical to Chlambase.org. Here is the proposal. Thank you for any help! Derek (talk) 18:42, 30 August 2018 (UTC) @EpochFail:

Just got back from Vacation. Yes check.svg Done BTW, I'm still active as EpochFail and my work as an OAuth admin is all volunteer so I do it from that account. :) --Halfak (WMF) (talk) 14:35, 4 September 2018 (UTC)

Mail[edit]

--Rosiestep (talk) 14:52, 20 November 2018 (UTC)

Your Thoughts[edit]

I noticed that you are the Foundation contact for Research:How much do Wikipedians in the US value editing Wikipedia? There has been resistance to the talk page requests sent by the researcher at their English WP talk page. Would you or the foundation have anything to add to the discussion there? Thanks! -- Dolotta (talk) 16:32, 19 December 2018 (UTC)

Hi Dolotta! I've responded from my volunteer account, User:EpochFail, since I'm not working on this in my official role as a staff member. Thanks for your ping. --Halfak (WMF) (talk) 20:58, 19 December 2018 (UTC)
Thanks for letting me know. Happy editing! -- Dolotta (talk) 22:43, 19 December 2018 (UTC)

Question regarding technical research for Wikivoyage[edit]

Hello. Apologies if you are not the right person to whom I should pose this question.
I am an administrator at the English Wikivoyage. There, we are considering a proposal to supplement redlinked articles with an automatic link to the corresponding Wikipedia article. However, there was some concern that adding such a link would disincentivize creations of these articles locally. We are a relatively small community, and creations of articles for travel destinations by new or anonymous users are a regular occurrence, and quite valued. We were hoping it would be possible to gather some kind of data on this. Beyond making the change for a trial period and comparing the number of page creations before and during, none of us have any ideas. Of course, not all page creations come from redlinks, and so a more sophisticated test would be ideal. Is it possible to do this within the Research framework? If so, what would we need to do?
Any help or resources you can give us would be immensely appreciated. Thanks, ARR8 (talk) 01:12, 31 March 2019 (UTC)

Hi ARR8! I could possibly help you with this. Generally, I would recommend running a trial for whole week long periods to address the periodic nature of week days and weekends. Just how many weeks you'd need to run the trial for in order to see if there is an effect is a more complicated question that I'd need to do some analysis to explore. Essentially we want to know how long the trial would need to run in order for us to get enough data to find any substantial change to be statistically significant. Do you have a sense for how much of a change would be meaningful? E.g. if article creation rates dropped off by 5%, would that be too much? Or would it need to be something more like 20% in order to matter? --Halfak (WMF) (talk) 21:10, 31 March 2019 (UTC)
Thank you! My guess is that ~8-12% would be a good thershold (corresponding to around half a standard deviation by some very cursory statistical measures), but some of our editors agree that 5% would be noticeable. ARR8 (talk) 03:20, 1 April 2019 (UTC)
ARR8 Can you show me how you arrived at that standard deviation? I could probably use that to do a statistical power analysis. --Halfak (WMF) (talk) 13:44, 1 April 2019 (UTC)
Sure. It's not a very meaningful statistic, but I used the auto-generated number of content page creations here (which may be your own creation?) going back two years. This would include all articles by every editor, though, not just destinations by new users, and maybe also includes non-mainspace page creations. ARR8 (talk) 15:33, 1 April 2019 (UTC)
ARR8 I see. This is useful actually we can make some good estimates from it. One thing that is strange though is that it doesn't look like there are *any* anon page creations. Could that be a bug with the data in this tool? (The tool is not my work but I helped a bit). --Halfak (WMF) (talk) 18:35, 1 April 2019 (UTC)
Definitely a bug. One of our editors came up with another set of numbers using a different tool, listing only destination articles. Maybe this is more helpful? ARR8 (talk) 22:41, 1 April 2019 (UTC)
I just tried my own query. https://quarry.wmflabs.org/query/34844 gets the daily article creations by anons for the last 30 days. I get a mean of 40 articles and a standard deviation of 22.7. If we ran the trial for 2 months, we could find statistical significance for changes bigger than + or - 8 articles per day (32 - 48 avg). We'd have to run the trial for a whole extra month to notice differences smaller than thank + or - 7.
That said, we could see if there is a huge change (e.g. going from averaging 40 articles per day from anons to 20 articles per day from anons), in 1-2 weeks.
Would that fit into the timescale y'all are thinking about for the experiment? --Halfak (WMF) (talk) 22:57, 1 April 2019 (UTC)
Sorry for the delay. Those times work fine, and I think the community would choose to run it for the extra month. Thanks again. What are our next steps? ARR8 (talk) 01:52, 5 April 2019 (UTC)

┌─────────────────────────────────┘

ARR8 Oh! One other question. I'm concerned about seasonal fluctuations. We often see changes in editing behavior due to kids going on summer vacation/going back to school. Or due to the winter holiday season. Judging from the graph of page creations, it looks like there aren't any major seasonal fluctuations that we should be worried about. The graph looks mostly flat from 2014 forward (except for a minor dip in early 2016 that seems to not repeat).
Otherwise, I think the next step is to schedule the change you want to make. From there we can project out when we'll want to do analysis to see if you want to reverse the change. I'm thinking that we'll want to have a look two weeks in, one month in, and then two months in assuming all is going well by that point. It shouldn't be too much trouble for me to run some basic analysis for you at those points. Just confirm the dates with me before you kick it off and I'll make sure I'm available. --Halfak (WMF) (talk) 13:31, 5 April 2019 (UTC)
I agree that seasonal variations don't seem to be a major factor. One source of concern brought up in the community, though, was that measuring anonymous-user page creations may not be a perfect proxy for redlinks-turned-into-articles, and it was requested that this be directly measured. Is this possible, or necessary?
In either case, we're ready to switch the template at any time. We have consensus to run the test. Let's say tomorrow, assuming nothing further can/should be done; how's that sound? ARR8 (talk) 03:49, 7 April 2019 (UTC)
ARR8: I'm not sure how I'd measure red links turned into articles. It turns out that's hard to get from the database. Generally, I'm thinking that if most anon-created articles come from redlinks, then any effect will be visible in the total article creations. If a small proportion of anon article creations come from redlinks, then (and this is a bit hazardous to say) maybe it doesn't matter if the automatic link causes a drop in such article creations. Either way, if it proves to be essentially, I have some ideas about how we might get at it after-the-fact. Otherwise, I think now is a fine time to kick off the change. Please note the time when you make the change here so I can reference it later :) --Halfak (WMF) (talk) 23:23, 10 April 2019 (UTC)
Sure. I made the change yesterday: diff and timestamp here. Thanks again. ARR8 (talk) 21:28, 16 April 2019 (UTC)


Hi, any update on this? ARR8 (talk) 23:30, 16 May 2019 (UTC)

ARR8, I can try to take a look at this next week. I'll post an update here when I get some analysis done. :) --Halfak (WMF) (talk) 07:50, 17 May 2019 (UTC)