IRC office hours/Office hours 2016-07-28
Chat on WMF Metrics Meeting
28 July 2016
18:00 - 19:00 UTC
[18:02:06] <guillom> Meeting is starting now; watch at https://www.youtube.com/watch?v=EbRLlhFS4f8
[18:02:47] <matt_flaschen> Happy anniversary, quiddity and everyone. :)
[18:03:34] <bd808> My hire date anniversary isn't until tomorrow ;)
[18:03:58] <i_jethrobot> bd808: so close!
[18:03:59] <guillom> MarÃa Cruz now giving the Community update
[18:05:36] <andrewbogott> Did I miss the anniversary slide? dang
[18:05:51] <rfarrand> aaron has 8 years andrewbogott
[18:05:53] <rfarrand> :D
[18:06:00] <guillom> Edward Galvez is now going to talk about Community Engagement Insights
[18:06:08] <bd808> AaronSchulz: thanks for staying so long and fixing so many things
[18:06:10] <andrewbogott> it's my 4th I think
[18:06:21] <guillom> Community Engagements Insights on Meta: https://meta.wikimedia.org/wiki/Community_Engagement_Insights
[18:08:18] <guillom> Neil Quinn now talking about high-level editing metrics.
[18:10:32] <i_jethrobot> Interesting spikes of new active editors in January each year...
[18:13:14] <matt_flaschen> Question: How is the number of mobile anon active editors estimated?
[18:13:28] <guillom> matt_flaschen: Queued.
[18:13:42] <guillom> (thank you for being our first question-asker!)
[18:13:59] <CKoerner_WMF> Am I seeing a slight dip in the summer months?
[18:14:28] <Pine> Maggie is today's feature? :)
[18:14:38] <guillom> Maggie Dennis now talking about Community culture: Harassment and healthy environments
[18:14:57] <matt_flaschen> Can we switch to the slide view?
[18:15:00] <guillom> CKoerner_WMF: Do you want me to ask this at the end of the meeting?
[18:15:11] <matt_flaschen> Thanks
[18:15:38] <CKoerner_WMF> @guillom No, that's ok. I can follow up later.
[18:15:45] <guillom> CKoerner_WMF: ok
[18:16:31] <guillom> Links from the slide:
[18:16:33] <guillom> Online harassment resource guide: https://meta.wikimedia.org/wiki/Research:Online_harassment_resource_guide
[18:16:44] <guillom> Online Community Conduct Policies: https://meta.wikimedia.org/wiki/Research:Online_Community_Conduct_Policies
[18:16:50] <MBeat> ty for links
[18:16:55] <guillom> Harassment consultation: https://meta.wikimedia.org/wiki/Harassment_consultation_2015
[18:17:03] <guillom> 2015 Harassment survey: https://meta.wikimedia.org/wiki/Research:Harassment_survey_2015
[18:17:07] <matt_flaschen> Thanks, guillom
[18:18:24] <guillom> Results of the harassment survey https://commons.wikimedia.org/wiki/File:Harassment_Survey_2015_-_Results_Report.pdf
[18:19:31] <pizzzacat> wow :(
[18:19:47] <guillom> yup. It's not just theoretical :(
[18:20:14] <pizzzacat> sad.
[18:20:23] <gwicke> re editor trends: one obvious alternate metric to look at is the number of *edits* (rather than editors); https://stats.wikimedia.org/EN/PlotsPngEditHistoryTop.htm shows a slight increase in the number of both authenticated & anon edits in 2015
[18:20:49] <guillom> Event Ban policy: https://meta.wikimedia.org/wiki/Event_Ban_policy
[18:20:49] <guillom> Friendly space expectations: https://meta.wikimedia.org/wiki/Grants:Friendly_space_expectations
[18:20:49] <guillom> Code of Conduct (draft): https://www.mediawiki.org/wiki/Code_of_Conduct/Draft
[18:20:49] <guillom> Conflict management pilot module: https://meta.wikimedia.org/wiki/Community_Capacity_Development/Conflict_management
[18:21:16] <i_jethrobot> Inspire Campaign on addressing harassment: https://meta.wikimedia.org/wiki/Grants:IdeaLab/Inspire
[18:21:17] <guillom> Inspire Campaign: https://meta.wikimedia.org/wiki/Grants:IdeaLab/Inspire
[18:21:24] <guillom> Hah too slow!
[18:21:31] <i_jethrobot> the double! : P
[18:21:34] <guillom> :D
[18:24:23] <guillom> Ellery Wulczyn and Nithum Thain now presenting their research on Modeling personal attacks on English Wikipedia: https://meta.wikimedia.org/wiki/Research:Detox
[18:25:08] <matt_flaschen> Thanks, Maggie.
[18:25:37] <K4-713> Yes. Thank you, Maggie! So glad we're talking about this.
[18:27:02] <neilpquinn> gwicke: yes, edits are definitely an option. But changes in number of human edits tend to be dwarfed by changes in bot activity, and the number of edits is a very crude measure of the amount of value added.
[18:27:45] <neilpquinn> so I'm particularly interested in Aaron's concept of persistent word revision, and in using a last-edited cookie to better estimate the number of anonymous editors.
[18:27:47] <AndyRussG> On the YouTube feed, Ellery is a lot softer than others...
[18:28:00] <gwicke> neilpquinn: the graphs break out bot edits, so that shouldn't be an issue; but, agreed that edits are not all created equal
[18:28:19] <matt_flaschen> Yes, it's a bit hard to hear him.
[18:29:01] <guillom> Demo: https://wikidetox.appspot.com/
[18:30:02] <brendan_campbell> sorry about the volume difference...ellery isn't speaking directly into the microphone. i will adjust
[18:31:13] <matt_flaschen> The 'punch' classification is pretty impressive.
[18:32:14] <i_jethrobot> Agreed.
[18:32:22] <Pine> I like the algorithmic approach. I had a variety of questions about possible sampling bias in the harassment survey. The algorithmic approach is imperfect too but opens up a lot of interesting possibilities for investigating prevalence and effects.
[18:32:27] <marktraceur> https://lut.im/5n6bVLIUNE/919k5WLncgzHkz24.png I'm sold, this is sophisticated
[18:32:42] <neilpquinn> gwicke: true, but even that filtering leaves a lot of bots in. If you look at the global edits graph from our last quarterly review (https://meta.wikimedia.org/w/index.php?title=File:Editing_department_%E2%80%93_Quarterly_Review_slide_deck,_2015%E2%80%9316_Q4.pdf&page=11), that filtered out all the flagged bots but the whole rise in 2015 and 2016 is
[18:32:43] <neilpquinn> unflagged bots, mainly on Wikidata.
[18:32:53] <rfarrand> haha! marktraceur
[18:33:06] <SMalyshev> doesn't recognize the Monty Python taunts though
[18:33:18] <marktraceur> Maybe it was trained on Shakespeare.
[18:33:34] <Pine> Hmm. I wonder if we could come up with an anti-snuggle for patrolling attacks that get flagged by the algo.
[18:35:07] <SMalyshev> "Your mother was a hamster and your father smelt of elderberries!" is 60% attack
[18:35:14] <Pine> Heh.
[18:35:21] <AndyRussG> brendan_campbell: thx!
[18:35:29] <K4-713> SMalyshev: :)
[18:35:53] <guillom> Joe Matazzoni now presenting new developments related to the Notifications feature: https://www.mediawiki.org/wiki/Notifications
[18:36:03] <gwicke> neilpquinn: I see; I guess the take-away is that edits are solid overall (possibly slightly trending up), but the magnitudes are in the margin of error
[18:36:25] <Vahid> Super interesting analysis on the harassment issue.
[18:36:43] <guillom> Vahid: For more info, see https://meta.wikimedia.org/wiki/Research:Detox
[18:37:21] <neilpquinn> gwicke: yes, I think that's accurate. halfak's work recently suggests that existing editors have been increasing productivity, so it looks like somewhat fewer editors are doing the same amount of work.
[18:38:24] <gwicke> nod�
[18:38:54] <neilpquinn> Pine: I quite like that ideaâ€”I'd use a tool like that.
[18:39:46] <Pine> neilpquinn: I get the impression that the researchers are heading in that direction, since they mentioned something about ORES integration.
[18:39:58] <Pine> guillom: are you our Q&A person today?
[18:40:08] <guillom> Pine: Yes, I'm JamesF for an hour.
[18:40:36] <guillom> Minus the class and British accent.
[18:40:42] <Vahid> thanks @guillom
[18:40:46] <Pine> guillom: cool. Would you please ask if there is a plan to create a Snuggle-like tool for patrolling edits that get flagged by the attack detection algorithm?
[18:41:07] <guillom> Pine: Queued.
[18:41:15] <i_jethrobot> Question for Ellery/Nithum: Does Wikipedia Detox evaluate text holistically, or does it basically answer the question, "Does this text contain an attack or an aggression somewhere in it?" I'm curious about whether it currently can or will be able to handle longer amounts of input.
[18:41:26] <gwicke> neilpquinn: what is currently the best place to look for edit stats that cover 2016 as well?
[18:41:38] <matt_flaschen> There was also some discussion about such potential applications of Detox at Wikimania.
[18:42:03] <Seddon> guillom: I have a question
[18:42:20] <guillom> i_jethrobot: queued
[18:42:28] <i_jethrobot> Thanks!
[18:42:40] <guillom> Seddon: tell me and we'll see if we have time ; there are 4 people lining up here :)
[18:42:53] <Seddon> guillom: Are there any plans to be able to do geolocated Notifications
[18:43:03] <guillom> Queued
[18:43:13] <ebernhardson> what would geolocated notifications be?
[18:43:39] <matt_flaschen> ebernhardson, I guess only showing notifications to people in a certain area, like https://en.wikipedia.org/wiki/Wikipedia:Geonotice
[18:43:41] <guillom> Presumably like geonotice
[18:43:48] <i_jethrobot> ^
[18:43:55] <i_jethrobot> That's what I thought, too.
[18:44:04] <matt_flaschen> Notifications are not currently supported as a messaging platform like sitenotice, thus there is no real need to filter by geo either.
[18:44:45] <matt_flaschen> But of course it's possible we would look at either or both in the future.
[18:44:47] <milimetric> we can redact edit summaries though, no? with rev_visibility?
[18:45:06] <ebernhardson> in terms of implementation, creating a piece in echo that uses geo to source users to deliver notifications to would be possible, although i don't know we actually record things like a geo location for users to calculate that against
[18:45:21] <ebernhardson> but also echo is not particularly optimized for delivering smething to thousands of users
[18:45:27] <Vahid> Are there plans to auto-ban the bad actors that belong to the group of few edits with high personal attacks (probable puppets)?
[18:45:28] <milimetric> sorry, rev_deleted: https://www.mediawiki.org/wiki/Manual:Revision_table#rev_deleted
[18:45:47] <guillom> Seddon: has your question been answered by the comments here, or should I keep it in the queue?
[18:45:57] <matt_flaschen> milimetric, yes: https://www.mediawiki.org/wiki/Help:RevisionDelete#Hiding_a_revision
[18:46:10] <guillom> Vahid: No, but there are possible plans to highlight them in anti-vandalism / patrolling tools.
[18:46:34] <i_jethrobot> Vahid - Edit filters are also another option.
[18:47:01] <i_jethrobot> https://en.wikipedia.org/wiki/Wikipedia:Edit_filter
[18:47:08] <Vahid> and then ban them?
[18:47:11] <guillom> Vahid: This relates to Pine's question though, which I'll ask in a few minutes
[18:47:34] <greg-g> we can't hear
[18:47:37] <pizzzacat> what was said?
[18:47:47] <greg-g> guillom: repeat the asnwer?
[18:47:57] <milimetric> "how do we make these tools work together? (harassment detection and notifications)
[18:48:11] <guillom> greg-g: Sorry, not sure which one you mean :/
[18:48:24] <greg-g> guillom: whoever was talking in response to Toby who wasn't on a mic
[18:48:42] <guillom> greg-g: ah, Amanda was saying Toby's question/comment was unclear
[18:48:48] <guillom> So he rephrased
[18:49:02] <Seddon> guillom: that's fine for me. I will update the phab ticket with that feedback :)
[18:49:08] <guillom> ok!
[18:49:08] <i_jethrobot> Vahid - The idea for making edit filters more sensitive to personal attacks / interpersonal communication / edit sumamries has been forwarded by some community members during the most recent Inspire Campaign.
[18:49:11] <urandom> providing that kind of feedback would also help people get around the detection
[18:49:16] <greg-g> guillom: thanks :)
[18:50:47] <matt_flaschen> If only there was some system in which comments were distinct objects structured into topics.
[18:50:58] <guillom> heh
[18:51:33] <matt_flaschen> I was actually considering asking how they identified comments in the current Detox work.
[18:51:45] <matt_flaschen> Vs. refactoring, editing someone's post, etc.
[18:51:46] <milimetric> yeah, /me thinks flow should get a lot more priority than it has
[18:52:28] <gwicke> urandom: a determined adversary will always be able to get around it, but it could still be successful at nudging the majority of angry users towards more civil language before hitting save
[18:53:02] <marktraceur> Just have clippy pop up and say "it looks like you're trying to be an asshole. would you like some help with that?"
[18:53:08] <urandom> ha!
[18:53:28] <milimetric> marktraceur: the clippy idea came up in four different groups at Wikimania :)
[18:53:35] <marktraceur> milimetric: Not surprising
[18:53:43] <milimetric> not saying you're not original, just saying - you're tapping into a real need!
[18:53:44] <marktraceur> We should use something else, though. Puzzly maybe?
[18:53:49] <rfarrand> clippy is an excellent role model
[18:54:11] <milimetric> clippy as punishment seems apt
[18:54:35] <milimetric> brilliant gwicke
[18:54:44] <K4-713> I support all these ideas.
[18:54:45] * milimetric adds clippy to dashiki dashboards
[18:54:58] <i_jethrobot> "instant nostalgia"
[18:55:07] <reled> Ok
[18:55:08] <i_jethrobot> ...optimistic, perhaps. : P
[18:55:36] <guillom> Great question by Robert about gender bias in rating
[18:56:33] <reled> Yeah
[18:56:43] <Pine> brendan_campbell: just for future reference, the camera facing the presenters has low brightness. Might want to check that for next time.
[18:56:43] <mooeypoo> Yeah
[18:56:52] <guillom> Sati asking about how the different teams working on harassment "stuff" comes together and integrates.
[18:57:19] <guillom> come*
[18:57:40] <guillom> Patrick Earley responding
[18:57:55] <matt_flaschen> Indeed. Thanks to Patrick.
[18:57:59] <brendan_campbell> Pine: currently trying to find the best solution to this. it's actually a lighting problem, not a camera problem. i've installed some stage lights that has helped the problem, but it's always a balance between properly lighting the stage, not washing out the projectors, and not blinding the presenters
[18:58:04] <mooeypoo> Isn't it a hard (technical mostly) question to answer in Wikipedia, though? the gender tags for users are used to identify users? I mena, I know that in gendered languages that is a bit more consistent, but I am not sure -- how good is that indicator in enwiki, for instance
[18:58:06] <brendan_campbell> you are right though!
[18:59:02] <i_jethrobot> brendan_campbell: Thanks for your effort in trying to find the right balance. : )
[18:59:09] <matt_flaschen> mooeypoo, I agree. I think a lot of people leave it default (unknown gender), either just because people don't change preferences in general or specifically for privacy.
[18:59:23] <matt_flaschen> At one point we ran a gender survey to get confidential data.
[18:59:47] <reled> Ok
[18:59:54] <reled> Good idea
[19:00:17] <mooeypoo> matt_flaschen and some people may also purposefully choose a certain gender to hide their own. Also there are issues with gender being a binary choice (we don't ask for gender in mediawiki, we ask for *language* preference - that can also affect the answers
[19:00:34] <matt_flaschen> Yep
[19:00:38] <mooeypoo> matt_flaschen, and I'm not saying it's impossible, I'm just pointing out these things complicate matters too.
[19:00:45] <milimetric> mooeypoo: we have reconstructed user name history (what names someone has been known by since they registered)
[19:00:57] <milimetric> one interesting way to analyze that is to see if name changes happened after attacks
[19:01:08] <guillom> Kitten time!
[19:01:10] <Pine> Quick guess: administrators are probably used to harrassment and brush it off, but the less experienced a user is the more likely they are to disappear if attacked.
[19:01:11] <milimetric> and then look at some of those and see if the implied gender of the name changes
[19:01:20] <K4-713> Hmmmm.
[19:01:28] <guillom> ... Or Thai food.
[19:01:30] <matt_flaschen> There's plenty of work still to be done, especially creating articles. Wikipedia may seem to have relatively complete coverage to an untrained eye, but that really just means "The kinds of English-language/my group/my culture" things I look up are usually there.
[19:01:37] <guillom> (no cats in the Thai food)
[19:01:42] <i_jethrobot> WikiLove for lunch!
[19:01:43] <rfarrand> no love
[19:01:44] <i_jethrobot> : P
[19:01:49] <matt_flaschen> But it's not "Wikipedia about the English-speaking world", it's "English-language Wikipedia". There's a lot of missing stuff.
[19:01:51] <varnent> I love this idea. :)
[19:01:54] <guillom> thank you all!