Community health initiative/Meeting notes

Add links
From Meta, a Wikimedia project coordination wiki

This page contains notes from a recurring meeting between Community Tech and SuSa/T&S about the Community Health Initiative. Newest notes on the bottom of the page. Scroll to bottom.

Thursday April 27, 2017[edit]

Trevor, Sydney, Patrick.

  • Notification blacklist
    • ‘Thanks’ or @mention spam as a form of harassment
    • Was on Community Wishlist last year, extension is built
    • Standard on many sites
    • Concerns:
      • Collaboration vs isolation.
      • Workaroundable — Using puppet accounts or asking a friend/swam to harass.
      • Is the blacklist public or private.
    • SuSa is preparing an analysis document of the discussion on the wishlist
    • Other options:
      • Warnings, not total blocks
      • Short-term muting (24 hours?)
      • Throttling
  • What is the office action policy
  • Data on blocks?
    • It’s out there, we’ll have to search
    • Special:Blocklist (raw data) : https://en.wikipedia.org/wiki/Special:BlockList
  • https://meta.wikimedia.org/wiki/Community_health_initiative/Draft_of_first_ENWP_communication

Thursday May 4[edit]

Trevor, Sydney, Caroline, Patrick, James

  • Finalizing (heh) Annual Plan milestones. My proposals:
    1. Increase the confidence of admins with their ability to make accurate decisions in conduct disputes, measured via focus groups or consultations
      • Concerns that Evaluation might be a risky focus area to begin, because the community is likely most excited about blocking tools.
    2. Release three features that increase the efficiency of dispute resolution for Wikimedia administrators.
      • Release three features which empower Wikimedia administrators to better enforce community policies and reduce abusive activity
  • What do we want out of our relationship with Jigsaw?
    • Jigsaw’s success is determined by good press, and they are not as familiar with out community as we are
    • We don’t want people to think Conversation AI is the cornerstone of our work, we have 4 focus areas and detection is one subset
    • Caroline is meeting with Jigsaw next week
    • Their API and data may be incredibly useful, their visualization is not as important for us to focus on at this moment.
  • BLP harassment under AHT team?
    • Yes, BLP harassment does fall under the purview of the Anti-Harassment Tools team
    • We will need to take this into account for the Reporting features — harassment doesn’t just occur on talk and user_talk pages.
    • Detection — typically a little more difficult, but we should still investigate it.
  • Sockpuppets
    • Discussions on what sockpuppets are
    • How do we model sockpuppet identification?

Thursday May 11[edit]

  • Notifications blacklist
    • Next steps — RFC by community member or Sydney?
    • General agreement that we should start with this as our first project
    • Do we discuss on on meta only? Or ENWP?
      • Start conversations on both ENWP and meta, and reach out to other wikis (langauges) if we’re informed about conversations starting elsewhere
    • Can we deploy this to test wiki?
      • Yes, we’ll just need to find a dev to ask to deploy it. (Trevor to ask Moriel)
      • Also, we could ask Elena to QA this feature.
    • Create a Sub-page for this project that contains all the info to date (phab, wishlist voting, maybe make a Q&A from the conversation.)
    • Yes, we want to reach out to some folks to start the RFC or VPT post.
    • A pre-announcement update to solicit input.
  • Special:Emailuser discussion
    • There is a ticket to update the default footer boilerplate
      • Should we update the ENWP footer boilerplate? Not now, we need more info.
    • Should we run a survey?
      • We have very little understanding (scale, severity) but we know it is a problem based on certain incidents.
      • Probably yes, let’s look into running a surveys.
      • Caroline to look into previous surveys we’ve run in the past. Edward is helpful and smart, Sydney will introduce.
      • Qualtrics (which we have an account) already takes care of our privacy concerns.
      • We should plan this on wiki?
        • Maybe, we don’t want to open the survey to be gamed, but this is a different type of survey, less risky.
  • How can we stay on top of other WMF product releases so we might know if anything could be interpreted by the community as affiliated with our work?
    • Or other hot button issues?
    • Look at Phabricator fields
    • Proactively reaching out to other teams to evangelize our work and get them interested in our work so they keep us in mind.
    • Community Liaisons could help too — they have their ears to the ground and we could ask them
      • Potentially we invite them to a monthly H proj meeting to provide an update.
      • Or send Syd to their meeting.
  • AP responses - wikihounding question
    • ‘Wikihounding’ is an ENWP unique term, so in the response we may want to explain that.
      • Persistent unwanted contact. ‘Stalking’
      • TBD: link to the ENWP definition?
      • Action item: Trevor to update answer, then post by EOD today.
    • Side note: in the future we’ll want to connect the terms used in harassment policies to standard terms (legal, industry-wide)
      • While still retaining room for wiki-specific types of harassment
  • Help from Wishlist subteam: Takedown tools, AbuseFilter
    • Niharika is looking at DMCA and CP takedown tools.
    • Leon is looking at AbuseFilter performance.

Thursday May 18[edit]

  • New hires!
    • David (full stack dev) starts May 30!
    • New Community Advocate selected! Offer to be extended soon.
  • How do we prioritize our work, in cooperation with the community?
    • Prioritized backlog as Google spreadsheet now, will be on wiki soon. To be determined if it is on ENWP or meta. Potentially Phabricator.
    • Place for us to consolidate previous discussions and requests
    • As product owner, Trevor is responsible for soliciting input from WMF staff and Wikimedia community on contents and prioritization
    • How do we include external projects and developers such as Jigsaw? (e.g. Jigsaw, volunteer devs, etc.)
    • Moving forward, do we want to keep our work in Progress on Phab or on wiki? We should reach out to Community Tech and get their input on what we should use for a project-level backlog.
    • On-wiki stuff can be consultation, announcements, and documentation. Phabricator can be our primary location for project/task management.
    • How can we clearly state where certain types of conversations should occur?
      • Not necessarily one perfect way to communicate out to users, we’ll have to learn and grow as we proceed.
    • Action item: Trevor to write proposal for documentation on wiki and Phab.
    • Aiming for end-of-May 2017 to post the backlog publicly.
  • Policy and Enforcement Growth - currently setting goals and plans for next year.
    • New Hire on SuSa will have a major hand in this work.
    • Draft document will be shared for proofreading and input soon
  • Newsletter!
    • Example newsletter: Wikipedia:The_Wikipedia_Library/Newsletter
    • Would be best if a volunteer could help run it.
    • GChat channel for news and new research?
    • No one on ENWP signed up for Newsletter yet, maybe just because it was a soft launch.
    • Wait and see. Chew on this idea.
  • Surveys updates
    • Caroline is writing a series of questions to ask all members of ArbCom
    • Writing a series of questions to send a larger Wikimedia community about harassment vs. annoyances
    • Have reached out to Edward for his assistance.
    • Do we have a vocabulary we should use to use a consistent voice in our user-facing communication?
      • We predominantly use peer review over a dictionary/glossary
    • When we survey, we need to demonstrate clearly what knowledge we’ll gain from the survey and how this information will be used to improve our work
    • James + Trevor love the concept of microsurveys

Thursday June 1[edit]

    • Introducing David!
    • How will we roll out Notifications Blacklist?
      • Hebrew Wikipedia has identified this feature and is discussing it on their Village Pump.
      • D: We shouldn’t have a rollout strategy, we should have a communication strategy. The earliest we can start talking about it, the better. If we approach this as a rollout question, then the discussion will be reactive about rollout, rather than the merits of the feature itself.
      • J: We also need to be honest about the technical perspective.
      • How does this work for cross-wiki notifications? We need to test and decide with the community.
      • When users ask about rollout, tell them we don’t know but we think this will be useful for people on all wikis, and why not. Be honest.
        • And that users that it is configurable (e.g. it can be disabled on specific wikis if needed.)
    • Privacy concerns for publishing findings from our user interviews.
      • Caroline talked with Erin from Legal about agreements, she is drafting up templates for interviewees sign. We’ll decide from interview to interview about which form is the best to use.
      • How will we synthesize and anonymize our findings when we want to share and publish them?
      • We want users who have been harassed to feel safe when they share their experiences with us, and that they can be honest.
      • Caro is establishing best practices for how to talk with interviewees who don’t want to open themselves up to further harassment.
      • Discussions where other people, minors, and the interviewee are threatened or are in harm should be shared with SuSa.
      • For our research we need to be able to clearly explain why our research is useful and what we expect to learn and build as a result of this research.
    • Wikimania
      • Our two submissions were rejected, we’ll need to figure out a contingency plan.
        • https://wikimania2017.wikimedia.org/wiki/Submissions/Reducing_harassment_on_Wikimedia_projects
        • https://wikimania2017.wikimedia.org/wiki/Submissions/Building_a_Better_Dispute_Resolution_System
      • In Mexico City we had an invite-only roundtable about Harassment. Something to think about for this year.
    • Prioritized backlog

Thursday June 8[edit]

  • Berkman Center workshop
    • Jun 28-30
    • “workshop that will explore harmful speech online at the intersection of algorithms and human behavior.”
    • Sydney and Trevor have been granted travel & lodging funds from the conference
  • Wikimania
    • How do we ensure we can successfully utilize our time at Wikimania to run unofficial roundtable and presentations?
    • Booths & posters (chairs and snacks) in the Community Village
    • Ask the organizers to book a spare room for a sit-down
    • Budget our time — be sure to take breaks and rests. Don’t over-extend ourselves.
    • After-hour conversations at dinners/mixers.
    • Make connections and schedule 1:1s for after the conference, not immediately at the conference.
    • Attend events that will likely have attendees that are related to our work (and then shamelessly self promote.)
    • Lightning talks & birds of a feather — Trevor to look into signing up.
    • Make the moster of our poster
      • Have a clipboard and sign-up sheet at our poster.
      • Maybe make a one-pager so folks can grab-and-go.
      • Stickers, pens, pins, fly swatters swag.
  • AbuseFilter
    • Trevor is working on a list of questions we want answered from our Community Consultations
      • Do users think AbuseFilter has the potential to alleviate harassing behavior?
      • What are the most requested changes to AbuseFilter?
      • What do users think about the warn functionality?
      • Should AbuseFilter be used to help community leaders intervene, or should the software intervene directly? (In some cases an unsolicited third party intervention could do more harm than good.)
      • Sockpuppet question
    • Part of a larger consultation?
      • Prioritization?
      • Defining harassment?
    • AbuseFilter transparency could be a concern.
    • Trevor & Sydney to begin crafting community consultation message
  • IdeasLab list
  • AHT Communication practices
    • Google docs vs. immediate responses
      • Try not to set expectations about off-hour responses (holidays, evenings, weekends)
      • But we want it to be a timely dialog.
      • We don’t want our responses to sound ‘by-committee’.
    • SLA for responding?
      • Not now.
      • Trevor/Caroline to check with Sydney (or James/Patrick if needed) if a question is loaded with ulterior motives.
        • If not, then T/C can immediately respond.
        • If so, then we think about crafting a response on google doc.
          • We can also respond with “Thank you for the comment, let me check with the rest of the team.”
    • When we post on-wiki communication, be sure we have blocked in our schedules/minds some time to focus on replying and understanding and discussing the comments.

Thursday June 15[edit]

  • How do we responsibly share examples/stories from ANI, ArbCom or elsewhere in our communications?
    • We need to be really cautious about linking to examples. By default, never link to a specific page or section within a page.
    • Redact names and identifiable information when we provide examples or quotes.
    • Be transparent about our opaqueness. Say “we pulled from several locations and interviews.” But be honest!
    • For common case examples we can be slightly less protective. Point to on-wiki policy pages.
    • Likewise, we shouldn’t ask users for specific examples of harassment.
    • Do we want to draft some disclaimers for our talk pages? (@Patrick to provide an example.)
    • Perhaps we bring this up at the symposium we’re attending later this month?
  • Policy and Enforcement Growth
    • Deliverables being drafted. Need to co-ordinate our research activities.
    • Caroline's ANI research will help both Tools and PEG
  • Who at the WMF is responsible for providing support for harassment at physical meetups
    • Event planners should have a Friendly Space Policy w/ enforcement (talk to Rachel Farrand or Nick Wilson)
    • When somebody asks for immediate help (on wiki or in person) to whom do we send them?
      • In person at events, we need to be prepared with who/where to direct users (should be aligned with the Friendly Space Policy)
      • Online: SuSa
  • Advancement report update
    • The grant mentions some specific community consultations. How do we want to report on this?
    • Collect pageviews, collect the number and type of posts and users who participate, count the number of calls the AHT team has made, individual Caroline interviews.
    • Prep for Wikimania and be ready to talk about our plans
    • Talk about Mexico City WikiWomen Camp
  • Strategy process
    • where is our work ranking in the consultations?
    • Keep monitoring but it is looking promising.
  • Jigsaw update
    • Data request from Nithum — Patrick and Kaldari are emailing.
    • Caroline met with Jigsaw, they could help us formulate queries when we have questions.

Monday June 25[edit]

  • Wikimania - get ourselves a booth/table! (See Ellie’s email: [CE] Community Village at Wikimania '17
    • AHT has a recurring meetings for Caroline, Sydney and Trevor to plan for our Wikimania activities
    • Patrick will submit a request for a booth for a combined SuSa and AHT booth
  • Hebrew Wikipedia & echo notifications blacklist (global releases)
    • Discussed on HEWP and Meta, we have some ideas of how to start the conversation back up and reach a conclusion (Sydney to own)
    • We’ll want to ping Johan and the translators mailing list about Echo notifications translations once development is complete — https://phabricator.wikimedia.org/T168902
    • AbuseFilter is also a global extension, so we need to be very conscious about how this will affect all wikis.
      • Trevor to own (and Sydney to help) in creating questions that are appropriate for a global audience and sending out invitations to participate.
    • Moving forward, it’ll be a case-by-case basis depending on which features (building on existing, creating new, etc.)
  • Caroline’s research blog
    • Origin: how do we report updates on research in progress. Idea: Post a daily(ish) blog on ENWP
    • Where should it live?
      • Talk page of CHI? Probably not best,
      • Subpage of CHI?
      • Caroline to pick then try it for a week or two then to re-evaluate.
    • We could announce its existence on the Village Pump, listservs, and other communication mechanisms.
    • Concerns:
      • How to keep it “tied” to our CHI pages on Wikipedia? But not jumble the communication streams.
      • All the posts should be cohesive, so seeing them all in sequence is important.

Thursday July 6[edit]

  • We discussed Fiscal Year 2017-2018 Quarter 1 goals for both the SuSa department and the Anti-Harassment Tools team. We'll publish them online soon.

Thursday July 13[edit]

  • Special:EmailUser mute feature
    • No data — do we need to measure how often the feature is used first?
    • T138165 & T138166
    • Wording for Q1 goal looks good: “Finish and release the Mute feature, which allows users to privately list usernames from which they do not want to receive Echo and email notifications and Special:EmailUser emails.”
  • We need to define harassment. How will we accomplish this?
    • What is a definition?
      • A series of definitions, a sliding scale, something to help assess threat levels and then guide our work on the abuse filter (ie if we can define harassment, and point to a scale of harassing actions, we can write code to help filter some of that out)
    • Reasons to define harassment
      • There are multiple, varying definitions across different wikis (internally and externally)- help consolidate these different definitions or explain and reinforce why there needs to be multiple definitions (e.g. does the WMF have a different one versus to EN:WP because of consensus, law, etc? Let’s figure this out)
      • Provide examples of harassment to help empower admins (this initiative will be done with admins and editors). Such as, what is a good example of wiki-hounding in action
      • Create a taxonomy and list of different kinds of conflict < harassment < abuse
        • There’s a difference between name calling and hate speech, but it’s good to define that name calling isn’t harassment but conflict, and hate speech isn’t conflict but abuse (more extreme than harassment)

Thursday July 20[edit]

  • Community survey results are in, let’s look at the numbers …
  • Notifications mute
    • Hits meta next Tuesday
  • Wikimania
    • Our notes: meta.wikimedia.org/wiki/Community_health_initiative/Wikimania_2017
    • Shared booth
      • Let’s make a “staffing schedule” for who sits when. Once the program drops, we can figure out our sessions and fill in gaps. Patrick will create a table.
      • Important to be at the booth during the breaks
      • We’ll leave leaflets at the table, too.
    • Functionaries mixer
      • James will invite us.
      • Saturday evening.
      • Bring appetite for poutine and beer 🍟 🍻
    • North American con Aug 10th
      • https://wikiconference.org/wiki/Submissions:2017/Building_MediaWiki_software_to_enforce_Page_%26_Topic_bans
  • Sprint 1 started
    • https://phabricator.wikimedia.org/project/board/2897/
  • Future of this meeting
    • Trevor will change calendar to Bi-weekly, alternative with sprint review

Thursday July 27[edit]

August 24[edit]

  • Update on the Mute feature(s) (Trevor)
    • en.wikipedia.org/wiki/Wikipedia:Community_health_initiative_on_English_Wikipedia/User_Mute_features
    • Building 2 separate lists for this quarter, maybe the ideal solution later
    • Potentially a volunteer user will create a feature to mute somebody via their profile, notification, etc.
  • Measuring Admin confidence (Caroline)
    • Caroline is making good progress on the list of questions. Format (focus group v. survey v. something else) to be determined by the end of next week.
    • If this is a survey, we will need to work with Legal to ensure the survey is up-to-snuff.
  • ANI Research methodology
    • Caroline is speaking with Dario today about getting Diego to help us with this project
  • UIH prototypes
    • Research has started, Trevor has made 2 prototypes based on ANI cases, is building a third (based on a much larger case)
  • Community Engagement Insights/Intent form
    • https://office.wikimedia.org/wiki/Community_Engagement_Insights/Intent_form
    • Trevor will apply for AHT team

September 7[edit]

  • Blocking tools
  • How do we instill a responsibility in all other Product Managers and Designers that Anti-Harassment is everyone’s responsibility?
    • Set a quarterly goal to achieve this?
    • Increase awareness
      • All-hands presentation?
      • Email wmf-all —T175306
      • Use Cross-org meeting to ask folks to take the message back to their respective
    • We should monitor what is being developed and proactively reach out to teams.
      • Quarterly goal meetings?
      • Quarterly goal slides / documentation?
  • meta.wikimedia.org/wiki/Community_health_initiative/Policy_and_enforcement_research#Experiences_with_ANI
  • User Interaction History Prototypes (Trevor)
    • One, two, and three
    • Next step: Trevor & Caroline to build digital paper prototype and then to solicit feedback
    • We need to be open to getting feedback from Sockpuppet hunters
  • Editing restrictions communication (Trevor)
    • Q1 goal is just to consult about per-page blocking
    • Too many topics?
    • Idea: Communicate about the problems to solve, suggest per-page blocking as one solution, ask for others.
    • Potentially also include our wikihounding tool?
    • Sydney to write a draft post, aiming to post next week.
  • How do we respond when users question our prioritization and consensus? (Trevor)

September 21[edit]

  • AHT / ComTech off-site in January?
    • SuSa’s is on 1/24, location TBD
    • Probably CommTech on 1/24 as well, we’ll miss SuSa folks
    • Trevor to check with Ryan about budget. Budget is due on 9/30
  • Q2 goals
    • User Interaction Feature is a big one
      • Analyzing reports quickly, with more certainty
      • Caroline has some prototypes
    • Mute features, looking at group settings for email muting
      • Metrics for usage, metrics for demand?
    • Communication strategy - how to goal-ify?
      • Create a written strategy/guidelines, published?
    • Dates around Quantitative AN/I, Version 2.0 - When to start?
      • October review of results, feedback, November new run
    • Qualitative survey scheduled to run early November
      • We’ll need a deployment strategy, use lessons learned from Admin confidence
  • Interaction History prototypes
    • Caroline working on wireframes
    • Look and feel - working with Nirzhar and Pau for design ideas
    • Cluttered vs. Clean - Wikimedians love a density of information
  • How to share ANI data that Dayllan generated
    • We will include an overview, but save the full analysis for October.
  • Results from Admin Confidence (so far)
    • 114 completed of 135 sign ups
    • Really useful comments in there!
    • We plan to take this into our prioritization
    • We plan to publish all the results on Meta/Commons/ENWP next week

October 18[edit]

First meeting in nearly a month!

  • AHT Project Updates
    • Timeline
      • Conversation on ENWP
      • User test completed, hope to do more
      • Also found Intertwined Contributions
      • MVP defined, may be built by the end of October
    • Editing restrictions (and larger blocking consultation)
      • Discussion has slowed down lately.
      • Per-page blocking could be used as an alternative to page protection
      • https://en.wikipedia.org/wiki/Wikipedia:Per-article_blocking
      • Feels like per-page blocking could be an easy thing to build a V1, release to smaller wikis, and see how the ENWP community would respond.
      • Next step (technical): Get an estimation of time required to build
      • Next step (discussion): roll into a larger Blocking discussion (Q2 goal)
    • Email group protection (newbies)
    • Survey analysis and discussion
      • Caroline is working with Melody to publish results on the Comm’s blog
        • Be careful of backlash that this was only on ENWP, not the whole community.
      • Next on-wiki step: May be posted on Caroline’s blog.
    • Proxy blocking bots
      • Trevor is researching bot policy on ENWP https://en.wikipedia.org/wiki/Wikipedia:Bot_policy
  • Interaction Timeline use cases (and design!)
    • Supporting 2 users vs. supporting more than 2 users
      • The amount of users this tool needs to support fundamentally changes how this tool needs to be designed.
      • Use case for supporting 3+: Battlegrounds, factions/collusion to POV push (usually coordinated off-wiki)
    • Cross wiki data in the Timeline?
      • Cross-wiki data in the tool could be useful to Stewards,
      • Primary use case will be local
    • We can always build different tools
      • Local vs. Cross-wiki
      • 2 users vs. 3+
  • Deployment plan for AN/I survey
    • How are we going to avoid having a skewed sample? (frequent vs. infrequent ANI users)
    • Still scheduled to go out in November, probably end of November.
  • Thinking ahead - Quantitative analysis round 2, need to schedule a review of first results
    • What’s useful? Useless? Tantalizing?
    • Cheap-as-free to get a larger data set (aside from just 2 months). Trevor to create a ticket to get this done in our next sprint.
    • Patrick to schedule discussion meeting (early Nov)
  • FYI — there is energy from the board to push for stronger on-wiki policies

November 2[edit]

  • ANI survey update, please?
    • Sampling concerns? Representative vs. directional
    • Deployment planning doc
    • Rollout date - Nov. 28
    • Still figuring out how to contact people who have only edited on ANI once and don’t return, as well as people who have never edited on ANI
    • We could potentially query for users who have N+ edits or have N+ tenure and have only edited on ANI once (or multiple times in a ~48 hour period)
  • Email prohibit features
  • Timeline prioritization

November 20[edit]

  • Blocking consultation
    • https://meta.wikimedia.org/wiki/Community_health_initiative/Blocking_tools_and_improvements
    • Who we will invite to the consultation
    • How do we decide when consensus has been reached so we can start development?
      • Demonstrate that we are begin open and receptive for everyone to join the conversation
      • We can use a combination of discussions elsewhere (Phab, wishlist, talk pages, etc.)
      • Depends on how much we’re changing — adding a new feature is less risky (and just needs support and lack of opposition), changing an existing feature (e.g. removing functionality)
    • The consultation is about
      • Helping us prioritize the problems to solve
      • Helping us prioritize the solutions to these problems
    • How do we help people understand that this is just a consultation about blocking tools, not the end-all?
      • Trevor to update the “What this page is about” section be explicit about what comes next
      • Retitle the page to CHI/Blocking_tools_and_improvements_(December 2017-January 2018) — Not yet, only if needed.
    • Will these tools be mandatory?
      • No, unless there is a technical reason to
      • We could potentially make tools per-wiki configurable (e.g. via MediaWiki message)
  • Cool research - A. Halfaker is working with researchers at University of Michigan on some POSITIVE research on civility - e.g. how civil interactions affect quality content creation
  • AN/I survey update
    • Locking questions, last chance for input
  • Sprint review meeting moved to Monday 11/27
  • Interaction Timeline update
    • Feedback is good so far, need to deliver on their requests/inputs before asking for more feedback. We don’t want to overburden them.
    • Group ping?

November 30[edit]

  • Objective 3 project (Communication strategy and guidelines)
    • Target date Dec 15 for outline (sneak preview)
      • Guiding principles
      • Strategy for general communication
      • Strategy for consultation/discussion
      • Strategy for surveys
      • Strategy for off wiki events
      • Public documentation
      • Slides for internal
  • Q3 draft goals
    • Tl;dr — Timeline, Blocking, research on Reporting system
    • We could ask our users what else we should research about the reporting system. Share the workload and decision-making with our users.
    • We will need to think up-front about “who will triage these reports” at the very beginning of the project.
  • Blocking consultation update -live on Dec 7
    • TBD if we will have a close date — we’ve set general milestones on the page (mid January, mid February)
  • Harvard Negotiation & Mediation draft report
    • Read & give feedback soon on the notes doc.
  • AN/I research
    • Qualitative Survey - in progress (survey)
    • Quantitative - review & update queries
  • Timeline prioritization https://phabricator.wikimedia.org/T179607

December 14[edit]

  • Update on ANI survey
    • Closed on Dec. 10 with 137 responses. Presentation plan being developed.
    • Reading/categorizing the free text responses will be happening soon. (Color like like we did with the Admin Confidence Survey.)
    • Will post on-wiki soon. Don’t want to commit to an unrealistic schedule.
    • Patrick and Caroline to work on timeline tomorrow
  • Reporting system project start (Caroline)
    • A lot of what we need to know is in the ANI survey results
    • How are we different/similar to other communities/platforms?
    • meta.wikimedia.org/wiki/Community_health_initiative/Reporting_system
    • May be worthwhile to do some user interviews with hyper-specific questions about reporting
      • Talking to ArbComm, admins who weigh in on AN/I (@sdyney maybe help assemble those people?)
    • Will revolve a large, centralized on-wiki discussion about making decisions.
    • Aside about concerns for scope creep — the reporting tool we build will receive non-harassment reports.
  • Update on blocking consultation
    • It’s underway on Meta, dewp, and enwp
    • Sydney and Trevor are responding to on-wiki questions/discussion
    • High focus on email.
  • Blocking wishes on Wishlist

January 11, 2018[edit]

  • Blocking consultation
    • Status update
    • Next steps? How to start winnowing to a decision?
      • Trevor to post an addendum to the feedback summary to date — both on the archive page and on primary talk page,
      • Sydney to MassMessage/Facebook/noticeboard/etc. a “hey, get your thoughts in because we want to move to the decision-making stage.”
      • Hint that we want to start winnowing to a list of the most promising tools/directions (but it’s not a vote)
    • Check with Legal?
      • Trevor to create of a list of potential security/privacy questions and check with Legal and James.
    • TBD: Think about creating a Blocking-tools Phab board.
  • ANI survey synthesis update
    • On track with the schedule! Free-text questions sorting exercise finished tomorrow.
    • Report draft begun
    • Dayllan’s data: Finding the most compelling data points for presentation
  • Reporting system
    • Lots of questions about Scope: How will we manage this regarding English Wikipedia/all wikis.
      • If we’re going to build something that will apply to other wikis, then we need to include them in the discussions and decision-making
      • Other wikis may take and use what we build for a different reason than originally designed regardless
    • Public vs. Private reporting?
    • This is not “replace ANI” but rather “improve the biggest pain points of all ways users report harassment
    • Name — “Harassment Reporting System” excludes other types of reports (copyvio, deletion, etc. — which is good) but may also keep out smaller scale user misconduct reports that we think should be covered
    • How do we design something that doesn’t get flooded?

February 8[edit]

  • Reporting system wiki page
    • https://meta.wikimedia.org/wiki/Community_health_initiative/Improved_tools_and_workflows_to_report_harassment
      • New section: here’s stuff we have worked on, some links, we did this survey
        • Could be a nice navbox.
      • New section: project updates, with subsections of calendar dates
      • Sydney to add these to the page
    • Let's establish a timeline for getting all the supporting research on site and who is responsible for each part, for the page documenting our ideas (hypothesis page), and timelines about holding the on wiki discussion/consultation after the pages are up. That way I can make announcements about the date for the discussion ahead of time.
    • Deliverables to community related to Reporting System (Google doc created by Sydney)
    • Existing timeline prepared by SuSa:
Rough draft for wider review February 16
Final draft of report February 21
On-wiki release of data, host discussion March 1
    • May be difficult to accelerate this, unless we want to release different parts (survey data, Caroline’s report, Dayllan’s data, Harvard report) at different times.
      • Post Harvard report on Commons this week or next, start by letting participants know first
      • All in attendance agreed to March 1 deadline
    • Caroline to spearhead writing out steps//do a product audit of our reporting spaces list, and then rewriting hypotheses  
  • Caroline posting on Community Health Pages more
  • Merchandise - requests need to go into Patricia Pena. SuSa’s putting forward a request that includes some schwag for Sydney to distribute to helpful folks.

February 22[edit]

  • Blocking tools update
    • Met with legal, all 6 projects are OK to proceed.
    • On-wiki discussion is progressing nicely.
    • Trevor and Sydney to discuss next steps for Blocking tomorrow
    • Mobile error message: https://phabricator.wikimedia.org/T165535
    • Datetime selector: https://phabricator.wikimedia.org/T132220  
  • Interaction Timeline
    • On track to hit a successful V1 release next week
  • Annual Plan
    • Targets — how do we make this measurable?
      • Difficult because we’re not certain exactly what we’re building.
      • Reach — number of users and/or wikis using our tools — or “tool is translated and usable in N languages” — Risk of this resting on volunteer translators
      • Trevor to make Outcome 1 Target 2 about i18n, or remove it altogether
  • Wikimania Program submissions
    • Due: Sunday March 18
    • Meeting scheduled for March 5 w/ Kaldari
    • Theme: Bridging knowledge gaps, the *ubuntu* way forward.
    • Pre-conference?
  • Reporting System research
    • Still mostly on track to release on March 1.
    • Harvard report should be uploaded imminently
    • Joe is working through formatting the data and creating visualizations.
    • Caroline and Danny are working through the in-depth report, may slip past March 1. We won’t hold up the other releases though.
    • User Interviews — needs to be written up (anonymized) and published on wiki?
    • Auditing user journeys of existing reporting workflows — will happen after in-depth report is published. Meeting set for next week or so?
  • Possible research project with Aaron Halfaker and Elizabeth Whittaker - they are looking for direction on a project around civility
  • Find a way to invite Christel into these meeting? Will require AM timeslot
  • CHI email list?

March 8[edit]

  • ArbCom project mapping
    • James is beginning work on looking into all of the ArbComs across all the wikis
  • Interaction Timeline
  • ANI write-up
    • Retrospective
      • Schedule post publication
      • Trevor or Patrick will reach out to Grace or someone from T&C to see if she’d like to be part of the retro
    • Next steps to get it on wiki
      • Some minor copy edits, Patrick  will schedule a cowork session(s)
      • Targeting release next week to ENWP
  • Wikimania 2018 (doc)
    • — Lightning talk for Timeline — Trevor
    • — Roundtable on the Reporting System — Trevor
    • — Resubmit the same Poster as last year — Trevor
    • — TBD: Workshop on the Interaction Timeline — Sydney
    • — New blocking tools & how we built them with community feedback — Sydney
    • — Something around WikiData harassment policy — Patrick
    • And anyone else can submit for a Lightning Talk (scheduled closer to July)
    • Submissions due next Friday!
  • Reporting system research
    • Document with links of wiki pages and spreadsheets
    • Caroline’s final projects: write-up of ArbCom interviews, audit of reporting workflows
  • Blocking tools update
    • Investigations started yesterday
    • Trevor posting update to talk pages today

March 22[edit]

  • Q4 Goals time y’all
    • AHT
    • SuSa/Support
    • https://www.mediawiki.org/wiki/Wikimedia_Audiences/2017-18_Q4_Goals
  • Community Health Metrics Kit
    • Working on a framework w/ Dana McCurdy of L&E
  • Policy on Wikidata is a go!
    • L. Pintscher is supportive, but needs to see more of a plan.
  • Blocking statistics — T190328  
  • James Q's
    • Emailed user in CU: T189943
    • Wikidata/Harassment reporting tool? Trial/Pilot?
  • Researcher JD and hiring

May 3[edit]

  • QCI slides
    • Feedback?
  • Interaction Timeline
    • Announcements
    • Trainings
    • Trevor to prepare a list of “features we might build next” w/ some commentary/explanation
  • Hiring news (if any)
    • Third AHT dev may get a good candidate from Matt F backfill, or will wait until August/October
    • Researcher job description is written, hiring panel is being assembled, kickoff is soon. Will report to Margeigh.
  • Wikimania presentations
    • Sydney, Patrick, Joe, Christel, David, Kalli, Trevor, Kaldari, & Danny are attending
    • Reporting system accepted
    • Blocking tools waitlisted
    • Metrics Kit waitlisted
    • Timeline rejected
    • Poster rejected — Trevor to write them and ask them why.
  • Metrics kit progress
    • Inspire Campaign
    • David and Dayllan can help reviewing technical feasibility of potential metrics & reviewing Inspire Campaign technical applicants
  • Granular blocking tools

May 31[edit]

  • We’ve invited Leila to discuss Research/Jigsaw and other Research department initiatives.
    • Upcoming research showcase will cover details of the harassment detection work with Jigsaw and Cornell: https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#June_2018
    • Verge write-up
    • Sockpuppet detection update
      • Models: where are we with it? User scores based on deep learning.
      • Auditability: develop feature based models.
      • When to put this in front of checkusers for feedback on the quality of the model (not as an immediately useful tool)
        • There is no risk to put this in front of the checkusers as long as we explain the details.
        • From the community perspective, if two usernames have signals of being similar we don’t want to line them all up.
          • How about we let checkusers enter a username and the system shows them the list of usernames that are “similar” to the input username.
            • That can work much more aligned with current practices.
            • Legit use of multiple accounts
      • Private or public, early thoughts?
        • Currently using only public data
        • Each project has its own values. For example, I can imagine dewiki would want to have it public, but enwiki may decide to do it differently.
        • Perhaps early on we should look at it early on as a system that people with special permissions can look at it.
    • TODO
      • [Leila] to look into maintenance and updating of the data-sets. And interaction with ORES. Send an email to the team about it.
      • Patrick (and maybe Trevor, schedule permitting) to meet with Halfaker about integrating Conversation AI API + ORES)
      • [Leila] How far are we from being able to have some sort of a feed that can help us monitor places/conversations that are becoming hard.
  • U. Virginia data science program looking for partnership
  • CHMK Winnowing (Patrick)
    • Inspire Campaign on Community Health Measurement in July
  • Reporting system — where are we? Next steps?
    • Role of adminship discussion - moderator role?
    • Plans for using Wikimania presentation for springboard
    • Routing system vs. redesign of notice boards and processes
    • Report quality improvement
    • Ways to get participation
      • Maybe write a Signpost article?
      • Banner?
      • Put something on ANI or AN.
      • Will start with regular tactics — VP, mass message, etc.

June 14[edit]

  • Takedown tools bug fixes are live on production
  • Timeline launch
    • Wikimania poster
      • Group decision: No poster. Concentrate our efforts on our booth.
    • Status updates?
      • Blog post
        • Sydney
        • Bery basic draft started
        • Goal: submit draft to Comms on 6/20
      • Adding Call-To-Action to timeline
        • Trevor
        • Doc: message to post on wiki, hoping to post on-wiki today.
      • Get links in tool lists
        • Patrick
        • Incl. commonly-used templates
        • End of next week
      • Get in the hands of friendly admins
        • Sydney
        • By 06/15 email written
      • James Hare’s ToolHub project
        • Trevor to talk to James about the Timeline (emailed)
    • Metrics
      • # Reports generated and used on wiki?
        • Initial report
        • Supplement to an existing on wiki case
        • Saved for further investigation
        • Make use https://tools.wmflabs.org/globalsearch/globallinks.php
      • Qualitative report/storytelling
      • # Amount of community engagement related to socializing and training
        • # contacted
        • # page views and # editors engaging with CHI (edits)
        • # Blog views, other social media #k
  • AHT Q1 goals
    • document
  • StackOverflow tomorrow
    • Topics:
      • (Both) How are you structuring teams to accomplish this work?
      • (For Wikimedia) How are y'all getting buy-in from volunteers who may be wary of "social-justice" projects?
      • (For Wikimedia) How are you approaching incivility and micro-aggressions (vs. direct harassment)?
      • (For Wikimedia) Any advice y'all have for us as we ramp up this project, things you wish you'd known, pitfalls to avoid, etc.?
  • Jigsaw press
    • Motherboard article
    • There may be more
  • TechCom RfC for partial blocks
    • https://phabricator.wikimedia.org/T2674#4282730
    • https://www.mediawiki.org/wiki/Requests_for_comment/Process
    • Next step: Trevor to meet/talk with Ryan

June 28[edit]

  • Wikimania prep!
    • Our round table: Building a Better Harassment Reporting System
    • What do we want to leave the roundtable with?
      • Identify a wiki that wants us to build tools for?
      • Learn how other wikis report harassment (document workflows) — draw on giant posterpaper?
      • Capture a list of the most important pieces of information that good reports contain?
      • Get people to sketch what an ideal system would look like?
      • Try to avoid personal stories from participants about harassment — frame the event by telling people that we are open to hearing stories in private but now we just want to talk about workflows.
    • Do we want to focus on Global or enwiki focus
      • Expect a global audience
      • Use this as an opportunity to find a wiki that would be really excited to be a pilot wiki for our work.
    • Preview the routing idea
    • Depending on the number of attendees, we’ll want to have one consolidated discussion OR split into groups — all discuss the same topic.
    • Next steps so we’re prepared by July 20
      • Trevor to start a planning doc, will share for input before PTO
      • After Wikimania, Trevor will take comments/suggestions/edits to the plan, collect any needed materials from SF office
      • Recruitment of people to come?

July 12[edit]

  • Welcome our special guest Maggie!
  • Framing
    • FY18-19 goals
      • Focus on one large project? The reporting system will be our major focus.
      • Blocking tools are not currently reflected in these goals
      • Danny will talk with Toby about swapping Output 2 (brewing tools) with blocking tools
    • Q1 goals
      • Idea for measuring block effectiveness: block escalation?
  • Wikimania is next week!
    • Roundtable on Friday: "Building a better harassment reporting system"
  • Timeline

August 9[edit]

  • Welcomes!
    • Ty
    • Morten
    • Alex
  • Background on the User Reporting System
    • Ty is interesting in this project as a way to improve T&S productivity and user experience
  • Community Health Metrics Kit - design and community feedback
    • Next steps: get community feedback
    • Next steps: Integrate Inspire Campaign results
    • Be prepared for the tool to prepare the sources/citations of where the data comes from.
    • Be prepared to answer when numbers change. Address the curiosity.
  • User Reporting System next steps
    • Traceability — document everything so all decisions can be traced to earlier evidence // at earlier stages
    • Translation? How can we keep the page static enough to translate but updateable enough. - Transclusion!
  • AHT is now an independent team from CommTech

August 21[edit]

  • Check-in on Q1 goals: https://www.mediawiki.org/wiki/Wikimedia_Audiences/2018-19_Q1_Goals#Contributors
    • In Q2, let’s try to set fewer goals to stay more focused.
  • Next steps for the Timeline
    • In-product explanation? Rename the tool? Nope.
    • Filters
      • Namespace
      • Chatty pages (e.g. public noticeboards)
    • Don’t wait for the design research position to be filled
    • Need impact metrics + narrative for the timeline ASAP — not just usage but impact. Are we making things better?
      • LinkSearch on English Wikipedia for people actually using this
      • Also used at least once on Functionaries-l
    • Next steps::
      • Trevor + Sydney to determine user testing plan
      • Prioritize some filtering features to bolster usability (separate signal from noise) and prepare design
      • Trevor to write a narrative of what we consider success to be.
  • Reporting system consultation next steps
    • Any successful new system will have more reports — how can we also solve for the bandwidth of the moderators?
    • Project on hold until 9/6
  • Metrics kit and design needs
    • Analytics wants a UI design
    • Prateek just started working with Community Wishlist. Can talk to him as well for AHT things.
    • Joe to send Danny an introduction email to send to him about this, bearing in mind that this is a new role.

September 6[edit]

  • Interaction Timeline doc next steps
    • What’s missing?
    • How to appropriately share with the C-levels?
      1. This 3-page document is good.
      2. Suggestion: Let’s finalize it, write a brief summary email, and send it out by September 14
      3. Idea: Monthly meeting w/ Maggie and Toby (and add Dayllan, David) about CHI.
      4. Action Item: Patrick to observe New Readers and report back with how we could convert one H project meeting a month to be higher level.
  • Blocks
    • Roadmap:
      1. Stages deployment tentative timeline (rollout page on wiki)
      2. Action item: Trevor to expand this page into an internal doc that is more descriptive to all pertinent WMF management and individual contributors.
    • Measuring block effectiveness
      1. Spreadsheet
      2. Add to partial blocks talk page
    • Third round of designs
      1. Wireframes post, no responses yet
      2. Talk with Arbitration Enforcement (AE) admins
    • Log discussion
      1. Started, low engagement so far
    • Release
      1. Mid October
      2. Will want to do usability tests w/ ITWP admins.
  • Admin Confidence survey next steps?
    • Reflections on Edward’s feedback (he hasn’t sent it yet)
    • Three ways to run survey:
      1. Carbon copy of 2017
      2. Add some new questions
      3. Completely re-tool it.
    • Questions we need to eventually answer: Reach out to the same users or new people?
    • Plan:
      1. Wait for Edward’s feedback, see if he suggests i, ii, or iii.
      2. If i or ii, run in September. If option iii then talk with Maggie and Toby
  • Claudia - onboarding and initial work
    • Can copy old onboarding doc for Caroline/David/Dayllan (with updates) (example)
    • Patrick to schedule meeting w/ Margeigh about first projects
  • Password strength

September 20[edit]

  • AHT Q2 goals (doc)
    • Due 9/20
    • Moving forward, how do we set CDP goals, not AHT + T&S
    • Slide deck link
  • Blocks roadmap
    • Epic ticket updated with latest requirements (sub-tasks) and sequence: phab:T190350
    • Maybe create a Phabricator board?
  • Metrics Kit
    • Meta project page
    • On-wiki Consultation has begun!
    • Talking w/ design
    • Giving data to people & informing how to interpret that data — both are important
  • Claudia onboarding update
    • 2 weeks of not tasking her with things
    • Then we’ll decide, based on our Q2 goals.
  • October CHI Retreat- pre-planning

October 4[edit]

  • Claudia’s first H meeting! :D
  • Partial blocks
    • QA testing - Ideal schedule - when should we push other staff members to test?
      • Oct. 16 - Available on test.wiki
      • Patrick will work on a list of folks outside T&S to invite to test
      • Patrick to schedule a T&S co-work week of October 15
      • Sydney can invite other admins to test on test.wikipedia
    • Measuring the effectiveness of blocks (meta page)
      • No CTA on the page to participate in the discussion. Should probably add one.
      • Trevor to add specific questions on the talk page.
    • Blocking policy
      • Blocking policy changes are already happening on ITWP.
      • How closely do we want to work with wikis to update their policies vs. let them do it themselves? TBD.
        • Perhaps just documentation about how other communities have changed their policies for the new tool.
      • Claudia to annotate this page for terms that may need to be glossarized.
  • Retreat plan
    • If there’s anything that could work in Day 2, please comment. Otherwise… 🔪✂️
    • AHT team is holding a 60-minute brainstorm session for other team members to generate their most-excited-by ideas
  • Metrics Kit update (meta page)
    • Consultation is on track, 8 folks participating good feedback
    • Design conversations — first version wireframe
    • Blocking / noticeboard activity / page protection metrics in a goal for AHT.
  • WikiCon North America
    • Sydney has a roundtable on policy: https://wikiconference.org/wiki/Submissions:2018/Aligning_technical_solutions_and_policy_to_combat_harassment_on_Wikimedia_projects
    • Also of interest: wikiconference.org/.../Anti-Harassment_and_YOU
    • Opportunity to get feedback on:
      • User Reporting System (the role of people who would be dealing with harassment reports.)
      • Partial Blocking
      • Trevor & Claudia to create a handout survey for people who would be interested
      • Unconference
      • Lightning Talk(s)

Oct 11[edit]

  • QCI
    • Alex E will be here first 30 minutes of this meeting
  • Recap from retreat
  • This is the last H project meeting. RIP. Live long and prosper. Will be transferred.

end of page[edit]

 

Retrieved from "https://meta.wikimedia.org/w/index.php?title=Community_health_initiative/Meeting_notes&oldid=19921010"