Community health initiative/Blocking tools and improvements/Suggestions ranking

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search

This table shows the rankings by the Wikimedia Foundation's Anti-Harassment Tools team for the suggestions generated by the on-wiki discussions related to improved blocking tools, from December 2017 to February 2018.

About this ranking system[edit]

We received 58 different suggestions over the four problems we've identified, listed in column 1.

  • Problem 1. Username or IP address blocks are easy to evade by sophisticated users
  • Problem 2. Aggressive blocks can accidentally prevent innocent good-faith bystanders from editing
  • Problem 3. Full-site blocks are not always the appropriate response to some situations
  • Problem 4. The tools to set, monitor, and manage blocks have opportunities for productivity improvement

In order to narrow the list into the top contenders so we can have a focussed discussion about which tools the WMF's Anti-Harassment Tools team should build over the coming months in 2018, we ranked each one on four criteria:

  1. User demand — how many people have mentioned/endorsed this and how much support has this suggestion received on wiki, Phabricator, and elsewhere? (1 = low support, 5 = high support)
  2. Potential impact — how likely will this suggestion actually curb harassment or solve the problem at hand? (1 = low impact, 5 = high impact)
  3. Technical complexity — how easy/difficult is each feature to build? (1 = difficult, 5 = easy) (We only sized the top 25 items due to time)
  4. Wikimedia ethos fit — how much does this suggestion align with Wikimedia's mission and pillars? (1 = low alignment, 5 = high alignment)

This was driven by professional experience and intuition, it is not a scientific ranking. The Total column is a combination of the scores of user demand, technical complexity, ethos fix, and double the score of potential impact. The Risk column was an additional optional column used to mark items that pose additional challenges.

Rankings[edit]

Problem ID Suggestion Phab User demand Potential impact Technical complexity Wikimedia ethos fit Risk TOTAL WMF team notes
4 - Usability 4-15 Improve the display of block notices on mobile web T165535 2 5 5 5 22 We will do this in addition to a new block tool
1 - Evasion 1-2 Block by combination of hashed identifiable information (e.g. user agent, screen resolution, etc.) in addition to IP - 2 5 3 5 Medium 20 Privacy policy questions, will need to check with legal. Seems very complex. We will have to create our own fingerprinting. Might be a duplicate of 1-1. Might need to build 1-3 at the same time.
3 - Breadth 3-5 Block a user from uploading files T6995 4 4 2 4 18 Certainly practical for some cases, where copyright is the main issue
1 - Evasion 1-1 Block by user agent (in addition to IP) T100070 4 3 3 5 18 Privacy policy questions (limit to CheckUser only?)
3 - Breadth 3-4 Block a user from creating new pages - 4 3 4 4 18 No schema changes involved.
3 - Breadth 3-15 User-centric AbuseFilter (to create custom, complex blocks) - 4 4 1 4 17 This is incredibly complex to build and complex to design. We will not be building this.
1 - Evasion 1-3 Hash identifiable data to surface as a percentage match to CheckUser - 2 4 3 4 17 Same as 1-1 and 1-2, this is promising, maybe we make this for non-CheckUsers if the data is hashed.
3 - Breadth 3-2 Block a user from all pages inside a specific category - 4 3 3 4 17 We would use category ID if the category is renamed. Could be built with 3-4, 3-5, and 3-3
3 - Breadth 3-3 Block a user from specific namespaces T179110 4 3 3 4 17 If we build this, building others (e.g. 3-2, 3-1) would be 'cheap as free'
1 - Evasion 1-6 Drop a 'blocked' cookie on anonymous blocks T152462 4 2 4 5 High 17 Need to check with Legal, may not be possible in European Union
4 - Usability 4-6 Allow admins to set a block date range via datetime selector T132220 3 2 5 5 17 We will do this in addition to a new block tool
4 - Usability 4-11 Special:Block could suggest block length for common policy infractions - 2 3 5 4 High 17 Easy to build, but likely a very difficult sell to individual wiki communities
3 - Breadth 3-7 Block a user from all pages, except a whitelist T27400 3 3 3 4 Medium 16 Potential for harassment for reporters of harassment. Some of this could be built into the new reporting system. The whitelist could either per user or shared.
3 - Breadth 3-10 Allow admins to specify exactly which permissions to block. T27400 3 3 3 4 16 Complex. Will have to check for error messages and make sure we're not giving permissions accidentally
1 - Evasion 1-13 Identify by editing patterns (e.g. time of day, edit session length, categories) - 3 3 1 5 15 European privacy laws don't like us aggregating this sort of stuff. Maybe Hash this?
2 - Bystanders 2-10 Model how users change their IP address to build more tactical blocking tools. - 1 4 1 5 High 15 This is a research project, not software development. Likely too risky for false positives
4 - Usability 4-14 Display if a user is currently blocked on another wiki on Special:Block - 3 3 2 4 Medium 15 We will need to decide the social impact of this before development. Could be an expensive query, so we'll have to decide the UI.
3 - Breadth 3-6 Block a user from all pages except talk pages T18644 3 3 3 3 15 Same as 3-3
1 - Evasion 1-14 Special:RangeContributions T145912 3 2 2 5 14
2 - Bystanders 2-9 Convert Twinkle and/or Huggle from gadgets to extensions, increase accuracy - 4 2 1 5 14
1 - Evasion 1-8 Proactive globally block open proxies T166817 4 3 2 2 Medium 14 Problematic for high risk areas in the world.
1 - Evasion 1-9 System to share identified open proxy IPs - 2 3 2 4 14
2 - Bystanders 2-1 Require accounts created in an IP range to confirm email address before editing - 4 2 3 2 Medium 13 Seems very complex, but effective if done in concert with other mechanisms
4 - Usability 4-1 System to display how many other warnings have ever been given to a user. - 2 3 1 4 13 We assume this would only count templated warnings. As these are applied by anyone with a fair amount of false positives, not sure how this would play out.
4 - Usability 4-16 Allow admins to ‘pause’ a block so the user can participate in on-wiki discussions - 3 1 2 5 12 Heavily requested by the German Wikipedia, but we're uncertain if it would it actually solve anything
2 - Bystanders 2-5 Throttle account creation and email sending per browser as well as IP address T106930 2 3 3 11
2 - Bystanders 2-6 Allow CheckUsers to compare hashed email addresses T117801 1 3 4 11
4 - Usability 4-2 Twinkle should know the appropriate warning template to use on that user. - 2 2 5 11
4 - Usability 4-10 Display a warning on the block page when admins are blocking a sensitive IP T151484 2 2 5 11
1 - Evasion 1-4 Global blocks for usernames T17294 3 1 5 10
1 - Evasion 1-10 AI that compares editing patterns and language to predict possible sockpuppets - 1 3 3 High 10 Hash is more transparent, blackbox AI is against ethos
2 - Bystanders 2-7 AI that recommends a block based on UserAgent, IP and/or email - 1 3 3 Medium 10 Dependent on UserAgent and/or email blocking
2 - Bystanders 2-8 Require two-factor authentication for edits in certain IP ranges - 1 3 3 10 Still as evadable as regular IP blocks
3 - Breadth 3-1 Block a user from individual pages T2674 4 1 4 10
4 - Usability 4-5 Allow admins to annotate previous blocks as accidental T46759 3 1 5 10 Commonly requested
1 - Evasion 1-5 Add "Prevent account creation" to global block T17273 2 1 5 9 Anti-spam?
3 - Breadth 3-14 Throttle edits to a maximum number per day/hour/etc - 2 2 3 9
3 - Breadth 3-16 Tool to prevent users from writing about themselves. - 1 2 4 9
3 - Breadth 3-17 User masking systems to obfuscate or ‘hide’ users from each other on wiki - 2 3 1 9
4 - Usability 4-3 Log bans like blocks - 1 2 4 9
4 - Usability 4-13 Block appeal process could be improved to reduce the work required for admins - 1 2 4 9 Is different on different wikis and language versions
1 - Evasion 1-7 Add a way to extend autoblock to longer than 1 day T27305 3 1 3 High 8 Maybe too many innocent bystanders
2 - Bystanders 2-2 Prevent blacklisted email addresses from new accounts - 2 1 4 8 Only useful with 2-1
2 - Bystanders 2-3 Require email address to be unique for edits in certain IP ranges - 2 1 4 8 Only useful with 2-1
3 - Breadth 3-11 Allow admins to temporarily revoke a users' autoconfirmed status. - 2 2 2 8
3 - Breadth 3-12 Require all edits by a user to go through deferred changes. - 1 2 3 8
3 - Breadth 3-13 Block that only expires when a user has read a specified page T18447 2 1 4 8
4 - Usability 4-4 Allow CheckUsers to watch specific IPs T21796 2 2 2 8
4 - Usability 4-7 Allow admins to set different expirations for blocking editing vs. account creation T65238 2 1 4 8
4 - Usability 4-9 Display block expirations in logs T148649 2 1 4 8
1 - Evasion 1-15 Extend Nuke to IP ranges - 1 1 4 7 Not a blocking tool, a cleanup tool
3 - Breadth 3-9 Block a user from emailing or pinging other users T104099 1 2 2 7
4 - Usability 4-12 Improved way to set mass blocks - 1 1 4 7
2 - Bystanders 2-4 Allow only whitelisted email domains for edits in certain IP ranges - 2 1 2 6 Only useful with 2-1
4 - Usability 4-8 Allow admins to oversight usernames while blocking them - 2 1 2 6
1 - Evasion 1-11 Identify by typing patterns (e.g. rhythm/speed) - 1 1 1 4
1 - Evasion 1-12 Identify by network speed - 1 1 1 4
3 - Breadth 3-8 Block a user from viewing Special:Contributions - 1 1 1 High 4

Discussion[edit]

Discuss the rankings or the top rated suggestions at Talk:Community health initiative/Blocking tools and improvements