Community health initiative/Blocking tools and improvements/Suggestions ranking
This table shows the rankings by the Wikimedia Foundation's Anti-Harassment Tools team for the suggestions generated by the on-wiki discussions related to improved blocking tools, from December 2017 to February 2018.
About this ranking system[edit]
We received 58 different suggestions over the four problems we've identified, listed in column 1.
- Problem 1. Username or IP address blocks are easy to evade by sophisticated users
- Problem 2. Aggressive blocks can accidentally prevent innocent good-faith bystanders from editing
- Problem 3. Full-site blocks are not always the appropriate response to some situations
- Problem 4. The tools to set, monitor, and manage blocks have opportunities for productivity improvement
In order to narrow the list into the top contenders so we can have a focussed discussion about which tools the WMF's Anti-Harassment Tools team should build over the coming months in 2018, we ranked each one on four criteria:
- User demand — how many people have mentioned/endorsed this and how much support has this suggestion received on wiki, Phabricator, and elsewhere? (1 = low support, 5 = high support)
- Potential impact — how likely will this suggestion actually curb harassment or solve the problem at hand? (1 = low impact, 5 = high impact)
- Technical complexity — how easy/difficult is each feature to build? (1 = difficult, 5 = easy) (We only sized the top 25 items due to time)
- Wikimedia ethos fit — how much does this suggestion align with Wikimedia's mission and pillars? (1 = low alignment, 5 = high alignment)
This was driven by professional experience and intuition, it is not a scientific ranking. The Total column is a combination of the scores of user demand, technical complexity, ethos fix, and double the score of potential impact. The Risk column was an additional optional column used to mark items that pose additional challenges.
Rankings[edit]
| Problem | ID | Suggestion | Phab | User demand | Potential impact | Technical complexity | Wikimedia ethos fit | Risk | TOTAL | WMF team notes |
|---|---|---|---|---|---|---|---|---|---|---|
| 4 - Usability | 4-15 | Improve the display of block notices on mobile web | T165535 | 2 | 5 | 5 | 5 | 22 | We will do this in addition to a new block tool | |
| 1 - Evasion | 1-2 | Block by combination of hashed identifiable information (e.g. user agent, screen resolution, etc.) in addition to IP | - | 2 | 5 | 3 | 5 | Medium | 20 | Privacy policy questions, will need to check with legal. Seems very complex. We will have to create our own fingerprinting. Might be a duplicate of 1-1. Might need to build 1-3 at the same time. |
| 3 - Breadth | 3-5 | Block a user from uploading files | T6995 | 4 | 4 | 2 | 4 | 18 | Certainly practical for some cases, where copyright is the main issue | |
| 1 - Evasion | 1-1 | Block by user agent (in addition to IP) | T100070 | 4 | 3 | 3 | 5 | 18 | Privacy policy questions (limit to CheckUser only?) | |
| 3 - Breadth | 3-4 | Block a user from creating new pages | - | 4 | 3 | 4 | 4 | 18 | No schema changes involved. | |
| 3 - Breadth | 3-15 | User-centric AbuseFilter (to create custom, complex blocks) | - | 4 | 4 | 1 | 4 | 17 | This is incredibly complex to build and complex to design. We will not be building this. | |
| 1 - Evasion | 1-3 | Hash identifiable data to surface as a percentage match to CheckUser | - | 2 | 4 | 3 | 4 | 17 | Same as 1-1 and 1-2, this is promising, maybe we make this for non-CheckUsers if the data is hashed. | |
| 3 - Breadth | 3-2 | Block a user from all pages inside a specific category | - | 4 | 3 | 3 | 4 | 17 | We would use category ID if the category is renamed. Could be built with 3-4, 3-5, and 3-3 | |
| 3 - Breadth | 3-3 | Block a user from specific namespaces | T179110 | 4 | 3 | 3 | 4 | 17 | If we build this, building others (e.g. 3-2, 3-1) would be 'cheap as free' | |
| 1 - Evasion | 1-6 | Drop a 'blocked' cookie on anonymous blocks | T152462 | 4 | 2 | 4 | 5 | High | 17 | Need to check with Legal, may not be possible in European Union |
| 4 - Usability | 4-6 | Allow admins to set a block date range via datetime selector | T132220 | 3 | 2 | 5 | 5 | 17 | We will do this in addition to a new block tool | |
| 4 - Usability | 4-11 | Special:Block could suggest block length for common policy infractions | - | 2 | 3 | 5 | 4 | High | 17 | Easy to build, but likely a very difficult sell to individual wiki communities |
| 3 - Breadth | 3-7 | Block a user from all pages, except a whitelist | T27400 | 3 | 3 | 3 | 4 | Medium | 16 | Potential for harassment for reporters of harassment. Some of this could be built into the new reporting system. The whitelist could either per user or shared. |
| 3 - Breadth | 3-10 | Allow admins to specify exactly which permissions to block. | T27400 | 3 | 3 | 3 | 4 | 16 | Complex. Will have to check for error messages and make sure we're not giving permissions accidentally | |
| 1 - Evasion | 1-13 | Identify by editing patterns (e.g. time of day, edit session length, categories) | - | 3 | 3 | 1 | 5 | 15 | European privacy laws don't like us aggregating this sort of stuff. Maybe Hash this? | |
| 2 - Bystanders | 2-10 | Model how users change their IP address to build more tactical blocking tools. | - | 1 | 4 | 1 | 5 | High | 15 | This is a research project, not software development. Likely too risky for false positives |
| 4 - Usability | 4-14 | Display if a user is currently blocked on another wiki on Special:Block | - | 3 | 3 | 2 | 4 | Medium | 15 | We will need to decide the social impact of this before development. Could be an expensive query, so we'll have to decide the UI. |
| 3 - Breadth | 3-6 | Block a user from all pages except talk pages | T18644 | 3 | 3 | 3 | 3 | 15 | Same as 3-3 | |
| 1 - Evasion | 1-14 | Special:RangeContributions | T145912 | 3 | 2 | 2 | 5 | 14 | ||
| 2 - Bystanders | 2-9 | Convert Twinkle and/or Huggle from gadgets to extensions, increase accuracy | - | 4 | 2 | 1 | 5 | 14 | ||
| 1 - Evasion | 1-8 | Proactive globally block open proxies | T166817 | 4 | 3 | 2 | 2 | Medium | 14 | Problematic for high risk areas in the world. |
| 1 - Evasion | 1-9 | System to share identified open proxy IPs | - | 2 | 3 | 2 | 4 | 14 | ||
| 2 - Bystanders | 2-1 | Require accounts created in an IP range to confirm email address before editing | - | 4 | 2 | 3 | 2 | Medium | 13 | Seems very complex, but effective if done in concert with other mechanisms |
| 4 - Usability | 4-1 | System to display how many other warnings have ever been given to a user. | - | 2 | 3 | 1 | 4 | 13 | We assume this would only count templated warnings. As these are applied by anyone with a fair amount of false positives, not sure how this would play out. | |
| 4 - Usability | 4-16 | Allow admins to ‘pause’ a block so the user can participate in on-wiki discussions | - | 3 | 1 | 2 | 5 | 12 | Heavily requested by the German Wikipedia, but we're uncertain if it would it actually solve anything | |
| 2 - Bystanders | 2-5 | Throttle account creation and email sending per browser as well as IP address | T106930 | 2 | 3 | 3 | 11 | |||
| 2 - Bystanders | 2-6 | Allow CheckUsers to compare hashed email addresses | T117801 | 1 | 3 | 4 | 11 | |||
| 4 - Usability | 4-2 | Twinkle should know the appropriate warning template to use on that user. | - | 2 | 2 | 5 | 11 | |||
| 4 - Usability | 4-10 | Display a warning on the block page when admins are blocking a sensitive IP | T151484 | 2 | 2 | 5 | 11 | |||
| 1 - Evasion | 1-4 | Global blocks for usernames | T17294 | 3 | 1 | 5 | 10 | |||
| 1 - Evasion | 1-10 | AI that compares editing patterns and language to predict possible sockpuppets | - | 1 | 3 | 3 | High | 10 | Hash is more transparent, blackbox AI is against ethos | |
| 2 - Bystanders | 2-7 | AI that recommends a block based on UserAgent, IP and/or email | - | 1 | 3 | 3 | Medium | 10 | Dependent on UserAgent and/or email blocking | |
| 2 - Bystanders | 2-8 | Require two-factor authentication for edits in certain IP ranges | - | 1 | 3 | 3 | 10 | Still as evadable as regular IP blocks | ||
| 3 - Breadth | 3-1 | Block a user from individual pages | T2674 | 4 | 1 | 4 | 10 | |||
| 4 - Usability | 4-5 | Allow admins to annotate previous blocks as accidental | T46759 | 3 | 1 | 5 | 10 | Commonly requested | ||
| 1 - Evasion | 1-5 | Add "Prevent account creation" to global block | T17273 | 2 | 1 | 5 | 9 | Anti-spam? | ||
| 3 - Breadth | 3-14 | Throttle edits to a maximum number per day/hour/etc | - | 2 | 2 | 3 | 9 | |||
| 3 - Breadth | 3-16 | Tool to prevent users from writing about themselves. | - | 1 | 2 | 4 | 9 | |||
| 3 - Breadth | 3-17 | User masking systems to obfuscate or ‘hide’ users from each other on wiki | - | 2 | 3 | 1 | 9 | |||
| 4 - Usability | 4-3 | Log bans like blocks | - | 1 | 2 | 4 | 9 | |||
| 4 - Usability | 4-13 | Block appeal process could be improved to reduce the work required for admins | - | 1 | 2 | 4 | 9 | Is different on different wikis and language versions | ||
| 1 - Evasion | 1-7 | Add a way to extend autoblock to longer than 1 day | T27305 | 3 | 1 | 3 | High | 8 | Maybe too many innocent bystanders | |
| 2 - Bystanders | 2-2 | Prevent blacklisted email addresses from new accounts | - | 2 | 1 | 4 | 8 | Only useful with 2-1 | ||
| 2 - Bystanders | 2-3 | Require email address to be unique for edits in certain IP ranges | - | 2 | 1 | 4 | 8 | Only useful with 2-1 | ||
| 3 - Breadth | 3-11 | Allow admins to temporarily revoke a users' autoconfirmed status. | - | 2 | 2 | 2 | 8 | |||
| 3 - Breadth | 3-12 | Require all edits by a user to go through deferred changes. | - | 1 | 2 | 3 | 8 | |||
| 3 - Breadth | 3-13 | Block that only expires when a user has read a specified page | T18447 | 2 | 1 | 4 | 8 | |||
| 4 - Usability | 4-4 | Allow CheckUsers to watch specific IPs | T21796 | 2 | 2 | 2 | 8 | |||
| 4 - Usability | 4-7 | Allow admins to set different expirations for blocking editing vs. account creation | T65238 | 2 | 1 | 4 | 8 | |||
| 4 - Usability | 4-9 | Display block expirations in logs | T148649 | 2 | 1 | 4 | 8 | |||
| 1 - Evasion | 1-15 | Extend Nuke to IP ranges | - | 1 | 1 | 4 | 7 | Not a blocking tool, a cleanup tool | ||
| 3 - Breadth | 3-9 | Block a user from emailing or pinging other users | T104099 | 1 | 2 | 2 | 7 | |||
| 4 - Usability | 4-12 | Improved way to set mass blocks | - | 1 | 1 | 4 | 7 | |||
| 2 - Bystanders | 2-4 | Allow only whitelisted email domains for edits in certain IP ranges | - | 2 | 1 | 2 | 6 | Only useful with 2-1 | ||
| 4 - Usability | 4-8 | Allow admins to oversight usernames while blocking them | - | 2 | 1 | 2 | 6 | |||
| 1 - Evasion | 1-11 | Identify by typing patterns (e.g. rhythm/speed) | - | 1 | 1 | 1 | 4 | |||
| 1 - Evasion | 1-12 | Identify by network speed | - | 1 | 1 | 1 | 4 | |||
| 3 - Breadth | 3-8 | Block a user from viewing Special:Contributions | - | 1 | 1 | 1 | High | 4 |
Discussion[edit]
Discuss the rankings or the top rated suggestions at Talk:Community health initiative/Blocking tools and improvements