WikiLoop/DoubleCheck/RfC:Levels for WikiLoop DoubleCheck Reviewers
New here? Not sure what WikiLoop DoubleCheck is? Check out this story on The Signpost.
Reviewing Wikipedia edits is an important activity to ensure content quality, but it could also be abused or misused. Existing major review or counter-vandalism tools (e.g. STiki or Huggle) made the tradeoff by restricting access to their tools to only a small group of trusted and privileged users. WikiLoop DoubleCheck, an open source web app for easier reviewing edits, want to explore an alternative model: just like everyone can edit Wikipedia, the DoubleCheck would like to allow everyone to review Wikipedia, while having mechanisms to address the concerns about reviewer competence and trustworthiness.
In this page we propose a design to allow reviewers with different levels to have different restrictions or powers.
Ideally we hope to complete the discussion by end of 2020-08-29 23:59, unless the community suggests a large scale change to this proposal.
We propose the following ladders and their restrictions or additional powers for WikiLoop DoubleCheck.
A DoubleCheck reviewer's credibility ladder will be determined by the Wikipedia permissions or # of DoubleCheck judgements or endorsements whichever comes higher.
The main rationale is that
Here is a list of our current or planned restrictions / power features for different levels.
STiki for vandal patrollers requires a 1,000 mainspace edit threshold or the rollback right for its use. (see: Wikipedia:STiki#Using STiki), supports only EN Wiki to start using
Huggle requires rollback permission or to be on a global white-list to start using
Request for comments
There are a few specific questions I want to seek community feedback on
Please leave your comments below. Feel free to use the templates
@Ed6767: Thank you, I will take that feedback and digest them, Thank you for your detailed comment. Really appreciate that! I like to address taht the reason those are high bars of edits means "even you don't have endorsement from others, you can gain trust by just reviewing". We also provide a allowlist-kind-of-feature: that is any two endorsements from L4(rollbackers) or one endorsements from L5(admins) will give the user a L3 power. This is equivalent to the Huggle/STiki's whitelist models except that to get whitelisted for Huggle/STiki one will need approval by its developer or a given group, but the WikiLoop DoubleCheck's model recognizes endorsements from already trusted people to establish such allowlist, and hence more decentralized and less controlled by WikiLoop DoubleCheck Xinbenlv (talk) 05:39, 22 August 2020 (UTC)
Separate Discussion 1: Number of Contributions needed for each Level
If we allow reviewers to gain power without needing to be on a given allowlist, or without endorsements from anyone, what numbers are appropriate to be set as bars? Xinbenlv (talk) 05:43, 22 August 2020 (UTC)
Mark D Worthen PsyD (talk) 18:07, 22 August 2020 (UTC)
First, I know only one language fluently, therefore please take my suggestions in that light, i.e., I respect others who know two or more languages. Suggestion #1: Ask a colleague who knows your native tongue and English well to proofread the English version for proper grammar, usage, syntax, etc. Only minor improvements are needed, e.g., "P1. URL-to-Undo: display a button taking the reviewer to the Wikipedia page before they manually undo bad the revision" (I am not sure what that sentence means), and "such as 5 or 10, to be discuss in Separate Discussion 1" ("discuss" should be "discussed").
Suggestion #2: Emphasize the either/or nature of the first two columns in the first table, e.g., Either reach Wikipedia User Level Or has # of DoubleCheck contributions. Mark D Worthen PsyD (talk) 18:07, 22 August 2020 (UTC)
It's important for the tool to prominently display whether there are unread Notifications or unread Talk messages for the user. If the user's edits are getting reverted, or if someone is leaving messages on their user talk, it's important for the user to be aware so they can defuse any potential conflict. If the user is unaware and they keep making more edits the situation may escalate badly, possibly resulting in anger between editors or even resulting in a block. Alsee (talk) 03:47, 24 August 2020 (UTC)
Request for Comments (Second version after your input)
Hi all, thank you for your kind discussion in the first RfC version. We hear many of you ask to simplify the user level structure and to avoid abuse while giving all users the convenience for discovering problematic edits.
We discarded our original complex model and create a new simplified model as followed:
- Basic Power: all users will have DoubleCheck's convenience features to discover vandalism and share their opinions, but will not have FAST revert ability.
- Super Power: Admins and roll-backers will have FAST revert ability, similar to Huggle and STiki.
- Endorsements can be granted by Admin, Roll-backers and other endorsed users to get super power. Endorsement can be revoked should there be abuse of power.
Note: FAST revert ability means reverting edits without a strict rate limit (same as Huggle).
We look forward to your feedback or questions, please comment below, thank you!
1. Does the simplified power tier model above look good to you?
- I don't understand why a person just wouldn't get rollback. That's a defined process with onwiki transparency and can also be revoked by anyone, rather than having to resort to say a block if there's a problem. Best, Barkeep49 (talk) 20:22, 19 November 2020 (UTC)
- Support, the proposed model seems good to me. Alexcalamaro (talk) 21:43, 19 November 2020 (UTC)
- Support, the model above seems sensible, though I think the Endorsement process might be more trouble than its worth. Still, I can see it being useful. Ganesha811 (talk) 21:51, 19 November 2020 (UTC)
- Comment: I only use rapid revert in the cases of extremely obvious vandalism, such as a string random characters inserted into a sentence or very egregious edits such as might slander a living person. When I first started using Wikiloop, I over zealously reverted and then had to backtrack. Since then my policy is to not revert without further investigation.
- Support the 1 and 2 point. Oppose for 3. If someone want super power they should simply request rollback for which we have established processes. ‐‐1997kB (talk) 02:05, 20 November 2020 (UTC)
- Support points 1 and 2. Oppose point 3. I don't see the use for point 3: if you want the ability to revert without rate limits, you should ask for rollback at PERM. JavaHurricane 04:03, 20 November 2020 (UTC)￼
- Support this system; not sure what is ment by super power. BEANS X2 (talk) 08:19, 20 November 2020 (UTC)
- Support point 1 and 2, Oppose point 3. Endorsements just seem like an overcomplication, when rollback could just be requested. A-NEUN (talk) 10:33, 20 November 2020 (UTC)
- Support point 1 and 2. I agree with others that getting rollback rights is a fine enough alternative process that we don't need point 3. - Whisperjanes (talk) 14:24, 20 November 2020 (UTC)
- Support point 1 and 2, Oppose point 3. I think point 3 is a little unnecessary, I think point 1 and 2 is good enough, and if the user does not fall under the category, I think they can just request for rollback, similar to the use of Huggle. Sophiajoanne (talk) 22:30, 20 Novemeber 2020 (UTC)
- ' Support' SHISHIR DUA (talk) 13:51, 22 November 2020 (UTC)
- Support points 1 and 2. Neutral on point 3 per above opposes. Justarandomamerican (talk) 19:12, 22 November 2020 (UTC)
- Support points 1 and 2. Oppose point 3 for the same reasons above. I think we can keep it simple and just tack on existing permission granting process within individual wikis. Robertsky (talk) 05:08, 24 November 2020 (UTC)
- Support points 1 and 2. Oppose point 3, for the reasons stated above. Fast Revert seems similar to Rollback rights, and we have an existing group for that. Info-Screen (talk) 16:16, 24 November 2020 (UTC)
- Support 1 and 2. Oppose 3. That's what administrators are for, I don't think other rollbackers should be trusted with the ability to grant that. Dylsss (talk) 23:09, 29 November 2020 (UTC)
- I echo Barkeep49's point and oppose any provision that would allow rollbackers to further grant other editors escalated privileges. I think auto-granting sysops and rollbackers the fast revert ability automatically would be fine. Best, KevinL (aka L235 · t) 06:21, 1 December 2020 (UTC)
- Support 1 and 2, Oppose 3: If rollback is the requirement (which is a good idea), keep it a requirement. Especially do not grant privilege-granting access just for being part of the rollback group (or worse, only for having been rollbacker-endorsed oneself). ToBeFree (talk) 22:04, 3 December 2020 (UTC)
- Thank you everyone for voicing your support and also some of your concerns.
- For everyone who has concern with Option 3. Here is my thinking: I think the reason we want to provide option 3 because we want to make this process as easy as possible. And super power here ONLY means revert faster than a slow limit. As you can see below, majority people think a 1 revert per 5min be too slow. But I will also think if use something like 1min per 1min, I also anticipate many to think it to be too fast. Therefore, to strike a balance, we say that while we want everyone to be able to revert fast, we need to assess the trust worthiness of a user in some simple way.
- Some of you asked, why don't the reviewer just get a ROLLBACK permission? Let me express my concern on requiring getting ROLLBACK permission: ROLLBACK power is actually a very very high bar, and once got it, it also gives user a lot of power globally. For example, en:Wikipedia:Rollback#Requesting_rollback_rights suggests "it's very rare for someone to get ROLLBACK permission with less than 200 main space edits. Also one have to demonstrate being able to appropriately warn user with bad edits." Therefore, ROLLBACK really isn't for everyone.
- Now, if you were an admin, do you want to give someone ROLLBACK permission, just because they have demonstrated some sense of responsibility on reviewing a few dozen or hundreds of review? I assume no, you will follow the requirement on en:WP:ROLLBACK and require the higher bar to be demonstrated.
- In practice, not many people have ROLLBACK permission. I counted the EN WIKI, in the past 12 months only around 36 users are granted ROLLBACKers. Requiring ROLLBACK permission has caused th tool Huggle to maintain a user whitelist who didn't get ROLLBACK but be approved by developers/maintainers of Huggle. STiki is the same. Essentially that approach by passed the ROLLBACK requirement but just make the tool developers a smaller group of people on who can approve who is on the whitelist or not. Well, why don't Huggle and STiki avoid using such a per-app whitelist? It's because the ROLLBACK is too high a bar and too few people currently have it, there are just too many people able to vandalise Wikipedia than people who can review it.
- And how do other popular tools solve it? The en:WP:Twinkle and en:WP:RedWarn, end up creating something bypassing entire ROLLBACK for people who doesn't have ROLLBACK permission.
- However, I think it's not very hard for a human to spot many common problematic edits just with common sense. This tool, the WikiLoop DoubleCheck, is trying to make it easy for everyone to use. It is based on an assumption just like wikipedia: everyone can review, and should be by default able to review, unless proven abuse. It also want to further allow cross-check, meaning for each revision, multiple opinions can be given by different people. Requiring getting ROLLBACK permission to revert fast, will either limit the amount of reviewers who are capable to user it, or push the reviewers to argue for a faster revert rate-limit, which also applies to anonymous users which might be abused by blocked or malicious users.
- Also, giving user a per-app privilege is a good way to give them a little incentive to contribute, also a longer runway to demonstrate they are indeed responsible and competency to conduct review and warnings appropriately, because we grant them the privilege that works across entire EN Wikipedia to many tools.
- In summary, that's why we propose the option 3. To make it easier to understand the trade off we are making here, there are 3 alternatives of Option 3, :
- 3-A. To have a whitelist for users to have non-thottled rate limit, but only developer of WikiLoop DoubleCheck, such as me, just like Huggle and STiki (I don't personally want that dictatorship though)
- 3-B. Remove such whitelist, making it available to only very small group (last time I check it was about O(10K)/a few tens thousands ROLLBACKers with only O(100)/ a few hundred active ROLLBACkers every month, to face about 30K vandalisms / damaging edits per month.
- 3-C. Use "Extended Confirm/Confirm": but it can be easily cheated to get by edit and self-revert, and also has nothing to with review competency.
- 3-D Create some on-wiki lower level trust - this is not likely to happen without database change or MediaWiki software change.
- 3-O.(the original option 3), to keep such whitelists, but allow Admins and ROLLBACKers to grand a lower level of trust before the full ROLLBACK is granted. Xinbenlv (talk) 23:22, 6 December 2020 (UTC)
- If tool developers have plans to give access to non-rollbacker also, then either 3-A or if tool dev don't want to handle such list then 3-C would be great options. But putting this burden on admins/rollbackers (3-O) seems to be illogical as if admin is actually reviewing someone and feel that they user can review recent changes then they can easily give rollback and in case of rollbackers I don't think should have power to give other user a rollback like access.
- About the rate limit, I think it would be great if tool use rate limit which are applicable on Wikipedia site. ‐‐1997kB (talk) 14:41, 10 December 2020 (UTC)
2. For none-FAST revert ability, will 1 revert per 5 minute be too fast or too slow?
- Seems way too slow to me. Like I don't huggle or sticki or do countervandalism and I can end up at more than 1 revert per 5 minutes. Best, Barkeep49 (talk) 20:22, 19 November 2020 (UTC)
- Too Slow, agreed with Barkeep. 1 revert per 1 minute seems about right. Ganesha811 (talk) 21:51, 19 November 2020 (UTC)
- I would think 1 revert every 30-45 seconds would be more reasonable -- it slows folks down enough to read, while not actually making it painful, Sadads (talk) 01:09, 20 November 2020 (UTC)
- Too slow. something like 2r/m would be ok. ‐‐1997kB (talk) 02:05, 20 November 2020 (UTC)
- Too slow by far. 1 or 2 reverts/minute is a better rate. JavaHurricane 04:03, 20 November 2020 (UTC)
- Too slow per above. A-NEUN (talk) 10:34, 20 November 2020 (UTC)
- Too slow, per above. Veracious (talk) 11:34, 20 November 2020 (UTC)
- Too slow, even when I did countervandalism by undoing edits and then manualy copy-pasting the warning templates I got more than that. 1 revert per 1 minute seems much more reasonable Asartea Talk (enwiki) 15:24, 20 November 2020 (UTC)
- Too slow. When you catch a run of obvious vandalism you can be processing them at 1 to 2 per minute. A cooldown of 5 minutes would likely make me find something else to do rather than enjoy the high of a good vandalism reversion run. Slywriter (talk) 16:58, 21 November 2020 (UTC)
- Too slow per Barkeep49. I think 5 minutes is a very very very looooooooong time to wait to revert an edit especially for a vandalism tool. Users could just use the Recent Changes log on the wiki and manually revert edits. P,TO 19104 (talk) 20:15, 21 November 2020 (UTC)
- Too slow. 1 or 2 reverts per minute seems better, since it usually takes around 30 seconds to a minute to undo obvious vandalism and warn a user. Maka⭐(talk) 04:27, 22 November 2020 (UTC)
- Too slow 2 per min would be much better. Robertsky (talk) 05:08, 24 November 2020 (UTC)
- Too slow (at least for large wikis). When doing counter vandalism work on enwiki, I easily made more than 1 revert every 5 minutes even without tool support. Especially when a lot of obvious vandalism happens. Info-Screen (talk) 16:16, 24 November 2020 (UTC)
- Too slow 1 or 2 per minute would be more appropriate for enwiki at least. Dylsss (talk) 23:09, 29 November 2020 (UTC)