Talk:Spam blacklist: Difference between revisions
Billinghurst (talk | contribs) →daviscountyutah.gov: Removed |
→Proposed additions: + apyoth.com |
||
| Line 65: | Line 65: | ||
* {{LinkSummary|vivirasturias.com}} |
* {{LinkSummary|vivirasturias.com}} |
||
vivirasturias.com is a travel and tourism agency that has a biographies section on its website (vivirasturias.com/biografias-asturias). On each page it has a copyright mark that says: "© EuroWeb Media SL. EuroSeb Media, SL es el único titular de los derechos de propiedad intelectual de textos y fotografías de vivirasturias.com", however in Wikipedia in Spanish we have verified that his articles are copies (plagiarism) of preexisting articles of Wikipedia (see [https://es.wikipedia.org/w/index.php?oldid=105449203#Otro_sitio_que_plagia_a_Wikipedia_y_pone_marca_de_copyright here]). Mainly from the editions in Spanish, Catalan and Asturian (in many cases in the versions of the December 2016), sometimes using automatic translators and others changing a few words. Despite the massive plagiarism, in a year the site was linked [https://es.wikipedia.org/w/index.php?title=Especial:BúsquedaDeEnlaces/http://www.vivirasturias.com&limit=650&offset=0&target=http%3A%2F%2Fwww.vivirasturias.com +600 times] in es.wiki, [https://ca.wikipedia.org/w/index.php?title=Especial:Enllaços_web&limit=250&offset=0&target=http%3A%2F%2Fwww.vivirasturias.com +230 times] ca.wiki, [https://en.wikipedia.org/w/index.php?title=Special:LinkSearch/http://www.vivirasturias.com&limit=100&offset=0&target=http%3A%2F%2Fwww.vivirasturias.com +50 times] in en.wiki, [https://ast.wikipedia.org/w/index.php?title=Especial:BúsquedaDeEnlaces/http://www.vivirasturias.com&limit=50&offset=0&target=http%3A%2F%2Fwww.vivirasturias.com +30 times] in ast.wiki, and it is also linked in other Wikipedias. In some cases linking directly to the home page. The site is using Wikipedia dishonestly to attract visitors to its website and then offer on each page its tourist services. Instead of requesting the same in each affected Wikipedia, I request its inclusion in the global blacklist. Especially because any honest editor (of any language) who does not know the plagiarism case can then use it as a reference or even think that it was Wikipedia that plagiarized the website. --[[User:Metrónomo|Metrónomo]]-[[User talk:Metrónomo|Goldwyn]]-[[Special:Contributions/Metrónomo|Mayer]] 05:35, 8 February 2018 (UTC) |
vivirasturias.com is a travel and tourism agency that has a biographies section on its website (vivirasturias.com/biografias-asturias). On each page it has a copyright mark that says: "© EuroWeb Media SL. EuroSeb Media, SL es el único titular de los derechos de propiedad intelectual de textos y fotografías de vivirasturias.com", however in Wikipedia in Spanish we have verified that his articles are copies (plagiarism) of preexisting articles of Wikipedia (see [https://es.wikipedia.org/w/index.php?oldid=105449203#Otro_sitio_que_plagia_a_Wikipedia_y_pone_marca_de_copyright here]). Mainly from the editions in Spanish, Catalan and Asturian (in many cases in the versions of the December 2016), sometimes using automatic translators and others changing a few words. Despite the massive plagiarism, in a year the site was linked [https://es.wikipedia.org/w/index.php?title=Especial:BúsquedaDeEnlaces/http://www.vivirasturias.com&limit=650&offset=0&target=http%3A%2F%2Fwww.vivirasturias.com +600 times] in es.wiki, [https://ca.wikipedia.org/w/index.php?title=Especial:Enllaços_web&limit=250&offset=0&target=http%3A%2F%2Fwww.vivirasturias.com +230 times] ca.wiki, [https://en.wikipedia.org/w/index.php?title=Special:LinkSearch/http://www.vivirasturias.com&limit=100&offset=0&target=http%3A%2F%2Fwww.vivirasturias.com +50 times] in en.wiki, [https://ast.wikipedia.org/w/index.php?title=Especial:BúsquedaDeEnlaces/http://www.vivirasturias.com&limit=50&offset=0&target=http%3A%2F%2Fwww.vivirasturias.com +30 times] in ast.wiki, and it is also linked in other Wikipedias. In some cases linking directly to the home page. The site is using Wikipedia dishonestly to attract visitors to its website and then offer on each page its tourist services. Instead of requesting the same in each affected Wikipedia, I request its inclusion in the global blacklist. Especially because any honest editor (of any language) who does not know the plagiarism case can then use it as a reference or even think that it was Wikipedia that plagiarized the website. --[[User:Metrónomo|Metrónomo]]-[[User talk:Metrónomo|Goldwyn]]-[[Special:Contributions/Metrónomo|Mayer]] 05:35, 8 February 2018 (UTC) |
||
==apyoth.com== |
|||
Cross-wiki spamming targeting cryptocurrency articles. Deferred to here from enwiki ([[w:en:MediaWiki_talk:Spam-blacklist#apyoth.com]]). [[User:Edgar181|Edgar181]] ([[User talk:Edgar181|talk]]) 13:51, 20 February 2018 (UTC) |
|||
* {{linksummary|apyoth.com}} |
|||
* [[w:en:Wikipedia:WikiProject Spam/LinkReports/apyoth.com]] |
|||
;Users |
|||
* [[w:ru:User:Алина Мирошниченко]] |
|||
* [[w:en:User:Эмиль Золя]] |
|||
* [[w:en:User:Елена Солнцевая]] |
|||
* [[w:en:User:Василий Маркин]] |
|||
* [[w:en:User:Софи Яковлева]] |
|||
== Proposed additions (Bot reported) == |
== Proposed additions (Bot reported) == |
||
Revision as of 13:52, 20 February 2018
- Proposed additions
- Please provide evidence of spamming on several wikis. Spam that only affects a single project should go to that project's local blacklist. Exceptions include malicious domains and URL redirector/shortener services. Please follow this format. Please check back after submitting your report, there could be questions regarding your request.
- Proposed removals
- Please check our list of requests which repeatedly get declined. Typically, we do not remove domains from the spam blacklist in response to site-owners' requests. Instead, we de-blacklist sites when trusted, high-volume editors request the use of blacklisted links because of their value in support of our projects. Please consider whether requesting whitelisting on a specific wiki for a specific use is more appropriate - that is very often the case.
- Other discussion
- Troubleshooting and problems - If there is an error in the blacklist (i.e. a regex error) which is causing problems, please raise the issue here.
- Discussion - Meta-discussion concerning the operation of the blacklist and related pages, and communication among the spam blacklist team.
- #wikimedia-external-linksconnect - Real-time IRC chat for co-ordination of activities related to maintenance of the blacklist.
- Whitelists
There is no global whitelist, so if you are seeking a whitelisting of a url at a wiki then please address such matters via use of the respectiveMediawiki talk:Spam-whitelistpage at that wiki, and you should consider the use of the template {{edit protected}} or its local equivalent to get attention to your edit.
Please sign your posts with ~~~~ after your comment. This leaves a signature and timestamp so conversations are easier to follow.
Completed requests are marked as {{added}}/{{removed}} or {{declined}}, and are generally archived quickly. Additions and removals are logged · current log 2024/06.
- Information
- List of all projects
- Overviews
- Reports
- Wikimedia Embassy
- Project portals
- Country portals
- Tools
- Spam blacklist
- Title blacklist
- Email blacklist
- Rename blacklist
- Closure of wikis
- Interwiki map
- Requests
- Permissions
- Bot flags
- New languages
- New projects
- Username changes
- Translations
- Speedy deletions
- snippet for logging
- {{sbl-log|17757056#{{subst:anchorencode:SectionNameHere}}}}
 |
SpBot archives all sections tagged with {{Section resolved|1=~~~~}} after 7 days.
|
Proposed additions
|
This section is for proposing that a website be blacklisted; add new entries at the bottom of the section, using the basic URL so that there is no link (example.com, not http://www.example.com). Provide links demonstrating widespread spamming by multiple users on multiple wikis. Completed requests will be marked as {{added}} or {{declined}} and archived. |
Beta Cluster spamming of 2018-02-18
auto24insurance24online.com
kourtney.fun
medium.com
infowars.com
letsgomurphys.com
—MarcoAurelio (talk) 11:34, 18 February 2018 (UTC)
Beta Cluster spamming of today
pasaranliga.com
ena.website
guaranteedppc.com
jayne.website
—MarcoAurelio (talk) 23:26, 15 February 2018 (UTC)
- Added. —MarcoAurelio (talk) 23:26, 15 February 2018 (UTC)
wikiwand.com
wikiwand.com
This is a clear wikipedia mirror, but also used as citation in wikipedia, e.g. [1] and frwiki and eswiki. enwiki seems to be clean. should the domain be blacklisted globally or locally only? -- seth (talk) 19:39, 5 December 2016 (UTC)
- @Lustiger seth: Hmm .. this certainly should not be used as a reference, but this site does have it's own article on many Wikis (w:en:Wikiwand and interwikis). I don't think this warrants blacklisting more than en.wikipedia.org (which should also not be used as a reference; en.wikipedia would not blacklist based on being unreliable alone, abuse is needed as excuse for blacklisting). Is the situation 'bad' (I don't see reference-use on enwiki). --Dirk Beetstra T C (en: U, T) 13:09, 8 December 2016 (UTC)
- hi!
- I guess at enwiki somebody cleaned up already (that's what I meant with "enwiki seems to be clean"). (Maybe there's an edit filter or a bot coping with the links in main namespace?) But in other wikis there are still some links left.
- I agree in that point that I don't see intended abuse. However, I see violation of our rules and want to stop that. Possibillities I see are: SBL, edit filter, a cleanung up and kindly inform bot. The SBL would be an easy and fast option. The most comfortable solution is the bot I guess.
- I guess, I start with an SBL entry at dewiki. An in a few hundred yea^W^W^W^Wsome point in the future I'll implement a more intelligent solution ... -- seth (talk) 23:03, 12 December 2016 (UTC)
- @Lustiger seth: this again seems to boil down to a proper rewrite of the spam blacklist extension - to be more edit-filter like. --Dirk Beetstra T C (en: U, T) 03:42, 13 December 2016 (UTC)
- Just noticed this conversation. I've removed quite a few links to this site on enwiki and do weekly sweeps of the http and https links. I hate to call it "abuse" as I'm almost sure 100% of the links were added by mistake rather than through deliberate spamming. They do clearly identify as a mirror; I'm far more concerned with the handful of mirrors that go out of their way to obfuscate the origins of their content. Kuru talk 22:27, 22 December 2016 (UTC)
- @Kuru: those might qualify for blacklisting as well. --Dirk Beetstra T C (en: U, T) 03:44, 12 February 2017 (UTC)
- Just noticed this conversation. I've removed quite a few links to this site on enwiki and do weekly sweeps of the http and https links. I hate to call it "abuse" as I'm almost sure 100% of the links were added by mistake rather than through deliberate spamming. They do clearly identify as a mirror; I'm far more concerned with the handful of mirrors that go out of their way to obfuscate the origins of their content. Kuru talk 22:27, 22 December 2016 (UTC)
- @Lustiger seth: this again seems to boil down to a proper rewrite of the spam blacklist extension - to be more edit-filter like. --Dirk Beetstra T C (en: U, T) 03:42, 13 December 2016 (UTC)
Comment COIBot as of today says: COIBot> 1755 records; Top 10 wikis where wikiwand.com has been added: w:en (603), w:de (86), w:fr (82), w:pt (76), w:es (73), w:hi (73), w:it (34), w:ru (31), commons (28), w:hy (20). I have set the domain to monitor, and recast for a link check. It is blacklisted at deWP and zhWP. — billinghurst sDrewth 08:08, 25 December 2016 (UTC)- 1873 records; Top 10 wikis where wikiwand.com has been added: w:en (639), w:pt (89), w:fr (88), w:de (86), w:es (79), w:hi (79), w:it (38), w:ru (32), commons (30), w:kn (28). — billinghurst sDrewth
itun.es
itun.es
Shortcut for itunes.apple.com. As the latter is a commercial site, I don't see why we should obscure these. It is from itun.es/us/iGunhb absolutely not clear even whether it is to a correct item. And though this particular is free, probably for many you have to pay. Not sure if this is needed. --Dirk Beetstra T C (en: U, T) 17:08, 5 July 2017 (UTC)
- Is there evidence of abuse or potential abuse? If it just a shortened url with no abuse, I am not sure why we would want to. There are numerous "internal" url redirects, if it is just a little shorter, and not abused, what do we care which?—The preceding unsigned comment was added by Billinghurst (talk)
- Is linking to this (or its master site) not by definition spam/promotion .. a lot of it will be single edits per editor (not a campaign). I am, on Wikipedia, a great opponent of url shorteners .. you don't know what you are linking to (and at least en.wikipedia 'forbids' using shorteners). --Dirk Beetstra T C (en: U, T) 17:49, 7 July 2017 (UTC)
- My comment was more related to the use of the shortener compared with the full url; not the provision of a link in itself. If a link is discouraged or banned, then of course we should ban the shortener. If the link is neither banned nor discouraged, why would we wish to prevent a non-abused linking process that just inhibits and somewhat confuses users, or prevents them from editing. If a specific wiki itself has a rule where a link addition is contrary to that local rule, they can and should use their blacklist. I don't see a requirement for a global ban unless there is broader problem. — billinghurst sDrewth 01:12, 8 July 2017 (UTC)
- Is linking to this (or its master site) not by definition spam/promotion .. a lot of it will be single edits per editor (not a campaign). I am, on Wikipedia, a great opponent of url shorteners .. you don't know what you are linking to (and at least en.wikipedia 'forbids' using shorteners). --Dirk Beetstra T C (en: U, T) 17:49, 7 July 2017 (UTC)
Question: is there further input in this? is it a problem? — billinghurst sDrewth 07:42, 6 February 2018 (UTC)
Godchecker.com
godchecker.com
Per en.wikipedia's WikiProject Spam remarks. Cross-wiki spam, ánd useless. --Dirk Beetstra T C (en: U, T) 09:39, 2 August 2017 (UTC)
See also w:Wikipedia:Administrators'_noticeboard/Archive291#Is_godchecker.com_blacklisted.3F_If_not.2C_how_to_make_it_happen.3F. --Dirk Beetstra T C (en: U, T) 09:43, 2 August 2017 (UTC)
- It is used at numerous sites without blacklisting. How do you see that it sits within the criteria for meta admins for us to add? — billinghurst sDrewth 03:01, 5 August 2017 (UTC)
- The xwiki report shows it has been added by administrator and other trusted users on other wikis. If it problematic at enWP, it may need to be locally managed. — billinghurst sDrewth 12:49, 5 August 2017 (UTC)
- Is there further evidence that this has been abused? In the past six months has there been a problem? — billinghurst sDrewth 07:40, 6 February 2018 (UTC)
- The xwiki report shows it has been added by administrator and other trusted users on other wikis. If it problematic at enWP, it may need to be locally managed. — billinghurst sDrewth 12:49, 5 August 2017 (UTC)
vivirasturias.com
vivirasturias.com
vivirasturias.com is a travel and tourism agency that has a biographies section on its website (vivirasturias.com/biografias-asturias). On each page it has a copyright mark that says: "© EuroWeb Media SL. EuroSeb Media, SL es el único titular de los derechos de propiedad intelectual de textos y fotografías de vivirasturias.com", however in Wikipedia in Spanish we have verified that his articles are copies (plagiarism) of preexisting articles of Wikipedia (see here). Mainly from the editions in Spanish, Catalan and Asturian (in many cases in the versions of the December 2016), sometimes using automatic translators and others changing a few words. Despite the massive plagiarism, in a year the site was linked +600 times in es.wiki, +230 times ca.wiki, +50 times in en.wiki, +30 times in ast.wiki, and it is also linked in other Wikipedias. In some cases linking directly to the home page. The site is using Wikipedia dishonestly to attract visitors to its website and then offer on each page its tourist services. Instead of requesting the same in each affected Wikipedia, I request its inclusion in the global blacklist. Especially because any honest editor (of any language) who does not know the plagiarism case can then use it as a reference or even think that it was Wikipedia that plagiarized the website. --Metrónomo-Goldwyn-Mayer 05:35, 8 February 2018 (UTC)
apyoth.com
Cross-wiki spamming targeting cryptocurrency articles. Deferred to here from enwiki (w:en:MediaWiki_talk:Spam-blacklist#apyoth.com). Edgar181 (talk) 13:51, 20 February 2018 (UTC)
apyoth.com
- Users
- w:ru:User:Алина Мирошниченко
- w:en:User:Эмиль Золя
- w:en:User:Елена Солнцевая
- w:en:User:Василий Маркин
- w:en:User:Софи Яковлева
Proposed additions (Bot reported)
|
|
This section is for domains which have been added to multiple wikis as observed by a bot.
These are automated reports, please check the records and the link thoroughly, it may report good links! For some more info, see Spam blacklist/Help#COIBot_reports. Reports will automatically be archived by the bot when they get stale (less than 5 links reported, which have not been edited in the last 7 days, and where the last editor is COIBot).
|
COIBot
The LinkWatchers report domains meeting the following criteria:
- When a user mainly adds this link, and the link has not been used too much, and this user adds the link to more than 2 wikis
- When a user mainly adds links on one server, and links on the server have not been used too much, and this user adds the links to more than 2 wikis
- If ALL links are added by IPs, and the link is added to more than 1 wiki
- If a small range of IPs have a preference for this link (but it may also have been added by other users), and the link is added to more than 1 wiki.
COIBot's currently open XWiki reports
| ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Proposed removals
|
|
This section is for proposing that a website be unlisted; please add new entries at the bottom of the section.
Remember to provide the specific domain blacklisted, links to the articles they are used in or useful to, and arguments in favour of unlisting. Completed requests will be marked as See also /recurring requests for repeatedly propo sed (and refused) removals. Notes:
|
coastcarevic.files.wordpress.com
coastcarevic.files.wordpress.com
In particular I want to reference: https://coastcarevic.files.wordpress.com/2015/09/coastline_32.pdf in https://en.wikipedia.org/wiki/Froggatt_Awards.
coastcarevic.files.wordpress.com is the official web site / blog for a state Government sponsored community engagement initiative.
It appears to be blocked due to the global block: \bfiles\.wordpress\.com\b, which I cannot find in the global block list ?
Aoziwe (talk) 11:39, 9 February 2018 (UTC)
- @Aoziwe:
not globally blocked. The domain was blocked a while back, for a couple of weeks or so, though is no longer, and is explicitly whitelisted at enWP these days anyway. Noting that the domain is blacklisted locally due to spam efforts. — billinghurst sDrewth 03:06, 10 February 2018 (UTC)
daviscountyutah.gov
daviscountyutah.gov
Hello. I would like to use that website as a reference, it's a county government website, but it has been blocked since 2015 because
has a redirect service that is being abused
Is the problem solved now? Thanks, October Ends (talk) 12:03, 20 February 2018 (UTC).
- @October Ends:
Removed from Spam blacklist. Also answered over at frWP. — billinghurst sDrewth 12:25, 20 February 2018 (UTC)
Troubleshooting and problems
|
|
This section is for comments related to problems with the blacklist (such as incorrect syntax or entries not being blocked), or problems saving a page because of a blacklisted link. This is not the section to request that an entry be unlisted (see Proposed removals above). |
derefer.unbubble.eu deblock
unbubble.eu
derefer.unbubble.eu
This authority is used 24.923 times in main space in dewiki!. It is used to clean up Special:Linksearch from known dead links, by redirecting them over this authority. It is hard to find a better solution for this task. --Boshomi (talk) 16:38, 24 July 2015 (UTC) Ping:User:BillinghurstBoshomi (talk) 16:49, 24 July 2015 (UTC)
- Please notice Phab:T89586, while not fixed, it is not possible to find the links with standard special:LinkSearch. in dewiki we can use giftbot/Weblinksuche instead.--Boshomi (talk) 18:04, 24 July 2015 (UTC)
- afaics derefer.unbubble.eu could be used to circumvent the SBL, is that correct? -- seth (talk) 21:30, 24 July 2015 (UTC)
- I don't think so, the redircted URL is unchanged, so the SBL works like the achive-URLs to the Internet Archive. --Boshomi (talk) 07:44, 25 July 2015 (UTC)
- It is not a stored/archived page at archive.org, it is a redirect service as clearly stated at the URL and in that it obfuscates links. To describe it in any other way misrepresents the case, whether deWP uses it for good or not. We prevent abuseable redirects from other services due to the potential for abuse. You can consider whitelisting the URL in w:de:MediaWiki:spam-whitelist if it is a specific issue for your wiki. — billinghurst sDrewth 10:09, 25 July 2015 (UTC)
- what I want to say was that the SBL-mechanism works in the same way like web.archive.org/web. A blocked URL will be blocked with unbubble-prefix to the blocked URL.--Boshomi (talk) 12:54, 25 July 2015 (UTC)
- It is not a stored/archived page at archive.org, it is a redirect service as clearly stated at the URL and in that it obfuscates links. To describe it in any other way misrepresents the case, whether deWP uses it for good or not. We prevent abuseable redirects from other services due to the potential for abuse. You can consider whitelisting the URL in w:de:MediaWiki:spam-whitelist if it is a specific issue for your wiki. — billinghurst sDrewth 10:09, 25 July 2015 (UTC)
- I don't think so, the redircted URL is unchanged, so the SBL works like the achive-URLs to the Internet Archive. --Boshomi (talk) 07:44, 25 July 2015 (UTC)
- afaics derefer.unbubble.eu could be used to circumvent the SBL, is that correct? -- seth (talk) 21:30, 24 July 2015 (UTC)
non-ascii are not blocked?
казино-форум.рф
I saw \bказино-форум\.рф\b in the page, so it's supposed to be blocked. However, I can link it: http://казино-форум.рф
It seems like all non-ascii links will be able to avoid blocking.
In Thai Wikipedia (where I am an admin), there are a lot of Thai URLs that we want to put them in the local blacklist but we couldn't because of the very same reason. --Nullzero (talk) 17:42, 18 February 2016 (UTC)
- This should go to Phab: quickly - that is a real issue. --Dirk Beetstra T C (en: U, T) 05:52, 21 February 2016 (UTC)
- @Beetstra: Please see Phab:T28332. It seems that you need to put
\xd0\xba\xd0\xb0\xd0\xb7\xd0\xb8\xd0\xbd\xd0\xbe-\xd1\x84\xd0\xbe\xd1\x80\xd1\x83\xd0\xbc\.\xd1\x80\xd1\x84(without\b) instead of\bказино-форум\.рф\b--Nullzero (talk) 20:00, 21 February 2016 (UTC)- *sigh* somehow the workaround doesn't work with Thai characters, so I don't know if
\xd0\xba\xd0\xb0\xd0\xb7\xd0\xb8\xd0\xbd\xd0\xbe-\xd1\x84\xd0\xbe\xd1\x80\xd1\x83\xd0\xbc\.\xd1\x80\xd1\x84will actually work or not. Please try it anyway... --Nullzero (talk) 20:24, 21 February 2016 (UTC)
- *sigh* somehow the workaround doesn't work with Thai characters, so I don't know if
- @Beetstra: Please see Phab:T28332. It seems that you need to put
Are these global, or local
Regarding my log of blocks, I have over the last days blocked almost 50 IPs whose only edits are hitting the blacklist (there is a related filter on en, that gets some hits). It makes the logs unreadable. My questions: a) is this a global problem, and b) if so, can we have a bot that globally blocks these IPs on sight (with withdrawal of talkpage access) so we de-clutter the logs. I block these for 1 month at first, as soon as they attempt to use one of the typical domains/links or when they add links to typical pages they tend to try. Is it feasible to have bot that gets access to these records and locks them globally? --Dirk Beetstra T C (en: U, T) 06:55, 19 October 2016 (UTC)
Note: seeing this (meta admin eyes only), it is global, there are many IPs with the same MO. So the request simplifies: can we have a bot that globally checks for this and lock the IPs on sight so we declutter the logs. For en.wikipedia, we have en:Template:spamblacklistblock as block reason/talkpage template for them. --Dirk Beetstra T C (en: U, T) 06:58, 19 October 2016 (UTC)
- @Beetstra: that is stewards' territory. There was no such bot when I had that hat, and if one existed now you would see it with the steward's right. So, no would be my guided guess — billinghurst sDrewth 03:43, 24 February 2017 (UTC)
- @Billinghurst: On en.wiki there is now a bot that blocks them, but maybe that should be extended (I notice that the list of regexes of that bot needs updating). --Dirk Beetstra T C (en: U, T) 07:47, 24 February 2017 (UTC)
- Yes, I know it. I am saying that this is a matter for stewards to globally block, rather than locally block IP addresses which is an admin task. — billinghurst sDrewth 09:06, 24 February 2017 (UTC)
- Note (@stewards who see this): I do have the impression that it actually helps. One of the domains that was hammered hard in the past has now disappeared from the lists, and although new domains pop up (which hence do not result in spamblackistblocks), and we still get daily hits, it is much less than before. Maybe worth to try this globally. --Dirk Beetstra T C (en: U, T) 03:36, 5 March 2017 (UTC)
- It might be worth doing. It's probably a bit more difficult to implement globally, since there is no centralized location for SBL hits, and we don't have any existing bots with steward permissions. – Ajraddatz (talk) 05:23, 5 March 2017 (UTC)
- It is a bit more difficult indeed, but I guess that loading the per-wiki spam-blacklist log for the last y hist every x minutes would be sufficient. IIRC, the bot on en.wikipedia pulls the log every 2 minutes, in that time the bot could do a full round on the ~800 wikis. It indeed would likely need steward permissions, but I am sure that there are people with a steward bit and that are bot operators, which could run the script that User:Anomie is running on en.wikipedia. --Dirk Beetstra T C (en: U, T) 10:23, 5 March 2017 (UTC)
- (or the logs should be made public - I don't really see a reason why these are admin-eyes-only anyway). --Dirk Beetstra T C (en: U, T) 10:24, 5 March 2017 (UTC)
- It might be worth doing. It's probably a bit more difficult to implement globally, since there is no centralized location for SBL hits, and we don't have any existing bots with steward permissions. – Ajraddatz (talk) 05:23, 5 March 2017 (UTC)
- Note (@stewards who see this): I do have the impression that it actually helps. One of the domains that was hammered hard in the past has now disappeared from the lists, and although new domains pop up (which hence do not result in spamblackistblocks), and we still get daily hits, it is much less than before. Maybe worth to try this globally. --Dirk Beetstra T C (en: U, T) 03:36, 5 March 2017 (UTC)
- Yes, I know it. I am saying that this is a matter for stewards to globally block, rather than locally block IP addresses which is an admin task. — billinghurst sDrewth 09:06, 24 February 2017 (UTC)
- @Billinghurst: On en.wiki there is now a bot that blocks them, but maybe that should be extended (I notice that the list of regexes of that bot needs updating). --Dirk Beetstra T C (en: U, T) 07:47, 24 February 2017 (UTC)
comune.viagrande.ct.it
comune.viagrande.ct.it
This is the official website of the Italian commune of Viagrande (d:Q478954) and the blacklist block it as "spam to viagra". In es.wiki, if the infobox takes the data of Wikidata the page is permanently blocked. --Metrónomo-Goldwyn-Mayer 23:09, 29 April 2017 (UTC)
- I have adapted the rule, it now should exclude the link: http://comune.viagrande.ct.it. --Dirk Beetstra T C (en: U, T) 06:54, 30 April 2017 (UTC)
This section is resolved and can be archived. If you disagree, replace this template with your comment. — billinghurst sDrewth 30 April 2017 (UTC) Problem about classic-rocks.it
classic-rocks.it
I had problem in adding link above to the web site www.classic-rocks.it because, in my own opinion, it counfuse this web site with the black list one brocks.it. How to solve problem? -- Pugliak (talk) 17:39, 28 October 2017 (UTC)
- @Pugliak: Removed — billinghurst sDrewth 03:48, 29 October 2017 (UTC)
Discussion
|
|
This section is for discussion of Spam blacklist issues among other users. |
Expert maintenance
One (soon) archived and rejected removal suggestion was about jxlalk.com matched by a filter intended to block xlalk.com. One user suggested that this side-effect might be as it should be, another user suggested that regular expressions are unable to distinguish these cases, and nobody has a clue when and why xlalk.com was blocked. I suggest to find an expert maintainer for this list, and to remove all blocks older than 2010. The bots identifying abuse will restore still needed ancient blocks soon enough, hopefully without any oogle matching google cases. –Be..anyone (talk) 00:50, 20 January 2015 (UTC)
- No, removing some of the old rules, before 2010 or even before 2007, will result in further abuse, some of the rules are intentionally wide as to stop a wide range of spamming behaviour, and as I have argued as well, I have 2 cases on my en.wikipedia list where companies have been spamming for over 7 years, have some of their domains blacklisted, and are still actively spamming related domains. Every single removal should be considered on a case-by-case basis. --Dirk Beetstra T C (en: U, T) 03:42, 20 January 2015 (UTC)
- Just to give an example to this - redirect sites have been, and are, actively abused to circumvent the blacklist. Some of those were added before the arbitrary date of 2010. We are not going to remove those under the blanket of 'having been added before 2010', they will stay blacklisted. Some other domains are of similar gravity that they should never be removed. How are you, reasonably, going to filter out the rules that never should be removed. --Dirk Beetstra T C (en: U, T) 03:52, 20 January 2015 (UTC)
- By the way, you say ".. intended to block xlalk.com .." .. how do you know? --Dirk Beetstra T C (en: U, T) 03:46, 20 January 2015 (UTC)
- I know that nobody would block
icrosoft.comif what they mean ismicrosoft.com, or vice versa. It's no shame to have no clue about regular expressions, a deficit we apparently share.
–Be..anyone (talk) 06:14, 20 January 2015 (UTC)
- I am not sure what you are referring to - I am not native in regex, but proficient enough. The rule was added to block, at least, xlale.com and xlalu.com (if it were ONLY these two, \bxlal(u|e)\.com\b or \bxlal[ue]\.com\b would have been sufficient, but it is impossible to find this far back what all was spammed, possibly xlali.com, xlalabc.com and abcxlale.com were abused by these proxy-spammers. --Dirk Beetstra T C (en: U, T) 08:50, 20 January 2015 (UTC)
- I know that nobody would block
- xlalk.com may have been one of the cases, but one rule that was blacklisted before this blanket was imposed was 'xlale.com' (xlale.com rule was removed in a cleanout-session, after the blanket was added). --Dirk Beetstra T C (en: U, T) 04:45, 20 January 2015 (UTC)
- The dots in administrative domains and DNS mean something, notably
foo.bar.exampleis typically related to an administrativebar.exampledomain (ignoring well-known exceptions likeco.uketc., Mozilla+SURBL have lists for this), whilefoobar.examplehas nothing to do withbar.example. –Be..anyone (talk) 06:23, 20 January 2015 (UTC)- I know, but I am not sure how this relates to this suggested cleanup. --Dirk Beetstra T C (en: U, T) 08:50, 20 January 2015 (UTC)
- If your suggested clean-ups at some point don't match jxlalk.com the request by a Chinese user would be satisfied—as noted all I found out is a VirusTotal "clean", it could be still a spam site if it ever was a spam site.
- The regexp could begin with "optionally any string ending with a dot" or similar before xlalk. There are "host name" RFCs (LDH: letter digit hyphen) up to IDNAbis (i18n domains), they might contain recipes. –Be..anyone (talk) 16:56, 20 January 2015 (UTC)
- What suggested cleanups? I am not suggesting any cleanup or blanket removal of old rules. --Dirk Beetstra T C (en: U, T) 03:50, 21 January 2015 (UTC)
- I know, but I am not sure how this relates to this suggested cleanup. --Dirk Beetstra T C (en: U, T) 08:50, 20 January 2015 (UTC)
- The dots in administrative domains and DNS mean something, notably
- I have supported delisting above, having researched the history, posted at Talk:Spam_blacklist/About#Old_blacklisting_with_scanty_history. If it desired to keep xlale.com and xlalu.com on the blacklist (though it's useless at this point), the shotgun regex could be replaced with two listings, easy peasy. --Abd (talk) 01:42, 21 January 2015 (UTC)
- As I said earlier, are you sure that it is only xlale and xlalu, those were the two I found quickly, there may have been more, I do AGF that the admin who added the rule had reason to blanket it like this. --Dirk Beetstra T C (en: U, T) 03:50, 21 January 2015 (UTC)
- Of course I'm not sure. There is no issue of bad faith. He had reason to use regex, for two sites, and possibly suspected additional minor changes would be made. But he only cited two sites. One of the pages was deleted, and has IP evidence on it, apparently, which might lead to other evidence from other pages, including cross-wiki. But the blacklistings themselves were clearly based on enwiki spam and nothing else was mentioned. This blacklist was the enwiki blacklist at that time. After enwiki got its own blacklist, the admin who blacklisted here attempted to remove all his listings. This is really old and likely obsolete stuff. --Abd (talk) 20:07, 21 January 2015 (UTC)
- 3 at least. And we do not have to present a full case for blacklisting (we often don't, per en:WP:BEANS and sometimes privacy concerns), we have to show sufficient abuse that needs to be stopped. And if that deleted page was mentioned, then certainly there was reason to believe that there were cross-wiki concerns.
- Obsolete, how do you know? Did you go through the cross-wiki logs of what was attempted to be spammed? Do you know how often some of the people active here are still blacklisting spambots using open proxies? Please stop with these sweeping statements until you have fully searched for all evidence. 'After enwiki got its own blacklist, the admin who blacklisted here attempted to remove all his listings.' - no, that was not what happened. --Dirk Beetstra T C (en: U, T) 03:16, 22 January 2015 (UTC)
- Hi!
- I searched all the logs (Special:Log/spamblacklist) of several wikis using the regexp entry /xlal[0-9a-z-]*\.com/.
- There were almost no hits:
- Of course I'm not sure. There is no issue of bad faith. He had reason to use regex, for two sites, and possibly suspected additional minor changes would be made. But he only cited two sites. One of the pages was deleted, and has IP evidence on it, apparently, which might lead to other evidence from other pages, including cross-wiki. But the blacklistings themselves were clearly based on enwiki spam and nothing else was mentioned. This blacklist was the enwiki blacklist at that time. After enwiki got its own blacklist, the admin who blacklisted here attempted to remove all his listings. This is really old and likely obsolete stuff. --Abd (talk) 20:07, 21 January 2015 (UTC)
w:ca: 0 w:ceb: 0 w:de: 0 w:en: 1: 20131030185954, xlalliance.com w:es: 1: 20140917232510, xlalibre.com w:fr: 0 w:it: 0 w:ja: 0 w:nl: 0 w:no: 0 w:pl: 0 w:pt: 0 w:ru: 0 w:sv: 0 w:uk: 0 w:vi: 0 w:war: 0 w:zh: 1: 20150107083744, www.jxlalk.com
- So there was just one single hit at w:en (not even in the main namespace, but in the user namespace), one in w:es, and one in w:zh (probably a false positive). So I agree with user:Abd that removing of this entry from the sbl would be the best solution. -- seth (talk) 18:47, 21 February 2015 (UTC)
- Finally an argument based on evidence (these logs should be public, not admin-only - can we have something like this in a search-engine, this may come in handy in some cases!). Consider removed. --Dirk Beetstra T C (en: U, T) 06:59, 22 February 2015 (UTC)
- By the way, Seth, this is actually no hits - all three you show here are collateral. Thanks for this evidence, this information would be useful on more occasions to make an informed decision (also, vide infra). --Dirk Beetstra T C (en: U, T) 07:25, 22 February 2015 (UTC)
- I am not sure that we want the Special page to be public, though I can see some value in being able to have something at ToolLabs to be available to run queries, or something available to be run through quarry. — billinghurst sDrewth 10:57, 22 February 2015 (UTC)
- Why not public? There is no reason to hide this, this is not BLP or COPYVIO sensitive information in 99.99% of the hits. The chance that this is non-public information is just as big as for certain blocks to be BLP violations (and those are visible) ... --Dirk Beetstra T C (en: U, T) 04:40, 23 February 2015 (UTC)
- I am not sure that we want the Special page to be public, though I can see some value in being able to have something at ToolLabs to be available to run queries, or something available to be run through quarry. — billinghurst sDrewth 10:57, 22 February 2015 (UTC)
- So there was just one single hit at w:en (not even in the main namespace, but in the user namespace), one in w:es, and one in w:zh (probably a false positive). So I agree with user:Abd that removing of this entry from the sbl would be the best solution. -- seth (talk) 18:47, 21 February 2015 (UTC)
Now restarting the original debate
As the blacklist is long, and likely contains rules that are too wide a net and which are so old that they are utterly obsolete (or even, may be giving collateral damage on a regular basis), can we see whether we can set up some criteria (that can be 'bot tested'):
- Rule added > 5 years ago.
- All hits (determined on a significant number of wikis), over the last 2 years (for now: since the beginning of the log = ~1.5 years) are collateral damage - NO real hits.
- Site is not a redirect site (should not be removed, even if not abused), is not a known phishing/malware site (to protect others), or a true copyright violating site. (this is hard to bot-test, we may need s.o. to look over the list, take out the obvious ones).
We can make some mistakes on old rules if they are not abused (remove some that actually fail #3) - if they become a nuisance/problem again, we will see them again, and they can be speedily re-added .. thoughts? --Dirk Beetstra T C (en: U, T) 07:25, 22 February 2015 (UTC)
- @@Hoo man: you have worked on clean up before, some of your thoughts would be welcomed. — billinghurst sDrewth 10:53, 22 February 2015 (UTC)
- Doing this kind of clean up is rather hard to automatize. What might be working better for starters could be removing rules that didn't match anything since we started logging hits. That would presumably cut down the whole blacklist considerably. After that we could re-evaluate the rest of the blacklist, maybe following the steps outlined above. - Hoo man (talk) 13:33, 22 February 2015 (UTC)
- Not hitting anything is dangerous .. there are likely some somewhat obscure redirect sites on it which may not have been attempted to be abused (though, also those could be re-added). But we could do test-runs easily - just save a cleaned up copy of the blacklist elsewhere, and diff them against the current list, and see what would get removed.
- Man, I want this showing up in the RC-feeds, then LiWa3 could store them in the database (and follow redirects to show what people wanted to link to ..). --Dirk Beetstra T C (en: U, T) 03:30, 23 February 2015 (UTC)
- Hi!
- I created a table of hits of blocked link additions. Maybe it's of use for the discussion: User:lustiger_seth/sbl_log_stats (1,8 MB wiki table).
- I'd appreciate, if we deleted old entries. -- seth (talk) 22:12, 26 February 2015 (UTC)
- Hi, thank you for this, it gives a reasonable idea. Do you know if the rule-hits were all 'correct' (for those that do show that they were hit) or mainly/all false-positives (if they are false-positive hitting, we could based on this also decide to tighten the rule to avoid the false-positives). Rules with all-0 (can you include a 'total' score) would certainly be candidates for removal (though still determine first whether they are 'old' and/or are nono-sites before removal). I am also concerned that this is not including other wikifarms - some sites may be problematic on other wikifarms, or hitting a large number of smaller wikis (which have less control due to low admin numbers). --Dirk Beetstra T C (en: U, T) 03:36, 8 March 2015 (UTC)
- Hi!
- We probably can't get information of false positives automatically. I added a 'sum' column.
- Small wikis: If you give me a list of the relevant ones, I can create another list. -- seth (talk) 10:57, 8 March 2015 (UTC)
- Thanks for the sum-column. Regarding the false-positives, it would be nice to be able to quickly see what actually got blocked by a certain rule, I agree that that then needs a manual inspection, but the actual number of rules with zero hits on the intended stuff to be blocked is likely way bigger than what we see.
- How would you define the relevant small wikis - that is depending on the link that was spammed? Probably the best is to parse all ~750 wiki's, make a list of rules with 0 hits, and a separate list of rules with <10 hits (and including there the links that were blocked), and exclude everything above that. Then these resulting rules should be filtered by those which were added >5 years ago. That narrows down the list for now, and after a check for obvious no-no links, those could almost be blanket-removed (just excluding the ones with real hits, the obvious redirect sites and others - which needs a manual check). --Dirk Beetstra T C (en: U, T) 06:59, 9 March 2015 (UTC)
- Hi!
- At User:Lustiger_seth/sbl_log_stats/all_wikis_no_hits there's a list containing ~10k entries that never triggered the sbl during 2013-sep and 2015-feb anywhere (if my algorithm is correct).
- If you want to get all entries older than 5 years, then it should be sufficent to use only the entries in that list until (and including)
\bbudgetgardening\.co\.uk\b. - So we could delete ~5766 entries. What do think? Shall we give it a try? -- seth (talk) 17:06, 18 April 2015 (UTC)
- The question is, how many of those are still existing redirect sites etc. Checking 5800 is quite a job. On the other hand, with LiWa3/COIBot detecting - it is quite easy to re-add them. --Dirk Beetstra T C (en: U, T) 19:28, 21 April 2015 (UTC)
- According to the last few lines, I've removed 124kB of non-hitting entries now. I did not remove all of them, because some were url shorteners and I guess, that they are a special case, even if not used yet. -- seth (talk) 22:25, 16 September 2015 (UTC)
- The question is, how many of those are still existing redirect sites etc. Checking 5800 is quite a job. On the other hand, with LiWa3/COIBot detecting - it is quite easy to re-add them. --Dirk Beetstra T C (en: U, T) 19:28, 21 April 2015 (UTC)
- Hi, thank you for this, it gives a reasonable idea. Do you know if the rule-hits were all 'correct' (for those that do show that they were hit) or mainly/all false-positives (if they are false-positive hitting, we could based on this also decide to tighten the rule to avoid the false-positives). Rules with all-0 (can you include a 'total' score) would certainly be candidates for removal (though still determine first whether they are 'old' and/or are nono-sites before removal). I am also concerned that this is not including other wikifarms - some sites may be problematic on other wikifarms, or hitting a large number of smaller wikis (which have less control due to low admin numbers). --Dirk Beetstra T C (en: U, T) 03:36, 8 March 2015 (UTC)
Blacklisting spam URLs used in references
Looks like a site is using the "references" section as a spam farm. If a site is added to this list, can the blacklist block the spam site? Raysonho (talk) 17:45, 5 September 2015 (UTC)
- Yes they can.--AldNonymousBicara? 21:56, 5 September 2015 (UTC)
- Thanks, Aldnonymous! Raysonho (talk) 00:07, 6 September 2015 (UTC)
url shorteners
Hi!
IMHO the url shorteners should be grouped in one section, because they are a special group of urls that need a special treatment. A url shortener should not be removed from sbl unless the domain is dead, even if it has not been used for spamming, right? -- seth (talk) 22:11, 28 September 2015 (UTC)
- That would be beneficial to have them in a section. Problem is, most of them are added by script, and are hence just put at the bottom. --Dirk Beetstra T C (en: U, T) 04:51, 4 October 2015 (UTC)
- Maybe it would seem more preferable to have "spam blacklist" be a compilation file, made of files one of which would be "spam blacklist.shorteners" — billinghurst sDrewth 12:15, 24 December 2015 (UTC)
- This seems like a nice idea. Would certainly help with cleaning up of it (which we don't do nowadays). IIRC, it is technically possible to have different spam blacklist pages so this is technically possible, just needs a agreement among us and someone to do it. --Glaisher (talk) 12:17, 24 December 2015 (UTC)
- Maybe it would seem more preferable to have "spam blacklist" be a compilation file, made of files one of which would be "spam blacklist.shorteners" — billinghurst sDrewth 12:15, 24 December 2015 (UTC)
@Beetstra, Lustiger seth, Glaisher, Vituzzu, MarcoAurelio, Hoo man, and Legoktm: and others. What are your thoughts on a concatenation of files as described above. If we have a level of agreement, then we can work out the means to an outcome. — billinghurst sDrewth 12:39, 25 January 2016 (UTC)
- I am somewhat in favour of this - split the list into a couple of sublists - one for url-shorteners, one for 'general terms' (mainly at the top of the list currently), and the regular list. It would however need an adaptation of the blacklist script (I've done something similar for en.wikipedia (a choice of blacklisting or revertlisting for each link), I could give that hack a try here, time permitting). AFAIK the extension in the software is capable of handling this. Also, it would be beneficial for the cleanout work, that the blacklist itself is 'sectioned' into years. Although being 8 years old is by no means a reason to expect that the spammers are not here anymore (I have two cases on en.wikipedia that are older than that), we do tend to be more lenient with the old stuff. (on the other hand .. why bother .. the benefits are mostly on our side so we don't accidentally remove stuff that should be solved by other means). --Dirk Beetstra T C (en: U, T) 13:05, 25 January 2016 (UTC)
- Is it really possible to have different spam blacklist pages? What would happen to the sites that use this very list to block unwanted spam? —MarcoAurelio 14:23, 25 January 2016 (UTC)
- It is technically possible. But this would mean that if we move all the URL shortener entries to a new page, all sites using it currently would have to update the extension or explicitly add the new blacklist to their config or these links would be allowed on their sites (and notifying all these wikis about this breaking change is next to impossible). Another issue I see is that a new blacklist file means there would be a separate network request on cache miss so their might be a little delay in page saves (but I'm not sure whether this delay would be a noticeable delay). --Glaisher (talk) 15:38, 25 January 2016 (UTC)
- Hi!
- Before we activate such a feature, we should update some scripts that don't know anything about sbl subpages yet.
- Apart from that I don't think that a sectioning into years would be of much use. One can use the (manual) log for this. A subject-oriented sectioning could be of more use, but this would also be more difficult for us. -- seth (talk) 20:49, 27 January 2016 (UTC)
- Another list for shorters would be a good idea. Also, a bunch of years ago Hoo wrote a script to find (and remove) expired domains. --Vituzzu (talk) 14:10, 22 February 2017 (UTC)
Before developers spend a lot of time on this, I would really prefer that they spend their time to completely overhaul the whole blacklist system, and make it more edit-filter like (though not with the heavy overhead of the interpretation mechanism, just plain regexes like the blacklist, but different rules for different regexes. --Dirk Beetstra T C (en: U, T) 10:08, 23 February 2017 (UTC)
External repositories
I recently found opensources.co's notCredible list. I think this, along with similar projects, could be a good addition to the list.
Also, it would be great if the updates could be automated.--Strainu (talk) 16:05, 10 February 2017 (UTC)
- It hasn't been our approach to proactively seek out lists of problematic sites, it has instead been reactive to problematic editing, typical examples are the redirect urls where we wait until they are spammed. To do that I would be prefer to see an RFC at meta and something notified to the wikis. Such a change needs to have a demonstrated value. Do we have an indication that we are being abused? Maybe we are better talking to @Beetstra: on whether we can utiise COIBot to simply add each domain to the bot's monitor list, at least as part of the proof of problem needing to be managed.
*If* were going to change to pro-active generated lists, I would like to see a re-architecture of how the blacklist is generated. It would be more useful to keep all such lists as their own entities and have a means to concatenate them at list generation time. I look at the indication of time that it takes for spamblacklist to roll out and anything that makes that longer and longer with little clear gain. — billinghurst sDrewth 11:36, 11 February 2017 (UTC)
- To note that I have picked 10 clickbait sites and having COIBot run some checks to see what may be there. — billinghurst sDrewth 11:40, 11 February 2017 (UTC)
Mentioning the sites here, in a LinkSummary template, makes a record in COIBot that they were reported here, and a report is generated. As anything can be spammed, I see little gain in having everything being monitored, even pre-emptively. We know that some spammers will create domains as they need them, and they cannot be pre-emptively monitored. Having a stronger detection mechanism may be nice, but since the linkwatchers are having a pain following everything, I don't think I will have resources to make a stronger mechanism (nor do I have time).
What sDrewth is suggesting is of what we want already for a long, long time: A proper rewrite of the spam blacklist. Developers unfortunately are busy with what WMF thinks is more important.</frustration> What I would like to have is a spam blacklist based on the EditFilter - take the current EditFilter, strip it completely of its interpretation function, and replace it with a simple field with regexes (formatted like in our blacklist). The only test that the system needs to do is to test, for each 'SpamFilter', whether the regexes match against the added external links. You then have the options (already available from EditFilter) to log only, warn, throttle, (warn and block), and block, add a custom custom ('hej <expletive>, you are using a redirect site, please adapt your edit and use the real site'). You can then fancy it up with more options if you like (add whitelisting, wikifamily-selection, namespace-selection, per-page exclusion, etc. etc.). It would give much more flexibility and control to the current spam-blacklist, and pre-emptive monitoring through log-only (or even warning) would be a non-disruptive option. Working a lot here and on en.wikipedia, I am sometimes surprised how much 'good use' there is of some links that nonetheless need to be blacklisted due to uncontrollable abuse. --Dirk Beetstra T C (en: U, T) 03:48, 13 February 2017 (UTC)
- @Hoo man: is there value transferring some of this into phabricator request for spam blacklist v. extrapolation of abuse filter? I remember there was some discussion about abuse filter needing some sort of morph, so not sure how we best address this to the developer community. — billinghurst sDrewth 03:53, 24 February 2017 (UTC)
- @Billinghurst: Sorry for the late reply, but yes please, bring this up on Phabricator. We have been talking about this change since at least 2013, but sadly nothing happened since (except for some discussions here and there). Cheers, Hoo man (talk) 10:40, 30 March 2017 (UTC)
Make all groups non-capturing
Currently, there are almost 200 capturing groups used in the blacklist. Because these capture, the regex engine has to devote extra resources to them, and because nothing is done with the groups, this extra expenditure is pointless. These groups can be made non-capturing by adding ?: just after the opening parenthesis. Note that if any group already has a ? following the opening parenthesis, the group shouldn't be touched. (The current coding is not actively problematic to my knowledge, so this is more an ounce-of-prevention, best practices/consistency thing; there are already ~160 groups that are non-capturing in the list.) Dinoguy1000 (talk) 21:28, 3 September 2017 (UTC)
- @Dinoguy1000: I noticed that User:Billinghurst has adapted all those regexes. --Dirk Beetstra T C (en: U, T) 05:01, 14 September 2017 (UTC)
- started, and noted to where. Will complete when I have a little more time. — billinghurst sDrewth 10:11, 14 September 2017 (UTC)
- Cool, I'd actually already forgotten about this request. If you're comfortable enough with regexes, I also noticed one or two groups that could be reduced to character sets (though I'd have to look through the list again to find them). There's probably other potential optimizations lurking in the list, too (though most of them are probably so minor as to not be worth worrying about from a purely optimization perspective). Dinoguy1000 (talk) 12:14, 14 September 2017 (UTC)
- @Dinoguy1000: You could just copy the whole page into a user-sandbox of yourself, and adapt them. A diff between the current version of the spam blacklist and your sandbox then gives us the opportunity to see what changed, and decide to copy it back into the blacklist if we don't see any problems. We do similar things with cleaning up old regexes that can go, or maintenance-type combination of multiple regexes into one. --Dirk Beetstra T C (en: U, T) 13:17, 14 September 2017 (UTC)
- I'm not broken up enough about it to do so at this time, I think, though I'll definitely keep the option in mind in the future. Dinoguy1000 (talk) 13:25, 15 September 2017 (UTC)
- @Dinoguy1000: You could just copy the whole page into a user-sandbox of yourself, and adapt them. A diff between the current version of the spam blacklist and your sandbox then gives us the opportunity to see what changed, and decide to copy it back into the blacklist if we don't see any problems. We do similar things with cleaning up old regexes that can go, or maintenance-type combination of multiple regexes into one. --Dirk Beetstra T C (en: U, T) 13:17, 14 September 2017 (UTC)
- Cool, I'd actually already forgotten about this request. If you're comfortable enough with regexes, I also noticed one or two groups that could be reduced to character sets (though I'd have to look through the list again to find them). There's probably other potential optimizations lurking in the list, too (though most of them are probably so minor as to not be worth worrying about from a purely optimization perspective). Dinoguy1000 (talk) 12:14, 14 September 2017 (UTC)
- started, and noted to where. Will complete when I have a little more time. — billinghurst sDrewth 10:11, 14 September 2017 (UTC)
Global Email blacklist
Hello, FYI: Email blacklist is available now. --Steinsplitter (talk) 13:10, 13 September 2017 (UTC)
How can I prove that I am not a paid editor?
I just imported 25 images from the website of the University of California at Santa Barbara and I'm sure that COIBot might note me as the exclusive user of this link and I've used it numerous of times on Wikipedian like at w:nl:Riukiuaanse mon and one Commons category is Commons:Category:Kokin kousei, Shinsen zeni kagami, I know the external linking policy and I have no COI, but how can I declare this before this link gets globally blacklisted and every mention of the University of California at Santa Barbara gets removed? --Donald Trung (Talk 🤳🏻) (My global lock 🔒) (My global unlock 🔓) 13:45, 15 December 2017 (UTC)
- Don't fuss COIBot's reports. It watches and runs reports, it makes no accusations. Users read reports and make assessments. You jump to conclusions. — billinghurst sDrewth 14:59, 15 December 2017 (UTC)
- I just don’t want a repeat of this and this (this reference was placed here years ago, and not by me), and I am currently planning on importing over 300 images from Art-Hanoi to Wikimedia Commons and importing a full book from that same website to Wikisource, I also used that website as a source on multiple wiki’s so I fear that if I do that it could get blacklisted, when I notified the removing party the last time he stated that I have “an obvious COI” but I can’t tell how or why other than using it as a source. Since I write about obscure subjects I will usually be the person to mostly use a link. Can I at least request that when Art-Hanoi or the University of California at Santa Barbara get reviewed by “spam fighters” that they will look objectively at what those links support instead of just blindly removing references? I don’t want the situation to repeat. --Donald Trung (Talk 🤳🏻) (My global lock 🔒) (My global unlock 🔓) 15:34, 15 December 2017 (UTC)
- @Donald Trung: Do you understand why that link was (considered to be) blacklisted? Do you understand why global locks, local blocks, abusefilters, etc. etc. are being applied? Do you understand why Arbitration Committees ban, block or desysop people? Since you keep failing to drop the stick (see this discussion and your signature), obviously you don't. Core policies apply to anyone, I do not understand why you think it is fine to sock (and even still show understanding for others who do). As I suggested elsewhere: Drop it. Edit. Communicate. Now! --Dirk Beetstra T C (en: U, T) 07:35, 19 December 2017 (UTC)
This section is resolved and can be archived. If you disagree, replace this template with your comment. — billinghurst sDrewth 23:07, 13 February 2018 (UTC) Wikidata Query and tinyurl
Hi,
This subject probably already have been discussed, but I can't find where. So I start it (again ?) here.
Wikidata Query SPARQL links are very longs. They use urlshortener from tinyurl, so these short urls are blocked EVERYWHERE on the WFM hosted sites.
There is a phabricator request to try to figure out how to change these short url, but I don't know how long it will take. So I write here to know if it is possible to block tinyurl only in mainspace ? Simon Villeneuve 12:34, 1 November 2017 (UTC)
- @Simon Villeneuve: The tinyurl is blacklisted as it is abused, along with hundreds of other url redirect services. In the global blacklist it is a hard rule, no finesse. That said, each wiki has the ability to work around this by use of their MediaWiki:spam-whitelist which can take complex regex,though no ability to whitelist based on namespace that I can see. There are phabricator tickets about WMF having a redirect service and you would do well to contribute to those, and when the community again votes on their priority projects for 2018, there will be your opportunity to advocate further.
Personally I am surprised that there isn't a tools project to allow for query to write its own url redirects in conjunction with a closed service at toollabs: similar to how petscan saves its queries, but maybe I don't understand the enormity of the task. — billinghurst sDrewth 08:42, 2 November 2017 (UTC)
- @Billinghurst:,
Thank you for the complete answer. Simon Villeneuve 15:38, 2 November 2017 (UTC)
- @Billinghurst:,
- @Simon Villeneuve: Please note that I would, probably vehemently, oppose using the whitelist on WikiData to whitelist a specific url-shortener for frequently used links (or any blanket whitelisting, for that matter). These url-shorteners cannot be restricted there to specific data, and hence be used in all fields, including in those which are rightfully blacklisted globally. One could then, e.g., replace the official website of a subject on WikiData with an abusive redirect link. That link then does get displayed on (all!) local wikis (if it is transcluded in a template, which many are). That means that a possibly harmful link will be displayed globally, ánd that all local wiki pages that transclude a certain property with a globally blacklisted link will be uneditable (as the wiki software parses the 'next edit' as an edit that adds a new link, resulting in the editor (confusingly!) not being allowed to save the page). --Dirk Beetstra T C (en: U, T) 12:41, 4 November 2017 (UTC)
This section is resolved and can be archived. If you disagree, replace this template with your comment. — billinghurst sDrewth 23:06, 13 February 2018 (UTC) Change to BLHandler
I have implemented here in our SBHandler gadget a change adding 2 possibilities for blacklisting:
- {{BLRequestRegex}}: template takes a parameter that is a regex. Is passed to the Blacklist unaltered.
- {{BLRequestLink}}: template takes a full link (not only a domain, e.g.), changes it to a regex, and passes that to the blacklist
- {{LinkSummary}}: as usual, just a domain.
The three are handled in order - if there are BLRequestRegex-es in the section, it will ONLY pass those (ignoring BLRequestLink and LinkSummary); if there is no BLRequestRegex but BLRequestLink-s, it will ONLY pass the parameters of BLRequestLink (ignoring the LinkSummaries), or it will only pass the domains in the LinkSummaries. (@Billinghurst:, see the Twitter request above). --Dirk Beetstra T C (en: U, T) 07:42, 19 December 2017 (UTC)
- Thanks Beetstra. Apart from using here, it will be useful in COIBOt's XWiki entries to capture the specific components. I have been doing a few more of these recently as the spambots have been using these extra links as good flags of problematic intent. — billinghurst sDrewth 11:12, 19 December 2017 (UTC)
- Oh, this only works on the requests on this page, on the /XWiki/domain pages the to-be-blacklisted information is parsed out of the title of the report. --Dirk Beetstra T C (en: U, T) 11:34, 19 December 2017 (UTC)
fiverr
fiverr.com
Hi all. There is article at ar.wiki about the website and because it is listed in spam blacklist it is not possible now to edit the article and this hinder improving its content. Any suggestions to solve this issue?--مصعب (talk) 23:00, 27 January 2018 (UTC)
- @Billinghurst: Hi. Can you help?--مصعب (talk) 00:29, 28 January 2018 (UTC)
- @مصعب: add the exact problem url, or a suitable regex of it, to w:ar:Mediawiki:spam-whitelist, either temporarily—whilst it is saved and embedded into the article—or permanently—if needed elsewhere. Noting that mediawiki is meant to recognise a previously saved url and not prevent saving of the article, so it will have been a significant change to have that hit the blacklist again. — billinghurst sDrewth 04:17, 28 January 2018 (UTC)
This section is resolved and can be archived. If you disagree, replace this template with your comment. — billinghurst sDrewth 23:04, 13 February 2018 (UTC) Thoughts about blacklisting .club/ .space/ and .website/
club
space
website
Call me a curmudgeon, however, the repeat and prolific amount of spambotting for .club, and now the propagating spam for .space domains has me thinking that for the vast bulk of wikis that these two top level domains are just junk that we should block early, and forcefully. I have yet to see a useful link in either of those domains, however, that could be me living a sheltered life. What are peoples thoughts, as we may just be seeing early opportunistic spam from the same set of spambots as they migrate their activities. — billinghurst sDrewth 23:03, 13 February 2018 (UTC)
- Agree. All domains with
.cluband.spaceshould be blacklisted. All these fake accounts already spam at id.wikisource. RaymondSutanto (talk) 17:06, 15 February 2018 (UTC)- I've done that some days ago on the Beta Cluster where (\.space and \.club) due to the enormous ammount of spam there. I see nothing positive comming from those. +1 from me. Specific links can be whitelisted if useful. —MarcoAurelio (talk) 18:31, 15 February 2018 (UTC)
- Also
.websiteseems to be getting spammed. See the Beta Cluster section above. —MarcoAurelio (talk) 23:27, 15 February 2018 (UTC)- Adding .website/ to subject line. Also noting that I blocked these top level domains awhile back at enWS, though it is easier achieve at such a site. — billinghurst sDrewth 01:04, 16 February 2018 (UTC)
- To note that I am flagging this discussion to enWP, deWP, frWP and Commons. — billinghurst sDrewth 01:20, 16 February 2018 (UTC)
- Adding .website/ to subject line. Also noting that I blocked these top level domains awhile back at enWS, though it is easier achieve at such a site. — billinghurst sDrewth 01:04, 16 February 2018 (UTC)
- Test first (I read here about a beta environment where this is being tested now) if blocking helps. My prediction is that the spambots will need less than a week to switch from club to com. Three weeks tops. If I can be proven wrong, go ahead. Alexis Jazz (talk) 01:49, 16 February 2018 (UTC)
- I think we should use a global abusefilter instead (which disallows only non-established users).--GZWDer (talk) 08:06, 16 February 2018 (UTC)
- Hi!
- AFAIK there hasn't been any major recent spamming in deWP via .club, .website or .space domains. Maybe at the weekend I can spare some time to search the replica database at wmflabs for those TLDs, just to see how many links would be affected. Or is there anybody else who has the time for this? -- seth (talk) 08:15, 16 February 2018 (UTC)
- @Lustiger seth: that would be useful in both seeing how many useful links are present, as well as spam that is present. Maybe it is the case that we are catching most of the spam with existing filters, and only a little bit of tidy up. It may be that the spam is only at certain wikis, that is not unknown. There is nothing urgent with this request, eyes wide open is far better with these cases. — billinghurst sDrewth 11:07, 16 February 2018 (UTC)
- quarry:query/24816 gives an overview of links to those TLDs in dewiki. Only 151 links in total for all three (http+https). —MisterSynergy (talk) 11:29, 16 February 2018 (UTC)
- Thanks. :-)
- So with these stats it's quite obivious that
.clubshould not be banned. And even the other two TLDs seem to be useful and it would be too much work to use the white list here, see quarry:query/24829 (enwiki: .space and .website). IMHO global blacklisting would not be a benefit. GZWDer suggested to use the edit filter. I guess this would be less destructive than global blacklisting. -- seth (talk) 12:30, 18 February 2018 (UTC)- Some (to all?) of the current spam is being caught by a global filter, which is why the question arises. So unless we start to get more successful breakthrough then there is no urgent requirement for an additional filter. If it becomes more problematic(ie. greater break through of existing filter) we can look to target it by new user spam global filter. Thanks for the feedback. — billinghurst sDrewth 13:03, 18 February 2018 (UTC)
- quarry:query/24816 gives an overview of links to those TLDs in dewiki. Only 151 links in total for all three (http+https). —MisterSynergy (talk) 11:29, 16 February 2018 (UTC)
- @Lustiger seth: that would be useful in both seeing how many useful links are present, as well as spam that is present. Maybe it is the case that we are catching most of the spam with existing filters, and only a little bit of tidy up. It may be that the spam is only at certain wikis, that is not unknown. There is nothing urgent with this request, eyes wide open is far better with these cases. — billinghurst sDrewth 11:07, 16 February 2018 (UTC)
- I think we should use a global abusefilter instead (which disallows only non-established users).--GZWDer (talk) 08:06, 16 February 2018 (UTC)
- question to all: are these genuine websites, or are they hidden redirect or front-ends. --Dirk Beetstra T C (en: U, T) 07:25, 19 February 2018 (UTC)