User:Billinghurst/coibot in action

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search

Example: active addition to COIbot

[07:39]	EdBever	whoadded wickedlondon.org
[07:39]	COIBot	6 records; Editors who have added wickedlondon.org: 87.194.66.9 (2), 92.24.196.5 (1), 78.147.62.33 (1), 79.121.254.230 (1), 78.147.56.107 (1).
[07:40]	EdBever	case new
[07:40]	COIBot	I have created case #60 for you (”).
[07:41]	EdBever	case 60 add link wickedlondon.org
[07:41]	COIBot	Link wickedlondon.org added to case 60: wickedlondon.org.
[07:41]	EdBever	case 60 add link theatreticketscheap.com
[07:41]	COIBot	Link theatreticketscheap.com added to case 60: wickedlondon.org theatreticketscheap.com.
[07:41]	EdBever	case 60 evolve
[07:43]	COIBot	List of users now is 87.194.66.9 2.98.245.247 92.24.196.5 2.98.247.18 78.147.62.33 79.121.254.230 78.147.56.107 78.147.62.76.
[07:43]	COIBot	I removed users from case 60. Remaining: 87.194.66.9 2.98.245.247 92.24.196.5 2.98.247.18 78.147.62.33 79.121.254.230 78.147.56.107 78.147.62.76.
[07:43]	COIBot	Testing ranges 87.194.66.9/24, 2.98.245.247/24, 92.24.196.5/24, 2.98.247.18/24, 78.147.62.33/24, 79.121.254.230/24, 78.147.56.107/24.
[07:43]	COIBot	List of links now is wickedlondon.org theatreticketscheap.com ur7s.com dutchcharts.nl lucidica.com moulinrougeparis.co.uk theater2k.com phantomoftheoperalondon.com seatplan.co.uk eiffeltowertickets.co.uk tubehotels.com jerseyboyslondon.org wewillrockyoutickets.org drurylanetheatrelondon.com companieshouse.gov.uk guardian.co.uk newstatesman.com wizardofozlondon.org uksport.gov.uk skanes.com hammamet.co.uk portelkantaouihotels.co.uk m
[07:43]	COIBot	I removed 6 links from case 60. Remaining: wickedlondon.org theatreticketscheap.com ur7s.com dutchcharts.nl lucidica.com moulinrougeparis.co.uk theater2k.com phantomoftheoperalondon.com seatplan.co.uk eiffeltowertickets.co.uk tubehotels.com jerseyboyslondon.org wewillrockyoutickets.org drurylanetheatrelondon.com companieshouse.gov.uk guardian.co.uk newstatesman.com wizardofozlondon.org uksport.gov.uk skanes.com hammamet.co.uk portel
[07:43]	COIBot	Autocase evolution for case 60 finished; expanded from 0 to 16 users and from 2 to 46 links.
[07:43]	COIBot	Users case 60: 87.194.66.9 87.194.66.87 87.194.66.188 87.194.66.201 87.194.66.233 2.98.245.247 92.24.196.5 92.24.196.162 92.24.196.212 2.98.247.18 78.147.62.33 78.147.62.76 79.121.254.230 78.147.56.107 78.147.56.143 78.147.56.186.
[07:43]	COIBot	Links case 60: wickedlondon.org theatreticketscheap.com ur7s.com lucidica.com moulinrougeparis.co.uk theater2k.com phantomoftheoperalondon.com seatplan.co.uk eiffeltowertickets.co.uk tubehotels.com jerseyboyslondon.org wewillrockyoutickets.org drurylanetheatrelondon.com companieshouse.gov.uk wizardofozlondon.org uksport.gov.uk skanes.com hammamet.co.uk portelkantaouihotels.co.uk monastir.co.uk paphos-cyprus.com 2ndcuddingtonscouts.o




[00:41]	Beetstra	prepare xwiki duhvoli.com.ua
[00:41]	COIBot	-> report xwiki duhvoli.com.ua 178.95.124.95 91.124.97.211 94.248.48.62 Ffederal Best Ace111
[00:41]	Beetstra	report xwiki duhvoli.com.ua 178.95.124.95 91.124.97.211 94.248.48.62
[00:41]	COIBot	Statistics report for link duhvoli.com.ua by 178.95.124.95 91.124.97.211 94.248.48.62 queued; see [[m:User:COIBot/LinkReports/duhvoli.com.ua]] & [[:en:Wikipedia:WikiProject Spam/LinkReports/duhvoli.com.ua]] (auto: - /).
[00:42]	Beetstra	(I killed the linksaver while it was processing that one
[00:44]	COIBot	Saved an XWiki report for isfana.ru (XWiki spam by 213.234.200.2 81.91.182.165).
[00:47]	Beetstra	!waiting duh
[00:48]	COIBot	1 records matching 'duh' waiting: 1 XWiki, 0 Local, 0 Redirect, 0 Poked, 0 Meta, 0 IP, 0 requested
[00:48]	COIBot	Waiting: duhvoli.com.ua
[00:48]	COIBot	On Wiki: 3 open XWiki reports and 5000 open Local reports.
[00:49]	Beetstra	sDrewth, if you see those reports with a borked name - put it to ignore, have a look what the real link was, and try to generate a report for that (e.g. like I did above for duhvoli.com.ua)
[00:49]	sDrewth	ok
[00:50]	sDrewth	I was just going to clean and close
[00:50]	sDrewth	or revert
[00:51]	Beetstra	Hmm .. also fine, but then they may reappear (which is good as well)
[00:52]	BLWatcher	Billinghurst edited [[:m:User:COIBot/XWiki/kravchyn.com.ua#Discussion]] http://meta.wikimedia.org/wiki/?diff=prev&oldid=3129072 (Reverted)
[00:53]	BLWatcher	Billinghurst edited [[:m:User:COIBot/XWiki/mediamatters.com#Discussion]] http://meta.wikimedia.org/wiki/?diff=prev&oldid=3129074 ({{ignored}})
[00:54]	Beetstra	wonders how long you will take over the thousands of open reports on en, sDrewth ...
[00:54]	BLWatcher	Billinghurst edited [[:m:User:COIBot/XWiki/mediamatters.com#Discussion]] http://meta.wikimedia.org/wiki/?diff=prev&oldid=3129077 (ignore)
[00:55]	sDrewth	what year are we at now?
[00:55]	Beetstra	
[00:55]	sDrewth	I take the elephant approach
[00:55]	Beetstra	They autoclose after some time ..
[00:55]	sDrewth	one mouthful at a time
[00:55]	Beetstra	I hope you have a big mouth
[00:56]	Beetstra	I would quick-scan .. some domainnames are obvious spam, leave the rest for later
[00:56]	sDrewth	one mouth, lots of chewing, lots of mouthfuls
[00:56]	sDrewth	I want to start pulling some of the stuff that we have from checkusers data into this, so we can get some better alerts
[00:57]	sDrewth	we are starting to identify user ranges, etc.
[00:58]	Beetstra	spam reports are good sources for finding socks as well
[00:58]	sDrewth	prepare xwiki mediamatters.com
[00:58]	COIBot	-> report xwiki mediamatters.com 71.192.160.147 Puffin 173.166.3.157 65.102.186.166 Jettparmer
[00:58]	Beetstra	All reports where a user is mentioned can be found with [[special:WhatLinksHere/user talk:Username]] or [[special:WhatLinksHere/user:Username]]
[00:59]	sDrewth	yes, but then I have to go to each wiki
[00:59]	sDrewth	even though enWP is that big elephant
[00:59]	sDrewth	report xwiki mediamatters.com 71.192.160.147 Puffin 173.166.3.157 65.102.186.166 Jettparmer
[00:59]	Beetstra	No, ALL reports are on meta ...
[00:59]	COIBot	Statistics report for link mediamatters.com by 71.192.160.147 Puffin 173.166.3.157 65.102.186.166 Jettparmer queued; see [[m:User:COIBot/LinkReports/mediamatters.com]] & [[:en:Wikipedia:WikiProject Spam/LinkReports/mediamatters.com]] (auto: - /).
[00:59]	sDrewth	k
[00:59]	Beetstra	local reports for all wikis go to meta, and only for en they also go to en
[01:00]	COIBot	Saved an XWiki report for duhvoli.com.ua (XWiki spam by 178.95.124.95 91.124.97.211 94.248.48.62).
[01:01]	Beetstra	those are the tricks to find related links and editors .. if you do a whatlinkshere for the user, you find for which links they were reported, and if you do a whatlinkshere for the link, you see if links were added together with other links
[01:02]	sDrewth	ok, I think. It is 1am, so my brain is trying to understand
[01:02]	Beetstra	And trick 3: Special:LinkSearch on the IP of the domain .. finds which links on one server have been reported
[01:03]	Beetstra	use: 'resolve <domain>' (or look in the report)
[01:03]	Beetstra	resolve duhvoli.com.ua
[01:03]	COIBot	The URL duhvoli.com.ua resolves to 91.209.206.50 ( http://91.209.206.50 ). See also: [ http://www.ping.eu/ns_whois/?host=duhvoli.com.ua ].
[01:03]	COIBot	Saved an XWiki report for mediamatters.com (XWiki spam by 71.192.160.147 Puffin 173.166.3.157 65.102.186.166 Jettparmer).


[01:09]	Beetstra	!info
[01:09]	COIBot	Beetstra: 2565 reports in 13 days, 03:46:09 hours (0.13/min) (0 normal, 554 xwiki, 1 ip, 0 meta, 144 poked). 30 edits in parserqueue. Last RC 03 seconds ago. Last LW 00 seconds ago. Report when overlap exceeds 25%. Reporting to #wikipedia-spam-t and #cvn-sw-spam (limited) and #wikipedia-en-spam (all); Reportlevels: RC is 0
[01:09]	LiWa3_3	LW: 99 days, 04:00:58 hours active; RC: last 2 sec. ago; Reading approx. 772 wikis; Queues: P1=0; P2=0; P3=0; W=0; A1=0; A2=0; M=0; Total: 56428507 edits (395 PM); 7288858 IP edits (15.5%; 51 PM); Watched: 46939320 (83.1%; 328 PM); Links: 2669113 edits (5.6%; 18 PM); 6120616 total (42 PM; 0.13 per edit; 2.29 per EL add edit); 704739 WL (11.5%; 4 PM); 77693 BL (1.2%; 0 PM); 9485 RL (0.1%; 0 PM); 20260 AL (0.3%; 0 PM)


sDrewth	coibot help

[20:58]	COIBot	USER/DOMAIN/LINK stats: whoadded [domain]; whatadded [user]; whenadded user [user]; whenadded link [domain]; domainsadded [domain or IP]; whatserveradded [user]; whereadded link [domain];
 whereadded user [user]; pageadded user [user]; pageadded link [link]; wherelisted [domain]
[20:58]	COIBot	REPORTING: getrevids user [user]; getrevids link [domain]; getrevids userlink [user] [domain]; report link [domain]; report meta [domain]; prepare [xwiki|swmt|xlinkbot] [domain]; report [xwiki|swmt] [domain] [userlist]|!|!ip|!all|!noip; report user [name]; report page [page]
[20:58]	COIBot	INFO: rights [user], admin/sysop/checkuser/bureaucrat/editor/rollbacker/founder/steward/importer/accountcreator/developer/oversighter [wikitype]:[lang]
[20:58]	COIBot	CASE: start: 'case new link <domain>' or 'case new user <username>', then 'case # evolve' (until stable) or 'case # evolve auto' (autostabilisation).
[20:58]	COIBot	SPECIAL: backparsing: 'backparse <wiki>:<lang> user <username>' and 'backparse <wiki>:<lang> page <pagename>'; 'special info' for parse status; 'use backparse' to make commander use this db, 
'ignore backparse' to ignore ('using backparse' for status), some commander-commands can be preceded with 'special ' to query db specifically


for !info Linkwatcher

LW: 1 day, 20:08:05 hours active; RC: last 5 sec. ago; Reading approx. 772 wikis; Queues: P1=0; P2=0; P3=0; W=0; A1=59263; A2=4278; M=0; Total: 1078071 edits (407 PM); 144643 IP edits (16.1%; 54 PM); Watched: 896035 (83.1%; 338 PM); Links: 64919 edits (7.2%; 24 PM); 128992 total (48 PM; 0.14 per edit; 1.98 per EL add edit); 5 WL (0%; 0 PM); 64 BL (0%; 0 PM); 0 RL (0%; 0 PM); 10 AL (0%; 0 PM)


P1, P2, P3 are the waiting lists for the parser. P1 for mainspace edits on XLinkBot-wikis, P2 for the rest of mainspace-pages, P3 for the rest of pages
[20:57]	W is the warning-queue (for the feed into XLinkBot)
[20:58]	A1 and A2 are the analyser queues, A1 for mainspace, A2 for the rest
[20:59]	and M is for the Messages waiting to be sent


[07:34]	siteinfo pfref.com
[07:35]	siteinfo pfref.com/tiny/GZzot
[07:35]	COIBot	Site pfref.com returns status: 200 OK - location: http://www.pro-football-reference.com/
[07:35]	COIBot	Site: 'pfref.com' on 'pfref.com' ('66.35.36.82') - base: ” on ”; Server: 'nginx/0.8.55'; Client response num: 1. - not a normal redirector - scanning frameset redirectors - scanning: tk, trcx, nettc, cocc
[07:35]	COIBot	Site: pfref.com on pfref.com (66.35.36.82); - no redirects detected.
[07:35]	quickcreate pro-football-reference.com
[07:36]	COIBot	Site pfref.com/tiny/GZzot returns status: 200 OK - location: http://www.pro-football-reference.com/play-index/tiny.cgi?id=GZzot
[07:36]	COIBot	Site: 'pfref.com/tiny/GZzot' on 'pfref.com' ('0') - base: ” on ”; Server: 'nginx/0.8.55'; Client response num: 1. - not a normal redirector - scanning frameset redirectors - scanning: tk, trcx, nettc, cocc
[07:36]	COIBot	Site: pfref.com/tiny/GZzot on pfref.com (0); - no redirects detected.
You can quickly add things to XLinkBot by typing "revertlist <domain> <reason>" (reason be a diff, a special:contributions or a local report, e.g.) 'revertlist <domain>' works as well, if there are link additions in the db, it will then list all the users that added the link.  But I prefer to make sure that I do not include regulars in those, COIBot adds ALL users it finds

[22:46] <+Beetstra> revertlist semseoguris.com see Special:Contributions/182.68.170.16
[22:46] <+COIBot> -> Revertlisted and logged addition of semseoguris.com (\bsemseoguris\.com\b) to User:COIBot/RevertList.css and User:COIBot/RevertListLog

COIBot should now listen to 'clear <domain>' .. removes the domain from the autoredlist and adds it to the whitelist


[01:07] <+COIBot> sDrewth (wikisource/billinghurst) is on trusted user list(*); commands: status; bl & wl search [user] [string]; ml search [regex]; channels; test [user] [string]; report user [name]; report #; *quit; *limit {0-100}; *bl & *wl add/remove [user] [string]; *ml add/remove [regex]; *reportlevel {0-4}; *report every #; *deteriorate; *lookbacktime; *join (rc/lw)channel [channel]; *part (rc/lw)channel [channel]; *
[01:07] <+COIBot> USER/DOMAIN/LINK stats: whoadded [domain]; whatadded [user]; whenadded user [user]; whenadded link [domain]; domainsadded [domain or IP]; whatserveradded [user]; whereadded link [domain]; whereadded user [user]; pageadded user [user]; pageadded link [link]; wherelisted [domain]
[01:07] <+COIBot> REPORTING: getrevids user [user]; getrevids link [domain]; getrevids userlink [user] [domain]; report link [domain]; report meta [domain]; prepare [xwiki|swmt|xlinkbot] [domain]; report [xwiki|swmt] [domain] [userlist]|!|!ip|!all|!noip; report user [name]; report page [page]
[01:07] <+COIBot> INFO: rights [user], admin/sysop/checkuser/bureaucrat/editor/rollbacker/founder/steward/importer/accountcreator/developer/oversighter [wikitype]:[lang]
[01:08] <+COIBot> CASE: start: 'case new link <domain>' or 'case new user <username>', then 'case # evolve' (until stable) or 'case # evolve auto' (autostabilisation).
[01:08] <+COIBot> SPECIAL: backparsing: 'backparse <wiki>:<lang> user <username>' and 'backparse <wiki>:<lang> page <pagename>'; 'special info' for parse status; 'use backparse' to make commander use this db, 'ignore backparse' to ignore ('using backparse' for status), some commander-commands can be preceded with 'special ' to query db specifically

[01:08] <+Beetstra> for LiWa, the lists are 'wl', 'rl', 'arl', 'nr' - all regexes
[01:08] <+sDrewth> but monitor is different from the syntax that you explained the other dayh
[01:08] <+Beetstra> 'bl' still exists in LiWa, but that has been replaced by the on-wiki revertlists after the bot request
[01:09] <+Beetstra> coibot has two monitor lists, one domain-based, and one regex based
[01:09] <+Beetstra> if it can go on the domain-based one, that is preferred (much faster)
[01:09] <+sDrewth> yes, but syntax for removal or searching?
[01:10] <+Beetstra> you have 'monitor regex <regex>' and 'monitor domain <domain>' to add ..and .. 
[01:10] <+Beetstra> eh
[01:10] <+Beetstra> ah:     if ($checkmessage =~ m/^monitor remove domain (.+)/ || $checkmessage =~ m/^monitor del domain (.+)/ || $checkmessage =~ m/^monitor delete domain (.+)/) {
[01:11] <+Beetstra> and
[01:11] <+Beetstra> ->     if ($checkmessage =~ m/^monitor remove regex (.+)/ || $checkmessage =~ m/^monitor del regex (.+)/ || $checkmessage =~ m/^monitor delete regex (.+)/) {


[01:13] <+LiWa3_1> Format of report line: [[lang:Pagename]] difflink [[lang:User:Username]] links; per link = added link (BL/RL/WL/AL/ARL) (1,2,3,4); 1 = linkcount for this user; 2 = linkcount for this domain; 3 = count this domain by this user; 4 = count wikis by this user for this domain.
[01:13] <+Beetstra> that one?  is hardcoded ...
[01:13] <+LiWa3_1> Format of warning line: WARNING: <reason> <domain> on <wiki/crosswiki> [[en:Special:Linksearch/*.<domain>]] (L: 21 (#links); U: 30 (#this last user); UL: 3 (#last user this link); ULW: 1 (#wikis this user this link); Users: 6 (#unique users); IPs 21 (#ip editors) (6 (#unique IPs); 6 (); 1 (#in range of last IP); <already saved>) Users: <list of users>.

[00:20] <+Beetstra> case 9 add user Beetstra
[00:20] <+COIBot> Case 9 - Added user 'Beetstra'.
[00:20] <+Beetstra> case 9 expand links
[00:20] <+COIBot> Links for case 9 now chardhamtours.in|pep.ph|abington.patch.com|2010.census.gov|forbes.com|facutinromania.wordpress.com.
[00:20] <+Beetstra> case 9 expand users
[00:20] <+COIBot> Users for case 9 now Beetstra|Beetstra|Eaglestorm|112.205.240.78|28bytes|Xqbot|3YTs|JmKissme|Beetstra|Beetstra|Eaglestorm|112.205.240.78|28bytes|Xqbot|3YTs|JmKissme|Beetstra|Beetstra|Eaglestorm|112.205.240.78|28bytes|Xqbot|3YTs|JmKissme|Beetstra|Beetstra|Eaglestorm|112.205.240.78|28bytes|Xqbot|3YTs|JmKissme|Gary Dee|Shennova|Ravichandar84|117.227.19.90|Verdulo|Rhatsa26X|Beetstra|Beetstra|Eaglestorm|112.205
[00:21] <+Beetstra> case 9 remove whitelisted users
[00:21] <+COIBot> Case 9 - Removed whitelisted users.
[00:21] <+Beetstra> case 9 show users
[00:21] <+COIBot> Case 9 - parameter users not set.
[00:21] <+Beetstra> eh
[00:21] <+Beetstra> case 9 params
[00:21] <+COIBot> Case 9 - parameters (6 items): owner, comment, maxlinkadditions, maxuseradditions, users, links.
[00:21] <+Beetstra> it is 
[00:21] <+Beetstra> case 9 show users
[00:21] <+COIBot> Case 9 - parameter users not set.
[00:22] <+Beetstra> case 9 remove whitelisted links
[00:22] <+COIBot> Case 9 - Removed whitelisted links.
[00:22] <+Beetstra> that does not make sense ..
[00:25] <+Beetstra> case restart
[00:33] <+sDrewth> prepare xwiki imarketingsolutions.info
[00:33] <+COIBot> -> report xwiki imarketingsolutions.info Burchkeith
[00:33] <+sDrewth> report xwiki imarketingsolutions.info Burchkeith
[00:33] <+COIBot> Statistics report for link imarketingsolutions.info by Burchkeith queued; see [[m:User:COIBot/LinkReports/imarketingsolutions.info]] & [[:en:Wikipedia:WikiProject Spam/LinkReports/imarketingsolutions.info]] (auto:  - /).

[00:35] <+Beetstra> case 10 remove users
[00:36] <+Beetstra> case 10 remove user Beetstra
[00:36] <+Beetstra> case 10 show users
[00:36] <+COIBot> Case 10 - parameter 'users' (76 items): Beetstra, Beetstra, Eaglestorm, 112.205.240.78, 28bytes, Xqbot, 3YTs, JmKissme, Beetstra, Beetstra, Eaglestorm, 112.205.240.78, 28bytes, Xqbot, 3YTs, JmKissme, Beetstra, Beetstra, Eaglestorm, 112.205.240.78, 28bytes, Xqbot, 3YTs, JmKissme, Beetstra, Beetstra, Eaglestorm, 112.205.240.78, 28bytes, Xqbot, 3YTs, JmKissme, Gary Dee, Shennova, Ravichandar84, 117.227.19.90,

case 10 remove whitelisted users
case 10 file  (this then saves the case to [[User:COIBot/Case/caseXX]]
file file file file file

[00:52] <+Beetstra> case new
[00:52] <+COIBot> I have created case #2 for you ('').
[00:53] <+Beetstra> case add link szqdyy.net
[00:53] <+Beetstra> case 2 add link szqdyy.net
[00:53] <+COIBot> Case 2 - Added link szqdyy.net.
[00:53] <+Beetstra> case 2 expand users
[00:53] <+COIBot> Users for case 2 now Xujie128.
[00:53] <+Beetstra> case 2 expand links
[00:53] <+COIBot> Links for case 2 now szqdyy.net|Centre%20de%20l'autisme%20et%20le%20développement.
[00:53] <+Beetstra> case 2 expand links
[00:53] <+COIBot> Links for case 2 now szqdyy.net|Centre%20de%20l'autisme%20et%20le%20développement.
[00:53] <+Beetstra> nah
[00:53] <+Beetstra> case 2 expand users
[00:53] <+COIBot> Users for case 2 now Xujie128.
[00:53] <+Beetstra> whatadded Xujie128
[00:53] <+COIBot> 7 records; Domains added by xujie128: szqdyy.net (6), centre%20de%20l (1).
[00:54] <+Beetstra> ah
[00:54] <+Beetstra> xujie128 added a f*cked up link
[00:54] <+Beetstra> heh
[00:56] <+Beetstra> case new
[00:56] <+COIBot> I have created case #3 for you ('').
[00:56] <+Beetstra> case 3 add user Billinghurst
[00:56] <+COIBot> Case 3 - Added user 'Billinghurst'.
[00:56] <+Beetstra> case 3 expand links
[00:56] <+COIBot> Links for case 3 now .

[18:46] <+Beetstra> 1) add it to the revertlist: https://en.wikipedia.org/w/index.php?title=User:XLinkBot/RevertList&diff=prev&oldid=474845259
[18:47] <+Beetstra> 2) add it to the OverRide list: https://en.wikipedia.org/w/index.php?title=User:XLinkBot/OverrideList&diff=prev&oldid=474845402
[18:47] <+Beetstra> 3) add it to the leveloverrule list: https://en.wikipedia.org/w/index.php?title=User:XLinkBot/LevelOverrule&diff=prev&oldid=474845372
[18:48] <+Beetstra> 4) if there is need, do a 'restart linkwarner' for LiWa3, so that it resets the blacklists, and if spammers come back in the next minute they will be reverted

[23:26] <+MER-C> whoadded cellularfactory.com
[23:26] <+COIBot> 23 records; Editors who have added cellularfactory.com: Ali raza00000 (14), 59.103.211.75 (6), 59.103.196.242 (2), Madnanbashir1 (1).
[23:43] <+MER-C> getrevids user Ali raza00000
[23:43] <+COIBot> revids w:en:472780691 w:en:473712774 w:en:473716770 w:en:473806007 w:en:475088583 w:en:475093191 w:en:475101577 w:en:475102966 w:en:475109217 w:en:475112924
[23:43] <+COIBot> revids w:en:475114206
[23:43] <+COIBot> 11 records shown.
[23:43] <+MER-C> revids w:en:472780691 w:en:473712774 w:en:473716770 w:en:473806007 w:en:475088583 w:en:475093191 w:en:475101577 w:en:475102966 w:en:475109217 w:en:475112924
... revision id output followed ...
 <+COIBot> 10 records shown.


[19:35] <+Beetstra> adsense far-cry-3.com
[19:35] <+COIBot> Detected for http://far-cry-3.com: No Adsense detected, No Google Web Bug, Clickbank IDs: cwillrath.

  • protected service

[22:53] * +Beetstra considers to explain sDrewth about 'coibot join channel <#channel>' and 'coibot part channel <#channel>

[21:55] <+Beetstra> !!create categories en.wikisource.org
[21:55] <+COIBot> Category [[Category:COIBot Local Reports for en.wikisource.org]] created.
[21:55] <+COIBot> Category [[Category:Open Local reports for en.wikisource.org]] created.
[21:55] <+COIBot> Category [[Category:Closed Local reports for en.wikisource.org]] created.
[21:55] <+COIBot> Category [[Category:Ignored Local reports for en.wikisource.org]] created.
[21:55] <+COIBot> Category [[Category:Stale Local reports for en.wikisource.org]] created.
[21:55] <+COIBot> Category [[Category:Local to XWiki reports for en.wikisource.org]] created.


[00:40] <+Beetstra> revertlist resolve 79.99.0.15 pump.pp4l.me type spam
[00:40] <+COIBot> -> Revertlisted and logged addition of resolve 79.99.0.15 (resolve 79.99.0.15) to User:COIBot/RevertList.css and User:COIBot/RevertListLog
[00:40] <+Beetstra> overrule resolve 79.99.0.15 6
[00:40] <+COIBot> Rule resolve 79.99.0.15 now overruled to level 6.
[00:41] <+Beetstra> overrule resolve 79.99.0.15 7
[00:41] <+COIBot> Rule resolve 79.99.0.15 now overruled to level 7.
[00:42] <+Beetstra> Do not go through start, you will not receive



[00:35] <+Beetstra> set teststring blah dea.thtoje.ws blah de.athtoje.ws blah
[00:35] <+COIBot> Teststring set to 'blah dea.thtoje.ws blah de.athtoje.ws blah'.
[00:35] <+LiWa3_1> Setting 'teststring blah dea.thtoje.ws blah de.athtoje.ws' set to 'blah'
[00:35] <+Beetstra> testregex d[\.|]e[\.|]a[\.|]t[\.|]h[\.|]t[\.|]o[\.|]j[\.|]e\.ws\b
[00:35] <+COIBot> Your regex 'd[\.|]e[\.|]a[\.|]t[\.|]h[\.|]t[\.|]o[\.|]j[\.|]e\.ws\b' did not match anything in 'blah dea.thtoje.ws blah de.athtoje.ws blah'.
[00:36] <+Beetstra> testregex d\.+e
[00:36] <+COIBot> Your regex 'd\.+e' did not match anything in 'blah dea.thtoje.ws blah de.athtoje.ws blah'.
[00:36] <+Beetstra> testregex d\.{0,1}e
[00:36] <+COIBot> Your regex 'd\.{0,1}e' matched ('de', 'de')
[00:36] <+Beetstra> testregex d\.{0,1}e\.{0,1}a\.{0,1}t\.{0,1}h\.{0,1}t\.{0,1}o\.{0,1}j\.{0,1}e\.ws
[00:36] <+COIBot> Your regex 'd\.{0,1}e\.{0,1}a\.{0,1}t\.{0,1}h\.{0,1}t\.{0,1}o\.{0,1}j\.{0,1}e\.ws' matched ('dea.thtoje.ws', 'de.athtoje.ws')


[02:25] <+Beetstra> read blacklist en.wikipedia.org
[02:25] <+COIBot> -> Blacklist from en.wikipedia.org read.
[02:28] <+Beetstra> When LinkSaver is idling, it takes a random wiki, and updates black/white/revertlists and saves them on disk .. Commander and all use that file to determine e.g. what is blacklisted for the 'wherelisted' command
[02:30] <+Beetstra> read revertlist en.wikipedia.org
[02:30] <+COIBot> -> Revertlist from en.wikipedia.org read.
[02:30] <+Beetstra> read whitelist en.wikipedia.org
[02:30] <+COIBot> -> Whitelist from en.wikipedia.org read.


[22:56] <+Beetstra> keyword adsense followed by a space and then the full id
[22:56] <+Beetstra> so here '3845661189639054
[22:57] <+Beetstra> so 'resolve 3845661189639054'
[22:54] <+MER-C> So how do I revertlist google analytics ids again?

[22:51] <sDrewth> revertlist resolve 188.165.237.46 # habbolatino.us keeps creating redirects to site

[00:55] <+Beetstra> top links
[00:56] <+COIBot> 0: Link: provinz.bz.it - count: 005166
[00:56] <+COIBot> 1: Link: tvbythenumbers.zap2it.com - count: 005263
[00:56] <+COIBot> 2: Link: dispatch.opac.d-nb.de - count: 006717
[00:56] <+COIBot> 3: Link: census.adrianfrith.com - count: 007994
[00:56] <+COIBot> 4: Link: viaf.org - count: 009005
[00:56] <+COIBot> Top 5 links reported.
[00:56] <+Beetstra> the 'likely not spam stuff' .. it would speed up LiWa3 if the domains are appropriately dontcounted or whitelisted ..
[00:58] <+Beetstra> do not count youtube.com
[00:59] <+Beetstra> Stuff you want to see, but for which counting is useless
[00:59] <+Beetstra> And just time consuming
[01:00] <sDrewth> basically stuff where I think that it is "leave to communities"
[01:00] <+Beetstra> facebook / myspace / youtube -> useless to count, many different users
[01:01] <sDrewth> >> 100 ?
[01:01] <+Beetstra> Yes, but stuff that you don't want to hide
[01:01] <sDrewth> okay, whitelist is whitelist, and anything that I say "Hmm" let the community decide but there is significant listing
[01:02] <sDrewth> restricted command?
[01:02] <+Beetstra> maybe more: added x times by ~x different users, x being > 50, or added y times, y > 250 .. ??
[01:02] <+Beetstra> Yes
[01:03] <sDrewth> dontcount = significant no. users, significant count
[01:03] <+Beetstra> Yep
[01:03] <+Beetstra> that command:
[01:03] <+Beetstra> top links
[01:03] <+COIBot> 0: Link: provinz.bz.it - count: 005166
[01:03] <+COIBot> 1: Link: tvbythenumbers.zap2it.com - count: 005263
[01:03] <+COIBot> 2: Link: dispatch.opac.d-nb.de - count: 006722
[01:03] <+COIBot> 3: Link: census.adrianfrith.com - count: 007994
[01:03] <+COIBot> 4: Link: viaf.org - count: 009005
[01:03] <+COIBot> Top 5 links reported.
[01:03] <+Beetstra> starts at about 5000 or so .. that is the top of the top
[01:04] <+Beetstra> With those, 'count links for users' is becoming work for the MySQL
[01:05] <+Beetstra> count links is milliseconds, count user is milliseconds, count how many users added link .. may become more .. and often that number does not tell anything anymore



[19:12] <Beetstra> whatadded 147.156.51.116
[19:12] <COIBot> 3 records; Domains added by 147.156.51.116: tortugasdemalditanerea.blogspot.com.es (2), títuhttp (1).
[19:12] <Beetstra> hmm
[19:12] <Beetstra> whereadded user 147.156.51.116
[19:12] <COIBot> 3 records; Wikis where user:147.156.51.116 added links: w:es (3).
[19:12] <Beetstra> getrevids user 147.156.51.116
[19:12] <COIBot> (1) -> 2012-12-03 07:53:10 es:Maldita Nerea http://es.wikipedia.org/w/index.php?diff=61911378&oldid=61888772 es:User:147.156.51.116 http://TÃtuhttp://tortugasdemalditanerea.blogspot.com.es/lo%20de%20la%20página%20o%20URL (0)
[19:12] <COIBot> (2) -> 2012-12-03 07:54:25 es:Maldita Nerea http://es.wikipedia.org/w/index.php?diff=61911385&oldid=61911378 es:User:147.156.51.116 http://http://tortugasdemalditanerea.blogspot.com.es/lo%20de%20la%20página%20o%20URL (74.125.225.107)
[19:12] <COIBot> (3) -> 2012-12-03 07:57:14 es:Maldita Nerea http://es.wikipedia.org/w/index.php?diff=61911406&oldid=61911398 es:User:147.156.51.116 http://www.tortugasdemalditanerea.blogspot.com.es (74.125.225.106)


pageadded domain

note. Note suere whether it just reports what is on enwiki or broader. No domain mentioned against output of page names.


test of Stopforumspam[edit]

<Beetstra> forumspammer TresaPoike
<COIBot> user 'TresaPoike' is listed. Last seen on 2012-02-26 08:57:27, frequency 1.


syslog and restart[edit]

syslog commander
syslog parser
syslog linksaver
syslog case
NOT syslog coibot

restart ...

commander (force new settings)
linkreporter (for lost link watchers)
linkparser
linkanalyser
diffreader

example of output from linksaver[edit]

[

18:35] <COIBot> Line 5: * 2013-1-15 @ 04:56:42 - start generating report for ses.com (2, Local Regenerate)
[18:35] <COIBot> Reported 5 items @ 2013-1-15 07:35:11 (9509 seconds ago) from syslog of linksaver.

(dumb questions followed, analysis...)
[18:45] <Beetstra> I saw that it is a couple of hours ago that COIBot saved something on-wiki
[18:45] <Beetstra> So I asked for the last 5 lines of the syslog
[18:45] <Beetstra> Last 2 lines of that, it is busy with saving ses.com, and it is doing that for 9509 second .. which is .. eh .. 2.5 hours?
[18:46] <Beetstra> That is too long .. so there is something that makes the linksaver hang .. likely too much work to parse it all
[18:46] <Beetstra> (though it might be that it is really hanging while waiting for information from the site or something like that)
[18:47] <Beetstra> so I decide to restart the linksaver, which means that 'ses.com' does not get saved .. so I manually check a bit
...
[18:35] <Beetstra> count ses.com
[18:35] <COIBot> COI: there are 0 records on link 'ses.com'; Linkwatcher: there are 273 additions of 'ses.com' and 290 additions of '*.ses.com'.
[18:36] <Beetstra> whoadded ses.com
[18:36] <COIBot> 290 records; Top 10 editors who have added ses.com: Satbuff (177), Lisa.raatikainen (14), Asdert (11), Sirius4 (11), Alphathon (10), WebCite Archiver (9), Dainis (8), C0re1980 (6), Rjh (5), Anyakaz (3).
[18:36] <Beetstra> whatadded satbuff
[18:36] <COIBot> 228 records; Top 10 domains added by satbuff: ses.com (177), lyngsat.com (11), sesbroadband.com (4), broadbandtvnews.com (4), advanced-television.com (4), france24.com (4), astrium.eads.net (4), satellites.co.uk (4), satelliteinternet.co.uk (3), onlinereporter.com (2).
[18:53] <Beetstra> if the linksaver is silent for > 1 hour it is worth checking what it does (it may really have nothing to do and parse blacklists etc., but that will be in the syslog .. if it is busy saving a link for >1 hour .. consider a restart)


Error output; [19:27] <LiWa3_2> Auto-restarted linkanalysers - seemed to be hanging (killed: ).

solved by add linkanalyser


Shoving linkwatchers

LiWa3_1 !quit

'show processes' shows which modules are running with their PID


Typical linksaver cycle[edit]

start process
  1. 2013-2-02 @ 07:14:35 - Server input - SAVE [hmr-brandambassadors.in] [Local spam on en.wikipedia.org by HMRBA] [1300078]. - took 0 seconds
<Beetstra> So the sequence in the log is a server input 'SAVE [<domain> ...'
  1. 2013-2-02 @ 07:14:35 - start generating report for hmr-brandambassadors.in (2, Local spam on en.wikipedia.org by HMRBA) - took 6 seconds
<Beetstra> a mention that it starts generating the report
  1. 2013-2-02 @ 07:14:41 - All sorted for hmr-brandambassadors.in - took 0 seconds
<Beetstra> an 'all sorted' remark
  1. 2013-2-02 @ 07:14:41 - generating hmr-brandambassadors.in for meta.wikimedia.org - took 1 seconds
  2. 2013-2-02 @ 07:14:42 - Selected users for hmr-brandambassadors.in are: HMRBA - took 22 seconds
  3. 2013-2-02 @ 07:15:04 - Saving report User:COIBot/LinkReports / hmr-brandambassadors.in on meta.wikimedia.org - took 10 seconds
  4. 2013-2-02 @ 07:15:14 - Saving report User:COIBot/Local / hmr-brandambassadors.in on meta.wikimedia.org - took 10 seconds
<Beetstra> then a handful of links for what it saves per wiki (generating ... for meta.wikimedia.org', selecting users, saving the linkreport, saving the local report)
  1. 2013-2-02 @ 07:15:24 - Parsing and saving User:COIBot/XWiki - 0 - 1359789324 - 1359789149. - took 1 seconds
  2. 2013-2-02 @ 07:15:25 - Parsing and saving User:COIBot/XWiki - Escaping: just did this in the last 1800 seconds(1359789325 - 1359789149). - took 0 seconds
<Beetstra> then if needed what it does with the XWiki table page
  1. 2013-2-02 @ 07:15:25 - Client FINISHED - SAVE [hmr-brandambassadors.in] [Local spam on en.wikipedia.org by HMRBA] [1300078] - sending 'SAVED [hmr-brandambassadors.in] [Local spam on en.wikipedia.org by HMRBA]' (1359789325 -> 1359789275 = 50). - took 0 seconds
  2. 2013-2-02 @ 07:15:25 - Server input - THANKS. - took 0 seconds
  3. 2013-2-02 @ 07:15:25 - Client finished - THANKS - sending 'REQUEST 1300079'. - took 0 seconds
  4. 2013-2-02 @ 07:15:25 - Server input - THANKSMESSAGE. - took 0 seconds
  5. 2013-2-02 @ 07:15:25 - Server input - THANKSMESSAGE. - took 0 seconds
  6. 2013-2-02 @ 07:15:25 - Server input - THANKSMESSAGE. - took 0 seconds
<Beetstra> And then a 'FINISHED' line - and a couple of communication lines
next process
  • 2013-2-02 @ 07:15:25 - Server input - SAVE [aigcorporate.com] [Redirect addition by Writejustified] [1300079]. - took 0 seconds

  • [15:02] <huh> When LiWa says (7, 294, 3, 1)
    [15:02] <huh> what do the numbers mean?
    [15:03] <huh> (2147, 832)
    [15:03] <Beetstra> 'this user added 7 links, this domain has been added 294 times, this user added this domain 3 times, and this user added this domain to 1 wiki'
    [15:04] <huh> thanks
    [15:04] <Beetstra> 'this user added 2147 links, this domain was added 832 times - I can't be bothered to count how often this user added this link, as it was already added quite often/the user added already many links'

    This is described in detail on LiWa (sorry for not checking there first). PiRSquared17 (talk) 05:06, 17 February 2014 (UTC)