Community Wishlist Survey 2022/Citations/More capacity for Citation bot

Random proposal ►◄ Citations The survey has concluded. Here are the results!

More capacity for Citation bot

Problem: Citation bot is very powerful, but needs hugely expanded capacity to meet demand and stop requests timing out.
Proposed solution: 1/ A new front end queuing system for Citation bot, similar to that used by InternetArchiveBot, which allows much bigger jobs and gives priority to single-page requests; 2/ Many more threads than the current limit of four; 3/ enhanced capacity for the Zotero servers on which Citation bot relies for filling bare URLs
Who would benefit: Editors trying to improve references, especially complex refs such as those to scholarly journals, whether on single articles or in batch.
More comments: This has been repeatedly discussed at User talk:Citation bot, but the excellent maintainer faces hurdles which he cannot surmount: see e.g. https://en.wikipedia.org/w/index.php?title=User_talk:Citation_bot&oldid=1064839514#And_failure_is_the_usual_option_again
Phabricator tickets:
Proposer: BrownHairedGirl (talk) 18:42, 10 January 2022 (UTC)[reply]

Discussion

I especially like the idea of having a queuing function so (assuming we cannot find additional compute capacity) at least it won't time out all the time. --Mblumber (talk) 01:13, 11 January 2022 (UTC)[reply]
For IABot, the main non-interactive bot runs on a VPS see Cloud VPS which anyone can request to have. IABot also uses Toolforge for interactive user batch requests. Citation bot could request a VPS (or multiple VPS) running multiple instances on the same VPS and/or multiple instances on the grid (Toolforge) ie. simply increase the Toolforge slot quota. However, I think the main bottleneck for Citation bot is Citoid/Zotero. -- GreenC (talk) 05:18, 11 January 2022 (UTC)[reply]
I concur with this proposal. -- TheInternetGnome (talk) 10:09, 11 January 2022 (UTC)[reply]
Resources are certainly needed in this area. For an alternative approach to getting people working on this, see also w:WP:VPT#Proposed Google Summer of Code project: expanding citations. Enterprisey (talk) 08:09, 12 January 2022 (UTC)[reply]
What would be good, is if the database consulting services of citoid and citationbot could be shared by both tools. Probably not that easy to do however, i suspect they are in different languages. —TheDJ (talk • contribs) 11:06, 12 January 2022 (UTC)[reply]
This. Enterprisey (talk) 09:55, 13 January 2022 (UTC)[reply]

Citationbot already pulls from Citoid results I believe. Izno (talk) 02:36, 17 January 2022 (UTC)[reply]
I think we should make Citation bot a global feature and we can translate the interface in many languages. Thingofme (talk) 12:23, 12 January 2022 (UTC)[reply]
It would certainly be nice to have more Citation bot capacity. Qwerfjkl (talk) 18:20, 26 January 2022 (UTC)[reply]

Voting

Support * Pppery * _{it has begun} 18:36, 28 January 2022 (UTC)[reply]
Support Congestion with this bot is so bad that editors are not creating articles out of frustration. If Citation bot is not given extra capacity, I predict that the legitimacy of this Community Wishlist Survey itself will be damaged. Abductive (talk) 18:46, 28 January 2022 (UTC)[reply]
Support Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 19:15, 28 January 2022 (UTC)[reply]
Support Qwerfjkl (talk) 21:56, 28 January 2022 (UTC)[reply]
Support —Bruce1ee^talk 08:47, 29 January 2022 (UTC)[reply]
Support Šedý (talk) 09:50, 29 January 2022 (UTC)[reply]
Support Lion-hearted85 (talk) 11:04, 29 January 2022 (UTC)[reply]
Support Terber (talk) 11:41, 29 January 2022 (UTC)[reply]
Support Aca (talk) 12:42, 29 January 2022 (UTC)[reply]
Support ACortellari (talk) 14:06, 29 January 2022 (UTC)[reply]
Support BSMIsEditing (talk) 15:04, 29 January 2022 (UTC)[reply]
Support Mbrickn (talk) 15:35, 29 January 2022 (UTC)[reply]
Support--Mahmud (talk) 19:42, 29 January 2022 (UTC)[reply]
Support TheInternetGnome (talk) 07:20, 30 January 2022 (UTC)[reply]
Support Lectrician1 (talk) 07:40, 30 January 2022 (UTC)[reply]
Support Thingofme (talk) 13:52, 30 January 2022 (UTC)[reply]
Support DGG (talk) 19:57, 7 February 2022 (UTC)[reply]
Support JPxG (talk) 00:37, 31 January 2022 (UTC)[reply]
Support Lfstevens (talk) 06:38, 31 January 2022 (UTC)[reply]
Support Qazwsx777 (talk) 09:37, 31 January 2022 (UTC)[reply]
Support Citation bot is great when it works, but can take forever to run e.g. on pages with hundreds of papers cited as references. If it breaks, you just have to run it again and hope it works this time. Parallel requests to data sources would help. Modest Genius (talk) 20:21, 31 January 2022 (UTC)[reply]
Support 15 (talk) 17:36, 1 February 2022 (UTC)[reply]
Support Stratocaster47 (talk) 12:24, 2 February 2022 (UTC)[reply]
Support. Alexcalamaro (talk) 19:15, 2 February 2022 (UTC)[reply]
Support WatkynBassett (talk) 20:45, 3 February 2022 (UTC)[reply]
Support - Darwin ^Ahoy! 20:50, 4 February 2022 (UTC)[reply]
Support paul2520 (talk) 02:24, 5 February 2022 (UTC)[reply]
Support I often use citation bot, and find it annoyingly slow at times and often times out for no apparent reason. I really don't know how BHG puts up with it, she's almost certainly the single largest user of it currently. Mako001 (talk) 03:40, 5 February 2022 (UTC)[reply]
Support —— Eric Liu_（Talk） 05:10, 5 February 2022 (UTC)[reply]
Support I have often seen the edits that Citation bot makes, but I have never been successful in getting it to handle a single page. Just today I have attempted to have it process one particular page about ten times over a period of six hours, but each simply hung until it timed out. There is a huge capacity problem here, apparently. --R. S. Shaw (talk) 05:27, 5 February 2022 (UTC)[reply]
Support Sir Proxima Centauri (talk) 10:36, 5 February 2022 (UTC)[reply]
Support — DaxServer (t · c) 18:24, 5 February 2022 (UTC)[reply]
Support Ayumu Ozaki (talk) 05:29, 6 February 2022 (UTC)[reply]
Support Redalert2fan (talk) 14:41, 6 February 2022 (UTC)[reply]
Support Tango Mike Bravo (talk) 18:40, 6 February 2022 (UTC)[reply]
Support — Eric0892 02:22, 7 February 2022 (UTC)[reply]
Support — Bilorv (talk) 21:57, 7 February 2022 (UTC)[reply]
Support Worldbruce (talk) 15:36, 8 February 2022 (UTC)[reply]
Support KnowledgeablePersona (talk) 23:28, 8 February 2022 (UTC)[reply]
Support Marcok (talk) 07:22, 10 February 2022 (UTC)[reply]
Support Nosebagbear (talk) 21:25, 10 February 2022 (UTC)[reply]
Support Jonathan5566^(talk) 14:43, 11 February 2022 (UTC)[reply]
Support DSparrow14 (talk) 16:55, 11 February 2022 (UTC)[reply]