Talk:Wiki labels

From Meta, a Wikimedia project coordination wiki
Jump to: navigation, search

Work log

[ refresh list ]


Welcome![edit]

Please feel free to post questions/ideas and other discussion about the Wiki labels system here. --EpochFail (talk) 18:02, 2 May 2015 (UTC)

"Wiki artifacts"[edit]

I believe people will not understand what is this. We should find a better word for describing this. Helder 01:28, 6 May 2015 (UTC)

I agree. "items", "elements", "components" could be considered. Cheers. --Stultiwikiatext me 22:03, 13 May 2015 (UTC)

Translation[edit]

https://fr.wikipedia.org/wiki/Utilisateur:Paannd_a/Brouillon --Paannd a (talk) 17:21, 24 May 2015 (UTC)

Translate tags[edit]

@Brackenheim: any idea why the content of Wiki labels/machine paths is appearing at Wiki labels#Machine-readable paths with raw <translate> tags? (it appears correctly at Wiki labels/pt#Machine-readable paths) Helder 04:45, 8 June 2015 (UTC)

@Brackenheim: ^. Helder 04:45, 8 June 2015 (UTC)
Hello Helder, unfortunately I don't know, why this doesn't work. It seems, that Wiki_labels/machine_paths/en is missing... Maybe you can find help here. Kind regards --Brackenheim (talk) 18:59, 8 June 2015 (UTC)
User:Steinsplitter fixed it right now. --Brackenheim (talk) 19:09, 8 June 2015 (UTC)

Edit quality campaign for ukwiki[edit]

Hy! How about turn on Edit quality campaign for w:uk:Вікіпедія:Оцінки? Or maybe we need some additional translation or config? You can find me at #wikipedia-ukconnect. --Yakiv Gluck (talk) 22:31, 3 October 2015 (UTC)

ping @EpochFail:. :3 --Yakiv Gluck (talk) 22:33, 3 October 2015 (UTC)
Yakiv Gluck Sorry. Saw the ping. We're working on it. Will ping when it is ready. White Cat might have a progress update to post. I'm waiting on him to run the sample through a few "prelabeling" scripts so that we can minimize the workload. --EpochFail (talk) 14:45, 4 October 2015 (UTC)
ok, no problem. Thanks for the answer. --Yakiv Gluck (talk) 15:31, 4 October 2015 (UTC)

Hi. A similar request about ruwp. Need turn Edit quality campaign. Localization was done (and 1, 2). @とある白い猫:--Iluvatar (talk) 20:09, 26 October 2015 (UTC)

"can start the campaign as soon as we have the two pages translated" - seriously?--Iluvatar (talk) 23:58, 10 November 2015 (UTC)
Hey Iluvatar, I'm not sure what the status is here, but I can review the notes we have in the meantime. It looks like we're nearly ready to go with running the campaign for Ukranian Wikipedia (see Phab:T114502). All of the components are blocked on me getting time to (1) merge some localization bits into our codebase for WikiLabels and (2) load the labeling data into the labeling databases. I suspect that I can get to this done today. I'll ping you here with an update at the end of my work day either way. --EpochFail (talk) 14:25, 11 November 2015 (UTC)
Hi Iluvatar, I've gotten the software updated to include the ukwiki translations. I'm now running our "prelabeling" script that filters the 20k workset down to something much more reasonable for evaluation within Wiki labels (mostly removing bot and sysop edits from the set). Once that is finished, I'll load up the remaining revisions into the labeling system. That'll be the last step before you can start work. I think i can get to in my morning (UTC-6) tomorrow. I'll ping again to let you know what's up. --EpochFail (talk) 22:13, 11 November 2015 (UTC)
Iluvatar & Yakiv Gluck I think we're good to go. See uk:Вікіпедія:Оцінки and ru:Википедия:Оценка. I used google translate to name the campaign. It should be a translation of "Edit quality (20k random sample, 2015)". If you've got some text you'd like me to place there, let me know and I'll make a quick update. --EpochFail (talk) 18:15, 12 November 2015 (UTC)
Thank EpochFail. ru: "Качество правок (20-тыс. случайная выборка, 2015)".--Iluvatar (talk) 05:37, 13 November 2015 (UTC)
EpochFail, oh so that's why the name sucks so much :) uk would be something like «Якість редагувань (вибірка випадкових 20 тис., 2015)». Perhaps Yakiv would think on better wording though. --Base (talk) 21:36, 13 November 2015 (UTC)
Yes check.svg Done for both. Thanks! --EpochFail (talk) 23:25, 13 November 2015 (UTC)

Wiki labels is translated into Norwegian[edit]

@EpochFail: I have translated the necessary pages for Norwegian (bokmål) Wikipedia at w:no:Wikipedia:Etiketter. I guess edit quality should be the first campaign? The English page is translated at w:no:Wikipedia:Etiketter/Redigeringskvalitet. We have not done any formal voting over whether we should join the project, there is a thread at our bazar w:no:Wikipedia:Torget#Wiki labels, but I guess the worse thing that can happen is that no-one shows up to the party! — Jeblad 15:48, 7 February 2016 (UTC)

Hey Jeblad! Thanks for getting that worked out. I made some notes on Phab:T126161 for what the UI translations we need to deploy there. Once you have those completed, I can take care of the rest. --EpochFail (talk) 17:10, 7 February 2016 (UTC)
Done. — Jeblad 17:49, 7 February 2016 (UTC)
Great! Thank you. Will ping here when I have the campaign ready. With my current workload, I expect that will be 1-2 days from now. --EpochFail (talk) 17:52, 7 February 2016 (UTC)
@EpochFail: Something missing from me? — Jeblad 12:42, 3 March 2016 (UTC)
Hi Jeblad. I'm sorry for the delay. I actually kicked off the campaign last week, but I forgot to ping here. You should be able to see the edit quality campaign load up at no:Wikipedia:Etiketter. I used google translate for the campaign title "Edit quality (5k balanced)". Let me know if the translation makes sense or if you'd like me to change it. --EpochFail (talk) 17:03, 3 March 2016 (UTC)
I'll take a look. ("Edit kvalitet" → "Redigeringskvalitet", I don't know what "Balansert" means in this context, "Campaigns" → "Kampanjer", "request workset" → "hent arbeidssett" or "Hent arbeidssett".) — Jeblad 17:04, 3 March 2016 (UTC)

input API[edit]

In addition to the gadget is there any input-API, I see there is a REST-interface for accessing data but is there one for creating it? --Abbe98 (talk) 11:19, 3 June 2016 (UTC)

Yes there is! We don't have a good reference for this yet, but you can see some of the details of the API by viewing server.js (the javascript used to access the server) or campaigns.py (the python used to specify the server routes relevant to campaigns). In order to perform any actions that modify data you'll need to have a the client do an OAuth handshake and store the relevant session cookie in future requests. See https://labels.wmflabs.org/auth/initiate for the endpoint that starts the OAuth handshake. --EpochFail (talk) 15:58, 3 June 2016 (UTC)

Wiki labels in cawiki[edit]

@EpochFail: What do we need in order to implement edit type campaign in cawiki? I've already translated ca:Viquipèdia:Etiquetes. Thanks --Gerardduenas (talk) 20:56, 8 November 2016 (UTC)

Hi Gerardduenas! Good question. We're experimenting with an improved sampling strategy right now where we target edits that have patterns in their comments that suggest certain types of changes were made. This helps us balance the labeling across under-represented classes (e.g. clarification and simplification). Can you link to a set of examples (~5) of the types of comments people leave when performing each of the edits in en:WP:Labels/Edit types/Taxonomy? --EpochFail (talk) 21:54, 8 November 2016 (UTC)
Copy Editing
rephrase; improve grammar, spelling, tone, or punctuation.
For enwiki, we looked for "copy edit" or "typo"
Clarification
specify or explain an existing fact or meaning by example or discussion without adding new information.
For enwiki, we looked for "clarify" and "clarification"
Simplification
reduce the complexity or breadth of discussion; may remove information.
For enwiki, we looked for "simplify" and "simplification"
Point of View
rewrite using encyclopedic, neutral tone; remove bias; apply due weight.
For enwiki, we looked for "npov", "pushing", and "neutral"
Refactoring
restructure the article; move and rewrite content, without changing the meaning of it.
For enwiki, we looked for "refactor", "relocation", and "reorganization"
Fact Update
update numbers, dates, scores, episodes, status, etc. based on newly available information.
For enwiki, we looked for "fact" and "factupdate"
Elaboration
extend/add substantive new content; insert a fact or new meaningful assertion (beyond a simple "fact update").
For enwiki, we looked for "add info", "extend info", "add some information", etc.
Verifiability
add or modify references and citations; remove unverified text.
For enwiki, we looked for "reliability" and "citation"
Disambiguation
re-link from a disambiguation page to the specific page.
For enwiki, we looked for "disambiguate" and "disambiguation"
Wikification
format text to meet style guidelines, e.g. add links or remove them where necessary; convert bold text to proper headers, etc.
For enwiki, we looked for "wikify" and "wikification"
Process
start or continue a wiki process workflow such as tagging (or un-tagging) an article with cleanup, merge or deletion notices.
For enwiki, we looked for "tagging", "cleanup" and "flagging"
@EpochFail: Here you have what you asked for, with a bit of delay, sorry ;). If you need anything else to set it all up, just ask! --Gerardduenas (talk) 18:05, 11 December 2016 (UTC)
Hi Gerardduenas. I've created a task for us to get the labeling campaign started. See Phab:T152965. I'll still need some examples of common edit comments left in cawiki for each of the edit types in the taxonomy before I can generate the sample. --EpochFail (talk) 15:29, 12 December 2016 (UTC)
Hi EpochFail, in the taxonomy page you have the catalan common edit comments. --Gerardduenas (talk) 18:50, 12 December 2016 (UTC)
Oh great! I didn't look closely enough. I'll get to work :) --EpochFail (talk) 19:40, 12 December 2016 (UTC)
@EpochFail:Has there been any progress in the deployment?--Gerardduenas (talk) 16:23, 27 December 2016 (UTC)
Gerardduenas, thanks for the ping. I'm sorry to say that I haven't had a chance to sit down with this yet. My volunteer hours have been stretched thin recently. I'll try to get this picked up tomorrow. Sorry for the delay. --EpochFail (talk) 00:28, 28 December 2016 (UTC)

┌─────────────────────────────────┘

Gerardduenas! I've made some progress. https://quarry.wmflabs.org/query/15115 contains a query to gather a sample of edit types based on your notes. Can you review it and the results to make sure I didn't make a mistake. You should see that each edit type covers two pages (200 revisions) of results. You can compare the "explicit_intention" to the "rev_comment" to see if the matching is working OK. If this looks OK to you, I'll start the process of loading the results into Wiki labels for manual labeling. --EpochFail (talk) 19:40, 28 December 2016 (UTC)

@EpochFail: I've made some corrections and added more rev_comments. A problem arrised, it classifies page renamings (that have orthographic purposes) as copy-editing, shouldn't the query be run against edits only? I think there is an error in the query, I've put some comments on the code. I wasn't sure if all the changes would be ok (especially all the regex) so I forked your query and made all the changes there. Here you have the fork: https://quarry.wmflabs.org/query/15119 --Gerardduenas (talk) 07:57, 29 December 2016 (UTC)
@EpochFail: If the word is between /* word */ in the edit comment, it shouldn't be categorized as so. /* */ Marks that the edit is done in a section and there are lots of sections named "exemples" (examples) which automatically categorises the edit as clarification. --Gerardduenas (talk) 09:20, 29 December 2016 (UTC)
Gerardduenas, I've modified https://quarry.wmflabs.org/query/15115 based on your work and your concerns about catching section headers. Can you review again and see if there are any other issues (or I've failed to account for some you've already raised)? --EpochFail (talk) 00:58, 30 December 2016 (UTC)
@EpochFail: I think everything is in order now. --Gerardduenas (talk) 10:18, 30 December 2016 (UTC)
I just made a modification to https://quarry.wmflabs.org/query/15115 to limit the results to edits to article pages. Once that query finishes, I'll load up the sample. Gerardduenas, can you give me a translation for the campaign name: "Edit type (2k stratified sample)"? --EpochFail (talk) 18:42, 31 December 2016 (UTC)
(update) The query finished and I need to go work on something else, so I just loaded the edits into ca:Viquipèdia:Etiquetes with the campaign name "Tipus d'edició (2k mostra estratificada)" which is what google translate gave me. Let me know if you want to change the campaign name and I'll make an update the next time I'm around. Otherwise, happy labeling! We're working on a good progress indicator, but for the meantime, you can check progress on the campaign by going to https://labels.wmflabs.org/campaigns/cawiki/46/?campaign=stats I've configured the system to get 2 labels per edit because that seemed to be necessary for English Wikipedia. So the campaign will be done when "labels" is 2x the number of "tasks". --EpochFail (talk) 19:01, 31 December 2016 (UTC)
Thanks for everything! I'll report back if we encounter any problem. Could we translate the tool to catalan somewhere? The campaign title is ok. --Gerardduenas (talk) 20:04, 31 December 2016 (UTC)
@EpochFail: I've translated the tool here and here could you apply the changes? --Gerardduenas (talk) 15:08, 5 January 2017 (UTC)

wikilabels in zhwiki[edit]

@EpochFail: see zh:Wikipedia:评级标记, How do create a campaign?--Shizhao (talk) 03:34, 6 December 2016 (UTC)

I just created one. You should see it there now.  :) --EpochFail (talk) 22:55, 6 December 2016 (UTC)
@Shizhao: ^^ --EpochFail (talk) 22:55, 6 December 2016 (UTC)
thx, I see--Shizhao (talk) 06:39, 7 December 2016 (UTC)

Need instructions on how to do this[edit]

I saw the note about http://labels.wmflabs.org/ui/enwiki/ and thought that I'd try it. I picked the "Edit type" campaign, because it had the smallest number.

First, I couldn't figure out how to classify "removed a category because it was deleted". So I abandoned that one.

Then I got a simple diff: the addition of an internal wikilink. "Wikification" sounds about right... although it'd be nice to find a description of what that's supposed to mean, so that I could be certain. But I'll be bold. Choose the item from the dropdown menu, click the 'Save' button... and get an error message: The field "semantic_labels" was not filled. Submit anyway?

I can't find anything on the page that seems to be called "semantic labels". Well, let's click a few things and see if we can figure out what it wants. The (i) next to "Information" doesn't do anything, so I guess I'm not going to get any information. I've got a new message, "No intentions chosen. Please add an intention or "other"." That's right underneath the dropdown menu where I've selected an intention. Does it want Added/Modified/Removed ticked? Well, I have no idea whether adding an internal wikilink is "added" or "modified" or so trivial that it's "none of the above". Oh, look. The green [+] is actually (i) and tells me that wikification is the right thing. And now there's a destructive-red trash can that looks like the world's scariest way to dismiss an interface message.

I'm giving up now. Feel free to ping me if you have documentation on how to make this work. WhatamIdoing (talk) 18:48, 16 January 2017 (UTC)

Hey WhatamIdoing. Sorry to not get back to you sooner. So, we've just made a transition off-wiki to a stand-alone and it looks like we're in a weird state where it's hard to find the on-wiki documentation from the stand-alone tool. We have a task for addressing that more directly, but that's been languishing due to a lack of dev support for our team (proposals in progress). Sub Phab:T139957 if you want to track progress on that. See en:Wikipedia:Labels/Edit_types for the information page for that campaign. There's another issue that you brought up. The "edit types" form has a usability issue. You must click the "+" button in order to add an "intention" to an edit. This lets you add multiple intentions. You're not the first to not understand this. Sorry for the trouble. --EpochFail (talk) 20:40, 2 February 2017 (UTC)

Wiki labels in etwiki[edit]

@EpochFail: Howdy! How can I make Wiki labels interface work in Estonian language? "kvaliteet ( 20k juhuslik valim)" -- this is the only thing in Estonian right now. Everything else, buttons, etc. loads in English. Interface was translated back in 2015 here Wiki_labels/Interface_translation/Edit_quality. Cumbril (talk) 16:04, 8 February 2017 (UTC)

Hi Cumbril! arg! I'm sorry about that. We need to take those pages down as they are out of date. We now use https://translatewiki.net/ (specifically https://translatewiki.net/wiki/Translating:Revision_scoring) Regretfully, it looks like the et langauage stuff didn't get added before we made our transition to translatewiki.net. I'd transfer your translations for you if I could, but you'll need to register on translatewiki.net and demonstrate that you can translate from en to et in order for the system to accept your translations. --EpochFail (talk) 16:20, 8 February 2017 (UTC)
That is totally OK, actually I'm already registered on Translatewiki and have done some translations. I will translate it there (or transfer already existing translations there). Cumbril (talk) 19:18, 8 February 2017 (UTC)
@EpochFail: I translated everything, but do you know where this string comes from – "Edit kvaliteet ( 20k juhuslik valim )"? It's not in Translatewiki. It's the string that is next to the plus sign, at the top, under Campaigns title. It should be "Redaktsioonide kvaliteet ( 20 000 kirjet, juhuslik valim )" if translated. Cumbril (talk) 17:04, 9 February 2017 (UTC)
Thanks Cumbril. I'll get those messages merged and deployed today. I'll also get the campaign name fixed. Thanks for that. --EpochFail (talk) 19:48, 9 February 2017 (UTC)
Yes check.svg Done --EpochFail (talk) 20:41, 9 February 2017 (UTC)
@EpochFail: Wow, with translated messages the interface looks really nice! Thank you. By the way, when we give labels to edits, do these labels improve machine learning algoritms immediately or someone has to manually update algoritms with new results? I notice that most of the time ORES scores are quite spot-on for etwiki. With the 20k sample size it will take long before we get all edits rated. Cumbril (talk) 20:50, 9 February 2017 (UTC)
Cumbril It doesn't get loaded into the models automatically. I'm happy to hear that the "reverted" model is working pretty good. That's a good sign that we'll be able to do even better with the labeled data. We do need to manually train new models with the labels that you submit. It looks like ya'll have completed 3273/4678 labels, so you're almost done! We only need labels this subset of the 20k because we run a "autolabeler" that flags all non-reverted edits from sysops, bots, etc. as good and then upload the rest to Wiki labels. Once you're done with this, we'll be able to deploy the mw:ORES Review Tool on etwiki. --EpochFail (talk) 22:11, 9 February 2017 (UTC)
OK, we will try to speed the rating process up. Can't wait to see how the ORES scoring will perform with training! :) Cumbril (talk) 10:56, 10 February 2017 (UTC)

@EpochFail: I just read your 22:13, 11 November 2015 comment above ("removing bot and sysop edits from the set") and would like to mention that etwiki workset still has bot edits in it. These edits don't comprise sizeable portion of all edits, because there are not that many bots running in etwiki, but they are 100% good edits (haven't yet seen any bot edit I would revert).
What do make up sizeable portion of all edits though are user namespace edits – etwiki has many novice users that prepare their articles in user namespace. User namespace edits are almost always (I'd say over 99% of the time) good – sysops wouldnt revert them even if users make errors, because it's part of their learning and experimenting process that takes place in their user pages, harming no-one. And there is very little to no vandalism taking place in user namespace in etwiki. When I see user namespace edit in the workset, I almost automatically set "Not damaging" and "Good faith" labels to it. --Cumbril (talk) 09:48, 15 February 2017 (UTC)

We shouldn't have bot edits in the dataset. I'll double check this and get back to you.
Re. user namespace edits, I'd like to keep them since vandalism can happen there too. We need to teach ORES what vandalism looks like, but we also need to teach ORES what non-vandalism looks like too. I think it's good that we teach ORES that users often make messy edits under their user page and that those aren't damaging. Even if we do see vandalism 5 or 6 times in user space, that will be enough to help ORES make sense. Do you think we'll see at least that much vandalism in user space? --EpochFail (talk) 00:00, 17 February 2017 (UTC)
Ping Cumbril ^^^ --EpochFail (talk) 00:01, 17 February 2017 (UTC)
@EpochFail: Definitely! I have seen vandalism in user space more than 5 or 6 times for the whole sample. I was just thinking that maybe there are too many user space edits in the sample. There are about 10 user space edits per every 50 task workset and 99% of the time I label them as non-damaging and good faith. Other thing - it would be very helpful if for user space edits the labeller could see who did the edit (i.e. user name or IP address). Because that is the most important thing in determining if the edit was harmful. User who edits his own user page can do almost whatever he likes, unless it violates Wikipedia policies. Article space harmful/bad faith rules don't apply to those edits. But someone editing other user's user page is often a sign of a bad faith edit, even if the edit itself (the content) is harmless per article space rules. Cumbril (talk) 07:57, 17 February 2017 (UTC)
I think the reason of those bot edits in the sample could be that they are done by bots not registered as bots in etwiki. Like this one: https://et.wikipedia.org/w/index.php?title=Dubno&curid=317408&diff=4585138&oldid=4585136. Cumbril (talk) 11:58, 17 February 2017 (UTC)
Cumbril, here's the query I used to grab the sample for etwiki. https://quarry.wmflabs.org/query/6182 This should get us a purely random sample of edits saved between October, 2014 and October, 2015. For the bots, I wonder if we could get those bots flagged ASAP so that we don't have this problem again with future samples. What's etwiki's policy on unflagged bot activity? As it stands, I think we'll want to label a new sample of edits pretty soon as this sample is already well over a year old.  :/ --EpochFail (talk) 15:27, 17 February 2017 (UTC)
EpochFail: OK, I see. So then you used some sort of script to filter out admin edits, bot edits etc? I'm wondering if it would be possible to reduce user space edits proportion in the sample next time.
Etwiki policy on bots is pretty lax. At least as long as the bot doesn't start to do massive edits. If it does one edit now and then, then no-one really cares. Of course we should flag the bots, I will bring it up.
I think it doesn't matter that the sample is over one year old. Vandalizing hasn't changed much. Cumbril (talk) 15:51, 17 February 2017 (UTC)
OK. Since we're almost done, let's stick with what we have for now. Next time we run the sample, we can make changes. Sorry for the confusion and weirdness. Thanks for your contributions :) I'm excited to get this deployed at etwiki. --EpochFail (talk) 16:07, 17 February 2017 (UTC)

@EpochFail: I changed data for etwiki in the table on the page Research:Revision scoring as a service/Word_lists, but Dexbot overwrote my changes. Can you please update information based on this diff: https://meta.wikimedia.org/w/index.php?title=Research:Revision_scoring_as_a_service/Word_lists&diff=16321095&oldid=16297074 ? Cumbril (talk) 09:13, 22 February 2017 (UTC) ┌─────────────────────────────────┘
Cumbril, thanks for pointing that out. I've filed a task so we can figure out what happened. See Phab:T158899. --EpochFail (talk) 19:52, 23 February 2017 (UTC)

@Cumbril and EpochFail: You need to add your updates in the subpages (For example: Research:Revision scoring as a service/Word_lists/et) The bot will bring that update to the main page otherwise it overwrites them. I need to make an edit notice there about it. Amir (talk) 22:20, 23 February 2017 (UTC)
@Ladsgroup: Thanks! In the hindsight it seems logical, but somehow I failed to get it before. I think the fact that I also changed "Contact person" field on the main page and this change wasn't overwritten by the bot added to my confusion. Cumbril (talk) 11:43, 24 February 2017 (UTC)

@EpochFail: I'm happy to report that etwiki Wiki Labels campaign is finished. The only redactions without label in the set are the ones with the message "code": "revision not found" (about 50 redactions). I have also been updating etwiki wordlist, and it is ready to be used. Can we deploy the mw:ORES Review Tool now in etwiki? Cumbril (talk) 13:08, 4 March 2017 (UTC)

Cumbril, sorry for the late response. I saw your ping and have prioritized getting the models built. With luck, we'll have the review tool deployed next week. --EpochFail (talk) 21:09, 8 March 2017 (UTC)
@EpochFail: No worries, I understand you have other projects and real life. You have been very responsive and resourceful so far, thank you for that! I'm excited to see ORES working in etwiki soon. Cumbril (talk) 21:24, 8 March 2017 (UTC)