Research talk:Post-edit feedback

Delivering post-editing feedback

Possible tools for delivering post-editing feedback include:

• Heavily modified CentralNotice banners, using an API call to determine editor attributes and delivery to every editor who meets the qualification.
• Talk pages, especially via bot
• Email
• Another method for delivering arbitrary JavaScript to editors (prospective)

I am moving these ideas to the talk page as the scope of this experiment is focused on instantaneous feedback displayed upon successful completion of an edit while all these methods are about delivering asynchronous feedback --DarTar (talk) 23:21, 8 May 2012 (UTC)

Idea

Noting this so we don't forget.... both the confirmation and gratitude messages could be used as hooks for MoodBar, where we ask people to rate their experience editing afterwards. Steven Walling (WMF) • talk 01:08, 8 June 2012 (UTC)

WikiProject invitation

Mockup of post-edit WikiProject invitation

Post-edit messages could be used for inviting users to join WikiProjects, ideally in combination with a simplified sign-up process (which shouldn't be too hard to do in a hackish manner if the WikiProject page follows a predictable format). This would likely not have huge impact without overall improvement to how WikiProjects work, but it would be very interesting to see what kinds of sign-up rates we can get.--Eloquence (talk) 01:55, 13 June 2012 (UTC)

Yes, task recommendations or invitations of this kind is definitely a followup item. The challenge in this case is discovery of what WikiProject an article is tagged with in sufficient time to deliver a notification post-edit. That information is in a fairly predictable format, but since it is encoded in wikitext via categories on the talk page, it may require the kind of processing that isn't ideal for such a real time notification, so another method of delivery may be more appropriate to test. Steven Walling (WMF) • talk 17:52, 13 June 2012 (UTC)

It should be a fairly quick API call, but it might be useful to pre-load it while the user is still editing the page.--Eloquence (talk) 20:49, 13 June 2012 (UTC)

Best idea, yet!!

Hi :-) I think that giving post-edit feedback such as described in the research documentation is the best idea that I've heard so far. I'm really interested in hearing the results. FloNight (talk) 22:52, 15 June 2012 (UTC)

Glad it's something that sounds appealing! The feedback is much appreciated. :) We are shooting to start the first phase of this test (the confirmation message) next Thursday, if there are no delays. Steven Walling (WMF) • talk 00:37, 16 June 2012 (UTC)

I started using a w:en:Fitbit last week. (The company is located in SF). One of the aspects of it that works for me is the feedback it gives. Be interesting to see if us being more interactive will reach out to some people and draw them in more. I'll be eagerly following the outcome of the phases of experimentation. FloNight (talk) 18:19, 16 June 2012 (UTC)

I saw one of these pop up today - "your edit has been saved" - and thought it might have been you guys... Nice one. Wittylama (talk) 02:45, 22 October 2012 (UTC)

User quality heuristics

The following are criteria for post-hoc filtering to ensure that our analysis takes into account only new good-faith editors:

• no blocks
• rejection-to-successful-edit ratio of no less than .80, where rejection is defined as either reverted or revision deleted
• no test accounts of team members (WMF accounts)
• no legitimate alternate accounts (will be in Category:Alternative Wikipedia accounts)

Additional factors to consider/filter for:

• users with no mainspace edits (e.g., all userpage, sandbox, etc.). This is not in itself a measure of low quality, but is not a sign of a user progressing into valuable contributions to the encyclopedia.
• no suspected sockpuppet tags on userpage. If a user has this without actually being blocked, we should assume good faith, but it's not promising
• the presence of warnings, especially level two and three standard warnings, on user talk page may be appropriate as a quality metric related to the rate at which someone is rejected. Theoretically, a user might have 9/10 of their edits stick, but do something in the minority of poor edits which is egregiously bad. However, the likelihood that editors would receive no corrections early on is pretty slim, and particularly bad users are blocked anyway.

Could you let us disable it?

I've no problem with the general idea, but perhaps a checkbox could be put into the user preferences to let people disable it if they don't like such messages? I find it a little annoying and unnecessary, personally, but as an experienced and heavy editor I'm clearly not the target audience. I've managed to make them go away by editing my user CSS file, but this feels like kind of a hack. Thanks. --69.244.72.161 14:53, 22 October 2012 (UTC)

Hi anon. We have chosen not to make this a preference so far, for the following reasons:

• No other site which includes these kinds of messages allows for a preference to hide it (Gmail and other Google applications, Twitter, Dropbox, etc.). This is because consistency is a key part of a confirmation message like this.
• Like these other sites, this confirmation is designed specifically to reduce annoyance and, after you've seen it for a while, to become something that your brain ignores as part of the normal functioning of the site. For now, I realize it probably seems very new if you're at all used to editing. But it only appears for two seconds, and I bet if you give it time, you won't see it after a while.
• Adding a preference would add to the bloat of checkboxes for individual features. The community and the WMF are actively trying to remove preferences that aren't used by many people, and we don't want to add a preference by default for every new feature.
• So far I have only seen a relatively small group of people either asking to remove it or actively hiding it. For instance, only 29 editors in English and 16 in German have chosen to add the CSS snippet. For power users like yourself, personal CSS is specifically designed to enable these kind of tweaks. When considering other options, I don't think it's fair to add yet another preference that would be visible to all the tens of thousands of other editors, but only used by a handful.

We said effectively the same stuff on the English and German Village Pumps, so apologies if that was repetitious in any way. Steven Walling (WMF) • talk 19:27, 23 October 2012 (UTC)