Research:Article feedback/Reader survey
The Wikimedia Foundation, in collaboration with editors of the English Wikipedia, is developing a tool to enable readers to contribute productively to building the encyclopedia. To that end, we started development of a new version of the Article Feedback Tool (known as AFTv5) in October 2011. Once the AFTv5 feedback forms were deployed in late December 2011, we began gathering data on how our editors, readers and team members felt about each design. This report is covers the survey responses we received from readers.
From 25 January to 13 February 2012, the call to action - a panel that appears after the feedback form has been used, normally encouraging readers to try editing - was replaced with a link to a survey that invited readers to give feedback on Version 5 itself. We gathered both quantitative and qualitative data in this fashion, using questions that are included in the appendix of this report, for each version of the form. In total, 1,472 people have filled out the survey, with an average of 491 for each of the three versions of the form. Our intention was to discover which version readers preferred before making a final decision on what would be used when the tool is finally deployed.
Key findings 
Of the readers who chose to fill out the survey, an average of 64 percent approved of at least one version of the form, while 13 percent disapproved - the remaining 23 percent were not sure about the tool's usefulness. Using only percentages, Option 1 and Option 3 were tied, with 66 percent approval for each. However, the net likes (the percentage who liked it, minus the percentage who did not) puts Option 1 in the lead, from a reader perspective with 55 percent net approval compared to 52 percent for Option 3. In addition, although Options 1 and 3 were tied for raw "likes", the survey for Option 1 gathered far more feedback, with 38.63 percent of the feedback coming from Option 1 compared to 33.83 percent for Option 3. Overall, Option 1 is narrowly considered the best design by our readers, with both raw and net data bringing it into the lead if one considers the substantially larger number of readers commenting on this design compared to the others.
Option 1 
"make it more eye-catching on your website. otherwise it is perfect."
"no free text - prefer options. its more quick"
"You aced it, as far as I can tell, it's nice and easy to use. "
|11% don't like|
|55% net likes|
Option 2 
"It's simple and understandable."
"It should be small. The bigger the size of the popup, the bigger the event seems."
"it's already quite good"
|13% don't like|
|46% net likes|
Option 3 
"Quick and easy and as a result I decided to do the quick survey, which took less than a minute!"
"You could improve the feedback form by completely removing it from the article. It's like having a popup in front of the article that I can't block. Why the hell is it here?"
"It was great. I don't think it needs changing."
|14% don't like|
|52% net likes|
Based merely on the percentage of respondents who approved of the design, Option 1 and Option 3 are tied, with 66% of readers liking each form. However, the Option 1 survey was responded to by more users - making up 38.63 percent of the responses, as opposed to 33.83 percent for Option 3 - meaning that, numerically, more readers liked Option 1 than Option 3. In addition, Option 3 garnered a higher dislike rate - 14 percent to Option 1's 11 percent. Using the net likes metric, Option 1 comes out slightly in the lead, with 55 percent of readers approving of the design compared to 52 percent for Option 3. In conclusion, Option 1 is the design readers prefer overall, albeit narrowly.
|Metric||Option 1||Option 2||Option 3|
|Percentage of likes||66%||59%||66%|
|Percentage of dislikes||11%||13%||14%|
|Percentage of unsures||22%||28%||20%|
- (1) Why did you fill in the article feedback form today?
I want to: (check all that apply)
- improve the article I was reading
- make a suggestion
- ask a question
- give praise
- report a problem
- share my opinion
- (2) What do you think of that feedback form?
- I like it.
- I do not like it
- I am not sure
- (3) Why? How can we improve that feedback form?
[add a comment]