Learning patterns/WLX jury benefitting from the wisdom of the Commons crowd
What problem does this solve?[edit]
- "Good judgment comes from experience, and experience comes from bad judgment."
- – Unknown author
In the second Wiki Loves Earth photo contest in Bulgaria, in 2016, the contest jury made a bad judgment, unintentionally selecting two photo montages in the Top 10 national selection.
Having announced the results,[1] the organizers (and other unrelated editors) proposed the winner photos for quality and/or featured images in Wikimedia Commons, and then all hell broke loose: The two images were recognized and qualified as 'fake',[2][3][4] and were disqualified from the international stage of the contest. The local team also disqualified these two images and informed their author, reviewed the ranking and selected and announced[5] two other images, replacing the disqualified two. This was an awkward and difficult situation from the organizational, public relations – and potentially legal – points of view. Shit happens.
Even the most carefully selected and dedicated jury members can make mistakes when they have to review hundreds and thousands of images within a short period of time. Even the best professionals in their subject areas, like ecology, biology, geography (for WLE) or cultural heritage (for WLM), and even the most experienced Wikipedians, may overlook a professionally produced photographic montage. While montages are not explicitly forbidden on Wikimedia Commons and can be useful and illustrative in certain contexts, it is true that they have practically zero value as encyclopedic photography which aims at documenting in a realistic way the natural and cultural heritage.
And we are humans; sometimes we miss the forest for the trees. It is not by chance that Eric S. Raymond has formulated the famous Linus's Law: Given enough eyeballs, all bugs are shallow.
What is the solution?[edit]
Of course, once the mistake is made, it's made, and admitting it, correcting it and trying to learn from it is the best possible action. However, the input from the Commons "crowd" in the discussions for quality and featured images, can be solicited next time in advance, rather than post factum.
A proposed idea, and a preventive measure for other mishaps of this kind, is the following: provide in your local selection procedure some time (like 7-10 days) for an extra "control loop". Let the jury review the images and shortlist a number of best candidates for the Top 10 (or Top-X, depending on the specific photo competition). Before the jury takes the final decision, let these shortlisted candidates for winning photos be reviewed by the Commons community in separate discussions using the quality and/or featured pictures processes.
This extra step is not diminishing the role of the jury. The feedback received in this way can help the jury sieve the photos once again and learn from the arguments given by experienced Commons members, and give them extra confidence in the accuracy of their decision, or occasionally identify a problematic choice.
Things to consider[edit]
- This step does not free the jury from their responsibility to "do their job". Shortlisted images for review in this "control loop" should be a reasonable number, like two or three times the number of the winners (e.g. for 10 winner photos, propose for community review no more than the top 20 or top 30 images shortlisted by the WLX jury).
- The voting period for featured images on Commons is 9 complete days counted from the nomination.[6]
- Only two active nominations for featured images by the same user (that is, nominations under review and not yet closed) are allowed. The main purpose of this measure is to contribute to a better average quality of nominations, by driving nominators/creators to choose carefully the pictures presented to the forum.[6]
When to use[edit]
- When the jury members participate in such a WLX jurying procedure for the first time, and would like to get an extra opinion.
- When the jury members are not professional photographers, and would like to get an extra opinion.
- When the jury members... would just like to get an extra opinion. :)
Further reading[edit]
Recommended further reading for this specific situation is contained in Piet Hein's grook (short poem)
THE ROAD TO WISDOM?
The road to wisdom? — Well, it's plain
and simple to express:
Err
and err
and err again
but less
and less
and less.
Other texts by Hein are included in Wikiquote.
Endorsements[edit]
- I just think that it is amazing to be able to admit a mistake, learn from it and share your experience with others. there is no shame in making mistakes, but sometimes people need to be told that directly. I wish we had some time for this in Wiki Loves Monuments and Wiki Loves Earth contests... -- アンタナナ 19:44, 5 October 2016 (UTC)
- I think every jury should have at least one experienced photographer who will take care about the technical quality of the images and Wikimedia Commons is certainly a good place to look for such people if the organisers in some countries face difficulties in finding them. However, it is much more practical and less time-consuming if this is done as part of the jury's work instead of doing it separately. Perhaps, we should add this as a hint in the guidelines for how to organise WLX on Wikimedia Commons and open a discussion with the community on Wikimedia Commons regarding the possible involvement of some users in the jury.--Kiril Simeonovski (talk) 13:24, 10 October 2016 (UTC)
- Www
Rr 103.67.158.236 06:55, 17 July 2022 (UTC)
