Wikimedia Fellowships/Project Ideas/A Digital Wiki Coach To Provide Focused Advice

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search
List of Project Ideas Digital Wiki Coach To Provide Focused Advice



To get more community feedback I've made public my fellowship idea - submitted earlier with my fellowship application.

Contents

Abstract - Simone The wikipedia Coach[edit]

Simone the Wiki Coach[note 1]

The single largest source of editors available for has always been at the new article creation. Yet increasing xenophebia excerbated by labor saving factors has culled this source of new editors. This proposal describes how this trend can be reveresed using a software coaching agent which can intervene on user's behalf: The Strategic WMF goals are framed in a social context and analysed. Two key dilemmas are considered within a game theoretic perspective and solutions are proposed to create an eusocial synthesis of concerns. The Media in the MediaWiki software's ecosystem is considered using McLuhan's tetradic method. The requirement are refined, taking form as Simone - a proactive autonomous agent. Her activity is illustrated in the scope of new user recruitment coaching. Driven by a goal to reform the community by promoting cooperative behaviour areas, Simone's overriding concern is to oversee good-faith editing without damaging the long-term user base of the project. The innovation of influencing editors not articles is explained. Finaly how Simone will scale and her long-term viability is outlined

Reframing the central questions[edit]

Editor loss, the gender gap, article quality improvement and low editor engagement are not isolated problems. They are form an interrelated hierarchy and share three main underlying causes.

  • Defective media architecture — the most precious resource in a wiki is human labour. Yet the MediaWiki software is highly deficent when it come to minimising to the bulk of both social & editorial tasks of building and maintaining Wikipedia. This creates both a steep learning curve and labour crunch and introduces social and technological barriers to new participation. [note 2]. Partial solutions based on user scripts, gadgets, robot and third party software only handle one facet of the problem and ulimately increase the devide between the core contributors and the new comers - a source for Xenophobia.
  • Piecemeal editorial policy — Many pages of policy, recommendations, essays, specialised jargon and actual individual interpretation developed in little pieces. This is also joined by a common law approach to administering a policy so complex and cumbersome that any argument can quickly become a war of attrition. [note 3]
  • Social and personal anarchy — WMF, developers, chapters, ArbCom, administrators, seasoned editors, patollers; newcomers < readers and offsite readers are all interacting without coordination or communication flowing back and forth and lacking a chain of responsibility to handle problems. As a result only crisis management in the form of top down intervention. (This is the most difficult place to change since it involves [note 4]

The chief contributing defect of MediaWiki is unplanned social scaling [1]. The piecemeal development of governing of editorial policy "has led only to the use of violence in place of reason, and if not to its own abandonment, at any rate to that of its original blueprint"[2] [3]. The violence is deletion of good faith work by domain experts and novice editors. Research shows that over half of articles deleted daily using a unilateral CSD article A7 "no indication of importance" are good faith work where as bad faith work such as spam, patent nonsense and vandalism together only account for less than 20% percent of articles deleted by the CSD. [4] Both Social and personal anarchy, anachronistic vestiges from the early days of the web which have consistently brought about the destruction of societies which have embraced them as values. [5] With the rise of social networks and blogging these values have undergone a reversal — as identity has played a central role in promoting transparency in the social age of the web and for the first time curing Gender Gap in virtual societies.

Of the three factors MediaWiki software is the easiest to change in particularly using a small stand alone robot. Editrial Policy, the rules by which wikis are governed are the next most easily changed though it does require a wider much larger group of individuals. Social change is harder to produce as it involves large masses of people who are largely unaware of the issues at hand. Piecemeal change in policy is often reactionary, populist and counterproductive[5]. However even at the social level change can be observed though it thousands of not millions of people who are the real stakeholder providing the bulk of funding, content and influence for WMF projects.

When problems are well posed their solution is much easier to find?

  1. How can the loss of active editor, due to conflict over editorial concerns, be reversed?
  2. How can women be encouraged to play a greater role in Wikipedia?
  3. How can wiki increase user's engagement when greater involvement requires less editing and more drudgery?
  4. How can articles quality be improved with limited access to domain experts?

the solutions should therefore involve

  1. Modify the media/editorial policy to cater for more editors.
    1. Find more editors.
    2. Try to keep each editor for longer.
  2. Since there are virtual communities where women participate more than men - they should be emulated.
  3. Reduce non-editorial tasks using technological solutions and produce new ways to engage all level of users.
  4. Provide a mechanism to allow a domain experts contribution to get have greater weight.

The editor's dilemma: Choosing content over community[edit]

The editors dilemma embodies a conflict between two cross-cutting concerns, building content and building a community. An informal statement is: "in a wiki an editor has to choose between two evils: accepts new content of poor quality or reject the new editors who are the long-term future of the project". The incentive for quality content is clear and easy to quantify but the cost of losing a potential editor's is marginalised since it is shared by the whole community.[note 5]

The tragedy here is that the most important requirement for a wiki's ongoing success — a large and diverse community is sacrificed in the name of quality. Luckily it is possible to untangle these crosscutting concerns and get the best of both worlds - quality content and new editors. To go ahead would inflict costs , risks and some impediments to the community but these would be offset by down the road the greater benefit. A wiki requires editors for many labour intensive tasks and so this rejection is not without a moral hazard.

However the editors dilemma is a strategic problem occurring over a long-term - to address it the tactical version must be resolved, this is the reviewers dilemma.

The reviewer's dilemma: Choosing well-formed edits over quality edits[edit]

The reviewer's dilemma is a version of the prisoner's dillema stems from the asymmetry of information between editor and reviewer. On the one hand is domain knowledge and on the other is the knowledge of policy. It is easier to write about what one know about, more difficult to review what you don't. In the absence of a formal review protocol a mandatory step in print encyclopedias — self-appointed reviewers are faced a few choices:

  1. revert.
  2. check the fact.
  3. defer to another reviewer.

As the information asymmetry increases the bias towards option one increases — reverting is a confirmation bias. Instead of making a potentialy erronius decision by reverting they restore a status quo. To support such a choice any editorial defect is cited . Though in a xenophobic situations standards for rejection are very low and arbitrary - "grammatical errors" or "too many red links". The domain expert will only see the default in response to his co-operation. If it happens a few more times he will learn that his work is not welcome, and tell his friends [6]. The simple statement of this dilemma is at every edits machine and human reviewers must choose or reject domain based edits (and resolve human conflict) based only on their expertise of editorial policy . In almost all cases of good faith work — rejection of edits will occur due to editorial concerns and not due to domain problems. Wikipedia's growing popularity means its errors will often be repeated in print, and on-line.

Incentives for Cooperation[edit]

Threat of retaliation
Kindness

As virtual communities mature they eventually experience a decline in participation. This is due to introduction of fitter competitors[note 6] which work better at attracting participant. Most virtual communities, unable to adapt to competition become zombified within a number of user generations. As a Wikipedia articles mature, contribution by domain experts becomes the bottleneck for progress on quality. Studies on the evolution of cooperative behaviour using iterated games show that

  • information asymmetry
  • anonymity
  • lack of communication

are the worst conditions for cooperation to evolve[note 7]. This theoretical warning is directly mirrored by conditions in the field - the requirements that editors work in the isolation on new articles in their private spaces.

The incentives of users are complex but they can be effectively modelled using for factors:

  • Information - shared property. (Too often ignored at notability debates)
  • Work - time investment
  • Reputation & Face - built overtime and lost though conflict.

Solutions comes in three main forms:

Improving incentives[edit]

Correcting payoffs so that even individuals pursuing their own selfish interests will prefer to cooperate. The way to change incentive is to either to provide greater rewards or to reduce the cost of for prefered behaviour. Negative reinforcement of bad behaviour is also possible. We can reduce the communication and coordination cost using technological solutions. See their benefits exceeds their costs in time, and work, coordination, and communication. So that good faith work remains a rational choice while still being able to address bad faith work effectively. The focus shifts to of cost of coordination, copy editing is put on the reviewer (for whom it is easy) and the cost of communication is shared.

  1. reducing costs of cooperation.
    1. making results more visible (information contributed * usage as opposed to edit count).
    2. Reducing coordination costs - the costs of arranging different people to do sub tasks in a complex goal. Good editing work can be done by a domain expert cooperating with an experienced editors. Yet long before a good editor will come along the newcomer will have to deal with CSD Afd and other notices. Simone can act as a matchmaker by engaging good editors in subjects they deeply care about, and by sending domain experts to articles where they are most needed and can best contribute.
    3. Reducing communication costs - the cost for SD and other a social actions has been reduced to help patrollers but the more complex task of communication remains a costly task. By making good communication almost trivially to launch an improvement protocol for an article.
    4. Reducing the asymmetry between experienced and inexperienced users. For example every day most new user's article's are deleted yet at Afd discussions where deletion can be contested, new comers are practically absent..
  2. increasing rewards.
    1. Let experts and new users operate by paying forward[note 8]
    2. Wikipedia has a problem to provide users with credit to editors. By tracking contributions it would be possible to determine and acknowledge a user's higher level contribution. So while we could make a leader board for biters and serial biters and provide them with sad kittens - we should also track good collaborators great communicators as well as coaching assistants and provide them with rewards and acknowledgments. For instance providing an executive coaching opportunity.
    3. invoke Simone using a Twikle extension page - to explain one or more policy issues . (c.f. the invocation API)
    4. provide a new users an Afd response via a single click response template transclusion. "I am under coaching and will (address the issue) as soon I know how. Please provide me with a specific
    5. patrolling - good faith work is hard to assess by new Page Partollers[7] users tools like Twinkle, Huggle, Kissle, and AutoWikiBrowser
  3. imposing greater penalties for non-cooperative behavior
    1. One example is to introduce a better[note 9] review protocol for good faith work:
      1. 3-5 inline tags must be introduced into the article by the reviewer.
      2. Notice allowing editors a week to respond by indicate willingness to correct the problems.
      3. Notice requesting the editor demonstrate his domain expertise by taking a quiz. (could be fun within the scope of user engagement).
      4. The editor may request a peer review for a second opinion.
      5. If the edit is not illegal the un reverted a copy or link to the edit will be given to the editor in his user space.
      6. Reviewers who do not comply will lose his privileges for 2 weeks. (cooling off).
    2. Increasing visibility - article deletion effectively hides the users edits while reversion hides the editorial problems. Both these allow a moral hazard of reducing potential penalties to the reviewer.

Improving game form[edit]

    1. Reducing anonymity - reputations cannot develop in a truly fully anonymous environment.
      1. Simone will assign users under coaching a unique avatar and advise how to link it to their signature.
      2. Simone will allow users to use a Facebook id to identify.
    2. Simone will bear the cost of coordinating repeated encounters within smaller groups of individuals. [note 10]
    3. Increasing visibility - article deletion effectively hides the users edits while reversion hides the editorial problems. Both these allow a moral hazard of reducing potential penalties to the reviewer.
    4. Using a social network identity would be even better to avoid deception.
    5. Repeated encounters between the same individuals are important to evolve from a reviewer's dilemma to a bargaining game.

Improving the diversity of the population[edit]

  1. an increase in circles of social and intellectual relatedness will improve capabilities of co-operation within each group.
    1. No more speedy deletion of for good faith work - speedy is unsuitable for good faith work. It send a bad messages to editors and reviewers.
    2. Restrict reviewing of non spam to categories which they have edited. This will coordinate reviewers and editors better and keep interaction within circles of greater relatedness.
    3. Invade the community with a new breed of editor who can dominate though co-operation.

Some of these ideas would create a gap which is where Simone's comes into the picture. It will be possible for a reviewer to request it to intercede in such situations.

  • Provide an example of software agent whose goal is positive social engineering rather than negative editorial automation.
  • Not just increase the number and life span of editors but increase the domain expertise will reduce the high cost or reviewing edit.
  • Reduce the need for subjective editorial decisions - which are usually handled badly[8] by human expert.
  • Create a minimal learning curve for acquiring editorial, social, technical and administrative skill.

I propose to provide positive alternatives to for instance templates for section and article under construction work easier use of problem tags, wider usage of incubation and requested article features. As these come to effect there would be far less need for a police state mentality and a return to the early meritocracy. I envision this happening socially in more and more places over time using technological means. This would make participating less stressful and more productive.

Certain of the early instigators of the more esoteric policies on Wikipedia have voiced regret once the protocols they innovated got wider and wider usage[note 11]. Other policies[note 12] solve localized problems by imposing unnecessary criteria on the whole community.

Wiki Evolution from Asocial to Eusocial[edit]

Biting newcomers

Wikipedians have been described as xenophobic in their attitude to new comers. Not surprisingly the Wikipedia jargon for this behavior is biting the newcomers. The roots of this asocial dynamic can be understood using simplified models like the reviewer's dilemma. This law of the jungle behaviour with its mad logic, brutish cunning and has evolved in countless encounters has. But this begs the question - could a new 'species' of Wikipedian invade and dominate the population? Considering that such invasion have been described by Sue Gardner in her address at Wikimania 2011.

To answer this question eloquently it is best to use a powerful social metaphor for the wiki and it is the beehive. Within the lens of beehive it becomes preeminently clear that a change of the asocial paradigm to the eusocial one is required. In the real world it means increasing userrelatedness and changing the form of conflict.

In a hive bees use chemical signatures to differentiate hive members from foreign intruders. In the Wikipedia settings behavioural signatures are used. These behavioral signatures have developed though iterated editing games in which the dominating strategy is tit for tat[9]. To be recognised as community member, a newcomer must make a protracted commitment of time and effort.[note 13]

Eusocial Communication

Like a real hive, a wiki requires constant policing to ensure that drones are not pursuing their own agenda at the cost of precious shared resources. In the hive these freeloaders once identified are restrained or destroyed and their progeny eaten. In the wiki sock puppets are unmasked using a check user and are then banned and their works systematically purged. The hive must deal with parasites (COI) and invaders (Paid Editors), and a degree of anarchy (ArbCom). Each generation is very closely related and the oldest, most senior maintains the status quo.

So one may argue that this is the natural order of things. New editors should be viewed with great suspicion -- Are they foreign agents? (Paid Editors) trying to subvert, or perhaps self-serving parasites? (COI). The default solution is one of extreme hazing - the old generations retaliate against individuals who might be suspected of hurting the group, until they learn to walk the line or depart. And the few who graduate will maintain the asocial dynamic. This is a form of total war — it is extremely expensive to wage and is uncommon in nature.[10]

However such solutions are unnecessary and no longer acceptable — it is time to groom another breed of Wikipedians who default to settling differences sporting within a "limited war" paradigm rather than the picric outcome of todays "total war".

Understanding Media in MediaWiki[edit]

In the early dates of Wikipedia it was a simple technology, it lacked experienced editors or an editorial policy. It had no established hierarchies of influence or personal stakes. All these things are signs of progress, yet they present impediments to contribution. Consequently the threshold cost of contribution was low[note 14]. However as the project matured it was modified piecemeal to address immediate concerns - without a guiding hand the changes have often been regretted in retrospect.

Marshal McLuhan in his seminal work understanding media[11] and later text innovated the tetrad as a tool for analyzing the impact of new media. By providing a tetradic analysis for Simone, it is possible to forcast the scope of her impact.

Tetradic Analysis[edit]

Simone Tetrad
  • Enhances - review of good faith edits for: article quality, editorial standards; user recruitment, and coaching; group collaboration under peer review.
  • Reverses - editorial complexity, community reduction, coordination cost, recruitment of new editors, policy complexity.
  • Retrieves - organic editing[12] , pay it forward editing[note 8] shorter learning curve. reduced cost of contribution.
  • Makes Obsolete - early selection of new articles; subjective human assessment of editorial level for articles.

Next Simone will be compared with some other media being utilised and developed today.

Simone and Normal Bots[edit]

Simone reuses existing bot frameworks but will be different in two senses:

  • It is aimed at altering user's editing behaviour and not to fix or revert their edits.
  • It is planned to preempt future problems rather than react to old ones.
  • It is planned to rival human assessment of editorial issues.

Simone and Abuse Filter[edit]

Simone will also build on abuse filter but different in three senses:

  • Abuse filter provides negative options to a few admins. Simone chooses from many positive choices for many editors.
  • It will provide many options in the cooperate category rather than provide one options in the default category.
  • Abuse filter is intended for abuse, but Simone is planned to cover a grey area of good faith work that borders on "abuse".

Simone and The Visual Editor[edit]

On the other hand MediaWiki developers are working to reduce the complexity of editing via the Visual Editor. While a visual editor will reduce the technical level of editing, but does not address any of the social issues discussed above. This solution may have an unintended result of increasing new user hazing. This could happen because new editors will be harder to assess on using technical signatures and will be judged more harshly on policy and style — which will appear to be more arbitrary.

Case Study: The Decline and Loss of Organic Editing[edit]

No other loss of functionality present a greater stumbling block than loss of Organic Editing [12][13]. Organic editing is the ability to create sentence by sentence a new article in main space. Organic also refers to natural growth of the articles. Yet today anyone bold enough to attempt to do this will be censured by patrollers and administrators at every save and informed that her work is not welcome. It will be nominated for speedy deletion on grounds of context, content or non notability. However notability is not an objective assessment, it is simply the harshest dictum to defend for a new editors. When the one editor complains they will be informed that their article may be exiled to the user space. It could apply for a visa when it reaches maturity. Unfortunately user space is not a place for natural collaboration. Your work gets no exposure and other new collaborators are often reverted there.

So why was organic editing lost? As predicted by [14] The number of edits in a day depends on existing article counts then since or as the number of articles grows much faster then the number of patrollers in a given week. And this leaves out the growing complexity introduced by a rise in quality

  • "Don't make many small edit - save them as one large edit".[note 15]
  • There are less red links being added by new editors
  • I reverted your good faith edit because it had many red links.

Corrective Action[edit]

Whenever a page is edited it will be analysed and stored into the index it with a list of current issues. Searching the index will allow Simone a greater level of intelligence. Running strategic queries it will enable planned proactive action rather than a reactive interventions.

Simone's explainable activities would be driven by different causes. However strategies will assist develop full and order behaviour and where multiple options exist it will allow choosing the option which has the highest priority.

  1. Simone objectively advises new users how they could best contribute. New users will, once again contributing with at a near zero cost - on the assumption of good faith work and that they would pay their dues once they have ascended up the learning curve. They would be more rapidly progress in skill and join rank as the next generation of our community - a generation that is less obsessed with unwritten rules, one which would not require wiki-lawyers.
  2. Expand the program to bring articles systematically to featured levels by providing editors with expert advice.
  3. As the Simone matures it could be directed [note 16] by users to instruct users of policy infringement and by project to direct their members to collaborate on hot spots.
  4. To reach a far greater audience and higher levels of engagement a reward based game mode would be developed.

Rationale[edit]

The project is made of software elements unified into a bot I will refer to as Simone. It will either work as an external user or will be integrated into the editor. Work will begin with sensor modules which can check a given revision's merit in various dimensions. These include a comparison with Gold Stadard and a Minimum standard.

technically these can range from simple rules to search wisdom of crowds ranking algorithms to more traditional statistical discriminative models.
  • Next it will select the highest priority issue from its queue.
  • The bot will summarize the top issue in a one line human readable report.
  • Next it will offer advice on how to rectify the problem.
  • Finally it will offer some alternatives and ask for feedback.

Illustration: advice sequence for new user[edit]

The following figures illustrate a sequence of advice notices that would be generated by Simone as it coaches Hildur, a new user from Iceland working on her first article on Kafka's story The Trial. The UI element shown is a talk page/user talk page based system that would be inserted by Simone.

new users coaching can start in several ways

  1. proactive detection of an anonymous editor / new article by a registered user.
  2. reactive intervention based on a biting incident sensor.
  3. invocation from Afd - i.e. a new suspended delete sentence. Either correct the quality issue within a week or be deleted. Simone will be used to carry out and enforce the probation - if successful the probation will end by removing the probation notice. If no action is performed the article will be deleted/userified.
  4. invocation by Huggle, Twinkle or a similar tool with Simone integration.

Step 1 - Join Wikipedia and apply for coaching to save your work[edit]

Current Goal: Bring the article to Threshold Level
Wikipedia Wants You!
Hello stranger, thanks for helping out with this article. We always need more helpers
Your report card for: The Trial
Current Status: The last submission has some editorial problems and since you are unregistered it is likely to deleted soon.
If you are planing to further improve this entry you can save your work by improving it. If you are interested in advice on how to improve it to the threshold level of inclusion choose one of the options bellow.
Article Status
Aspect Overall Score Δ Threshold
Stop x nuvola.svg Context
+5 20
Content
+3 15
Notability
+3 8
References
+5 15
Style
-3 0
Social
0 0
  • Your Mission — Sign up for coaching.
Action Items
Options Accept Reject
Register for coaching accept reject
To mark the page as "under construction" for a day accept reject
I want to view a video on the policy and the advantages of coaching. accept reject
Please don't bother me, I know what I'm doing. accept reject
I've done all that I can. accept reject
( opting out | feedback | about )


Logs[edit]

Simone believes that one should always be testing. She is careful to make her logs useful for analysis using R as well as for keeping track of action that needs a followup. On creating the above message Simone will append to its log:

  • <NewUserEvent>
<Source>1.234.44.215</Source>
<revId/>486369447<revId/>
<Sensor>unregistered edit</Sensor>
<Opt id="100"/><Opt id="101"/><Opt id="105"/> <Opt id="202"/><Opt id="230"/>
<Opt id="G20"/>
<NewUserEvent/>

This log includes:

Notes[edit]

  • Points out that there is a problem with the edit.
  • Advises the use to join sign the user.
  • Can allow measurement of interest in anonymous editing.
  • Option 2 will place an organic editing template in the page with one day expiry. It will also place a notice on the talk page.
  • This user element would be further customised for capturing new users based on stylometric analysis of their input. To consider gender and nationality.
  • Most of the options are just hooks to get the user to sign up. For example I've done all that I can will inform her that she can do much more - Wikipedia has mistakes she can correct and so on.
  • Depending on integration with MediaWiki
    • If in the submission loop the advice will expire immediately if ignored.
    • If it is just put into the IP user page - unregistered advice expires at is never clicked on would expire after 3 days and marked as a bounce.

Step 2 — Learn about context[edit]

Current Goal: Bring My First Article to Threshold Level
Simone Wiki Coach
Hildur, thanks for enrolling for wiki coaching.
Your report card for: The Trial
Current Status: Your article is so short that it could be marked for speedy deletion under article A1 - which means that it does not provide enough context to justify its inclusion in.
Your article: on The Trial has been temporarily protected from speedy deletion. It is a good start – but 50 words is considered too short for a Wikipedia article. Once you add at least another 100 words, it would qualify this article as a stub article.
Article Status
Aspect Overall Score Δ Threshold
Stop x nuvola.svg Context
+5 20
Content
+3 15
Notability
+3 8
References
+5 15
Style
-3 0
Social
0 0
  • Your Mission — fix context
Action Items
Options Accept Reject
you can expand the article - don't forget to include details or technical terminology. accept reject
you can let add a template to protect it while you work accept reject
you can move your article to your user space accept reject
you can see an explanation video tutorial about context accept reject
to dismiss this matter for now. accept reject
( opting out | feedback | about )


  • There are many ways to improve context.
    • Adding info boxes
    • categorising
    • Writing sufficient details.
  • Simone will be basing it context score on minimally unique word n-grams. This will quickly indicate if the article if covering something new or not. Even if there are no unique terms it will also use word vector cosines to identify the closest articles.

Step 3 — Learn To Wikify Text[edit]

Current Goal: Bring My First Article to Threshold Level
Simone Wiki Coach
You are making good progress, the edit brings the content score to 4/100. But you article is an orphan.
Your report card for: The Trial
Current Status: speedy deletion A1 - your stub lacks context.
To fix this all you need to do is add some internal links. We call this processes wikifying.
Article Status
Aspect Overall Score Δ Threshold
Context
+5 20
Stop x nuvola.svg Content
+3 15
Notability
+3 8
References
+5 15
Style
-3 0
Social
0 0
  • Your Mission — Wikify some text
Action Items
Options Accept Reject
to wikify a word use braces like this [[word]]. accept reject
for a how to video tutorial about why it's important to wikify articles. accept reject
to get a recommendation on what to wikify click here. accept reject
to dismiss this matter for now. accept reject
( opting out | feedback | about )


Notes[edit]

  • Syntax should be taught from simple to the more complex.
  • If difficulties arise for engaging users with pedagogically significant advice or goals Simone could review a user's edit history to gauge the existing skill set.

Step 4 — Learn about Notability[edit]

Current Goal: Bring My First Article to Threshold Level
Simone Wiki Coach
great your content is now scoring 7/100
Your report card for: {{{ArticleName}}}
Current Status: Your article has low notability and may be marked for speedy deletion under article A7 or A9 - which means that it does not indicate the importance of the subject to justify its inclusion.
Your article: on The Trail has no references to external sources - add references and wikifi the text so you can better demonstrate what other articles your work touches on – these will quickly boost your score by increasing the visibility
of your articles notability.
Article Status
Aspect Overall Score Δ Threshold
Context
+5 20
Content
+3 15
Stop x nuvola.svg Notability
+3 8
References
+5 15
Style
-3 0
Social
0 0
  • Your Mission — demostrate the article importance for inclusion in an encyclopedia.
Action Items
Options Accept Reject
you can expand the article accept reject
you research and insert references from print sources. accept reject
you research and insert references from online sources. accept reject
you can watch a tutorial about notability and reliable source. accept reject
to dismiss this matter for now. accept reject
( opting out | feedback | about )


note: at this stage we would like to teach the user an editorial skill set as well as a social one.

  • the Editorial requirement that she must show that the article is important enough to be included in Wikipedia. This will require more work, the less notable the article is
  1. don't include it to better answer the issues of notability

However there are probably more than 5 options to fulfil this mission. If we are able to collect 20 options and track each options according to its success — Simone can optimise this advice. I.e. She will pick 5 options at random out of the 20, respecting each option's probability of success.

Step 5 — Learn to add categories[edit]

Current Goal: Bring My First Article to Threshold Level
Simone Wiki Coach
great your content is now scoring 7/100
Your report card for: {{{ArticleName}}}
Current Status: Your article has low notability and may be marked for speedy deletion under article A7 or A9 - which means that it does not indicate the importance of the subject to justify its inclusion.
Your article: on The Trail has no references to external sources - add references and wikifi the text so you can better demonstrate what other articles your work touches on – these will quickly boost your score by increasing the visibility
of your articles notability.
Article Status
Aspect Overall Score Δ Threshold
Context
+5 20
Content
+3 15
Stop x nuvola.svg Notability
+3 8
References
+5 15
Style
-3 0
Social
0 0
  • Your Mission — demostrate the article importance for inclusion in an encyclopedia.
Action Items
Options Accept Reject
you can use hot cat accept reject
you can create a new category. accept reject
you can use the category recommendation too accept reject
you can watch a tutorial about categories accept reject
you can to dismiss this matter for now. accept reject
( opting out | feedback | about )


  • option number four would be supported by an automatic categorising tool

Step 6[edit]

Once threshold is reached the article will be safe to remain in main space. The user will have a positive payoff for her work in the form of a permanent contribution to wikipedia. But there is still much more that the user can still learn. By completing the new user's coaching she be able to create new articles without significant risks of deletion. Thus Simone will continue to instruct the user and cover more points such as

  1. how to use her talk page.
  2. how to use the user space sand box.
  3. how to customise their user space for eusocial communication.
  4. how to view requests for creation.
  5. how to interact with reviewers using wiki love.
  6. the new article creation wizard.
  7. how to get a human coach
  8. how to join project.
  9. how to interact with templates
  10. how to use Template:Inuse-section
  11. how to join project.
  12. how to improve their user page.
  13. how to make tables
  14. how to watch pages
  15. how to insert images
  16. how to delete old/broken references

the curriculum will include trails based on

  • mediawiki software trail
    • e.g.how to use preferences to add some gadgets
    • how to master the rest of the wiki syntax
  • wiki's policy trail
    • core editorial policies
    • recommended best practices (Simone good Wikipedian training)
    • netiquette - socially accepted behaviour

Advice Strategies for Other Use Cases[edit]

Advising only on new users might seem to be a more focused approach but it is deficient in several areas

  • it will not mobilize sufficient human resources to reverse the underlying reasons for editor loss
  • it will be restricted to situations where machine judgment is better than human Judgment.
  • it will miss on some of the greatest benefits that Simone can provide - discounting coordination and communication costs.
  • it will mean that experienced users will not be familiar with the possibility of invoking Simone to place new articles and users under probation.

I will present a some strategies for executive coaching, article improvement. Since there exist a very low activity admin coaching projects this is also a candidate for intermediate level of editors. It can stress finer points of policy.

Retention and Engagement for Seasoned Users[edit]

Loss of seasoned users is the flip side of new user recruitment. Experienced users may run out of information to contribute or they may end up being promoted into role that are less satisfying and experience burn out. Recent studies now provide deep knowledge of this problem. Studies show that under the right conditions these users can be retained longer within the community.

By adding a burn out sensor users who fall out of the fold can be contacted and offered to received executive editor coaching acknowledging, perhaps for the first time, their superior status. There is a cost of coordination of

Simone is goal and mission based. She will, based on feedback provide them with tasks. These will involve work that requires human touch and is therefore harder for Simone to do.

  • Peer review in their area of editing.
  • Peer review in their area's of expertise.
  • Crowdsourcing of articles for importance and quality[note 17].
  • AFC in their areas of expertise.
  • Quality improvement missions.
  • Translation missions based on criteria in the next section.
  • Missions will be varied, including technical task and social tasks.


Current Goal: join the MIB
Simone Executive Coach
Do you have what it takes?
Your report card for: outstanding achievments
Current Status: Your many contribution have singled you out as an outstanding community member.
Why not take it to the next level? If you join the executive coaching you will be able to fulfill your potential and join the editor elite. I will give you some high impact missions.
Article Status
Aspect Overall Score Δ Threshold
Context
+5 20
Content
+3 15
Notability
+3 8
References
+5 15
Style
-3 0
Social
0 0
  • Your Mission — Score on five introductory missions to unlock your Executive Coaching Badge
Action Items
Options Accept Reject
tagging a mentored users' first article with inline problem tags. accept reject
Correction of references to web sites that are broken. accept reject
Translation missions based on criteria in the next section. accept reject
Learn how to join executive coaching. accept reject
Help being an article to featured level accept reject
( opting out | feedback | about )


  • executive coaching will complement the new user coaching program in places where Simone is unable to assist.
  • report cards in executive coaching will be used to give a the tasks a game-like feel.

Quality improvement[edit]

Article quality improvement is the primary driver of Simone's behaviour. However if user have completed new user coaching they can either select another module or stay with the quality improvement module which will give feedback on editing progress.

Simone will still detect problems and direct users to fix them.

  • Outstanding inline tags in the users previously edited articles.


Current Goal: Quality Improvement
Simone Quality Coach
Do you have what it takes?
Your report card for: Quality improvement
Current Status: It's time to do some work on improving article quality!
Quality improvement is not only one of this wiki's strategic goal - it is a part of how community members view your contribution. If you help bring an article to good or featured will count in your favour in many situations.
Article Status
Aspect Overall Score Δ Threshold
Context
+5 20
Content
+3 15
Notability
+3 8
References
+5 15
Style
-3 0
Social
0 0
  • Your Mission — Select an article to improve and upgrade it from C to B level to receive a QI Badge.
Action Items
Options Accept Reject
Place a section for improvement in a C level article. accept reject
Introduce 3 new references in the article. accept reject
Look at suggestions for article to improve. accept reject
Learn more about Quality Improvement by watching a video. accept reject
Help being an article to featured level accept reject
( opting out | feedback | about )


  • Quality improvements will be able to select Users who can translate sections from another language where the other language has a better article.
  • report cards in Quality improvement will also be used to give a the tasks a game-like feel. This include offering badges and barnstars for successful missions.

Simone's UI[edit]

Simone's U.I. is designed to expedite coaching activities:

  • Simone develops a one on one relationship with the student.
  • It monitor's the student's performance using the dash-board
  • It defines a specific goal to work on.
  • It breaks down the goal into missions which can be accomplished in a single editing session
  • It provides feedback on previous activity.
  • It engages the users by providing intervention feedback in the form of advice.

The UI used in the illustrations would be

  • A/B optimized for greater usability.
  • Streamlined to make better use of space.
  • Adding quiz widgets to get a tighter feedback loops with even more engagement.
  • Adding media widgets for audio/video/text
  • Adding a IM chat widget for interaction with human coaches.

Floating UI

  • supplemented with a gadget to place the UI in a coaching tab to make access to coaching easier and more personal.
  • Later a more dynamic floating UI may be developed above and to the side of the editor/browse window.
    • This would allow Simone to inspect edits in real-time as well as show videos to user's while they edit.

More so when it comes to web-based applications and even more so in the context of extending MediaWiki. MediaWiki is years behind normative content management systems, which provide simple UI widgets and an easy extension mechanism. The WMF claims to be open source, and constantly solicits help from volunteer developers. However in recent years the practice has become increasingly less open, there is little trust in community submissions and therefore little help from volunteer developers. Recent discussion on wiki-techL has highlighted that it can take over two years to review 3rd party plugins -- long enough for them to become obsolete.

I have made some paper UI design but the Template above represent a second iteration of these ideas. While they could be vastly improved and streamlined this should be done within the scope of the project itself. As mentioned above I would propose short development cycles with testing using real against real users and getting feed back from active coaches.

Some levels of integration:

  1. Using a Message based U.I. and no integration.
  2. Minor MW and UI integration (gadget)
  3. Major MW integration (extension)

It would be expedient to integrate into Simone a capability for collection information using forms. This is part of a use case for collecting feedback in surveys. Another use case would be administering tests to verify a user's progress before a graduation event. If this is identified as important such a component will be integrated into the UI.

Simone's Invocation API[edit]

Simone will also have an refferal mode. It would be possible to introduce a referral template into the page by a reviewer to invoke Simone's presence and ask her to present certain advice to an individual at a page. The rational for this is to allow quick integration of Simone into tools like Huggle, Twinkle etc. This will in effect introduce positive alternative into these tools. This will have the effect of changing the standard tactics from total war to limited war in the good faith cases.

Scaling[edit]

The project can scale in four dimensions:

Technological scaling - data based sensors[edit]

Simone's information architecture is designed to grow organically with the MediaWiki community it advises. Technological scaling of this sort involves an increasing discrimination capability (more points in space); and using feedback to fine tune a sensor's activation threshold (e.g. gaugeing the notability of a Soap opera episode article is not the same task as an article on a war.

This type of scaling requires creating sensors that become more refined as the wiki grows.

Simone's response to the above strategy is to provide smarter data driven sensors. The data for these will come from sampling the wiki so they will be normative rather than proscribing what may become unrealistic utopian criteria. By considering more data and by clustering this data to more levels it is possible for the same basic sensor to track changing dynamics as wikis evolve over time. Both positive and negative feedback mechanism can then be used to adjust the behaviour signatures which it detects .

The sensor developer's goal is Pareto efficiency. With a robust detection of Context, Content, Notability, References , Style and Social behavioral signatures in user edit and talk Simone will to preempting 20% of the most socially damaging reversion/deletion events which should dramatically reduce community oversight by about 70-80%.

The handling of remaining incidents would require human intervention and provide greater engagement. If not then once such a milestone is achieved the method can be refined by adding sensors and refining existing sensors to track the top 20% of remaining issues. This is likely since deeper underlying problems (such as cultural and gender gap issues) may be obscured by more visible issues.

Sensors could also be added to address additional (non editorial) concerns so long as the phenomena is discernible using techniques of discourse analysis. This simplest should involve retention of experienced editors.

In cases where a rules and pattern are too weak to detect issues (Such as neutrality) it will be necessary to produce a collection of good/bad edits and train models to discriminate their issues. Building some "Gold standards" will be one of the project's mile stone since it will involve community expertise and feedback.

Also if such issues become a priority the community will be requested to assist by tagging problematic revisions. Though this can be accelerated by earmarking a budget to outsource corpus development by outsourcing the work online using Amazon mechanical Turk at a token cost per tagged revision. This approach has been reported to be successful in developing an anti spam corpus.

Some initial sensors that would be developed:

  • biting sensor - incidents of new user biting.
    • bite scale from 1-10.
    • biter's leader board to mitigate biting and identify delinquent bitters.
  • new user sensor
  • new editor sensor
  • new article sensor
  • afd sensor
  • speedy sensor
  • article information sensor
    • determines quantitive the amount of information in this article.

Community Scaling – Advice[edit]

Sensing problems is not enough – Simone needs to offer useful advice. Since this will be the most visible aspect of the project it needs to be developed with great sensitivity to users and community acceptance. Advice defaults will provided but this will only be a starting point. Again community input will be requested on this subject. Suppose the community solicits our projects with 3 advice notices for a certain issue - say lack of Citations. Simone would systematical test these messages and score them based on their success (i.e. check how each advice correlates with the sensor score for Citations in following edits.) Tracking advice - and correlating using R in a DB will be another Project milestone.

Sample Advice for Context Issue
Probability Advice
0.5 Write another 120 words
0.25 Add an info box template
0.0625 Add a category.
0.0625 Add the article skeleton using section titles.
0.0625 Add categories.
0.03125 Add project description in talk page.
0.03125 Add project stub.
0.03125 Add an interview link to a version in another language.
  • Advice is offered in the scope of missions which are grouped under a single goal.
  • Goals are collected in strategies.
Sample Strategy for New Users
Sensor Definition Goal Mission
S1 Threshold Provide Context
S2 Threshold Provide Content
S3 Stub Provide Content
S4 Stub Notability
S5 Stub Add project description in talk page.

Deployment and Scaling on Language[edit]

To speed Simone's' community acceptance, default advice will be provided to the initial project by the developer. This is perhaps the most labour intensive task -- primarily because Simone would like to support new user in their first thousand edits to bring them into relatedness with other Wikipedians. To provide meaningful and engaging advice different editing issues would have to be explained in text and video.

Once Simone has stabilised through three or four initial development cycles it will be ready for its initial deployment. However Simone won't need all this media in order to go online. IT will co-operate with talented coaches to develop more advice for new situation. It will be done by issuing a "Request for Advice" - a crowd sourcing request to the community. This is a strategy to increase the number of stake holders involved in the project and who wish to see it succeed. The best scoring advice will periodically supplant the default advice and undergo translation. For example:

  • a sandbox based tutorial
  • a in-depth study on making references

Since the initial default advice will be in English, these will be Internationalized by translating using the resources of

TranslateWiki. This would involve:

  • Making its sensors cross lingual.
  • Translating its templates
  • Translating its advice.

I have consulted with the WMF developer of TranslateWiki which handles the Internationalization and localization of MediaWiki and he recommended for the translation of the templates:

Scaling on Projects[edit]

For example Wiktionary has quite different editorial guide lines from Wikipedia as do wiki source and Wikibooks. Scaling Simone for work on new projects requires:

  1. determining a socio-editorial policy using the deletion/reversion history and deriving concrete list of violation.
  2. sending out a Request For Advice to the project's community.
  3. beta testing the behaviour on a control group.

Thus Each project could adapt Simone to its needs.

Metrics[edit]

As already explained metrics are at the very heart of this projects - we call the sensors.

  • Simone will generate detailed logs of its activity.
  • Reporting will be generated from the logs.
  • Simone will be configured initially to have a control group which will not get the benefits of coaching.

Simone will provide periodic reports with summary statistics on

  1. incident (selected sensors correlating to high level events).
    1. new user biting
    2. speedy nominations
    3. article improvement from C to B and other levels
    4. user projected activity based on recent edit history[note 18]
  2. incident response
    1. new user biting interventions.
    2. speedy interventions.
    3. user coaching enrollment.
    4. editor's leaving coaching.
  3. long-term change.
    1. new editor acclimatization to the community.
    2. editor under coaching mass action. The overall edit count * positive change due to these as seen by sensors.
    3. morbidity rates estimates for editors under coaching and the control group.[note 19]

Long term - Sustainability[edit]

No claims about the long-term of the project only suggestions. I hope it is taken over by the WMF development team and more tightly integrated into MediaWiki whose shortcomings it addresses. However, since it is not the traditional model for robot based tools, the more realistic community process will be explained in the different roles of different stake holder involved.

Role of Developers[edit]

Today Bots developers are the most influential members of the community. Within the limited scope of their projects they provide more agile solutions than the WMF. They are also better aligned to the community's needs because not only must they get approval to run their bots from committee, they must then respond to issues raised by users.

Simone will be created in short development cycles as a MediaWiki based project - using git for source control, labs for test deployment, Bugzilla for bugs, code review for reviewing external patches once these are provided, translate wiki for internationalization.

Bots are mostly constructed from a few exiting frameworks and thus there exist a possibility of a high mobility within the community if a better technology is introduced. It some areas, such as ranking articles for completion and importance, Simone would provide a significant technological edge. This suggests that there is potential for pooling resources with exiting robot operators.

She will be deployed and start collecting data early on and generating reports. Her nature is planed to be extension. This means that the community will be given freedom to contribute to this project.

To promote Simone amongst developers:

  1. Her code will be available as open source.
  2. Data analysis scripts will be provided.
  3. Documentation will be provided on installation.
  4. Evangelism at Wikimedia, and Hackathons and other conferences would boost this community.
  5. After the end of the fellowship the volunteers will be found to take over and maintain the project.

Role of Administrators[edit]

The stake of administrators in running Simone - is that she will oversee editing in a the grey area between spam and good edits. This will provide a massive reduction in patrolling.

After undergoing the robot approval process Simone will be operated on Wikipedia. As usage will increase responsibility to operate it will be transferred to qualified Administrators.

Simone would be primarily controlled by a configuration files. If required a web-based interface could be added too based on edit filter. Administrator will:

  • fine tuning its sensors and their thresholds.
  • adjusting corpuses for normative standards
  • add remove or change strategies
  • add remove or change defining goals, mission, and advice hierarchies.

Role of Policy Makers[edit]

There is a number of suggestions proposed to policy in this proposal. These changes will be performed informally by Simone. As well-tempered users invade the community these policies will gain weight and gradually replace the existing practices.

A key assumption is that as Simone influence will be felt by policy makers. In this view when they decide to repel anti-social policies Simone will be reconfigured to fill in the resulting vacuum. This will allow to reform current policy to a shorter and clearer set of guidelines — more suitable for a larger more diverse society of contributors.

Role of Human Coaches & Domain Experts[edit]

Human Coaches will be asked to provide expert advice on coaching and help for debugging based on the open source doctrine of many eyes find bugs faster. Thiers stake in Simone is that it will make coaching more fun and productive. For humans the tasks will be less technological and more of a social activity.

Using a project page they will be asked to provide original and alternative texts for advice. They will be invited to suggest new missions, goals, strategies and sensors for current development cycles. Once these suggestions are provided they would be added into Simone as part of the next development cycle

These will be use by developers and administrators.

Domain experts can be paired with more seasoned editors to work as pairs. One contributing domain knowledge, the other providing wiki know how. This is another situation of changing the game from zero-sum to win-win rule.

Role of Projects within a MediaWiki instance[edit]

Specific project in Wikipedia such as biography of living persons have their own needs. Simone will be able to address these projects as a group, recommend new users to join the projects teach people the rules required in these and arrange collaboration on top priority articles. This coordination will allow a participating project to get boost in personal and to focus them on work in teams on the articles of greatest importance. This is something of a lost capability now that matching articles to individuals. Simone's user Soliciting module could be fine tuned from the start to training users in their areas of expertise in a socially acceptable way.

Role of The Researchers[edit]

Simone will provide reports on its activities. Its high level strategies would have better results of based around research carried by experts in this field. By providing better answers to the key theoretical questions it will be possible to define better goals and sensors and preemption strategies for Simone to pursue.

For example if there is more knowledge on conflict situation Simone could be instructed to preempt conflict [note 20]; existing user retention.

However, if researcher were to discover a better model of the editing and review game or a more suitable incentive paradigm these would be used by Simone to coach better editors for tommorow.[note 21]

  • life span in days/edits of editor generations.
  • percentage of new articles deleted under each SD clauses.
  • percentage of new articles deleted under other clauses.
  • degree of communication before reversion event.
  • time from first edit to becoming an administrator.
  • persistence of unregistered editors

Community Role[edit]

The most important role for Simone is that of community interaction. As users work with her she will become wiser and able to give better advice. Maintenance of advice as Wikis' policy changes is one major role of the community.

Footnotes[edit]

  1. this and other avatars used for Simone are based on Wikipe category are taken from commons and were contributed by their respective creators.
  2. This is easiest to correct as it involves less than 100 developers who are well-coordinated using multiple communication networks - the barriers however are design and cost
  3. These are much harder and slower to change as these involve about two to twenty thousands active community members every month and a process that relies distributed process that is unpublicised and is consensus based.
  4. Social problems exist at a level where there is too much friction and too small opportunities for communication to effect change by individuals.
  5. Value in the form of new or improved articles is immediately available for all. An increase in community size affects the community members indirectly though allocation of tasks and resources. New editors do not perform much communal tasks. Because each member only does as much work as they wish to they can personally avoid the consequence of a reduced population. The brunt of the cost is paid by the community.
  6. a nieche wiki dedicated to covering a specific knowledge domain or an alternate community lacking the social complexity required for editing Wikipedia.
  7. three routes to altruism are
    • Kin Selection - individuals whose interest are closely related, whose cooperation brings them even closer based on mutual interests.
    • Direct reciprocity - trading favors in paired relationships within a multi-round games with a capability for individuals to punishing a defector.
    • Indirect reciprocity - multi rounds under anonymous conditions.
  8. a b protected editing under a view that the editorial problems will be corrected by editor as they gains experience. Paying forward means that the cost of the learning curve will be paid later to future editors. Pay it back means making they new editors pay for their mistakes like reviewers had to
  9. this protocol operates by
    • reducing information asymmetry
    • adjusting personal utility to favour cooperate rather than delete/defect
  10. Without research on the subject, my own informal tests indicate that coordination is the greatest stumbling block to cooperation. Followed by communication.
  11. Conflict of interest, Arbitration
  12. No Original Research
  13. One has to develop a user page, learn to respond to policy challenges laced with Jargon in like kind and develop an edit count.
  14. where by cost I mean the amount of work time and learning required to successfully contribute an article
  15. it is often explained as straining the MediaWiki software. But Media Wiki only store a limited history in the database - so this is in realty only are patrolling problem
  16. using an api; a tool in the toolbox or talk page templates
  17. autamated quality assessment can be done faster and at a higher level of objectivity by machine. Importance is contextual and thus less suited for machine assessment. But capturing both it is possible to make use cross validation in a repeated article rating game in which editors get a reputation by self select though accurate ranking in a knowledge domain. Accordingly their votes on importance are boosted or discounted accordingly.
  18. inactivity would be statistical estimate of the users expected future contribution base on a poisson distribution with the user's historical
  19. since Simone is expected to affect the community as a whole change in the control group may be significant
  20. I omitted the details in deference to Dispute resolution
  21. a list of same game theoretic motivated questions on wiki editing behaviour together with raw Simon data would be provided if the project is funded.

References[edit]

  1. A Group Is Its Own Worst Enemy by Clay Shirky
  2. The Open Society and Its Enemies, Vol 1 & Vol 2, P.161 Karl Popper
  3. Where Wikipedia has gone wrong? what we can do to bring it back on track? by Dror Kamir Wikimania 2011 Haifa
  4. Participation in Wikipedia's Article Deletion Processes by R. Stuart Geiger and Heather Ford
  5. a b The Tyranny of Structurelessness by Jo Freeman (1970).
  6. Gustave Flaubert Laughs at Wikipedia Alan Shapiro Critical Point of View Amsterdam
  7. http://meta.wikimedia.org/wiki/Research:New_Page_Patrol_survey/WMF_report WMF New Page Patrol survey
  8. Super Crunchers - Ian Ayres Random House
  9. The Evolution of Cooperation - Axelrod, Robert Basic Books, 1984 0-465-02121-2
  10. The Logic of Animal Conflict - J. Maynard Smith, G.R. Price Nature 246 November 1973
  11. Understanding Media: The Extensions of ManMarshall McLuhan McGraw Hill, NY, 1964; ISBN 1-58423-073-8.
  12. a b Lost Functionalities
  13. Wikipedia reconsidered - Everything You Know About Wikipedia Is Wrong by Mindspillage, Fluffernutter, Ironholds, Kim Bruning, James F. and Tom Morris. Wikimania 2011 Haifa
  14. E. Goldman. Wikipedia’s labor squeeze and its consequences. Journal of Telecommunications and High Technology Law, 8, 2009.

Submitted by[edit]

Oren (talk) 13:43, 3 March 2012 (UTC)

Endorsements[edit]

This section is for endorsements by Wikimedia community volunteers. Please note that this is not a debate, vote, or poll, but is rather a space for volunteers to describe in detail why they think a project idea is of value. If you have concerns or questions rather than an endorsement to make, please use the idea Talk page. Endorsements by volunteers willing to work in collaboration with a fellowship recipient on a project are highly encouraged.

Support Support, I believe implementing this idea is very helpful for giving a smooth start to new wiki editors, and for saving expert editors' time spent on coaching new editors. --Ciphers (talk) 05:12, 21 March 2012 (UTC)