Research talk:Understanding Wikipedia Coordination Spaces and Costs

Something you may have missed

Though there is little information in the body of this proposal I thought I'd comment. I'm interested in the context of a game theoretic investigation of Wikipedia editing. I'm interested in estimating costs of coordination and communication. I've so far come up with a number of question related to the cost of coordination.

You seem to be looking at primarily at coordination for deletion of articles. There are two spaces which would shed more light on this subject. The first is the coordination of tasks during of editing, use of inline tags, page tags, article talk, user spaces and so on. A second are is the development of policy with quite different challenges. OrenBochman (talk) 14:24, 23 April 2012 (UTC)

Hi Oren,

Thanks for taking the time to leave a comment. I just saw this now (I don't get email notifications though I'm watching this

page). Your proposal looks interesting.

It's good to get suggestions, so thanks for that. Previously I did some research using article Talk pages. It's written up in a few of my papers, mainly :summarized in “Understanding and Improving Wikipedia Article Discussion Spaces.”. Definitely the research :on policy, editing templates, and user spaces is relevant too! So far I'm reading others' papers on these but not looking at them in detail. I definitely think :that the approaches I'm taking would be relevant for policy research.

Jodi.a.schneider (talk) 21:25, 28 April 2012 (UTC)

Introduction of self

Hello! I talked to Dario and it looks like it will make sense for me to be your contact for this project. I'll be posting some notes shortly. --EpochFail (talk) 20:00, 4 May 2012 (UTC)

Research Questions

Would you add your research questions to the project page? That will help viewers understand your needs an concerns. --EpochFail (talk) 20:02, 4 May 2012 (UTC)

Here's what I've added regarding the purpose/RQ's of the interview research:

http://meta.wikimedia.org/w/index.php?title=Research%3AUnderstanding_Wikipedia_Coordination_Spaces_and_Costs&diff=3727153&oldid=3714151 Jodi.a.schneider (talk) 11:02, 8 May 2012 (UTC)

Looks good to me. --EpochFail (talk) 13:27, 9 May 2012 (UTC)

Emailing users instead of talk posts

This sounds reasonable to me. RCom has a history of advising against this approach since it is difficult for Wikipedians to track, but considering that you are planning to get in contact with only 10 users, I imagine this to be much less of a burden. Can you post an example message that you'll be sending? --EpochFail (talk) 20:11, 4 May 2012 (UTC)

Out of curiosity: What do you mean, "difficult for Wikipedians to track"? Difficult to know who has been contacted? Or...? Jodi.a.schneider (talk) 11:03, 8 May 2012 (UTC)

Here is a past message that I have used, from my last round of interviews

> I'm looking for Wikipedians to interview about the deletion process. I

> envision a 30 minute skype or phone conversation. I'm interested in

> learning about what works well in AfD discussions, any frustrations you

> have with it, and why you generally do or don't !vote in AfD.

>

> I hope to talk with Wikipedians with a wide variety of experience

> editing (from newcomers to EN-WP, to regular EN-WP editors, to admins,

> especially admins who close discussions), with people who spend little

> time commenting in deletion discussions, as well as those who do.

>

> Would you be willing to talk with me? Let me know the best times for

> you; you can reach me at jschneider at pobox.com or with the info

> below.

Jodi.a.schneider (talk) 11:08, 8 May 2012 (UTC)

Email messages are difficult to track because only administrators have access to the Special:EmailUser logs while many processes have been built around tracking edits to pages (including edits to talk pages) (e.g. en:Wikipedia:Huggle). In other words, when you edit a user_talk page, the community can observe your activity much more easily than when you send an email.

Ok. That's what I thought you might mean. Particularly for small studies, that ease of observability is a threat to the anonymity of participants. So other researchers have deliberately avoided starting Talk page conversations with participants for that reason. Jodi.a.schneider (talk) 04:01, 9 May 2012 (UTC)

I'd recommend that, since you'll be emailing users directly, you alter the above message to address the individual and use the opportunity to immediately identify yourself and your institution as well as link to your project proposal here. --EpochFail (talk) 22:08, 8 May 2012 (UTC)

Yes, more concretely, this time around, I'm planning:

Subject line: Help improve the Wikipedia deletion process

My name is Jodi Schneider, I'm a fellow Wikipedian, as well as a researcher at the National University of Ireland. I'm studying Wikipedia's deletion discussions and am contacting you because of your participation in the Articles for Deletion process. My work aims to support decision-making and identify opportunities for helping new community members. Your opinions and experience would be very valuable to me in this work.

I'm looking for Wikipedians to interview about the deletion process. I envision a 30 minute skype or phone conversation. I'm interested in learning about your experiences with Wikipedia, and especially article deletion. You can read more about my project at

http://meta.wikimedia.org/wiki/Research:Understanding_Wikipedia_Coordination_Spaces_and_Costs

I hope to talk with people with a wide variety of experience editing Wikipedia.

Would you be willing to talk with me? Let me know the best times for you; you can reach me at jschneider@pobox.com or with the info below.

Best,

Jodi

WP:Jodi.a.schneider

skype:jodi.a.schneider

http://jodischneider.com/jodi.html

Jodi.a.schneider (talk) 05:21, 9 May 2012 (UTC)

I only have one very minor (nit picky) suggestion. I worry that the subject "Help improve the Wikipedia deletion process" is too general. Would you mind including something in the subject to suggest this is a study and that you'd like to perform an interview? (e.g. "Interview about Wikipedia deletion process" or "Study of Wikipedia deletion process") I suspect that many users would appreciate knowing that the email is a solicitation for research before opening it. Otherwise, this looks to be ready for the rest of RCom. I'll ping them to see if we can close this review. --EpochFail (talk) 13:22, 9 May 2012 (UTC)

Sure, how about "Interview to help improve Wikipedia's deletion process"? Otherwise, "Interview about Wikipedia deletion process" is fine -- but it doesn't make it 100% clear why somebody might want to participate. Thanks also for the email with samples of privacy policy. I'll take a closer look soon, that's quite useful! Jodi.a.schneider (talk) 13:25, 9 May 2012 (UTC)

IRB or similar approval

Does your institution have a review board that you have to acquire approval from in order to perform human subject research? If so, please post details. --EpochFail (talk) 20:11, 4 May 2012 (UTC)

Not for this sort of research. Very limited human subject research occurs in my department (we're a computer science research institute), and I have been advised that the university IRB primarily handles medical research. Jodi.a.schneider (talk) 21:03, 4 May 2012 (UTC)

Consent form / Privacy policy

Will you be presenting potential subjects with a consent form or a description of how information they share will be treated and kept private or made public? This is highly recommended. And I'm hoping you can link to the form/policy from this project description. --EpochFail (talk) 20:11, 4 May 2012 (UTC)

Yes, of course. Is there an example form from previous researchers to base this on? Jodi.a.schneider (talk) 21:24, 4 May 2012 (UTC)

Note, gentle future reader, that Jodi and I covered this via email. Some of the stuff that was shared was copyrighted.  :\ --EpochFail (talk) 13:25, 9 May 2012 (UTC)

Links to samples that are online:

Grouplens example consent form. Jodi.a.schneider (talk) 11:45, 10 May 2012 (UTC)

Research protocols

Research protocols may also be helpful. Examples from:

• Stuart Geiger
• Aaron Halfaker

Previous contact

Last year, for an earlier iteration of this project, I interviewed community members, soliciting interviewees via the Village pump and listservs (WikiEN-l, Wikimedia IE, and wiki-research-l).

I explained the project to interviewees, as well as that the information they provided was intended for my understanding and to be reported in my research papers, confidentially and anonymized. I later asked if interviewees would be willing to give feedback as my research progressed; I plan to follow up with these people, as well as with others who had earlier volunteered to be interviewed. Now that my project has advanced, however, I also require feedback from novice participants in AfDs. Jodi.a.schneider (talk) 21:24, 4 May 2012 (UTC)

Our consent form

The approved version of our consent form Jodi.a.schneider (talk) 18:55, 19 May 2012 (UTC)

Related research

Hey, the methodology looks sound and respectful of community time and energy here. I did want to point out that Maryana and I at the Foundation have done some very similar research about AFD and PROD notice effectiveness, which includes several interviews with new editors where they talk about the various processes. Steven Walling (WMF) • talk 18:20, 9 May 2012 (UTC)

That's really interesting, Steve! Thanks for the pointer, and for the feedback! :) I'll get in touch to see if there's anything you can share from the interviews. Jodi.a.schneider (talk) 12:01, 10 May 2012 (UTC)

Coordination and Communication Costs

Hi, Thanks for the help. I have made some significant progress with the articles you sent me. I have started developing TronBot on Wikipedia to try to provide a tech-discount for article creators at for Afd/Csd cost. I am also considering incorporate the statistics on CSD into the UI. However I have made some progress conceptually and would like to share this with you.

• Do you have background in Mathematics/Game Theory?

I am interested in estimating Coordination and Communication Costs. My interest naturally stems from the game theoretic analysis of Wikipedia and editing. These estimates are important for a few reasons.

• They can establish more accurate preferences - as reflected by player utilities (preferences) and subsequently their payoffs.
• More practically they will help to design better incentives for fostering cooperation.

I'd like to know how you (and perhaps others) define what you consider as Communication and Coordination. I'd like to share my own defintions at my overlong page at Meta. - You might want to have a look at 3.5.2.3 Coordination at my user page OrenBochman (talk) 10:13, 10 May 2012 (UTC)

Hi Oren, good to see you've started on TronBot. I don't really understand what you mean by a tech-discount, though I guess maybe it's a summary of the statistics? Or some aim at reducing coordination costs? Yes, I have a math background and know a little bit about game theory. One really important paper to read, from a coordination/cooperation perspective, is Travis Kriplean, Ivan Beschastnikh, David W. McDonald, and Scott A. Golder. 2007. Community, consensus, coercion, control: CS*W or how policy mediates mass participation. In Proceedings of the 2007 international ACM conference on Supporting group work (GROUP '07). http://dx.doi.org/10.1145/1316624.1316648 :

CSCW researchers have long understood that articulation work is critical for enabling collaboration [15, 16]. We claim that the Wikipedian community is strengthened by the difficult work of seeking consensus and that such work is perhaps the most fundamental articulation work done within Wikipedia. Yet, some researchers still consider these aspects of collaborative work “costly” or even “unproductive” (c.f., [9]). In the early 1980’s, McGrath [13] challenged the common psychological notions of “process loss” in group tasks, pointing out that without social and interpersonal support, groups often disintegrate. Clearly, some elements of the coordination overhead can be mitigated. However, a mitigation technology must understand what function the overhead serves and be careful to support it. Simply eliminating the “overhead” can be devastating.

I think it's a little hard to bridge the definitions of cooperation from game theory with those from HCI communities like CHI, GROUP, and CSCW. If you can do this, though, it will certainly be useful! I've taken a brief look at 3.5.2.3 Coordination on your user page and will reply about those parts on your Talk page there. Jodi.a.schneider (talk) 10:50, 10 May 2012 (UTC) (see this diff for those comments--just so they're easier to find later. Jodi.a.schneider (talk) 11:30, 10 May 2012 (UTC))

Thanks for the info. I've also reposed on my talk page and updated the Communication & coordination at my user space.

I've looked over the paper listed above. It has mentioned some power plays which I have incorporated into my wars of attrition section. I am aslos going over other CSCW papers to see if there is a more rigorous treatment of Cordination and Communication.

the tech-discount mentioned in TronBot is an attempt at mitigating conflict over article quality using an objective norms derived from featured and good articles. OrenBochman (talk) 11:35, 11 May 2012 (UTC)

RFC: Mark this project "reviewed"

I've started a discussion on the RCom mailing list asking for concerns to be raised by next Monday, May 14th. If nothing is brought up there or here by that point, I'll mark this project reviewed and advise Jodi et al to continue with the project. Please voice your support or opposition below. --EpochFail (talk) 20:19, 10 May 2012 (UTC)

• support --EpochFail (talk) 20:19, 10 May 2012 (UTC)
• support -- this researcher has already provided invaluble information on Afd. I'd be glad to see her work continue. OrenBochman (talk) 13:53, 11 May 2012 (UTC)

Given the support here and on the mailing list, I'm closing this review and advising that Jodi proceed the planned recruitment of research participants. Thanks for your participation, folks! --EpochFail (talk) 20:55, 18 May 2012 (UTC)

Counting AfD votes & close decisions

en:User:Snottywong has some tools for AfD. Jodi.a.schneider (talk) 10:15, 28 May 2012 (UTC)