Research:Codex/Methodology

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search
Correct.svg This page is currently a draft. More information pertaining to this may be available on the talk page.

Translation admins: Normally, drafts should not be marked for translation.


Scope[edit]

  • Anything that may provide insights about how to build, nurture and grow successful online communities who produce and curate free educational resources.

Differences with traditional literature reviews[edit]

Vested interest[edit]

Contrary to scholars, who may strive to remain neutral towards the subject of their research, Wikimedia communities have a vested interest in understanding how they function, in order to survive, thrive, and work towards the shared vision of the movement. Having an interest in a specific outcome doesn't mean abandoning the scientific method; however, it may play a role in prioritizing certain research questions over others, if they may provide insights more relevant to the communities' interests, even though they may not be as interesting or novel from a scholarly standpoint.

Acknowledging this vested interest is also critical in identifying and mitigating biases and prejudice. This is particularly important since this document is intended for a broad audience composed not only of academics, but also of practitioners.

Audience[edit]

Academic literature reviews are usually written for the benefit of other researchers, notably newcomers to the field. Depending on the subject of research, literature reviews may also be written with practitioners in mind, but often as merely a positive side-effect.

This initiative aims to benefit practitioners first, and researchers second. Therefore, a particular effort is made to use jargonless language as much as possible. Some complex topics obviously require specialized knowledge and vocabulary; such concepts are explained in simple terms if possible, and refer to relevant Wikipedia articles for further reading. The Codex also includes a glossary of common terms used by academics in the relevant fields of research.

Inclusion criteria[edit]

Authors of traditional literature reviews define and specific protocols to determine the extent of their work. For example, systematic reviews may select all publications matching a specific keyword in a given citation database for a specific period.

Reviewers outline inclusion criteria to make sure their review is comprehensive, but also to restrict the scope due to practical concerns: they need to be able to review their selection in a reasonable time frame with limited resources. They also often restrict their selection to peer-reviewed research, to follow academic standards.

This initiative doesn't have the same restrictions. Here we're interested in anything that may provide insights about how Wikimedia communities function and how to support them. This may include scientific articles published in peer-reviewed journals, but also books, conference proceedings, blog posts, or research reports. The document may mention Wikipedia explicitly, but it may also be more generally about online communities, social psychology or technology.

Also, our effort here doesn't aim for exhaustiveness. Some research areas related to Wikimedia sites, while worthwhile scholar endeavors, don't provide insights relevant to our interests. This notably includes all the research that merely uses Wikipedia's content as a corpus or data source, to answer research questions unrelated to online educational communities.

Living reference[edit]

Traditional literature reviews habitually result in a published artifact, usually an article in a peer-reviewed journal. Such publications are valuable, but as snapshots in time, they're also inherently out-of-date before they're even published. This drawback may be mitigated by considering successive literature reviews, if they exist.

The goal of this Codex is to be a living, up-to-date reference to the state of knowledge about online educational communities. When recent research provides new insights about a particular topic, updating the Codex will be as simple as editing the relevant wiki page.

Limitations[edit]

Timeline[edit]

The quantity of research to be potentially reviewed as part of this effort requires an important time investment. As an first step, work is currently (as of early 2016) focusing on topics of higher importance to the Wikimedia Foundation (contributor roles and system dynamics). Many stub pages have also been created on other topics, to make it easier for casual contributors to participate.

Specialized knowledge[edit]

The broad scope of this project makes it difficult for its contributors to be familiar with all the disciplines, theories, and methods involved in the multitude of cited publications. Therefore, there is a risk that some research be misunderstood.

The open, collaborative and iterative nature of wikis may alleviate this issue by encouraging other contributors (who might be more familiar with a specific topic) to improve on the original text. Calls for collaborative proofreading may be made on the wiki-research mailing list, to invite other researchers to edit the pages directly, or use talk pages to discuss misinterpretations and approximations.

Guidelines[edit]

  • A topic page should only include content that has been previously published, whether it's in a peer-reviewed journal, a conference paper or on a wiki page. Exceptions must be argued on the associated talk page.
  • A topic page must be a direct subpage of Research:Codex.
  • A topic page must include a summary section called "Summary", a section listing research questions called "Research questions", and a section listing open research questions called "Open questions". This allows for easy section transclusion.
  • A topic page may include a section called "Current research" listing related ongoing research efforts, and a section called "Sources to add" listing references that have not yet been included in the text.
  • A topic page should include a references section called "References".
  • Statements on a topic page should be dated as often as possible; e.g. "Based on data from 2004–2009…" or "As of March 2011…".
  • Statements on a topic page must not extrapolate from the results they cite. Statements should use credibility indicators to indicate the trustworthiness of the claim; e.g. "It is likely that…", "It is possible that…", "It is argued that …".
  • References should use a standard citation template, and should include a link to the WikiPapers page, Acawiki page, or Newsletter review for the resource (using {{Wikipapers}} or {{Acawiki}}), if they exist.
  • The primary link of a reference should point to the full text of the source, if publicly available.
  • A Codex page must include the {{Research codex}} header template, the {{Research codex navigation}} navigation footer template, and be placed in the Research Codex category.
  • Sections and templates should be placed in the order outlined on the page template.
  • A topic page is considered a Codex page. Codex meta pages (e.g. Background, Methodology, Glossary, and Overview pages) are not considered "topic pages".
  • These guidelines may change over time.