Talk:Wiki Scholarly Journals

Maybe this is an idea already discussed in "wiki space" already. But I have "only" been a contributor to wikipedia (english) for a bit over a year. I also happen to be a university professor.

The concept of peer review [1] is a foundational one in scholarly journals -- in which contributions are "reviewed" by external referees prior to publication. There are a few electronic journals that have been established which use a formal peer review process prior to "publication" in electronic media.

Let me analogize (I apologize in advance, I am an engineering professor not a computer scientist). My understanding of the concept of open source software, and also to some extent of the wikipedia and other wiki foundation projects is that they rely on the concept that many "eyeballs" improve the product (the converse of "too many cooks spoil the broth"). Suppose a WikiJournals project were organized in which scholarly articles (with appropriate footnoting, etc) were downloaded and subject to the "review" of the wiki-universe. I am impressed at the relative lack of errors in the wikipedia articles that are in areas I am highly knowledable about.

Peer reviews in scholarly journals are conducted by selected reviewers known to an editor. Could it not be that a WikiJournal would attract a community of knowledgeable contributors and reviewers (and unlike traditional journals -- either electronic or print -- the nature of Wiki permits intensive dialoging (and very import to me -- it seems -- archiving and potential for redaction of changes). My observation of the dynamics in wikipedia is that except for politically/culturally charged topics, after a while the edits do die off.

Of course such a project would require some "early adopter" volunteers amongst the scholarly community.

I look forward to comments.

--Professor Water 22:48, 5 Jul 2004 (UTC)

---

Personal opinion: I think there will be problems with the "ownership" of articles. Editing an encyclopedia article which is supposed to be general is different to editing an article where someone else put his personal style and research results in it. How do you think a scholarly journal could benefit from using a wiki? Do you know http://plos.org/ ? A german wikipedian had the idea of creating a wiki for abstracts (and translation of abstracts) of scholarly articles, I gave him a pointer to this page here. Maybe this could be also discussed in the context of the proposed Wikiversity and Wikiresearch. --Elian 17:52, 6 Jul 2004 (UTC)

In review articles there is less personal style. A scholarly journal with review articles and/or abstracts could benefit a lot from using a wiki but I doubt that you can list your wiki edits in your publication list :-( on the other hand Wikipedia does have a huge w:impact factor ;-) By the way I really think of funding a journal of our own within the next years (some kind of WikiReader WikiResearch) -- Nichtich 21:08, 6 Jul 2004 (UTC)

---

See http://www.livingreviews.org/. These journals contain peer reviewed, open access and regularly updated review-articles. It's only a little step further to do it with a Wiki. Are you thinking of using a Wiki for the internal process of peer review or publically available? -- Nichtich 21:08, 6 Jul 2004 (UTC)

---

WikiAbstracts could also be started with a couple of motivated (PhD) students. WikiReviews is nothing but excellent Wikipedia articles with a full bibliography treating research topics. -- Nichtich 21:27, 6 Jul 2004 (UTC)

---

Thanks for the above comments. A few reactions -- 1) http://plos.org/ is a "classical" electronic journal (see also for example British Medical Journal -- http://bmj.bmjjournals.com/ ). In this model, papers are still submitted, reviewers are selected by an editor, and a decision is rendered. This relies in the editor being able to select knowledgeable reviewers and also in the editor him/her self being unbiased. A WikiScholarlyJournal would open up review to the wikiuniverse. Would this work -- the empirical evidence from w:Wikipedia seems to be that "on average" open reviewing leads to a good product. Thanks for the link to http://www.livingreviews.org/, however it is a bit similar to plos, although the articles appear to be envisioned as somewhat more dynamic, although the review process is still a static one.

2) It might be that a WikiReviews would be the way to test the concept out -- open, NPOV reviewed articles. The "History" function of the wiki seems to be one that is amenable to citation for the primary author and contributors to review processes of articles. As an academic myself, this is not something that I would ever advise a junior (nontenured) faculty member to engage in, however it might lead to a fundamental creation of a new vehicle for dissemination and evaluation of knowledge.

3) There is certainly some intersection between my suggestion and Wikiresearch that is worth exploring. --Professor Water 00:30, 7 Jul 2004 (UTC)

Sorry for the 10+ years delay for me to find this page. There is now a project in Wikiversity which works as a open-access journal where works undergo peer review: Wikijournal. Mikael Häggström (talk) 11:33, 16 August 2016 (UTC)

Wikiresearch << >> Wiki Scholarly Journals

I think this is sort of what I meant with Wikiresearch :) Let's merge, okay? See also this thingy about Wikiresearch. Only, to think of it is as a journal is kind of redundant, I think, wikis are more or less timeless. Akagu 20:58, 8 Jul 2004 (UTC)

Akagu 20:58, 8 Jul 2004 (UTC)

---

xx

It seems to me that Internet Think Tank should also be merged into the same project as well. Why limit WikiResearch or Wiki Scholarly Journals to scientific research, when it can include research about politics and anything else? - TalkHard, 6 Aug 2004

This seems to be a hard science vs. soft science question -- Nichtich 09:49, 7 Aug 2004 (UTC)

---

All of the above have a lot of similarity with an idea that I've been toying with over the past year. I figure it doesn't hurt to expand on the idea here, since it can always be edited out later if it's of no use.

The focus of the site I had in mind was to present specialist information (both original, and historical) which I define as that which requires significant understanding of prior knowledge. For instance, it is not expected that you could understand non-linear control systems, without a thorough understanding of calculus, or linear control systems for example. This differs from the wikipedia, where each article can generally be read and understood by itself.

Because the information on the site is specialised, there are two main uses I could think of. There is the open-learning area, which I think is probably adequately covered by a number of other sites, and the research aid which has relatively little competition. As a researcher myself, I find that there are numerous problems with existing research tools which could be overcome by this site. These include

Finding research in related areas: Currently, this involves finding the correct paper with a guess the keyword type search. Then once a paper is found, you chase up its citations and things that cite it. Many of these citations are only in related areas, and not of specific interest. Also, important papers can easily be missed if they involve the same sort of problem, but are in a different application, as there is not a lot of cross-disciplinary citation(even between very similar areas such as statistics and machine learning).

Understanding new papers: Papers on similar problems in different subject areas, or by people with different technical backgrounds are often difficult to understand because of different assumed knowledge, or notation, missing lines or unexplained working, or generally poor writing. Also, there is a lot of repetition of ideas within papers, and often a lot of these need to be read fully to work out what (if anything) is new to the paper.

Paper publication: Traditional paper publication is a slow process, requiring fairly extensive surveys of the literature (which are very time consuming), writing an introduction to the problem (which has often already been described better in other papers), repeating information from other papers (necessary for the paper to have a chance of being understood by readers), and then finally the main text of the paper. Also, writing the papers usually require style files which are different for each journal. Then the paper needs to run the gauntlet of reviewers, and the wait can be quite long.

The above problems could (eventually) be solved using a wiki, with main articles which can have edits suggested by everyone, and a tree like structure consisting of directed links. For instance, an article would have a special backwards link to other articles which need to be understood before the current article makes sense (the wiki could check that there aren't any loops). The articles could then be arranged in topics (not necessarily related to the nodes of the tree), and ranked by "popularity" to make it easy to find articles directly related to the area in which you are interested. Original research could be submitted either as a new article, or a "brain storming" article where people pool ideas until the problem is (hopefully) solved.

One of the more difficult problems faced by this type of wiki will be getting people to contribute. This is because most people who would contribute original research have jobs which reward the recognition provided by journal publications. This is, in fact, probably the only reason people publish in a journal, as opposed to web publishing. If this wiki somehow provides an equal level of recognition (I propose a karma like score which I will elaborate on later), then I don't see anybody having reason to publish anywhere else (although it might take quite a while for sufficient numbers of people to start using it).

Detailed description: This is a sort of page by page description of how I would see the site working. It is incomplete in some areas since certain problems/features only occurred to me as I was writing this, and I haven't had time to come up with a solution.

 FRONT PAGE: This would contain links to the main SUBJECT AREAS (such
 as Mathematics, Chemistry, Philosophy, Geology, Physics, etc.)  It
 would also allow someone to LOGIN to their homepage.

 SUBJECT AREAS: The name of the subject area would be at the top of the
 page, with a link to the REVIEW article for that area. Separate
 lists for subtopic SUBJECT AREAS, normal ARTICLES and BRAINSTORM
 articles should be present, and ranked by popularity (number of
 people subscribed).

 If the user is logged on, the user should only be shown articles
 that he has not ignored, and be given the option to ignore articles
 for next time, or to turn off the ignore option and see all of the
 articles. The ignore option will allow user to easily see new
 articles without having to wade through a list. Articles which the
 user has subscribed to should be made bold.

 Also, if the user is logged on, they should have the option of
 selecting three articles which belong in a new sub-topic, or to
 move an article to another subject (if the number of articles left
 is two or less, then the current sub-topic is folded back into its 
 original subject area), or to make a symbolic link from one area
 into another subtopic, etc.

 Finally, the user should have the option of adding a NEW DOCUMENT
 for review.

 LOGIN: This page would ask for certain details about the user
 (name, e-mail would be compulsory. Affiliation, country and
 state might also be useful. After entering this information,
 an account is created, and they are taken to the OPTIONS page.

 HOMEPAGE: This contains a prominent link to the OPTIONS page, as
 well as separate lists for alerts (significant changes to subscribed
 articles and new articles in subscribed topics), as well as
 subscribed articles and subscribed topics.

 OPTIONS: Contains various options for configuring the site. I haven't
 thought too much about what this might contain yet.

 REVIEW: The review article for a subject area should briefly describe
 the subject (or problem) and any sub-topics (with links to their
 SUBJECT AREAS). If the subject is at the top level (so no
 sub-topics.. something like corner detection in machine vision might
 be an example), it should give a brief review of all of the
 articles.  A list of all uncited articles in the subject area should
 appear automatically at the bottom of the review, to indicate that
 they should be mentioned in a later update. The review might also
 have links to the users and organisations with the largest "karma"
 in this area (although perhaps this will be more useful on the
 SUBJECT AREA page).

 ARTICLES: These are the informational pages of the site, discussing
 completed research. 

 Each article would start with a list of documents with backwards
 links (just normal links, but are specified differently in the wiki
 source, to imply that the documents are required to be understood
 before the current document can be). The links would only be to the
 highest level links, so for example if a "dynamics" article required
 "calculus" and "arithmetic", and "calculus" already required
 "arithmetic", then only a link to "calculus" would be included in
 the back links. Back links to articles that the user has not claimed
 they "understood" (see end of the ARTICLES description) should be
 displayed in bold case.

 Afterwards, there would be a link to the REVIEW page for the
 article, which would take the place of an introduction and/or
 problem definition in a printed journal. Following this would be the
 main text of the article. Links to similar articles may be included
 using normal links.  Proofs and other arguments should have symbols
 next to them (sort of like zooming in and out) which allows more or
 less proof detail to be presented, since if you are just skimming
 the article, seeing all of the detail may not be helpful. There
 could also be links to alternative proofs of the same theorem.

 Finally, at the end of the article, there should be a list of
 articles directly depending on this article (all articles with
 backwards links to this article). There should also be buttons which
 allow people to ask a NEW QUESTION about the article (sort of like a
 software bug report) or to suggest an EDIT to the paper (which can
 only be done after they have reviewed the existing edits, unless it
 is a minor correction). They could also ANSWER QUESTIONS, VOTE on
 suggested changes, or click on a text-box to indicate that they
 fully "understand" the article.

 EDIT: Allows the user to suggest an edit to an ARTICLE. Because the
 subject areas covered by the wiki are quite technical, it can be
 very easy for errors to be introduced which can invalidate the
 entire article (especially in mathematics). For this reason, while
 anyone can suggest changes, some sort of peer review is necessary
 before they can be implemented. This is accomplished by requiring
 a VOTE on new changes (unless they are of a minor nature, such as
 spelling, or rewording sentences).

 The first step is to ask the user whether the article is a 
 spelling/rewording change, or a more major change. The first would
 not need peer review, but would get little KARMA.

 The next step is the actual editing.  I'm only slightly familiar
 with the current wikipedia editing facility, so the existing editing
 tools may already be better than that suggested here. First, the
 existing text would be placed in a text-box, and allow the user to
 add editing marks surrounding the sections that need to be
 edited. When this is finished, the contiguous sections are displayed
 one at a time, with text-boxes underneath each to contain the
 suggested change as input by the user. Finally, the change is either
 implemented, or an edit page is created where it can be VOTEd on.

 VOTE: Voting on suggested changes to an article has two purposes.
 Firstly, it ensures that the changes have been vetted by an external
 source, so the changes are less likely to be flawed.  Secondly, it
 allows feedback on the quality of the change to determine how much
 karma the author should be given if the change is implemented.

 The vote page would 1) show the contiguous regions of the original
 text, followed by the suggested change. 2) show a list of comments
 related to the proposed change, and give the user the option of
 ADDing TO DISCUSSION. 3) A voting ballot box, where the user can select
 whether making the change is a step in the right direction, and to
 rate how important the change is to the entire article. After a
 certain time and/or percentage of subscribers have voted, the change
 is implemented according to the results of the vote.

 ADD TO DISCUSSION: Prompts for the text of the discussion.

 NEW QUESTION: Prompts for the text of the question to be asked.

 ANSWER QUESTIONS: If a change has been made to the document (or if the
 answer appears in a previous document the inquisitor should have
 read but didn't), then this information should be communicated to
 the person who asked. When going to the answer questions page, the
 user should either select a link to the EDIT page containing the
 answer to the question, or type in the number of the article
 containing the information. An alert is then sent to the questioner
 stating that the question has been answered. Clicking on this alert
 would send the user to the question page, along with the answer, and
 a checkbox should appear asking if the question had been adequately
 answered.  If the user clicks no, then a textbox appears where the
 questioner should explain what is wrong with the proposed
 answer. Otherwise, the question is removed.

 BRAINSTORM: In a brainstorm, a problem of some sort is defined, and
 people suggest ways for the problem to be solved. Sometimes people
 are only able to solve part of a problem, and so this work would be
 less likely to be published. This type of article makes it easier to
 publicly collaborate, and get recognition (or karma) for partial
 solutions.

 The brainstorm page would actually be a collection of linked pages,
 with threads. The first page would be the parent page. This would,
 at the top, have a notice if the brainstorm had been turned into an
 article (see BRAINSTORM TO ARTICLE) and be followed by a list of
 prior articles (articles with backward links) and description
 of the problem and possibly a partial solution (written by the user
 who started the brainstorm).  Directly beneath this would be a list
 of threads, with brief descriptions and links to the appropriate
 thread page (each of which would be very similar in format to the
 main page, but fewer buttons... for instance, the BRAINSTORM TO
 ARTICLE link might only be on the parent BRAINSTORM page). Each
 thread would correspond to a different approach to solving the
 problem. Here, there should also be two links. An ADD TO BRAINSTORM
 button, which allows the user to add a new thread, and a BRAINSTORM
 TO ARTICLE link, which gives the user the option of selecting nodes
 from the BRAINSTORM pages, which will be modified to turn into an
 article once the problem has been solved.  After this, follows a set
 of short notes and comments. At the bottom, a button should be
 provided to add a comment, and possibly checkboxes could be supplied
 next to each comment to indicate the usefulness of each comment
 (this would only be useful if the number of comments is very high,
 so this might not be necessary).

 ADD TO BRAINSTORM: Allows you to add a thread to the BRAINSTORM. This
 thread should be more substantial than just a comment, or a citation
 of some reference. When this page is selected, it should prompt for
 a short title, a short description, and then the main text of the
 article. At the bottom there should be two buttons. One just saves
 the thread, but the other saves the thread and automatically goes
 to the BRAINSTORM TO ARTICLE PAGE (in the case where the thread
 added completes the solution to the brainstorm problem).

BRAINSTORM TO ARTICLE: Once the BRAINSTORM problem has been solved,
 the solution (which may span multiple threads) needs to be
 consolidated into the easily readable form of an ARTICLE. The first
 step is to allow the user to choose the threads from the BRAINSTORM
 which make up the final solution. These are then collated into one
 document, which the user then edits to produce the final text. An
 article is then created, and the user is taken to the REVIEW
 BRAINSTORM page.

 REVIEW BRAINSTORM: Once a BRAINSTORM has been turned into an article,
 it needs to be reviewed to ensure that the article adequately
 answers the BRAINSTORM problem, and that contributions from the
 BRAINSTORM have been correctly and completely cited. For the first,
 the user needs to be presented with a checkbox to indicate whether
 they think the BRAINSTORM problem has been adequately solved.  For
 the second, people who subscribed or contributed to the BRAINSTORM
 should be sent an alert asking them to check that the article has
 been adequately cited. I haven't quite worked out how to handle
 adding an attribution to a thread if one (or more) is missing. There
 should probably also be a check that the article to which a citation
 is suggested is older than the article itself. There also needs to
 be some way of portioning the credit for the article, also probably
 by vote.

 NEW DOCUMENT: Prompts for the title of the document, whether it is a
 new article, new summary of existing work, or a brainstorm.  It
 should also give the option of producing a "secret" article, which
 would allow people to work on an article (along with any nominated
 collaborators) amongst themselves, prior to eventually making the
 article public. If the article is secret, a URL for the article is
 displayed so the author can access it later. It then prompts for the
 text of the document, and creates it.

 KARMA: I don't have a complete idea of how "karma" should be handled,
 and this will probably need significant tweaking.  There will
 probably also be strategies found to increase karma in ways not
 intended. The general philosophy behind karma is that it measures
 expertise in a particular area. It should increase with the number
 of articles added (with greater emphasis on higher quality and
 original articles) as well as contribution to discussion and
 reviewing of papers.

 The total "karma" of a person in a particular subject area would be
 made up of several components. The first, and largest, would be
 contributing new articles and significant improvements made to
 existing articles. The score for this would be proportional to the
 number of subscribers (although perhaps some other monotonically
 increasing function with decreasing slope. There might also need to
 be a minimum number of subscribers before any contribution starts,
 to prevent people adding many stub articles to gain karma).

 The second largest component of karma would be due to adding review
 articles. Each article should generate a fixed level of karma
 (perhaps the equivalent of 1/10 of an average original article) once
 the number of subscribers has reached a certain minimum.  This
 should not be proportional to the number of subscribers because this
 does not necessarily reflect the quality of the review, but the
 popularity of the subject that was chosen to review.

 The next largest karma component would either be for contributing to
 a BRAINSTORM or adding a discussion to an EDIT page of an ARTICLE.
 Both of these would be measured as a total number over the previous
 six months, and either capped at some level, or allow discussion
 points to be weighted (allow people to have a checkbox next to each
 item to choose whether it was a useful, indifferent or unhelpful
 suggestion), so that inane comments could generate negative karma.

 Karma should also be generated for spelling/wording corrections and
 questions, but since these are not moderated in any way, they should
 definitely be capped, so that the total amount generated this way
 does not exceed a certain amount.

 Finally, the number of articles subscribed to or "understood" should
 not affect the karma at all, since it is trivial to subscribe to all
 articles, without contributing anything, or demonstrating any
 knowledge of those subject areas.

 HELP: I would like to see help pages appearing automatically when a
 logged on user enters a new area of the site. For instance, if
 someone is adding an ARTICLE for the first time, then some
 information describing what they are expected to do would appear.
 The pages would also appear as an index off a HELP link somewhere on
 each page. The individual HELP pages could be edited in the same way
 as an ARTICLE, except that even spelling and wording changes would
 need to be VOTEd on.

Another feature which the site might eventually contain is an employer/ment finder. Here universities or industry might submit an employment offer with a specified minimum "karma" (or subject knowledge score or whatever measure it's called) in a particular combination of subject areas, along with salary information, etc. The site could then compare all of the user's qualifications and job requirements against those of the job, and issue an alert to the user when a match is found. This might be good as a paid service for the employers placing the ad, to raise revenue for the site (if this service were free, it seems to me it could easily be abused by spammers). I don't expect this feature would be used much until after the site has taken off though.

I think the best way forward from this might be to use the above points as a starting point for discussions, and get some suggestions for improvements (on which I'm sure there will be many), and try to agree on some detailed structure for the web site. Then once this is done, we just need to trick people into implementing it in actual code.

Tristrom Cooke: August 20

Academic Books?

I don't know if this is the correct place for such a comment, if it is not my apologies. I have been working with a project 'Old Tibetan Documents Online' where we use a wiki that has to be logged in to, and the results are published as a book and webcite. In general the wiki component has been extremely effective. I think this has great potential for other research projects, and that it would be advantageous to not have to rely on the small amount of economic support from a university that the last project required. I have an active fantasy for a similar Old Burmese project. While I find a fair amount of discussion on the development of wiki-journals, and there is of course wiki-books for books. I cannot find any discussion of this sort of wiki research project, where for starters it makes sense to have users of the project use a different interface than editors, even if these two groups have the same members. I would welcome any thoughts. You can find me on the English language wikipedia as 'Tibetologist'.