Research talk:Guidance for NPOV Research on Wikipedia/Draft

Instructions

Based on your relationship to this draft Guidance for Researchers, we have a few questions that we would like you to consider. You may use the "Add topic" button to provide your feedback.

Some general notes after the August round of feedback:

• Thank you to all the folks who left feedback here. We truly appreciated that you took the time to read, engage, and make suggestions. If we missed anything, do not hesitate to let us know (it was an accident and not intentional).
• You can see the full set of changes to the Draft via this diff.
• We want to also acknowledge feedback from @Hfordsa and the thoughtful response from @Rubberpaw and provide a few more details. As stated in the Draft, our goal is "to lower the barriers to entry and foster high-quality inquiry by providing practical guidance for researching Wikipedia’s approach to neutrality." We are not intending to circumscribe what research is or is not legitimate – i.e. description and not rules. We do have a positionality though and feel it is important to point out what we see as some common pitfalls in existing research, though disagreement with that is always welcome. To further clarify this, we also added a paragraph under "What are we proposing to do?" that further explains our position.

--Isaac (WMF) (talk) 21:56, 1 October 2025 (UTC)

If you are a researcher

• What information did you wish to see in the Recommendation section that is currently missing? Please help us understand why including this information will be helpful for you.

1. One question I have is whether NPOV itself is a clean concept or a concept that could use clarification itself among Wikimedians? To the extent that there is (continued) evolution in the concept possible, that would be great to know -- potentially something that research could help surface! --Connieatwork (talk) 19:20, 8 August 2025 (UTC)

   @Connieatwork that's an interesting question but we considered it outside of the scope of our Guidance because there is another workstream that's exploring something similar. We linked to it under 2.3.1. Policy interpretation variance but adding here for simplicity: Analysis of Neutral Point of View Policies across Wikipedias Isaac (WMF) (talk) 21:30, 1 October 2025 (UTC)

2. The research community is not a monolith: The draft states that "Wikipedia is not a monolith," and this same attitude needs to be extended to the audience of the guideline. Many of my suggestions (below) are an attempt to address the fact that I believe this draft is oriented to a specific set of researchers rather than a guideline that speaks to the entire "research community." In fact, I think that some of the concerns that the guideline identifies (that NPOV is situated in specific contexts and can be misconstrued when extracted from that context) are the same ones that humanities scholars and humanities-adjacent social scientists have long been pointing out, but are not represented in the current draft. But luckily it is a draft and obviously can be fixed :) Here are my goto suggestions for research that does this kind of work. . As the last example (Jemielniak 2020) states: "the more quantified and powerful the datasets we use, the more important it is to use qualitative tools that improve our understanding of the outcomes."

In terms of the actual audience that the guideline is addressing, it currently reads as if it is written only (or primarily) for quantitatively-based researchers. For example, when I look at the recommendations for methods, I see recommendations concerning "source rating systems," using "mixed methods" or "multiple methods and triangulating the results," "measurement approaches," content methods that include "sentiment analysis," "classification models," "slant indexes," "surveys," and using LLMs for analysis. So a generous reading of this is that the current draft is just missing some of the insights on how to address NPOV context's by pointing to methods that are designed to analyze context (discourse analysis, textual analysis, interface critique, ethnography, policy analysis, and historical methods). Much of this work is listed on the Critical Wikimedia Research Bibliography.

But a less generous reading of the guideline comes from the fact that are two specific examples where aspects of humanities research are explicitly stated to be avoided. For example, §4.2.4 "Seek to measure trends and not moments in time" states "Measuring adherence to NPOV in a moment in time, especially if it is in a rapidly-developing topic area, may lead to conclusions that would be different if a later snapshot were used or if the state of content were compared over a duration of time." However, this is precisely the type of analysis an Internet or Web historian might be interested in, where they specifically want to know how the content of an article came to be at that specific moment through the practices associated with NPOV. This is perfectly valid research.

Likewise the guideline suggests to "Be cautious about making normative statements about editor behavior." I understand that it says to "be cautious", but then the following sentence says "avoid defining a certain set of editors or editing behaviors as normative" which then implies "aberrant" behaviours. And yet, making normative arguments (claims supported with robust evidence) that evaluate the degree to which behaviours and collective actions contribute to the meaning of Wikipedia's culture is precisely the work that humanities scholars are trained to do, and many are trained to do so carefully. Perhaps what the guideline is pointing to is research that makes normative claims only, but it does so without being theoretically grounded or supported with evidence. If so, some further specificity might help assuage any misunderstandings about this recommendation.

What I am highlighting is that while the draft is addressed to the "research community," it is functionally targeted towards a specific subset of research disciplines that focus on quantitative analysis. If this is the case, then the guideline would benefit by being explicit and transparent about this intended audience, rather than addressing the research community as a whole.

Likewise, if the point is to help researchers in fields where quantitative methods are more the norm and may not initially consider contextual information as part of the analysis, then I think the white paper would be more successful by pointing to researchers who have laid some of the ground work for doing research on the context of Wikipedia's policy environment. --Textaural (talk) 01:58, 18 August 2025 (UTC)

Thanks @Textaural for these extensive thoughts and links and also offering through our office hours to further review the paper. Speaking personally, I consider quantitative researchers a core audience (but not the only audience) for this white paper because quantitative research often is more reductive by design and so especially tricky in the space of NPOV where context is so important. But we heard you that it's important to be more specific about how qualitative methods can be important in this space. If you are willing to give us concrete proposals for how we update the text for a future update, we would be happy to consider your proposals. For now, we made some minor changes in this direction and wanted to publish an update, but I don't consider this question closed. Isaac (WMF) (talk) 21:31, 1 October 2025 (UTC)

@Isaac (WMF) I agree that the guidance is oriented towards quantitative research and I think the newest version does this better than it was before. And in terms of concrete proposals for how to recommend doing the work of building context into research designs, I have a few comments about the newest version than lean into this request.

Concepts: §2.3.4 "Early in the history of the project, rules for sourcing, content, audience, and tone were still being developed. Research indicates that policies have stabilized in recent years." My research team confirmed Hwang & Shaw (2022) (thumbs up for including this work) -- but we found that the authority of these rules to be locked in mostly by 2011 –– See Figure 2. But importantly in terms of concrete actions, part of our research design required for us to rigorously define what we meant by "authority" before making this claim. "Rigour" from my disciplinary perspective means engaging with the literature both about Wikipedia and more theoretical texts that treat authority as a concept -- which means acknowledging that "our concepts are founded in sociopolitical systems that are far more complex than would be indicated by treating them simply as linguistic communities organized around specific key concepts” Balbi et al citing Koselleck. I think this last point of that quote directly addresses the limitations of treating Wikipedian concepts of neutrality in isolation. So a concrete proposal would be that research on NPOV would benefit from being specific about which Wikipedian meanings they are enlisting, and which sociopolitical contexts are making those concepts important to study. For example, the reason to study neutrality today is different than it was in 2015. Why? That's for the researcher to rigorously explain in the research design.

Deep reading: In § Assess edits and editor actions in context. it now says "Articles can be renamed, moved, or recategorized as part of maintenance activities in ways that make it difficult to connect current article state with past article state." This is extremely important and not at all obvious to anyone just starting to research Wikipedia, or who assums that a datadump or scrape represents how the page came to be -- and this is the point the guidance is making and is invaluable. In fact, this was one of the major obstacles I had when trying to reconstruct how a project page came into being over time to tell the history of the GGTF.

This points to my second proposal in reference to the following sentence: "Countermeasures against this kind of error include engaging in deep reading or other more qualitative examinations." In my field we sometimes call this "close reading" (which is a specific methodology), but in the case of what the guidance is actually describing, I think the more appropriate framing comes from "digital methods" which is based on "following the medium," which offers "the researcher a strategy to cope with the ephemerality and instability of the Web, where a new feature, a changed setting [...] could stymie longitudinal studies" Venturini et al 2018.

The article provides a checklist of things for researchers to consider when they are doing digital methods. Given that nearly anyone who studies Wikipedia (unless they are doing only interview work) engages with the site in some way -- and are therefore doing some form of digital methods –– I heavily suggest the linked article should be highlighted as a primer on how being rigorous with Wikipedian content also means understanding how its content is shaped by shifts in the medium's form - regardless of whether it is for quantitative or qualitative research. You are already pointing this out, I would just point to the researchers who have done this. Textaural (talk) 21:02, 13 October 2025 (UTC)

1. response_2. your_name_or_signature
2. response_3. your_name_or_signature
3. ...

• Is there a part of the whitepaper that is unclear to you? If so, please point to it and give us some information about what makes it unclear to you or what questions it raises for you.

1. Very useful document for researchers interested in NPOV! Thank you :) One comment I have is with respect to section 2.2, § "NPOV means neutral editing, not necessarily neutral content."
   I understand what this sentence is supposed to mean, but I think that it will fail to convey its intended message to people who are not already familiar with WP and NPOV. Intuitively, I understand here that the end goal of the NPOV policy is not for editors to collectively produce neutral content (that is, content that appropriately reflects all sides to a debate / perspectives on a topic, in due proportion to their relative treatment in the available set of reliable sources), but for editors to "edit in a neutral way". Without further refinements, this suggests that NPOV requires Wp editors to be neutral in their editing (that is, not exhibit bias). This is the opposite of what NPOV actually requires: editors should write with a neutral *tone*, and accept in the body of the article all competing views or theories that are appropriately supported by the available sources -- but they are not expected to write without bias.
   In practice (as the document later recalls), NPOV on controversial topics is always reached or approximated through the aggregation of *biased* but *reliable* individual edits. As a result, the nature of the NPOV policy is to frame editors' edits and interactions in a way that does foster the development of neutral content (as strictly defined above). The essay on which this section is based tries to make it clear that "the 'Neutral' in NPOV is a term of art that refers to an editorial attitude and mindset; it is not a true 'point of view'" on the part of the editor trying to adhere to NPOV. This distinction makes the message of the essay consistent with the view I develop above (but then this section should insist on this distinction) -- but I don't see this as an intuitive way to convey the message. Alternatively, I think that one could more intuitively convey the same message by presenting things differently: NPOV does not mean neutral (or "unbiased") editing, it means that contributors collectively strive to produce neutral content (and then the section specifies what "neutrality" precisely means in the context of Wp, that is appropriately reflecting all sides to a debate / perspectives on a topic, in due proportion to their relative treatment in the available set of reliable sources). SalimJah (talk)
2. Definition of NPOV, potential inconsistency/issues - thanks also for the interesting paper! To echo SalimJah's comment above, I also found section 2.2 a little unclear and/or potentially at odds with itself at times. For example, (a) consider the guidance provided in the paragraph related to the NPOV facets of tone and due weight. In the sentence that seems to align with tone, it discusses: "editors should avoid biased language." However, it's not clear to me that tone and biased language are the same thing, whereas 'opinionated' language might be closer. Or (b) consider the use of objective/objectivity: Under Freedom from Editorial Bias: "Wikipedia editors and should be written using a tone that aims to be encyclopedic, that is objective and neutral in tone" versus "The baseline for NPOV is heavily influenced by reliable sources and referencing guidelines that every Wikipedia language adopts, not by some abstract, unattainable ideal of objectivity" (emphasis added).
   This can pose a problem not just because maintaining clarity around the terms such as bias, objective/objectivity, neutral/neutrality, balance is super tough -- which is required in the establishment of metrics and measurement -- but also because these terms may clash with a discipline's own discussion and use of these terms. So, for example, when coming from a historical background, there is the possibility of maintaining the pursuit of objectivity as desirable and consistent with being non-neutral (See Thomas Haskell, Objectivity is Not Neutrality (2000)), and there is a larger context in which these terms are attempted to be clarified (e.g. Historikerstreit] -- sorry, I can clean up my refs and make them proper hopefully later! :) ).
   Another way to put it is from something that SalimJah wrote above: "In practice (as the document later recalls), NPOV on controversial topics is always reached or approximated through the aggregation of *biased* but *reliable* individual edits." This is basically an attempt to summarize the epistemological theory at the heart of Wikipedia; if you agree with this characterization, then clarifying it as such could ease the approach of some researchers? Anyways, some thoughts! --Connieatwork (talk) 19:01, 8 August 2025 (UTC)

   I wholly agree with the conclusion that you come to here. Researchers bring their discipline's understanding of phenomenon/concepts to the table in ways that complement the meanings Wikipedian's have created for themselves. Also the last point means that NPOV studies are entangled with Wikipedian practices and ideas about consensus, something that someone (it's me) have studied. Textaural (talk) 01:21, 19 August 2025 (UTC)

   @SalimJah thanks for this deeper read on the idea of "neutral editing". For context: a common issue we see in quantitative analyses of neutrality on Wikipedia is that researchers will e.g., compare the sentiment of content about two different political parties with the assumption that they should be equal. This is the "neutral content" approach and we found the essay that we cited to be a nice summary of why that is a misinterpretation of the policy. I agree with you that "neutral" is not a term for which folks have consistent definitions. In that case, we shifted towards "impartial summary" as hopefully easier to understand but we've largely kept the terminology because I think it is valuable to build on these community perspectives (and re-use their terminology where possible). I've also added a sentence that references https://meta.wikimedia.org/wiki/Neutral_point_of_view and makes the point about not every edit needing to be "neutral" but the importance of the collective goal.

   Additionally, thanks for these thoughts @Connieatwork and follow-up @Textaural. In writing this, we've chosen to largely stick to the language of the NPOV policies of the English Wikipedia community (given its influence and that we are writing in English) rather than introduce our own terminology. This might mean that some of the terms are imperfect but admittedly there is no consistent terminology in this realm. We'll remove the term "objective" though, which was a deviation as en:NPOV largely avoids this term. We've also added a note on terminology to call this out towards the top of the guidance – i.e. some of these key terms have very different meanings across disciplines and language communities, but we're seeking to ground our terminology in that of the Wikipedia communities. Isaac (WMF) (talk) 21:36, 1 October 2025 (UTC)

   Agreed! The document reads much better now. Thank you :) SalimJah (talk) 08:17, 2 October 2025 (UTC)

   I appreciate that you've added in the multiple meanings, and I think that (as I mentioned above) providing context for NPOV also means providing context for what neutrality means both internally and externally to Wikipedia – as per the requirement of the research design to successful answer the research questions being pursued. Textaural (talk) 21:06, 13 October 2025 (UTC)

3. Needs more clarity/support for define the problem that guidelines is attempting to address: In the "Executive summary," the draft makes the following claim: "Writing Wikipedia with a neutral point of view is one of the five pillars of Wikipedia.[1] Although the topic is core to Wikipedia, it is relatively understudied considering hundreds of research studies are published annually about the project." The draft then references Asikin-Garmager et al. as evidence to support this claim. If the text is this pre-print [1], then there is an issue because I do not see where in this paper it describes the lack of research specifically on neutrality. Maybe I missed it, but this section would be more helpful if there is a direct quotation that could be used to help clarify the position.

Even though the cited paper and the claim (seemingly) don't match up, I believe in the intuition that NPOV research makes up a extremely small fragment of research being done on the Wikipedia each year. I have similarly found this to be the case when I have built my own literature reviews about Wikipedia policy topics. But with this in mind, I would argue that all of the pillars/principles are largely overlooked. What is the argument that the white paper presents for drawing attention to studying NPOV specifically, and not the other principles or pillars? I ask because I assume that NPOV researchers will want to provide a grounded reason to legitimize their research as relevant (and not simply just based on a gap on the literature), and they can cite the white paper to do so. What is the crisis of NPOV that makes it so important to study now?

If the goal of the guideline is to assist researchers in studying NPOV by identifying a gap in the literature, then there is also a responsibility of the white paper to highlight research that has been done on the topic – and I think the executive summary section should lead (perhaps in a second paragraph) with a summary of what has been studied, especially those studies that demonstrate how to study NPOV in a nuanced way. This would be an immediate benefit to researchers on this topic.

In this regard, I see that some of this is communicated in §2.3 "Dimensions and Factors influencing NPOV," but this is just one kind of research. Other researchers have examined the meaning and consequences of NPOV, especially on how it is translated as informal and formal Wikipedian practices. A lot of this work has been directly and indirectly addressed in a cluster of studies that (explore the juxtapositions of Wikimedia policies and practices) [2]. I highly suggest that the guideline reference some of this work as it can serve as models for studying policies in context - a concern that seems to animate the reason for creating the guideline.

Request for further clarity distinguishing between NPOV as a codified policy, NPOV as in/formal practices, and neutrality as principle The draft states "studying NPOV requires in-depth knowledge both of Wikipedia’s inner workings and of the intricate ways in which neutrality is defined and operationalized within the project." I completely agree. However, how neutrality and NPOV are defined by Wikipedians AND how they are operationalized can be distinctly different, and the juxtapositions (and contradictions) that emerge from what is said to be NPOV is (Point 1 below) and how it is invoked in practice (Point 2 below) and what this has to do with neutrality (Point 3) has been a source of fruitful research questions.

Point 1: I understand the desire to clarify what NPOV is for researchers doing work in this area. And my sense is that the focus of the guideline is to help those researchers who measure neutrality and require a definition to ground such measurements. However, the guideline should perhaps avoid attempting to redefine NPOV since the NPOV policies themselves already do this. I emphasize this because while NPOV is a relatively stable policy/principle, it is subject to change (even if it is slow). Perhaps then it would be better to reinforce the fact that if researchers are studying NPOV, that it would help their research be more rigourous if they go beyond quoting the summary section of the policy by either spending more time understanding how the policy is used in practice and/or relying on the support from other academic literature.

Point 2: Beyond this concern, I think there is also the risk that in the attempt to define NPOV for researchers, the guideline presents these attributes as constituting the meaning of NPOV, while there are inferred notions of NPOV that are taken up with consensus practices (as Connieatwork alluded to). Researchers (including myself) have explored how the ideas of NPOV get taken up in conversations across Wikipedia in more direct and indirect ways that become part of the cultural context of how Wikipedians respond to crises and controversies. In other words, NPOV is definable beyond the confines of bias critique, source critique, and rule adherence.

Point 3: [added after reading Connieatwork's comment] This complication points to how studying NPOV gets mixed up in broader discussions of neutrality as a principal that has a long intellectual history and various contexts. Researchers may therefore choose to study NPOV from this perspective, similar to how Nate Tkacz studied the conceptual and political genealogy of Wikipedia's principle of openness.

So while some NPOV research may examine whether Wikipedia's content is neutral (according to its policies or other metrics), other NPOV research will study the construction and changes to NPOV itself. For example, the empirical research that I used to support my history of the Gender Gap Task Force showed how the concepts of neutrality from NPOV were adapted and mobilized to legitimize a specific idea of who could be a Wikipedian. My co-authors and I also found that the Wikipedian notion of neutrality is entangled with the documentary form of policy consensus, and the transformation of ad hoc wikipages into formalized policy pages tends to present a neutral authority of the document that obscures the actual authority that was used in policy development.

So, I do see a utility in helping researchers navigate the Wikipedian definition of NPOV. But if it remains prescriptive and singular, then the guideline risks taking over a component of analysis that is the responsibility and domain of researchers.

Section 4.3: Guideline or imperative?: Despite this being a guideline, many of the statements in Section 4.3 come across as imperatives rather than suggestions, and some of these demands overlook some of the important circumstances of doing Wikipedia research.

"Communicate your research early and often." I think that this is good advice in theory, but not all research makes sense to communicate to broader publics early on. Some research requires years of tinkering and circling around research questions until it gets to the point where it can be "communicatable" in a sensible way. Perhaps it is better to say "practice communicating your research early and often". Otherwise I think the suggestion of creating a Wikimedia page is great, and I think the terminology of "encourage" is the right tone.

"For the large majority of NPOV related research findings, insights and recommendations, Wikipedia communities on the ground are the correct and priority audience for communicating the research findings to." Unfortunately, Wikimedia communities are only one of many audiences that researchers may choose to prioritize for communicating research findings. They may need to prioritize communicating to their peers, to their funders, or their institution in order to maintain their research career and their employment. However, I think if the researcher is being funded by Wikimedia and the agreement of the funding is to explicitly communicate to the Wikipedia communities, then such a prioritization makes sense. However, not every researcher can be funded by Wikimedia.

"Always share back with the Wikimedia research community." I appreciate the sentiment of building and contributing to the research community. This is important, but given that this is a "guideline," I do not think that using the imperative "always" is appropriate here. Indeed, some researchers' professional and life commitments may significantly limit their ability to contribute to research communities more broadly (I am thinking of those peers of mine who have heavy teaching and service contracts). Perhaps "Contribute what you can to the Wikimedia research community" is more in line with the ethos of Wikipedia itself.

"Communicate in ways that strengthen Wikipedia. […] Critiques are valued but ideally are paired with constructive recommendations, are replicable, leave space for feedback from Wikimedians, and do not overstate conclusions." I agree with the general sentiment, but there are some caveats here, especially in terms of framing what is valuable to whom. I think it would go a long way to state that "Critiques are valued [by Wikipedia and the WMF] but ideally are paired…". I suggest this as researchers may be beholden to the general public (either through funding or institutional connection), and the critiques they produce may be in and of themselves valuable to these groups -- and not necessarily Wikipedia. Additionally, some research traditions do not have imperatives to provide solutions. Their expertise (and broader social value) is precisely in the production of critiques. Likewise, the complexity of the issue may not yet present a clear and actionable recommendation and the creation of such a recommendation demands its own research project. Of course, while the use of "ideal" in the draft covers all this, it would be encouraging to see the guideline make some space to acknowledge these other research traditions, disciplinary distinctions, and circumstances. --Textaural (talk) 03:30, 18 August 2025 (UTC)

@Textaural a few thoughts on your first and third set of points. I think the second set will be better addressed when I have had more time to consider another set of updates per your feedback higher up in the page (thanks for your patience):

• Citation around understudying – the citation is actually for the total number of studies and not the claim about under-studying of NPOV (see the citation's Executive Summary). If you know of anyone who has made the claim about understudying more robustly than us, please let me know though and I'll gladly update.
• "What is the crisis of NPOV that makes it so important to study now?" I would agree that we would love to see more research in general that is grounded in assessing Wikipedia's pillars/policies. The reason for the particular focus on NPOV is best described in this blogpost: https://diff.wikimedia.org/2025/03/27/strengthening-wikipedias-neutral-point-of-view/
• "I think the executive summary section should lead (perhaps in a second paragraph) with a summary of what has been studied, especially those studies that demonstrate how to study NPOV in a nuanced way." This initial draft was done largely over the period of a few months and, coming in, we had the choice of structuring as a literature review (akin to our Knowledge Gaps Taxonomy) or a white paper (akin to our privacy white paper). We chose the latter given the time constraints and large scope (my personal experience is that a good literature review is quite worthwhile but really takes many many months). We obviously read many papers in preparation for this writing but were not ready to claim anything close to completeness. That said, we agree that there is some exemplary work out there and that being aware of it can greatly help researchers. We have tried to incorporate these citations throughout (where relevant) and hope to continue to add to them as we discover / are made aware.
• "Despite this being a guideline, many of the statements in Section 4.3 come across as imperatives rather than suggestions..." We have kept the more imperative style because we found that it makes the recommendations more intelligible, but have tried to clarify the overall intent and soften in certain key areas as well.

Isaac (WMF) (talk) 22:09, 1 October 2025 (UTC)

@Isaac (WMF) Thanks for taking the time to read through and incorporating a number of my suggestions. In terms of understudying policies, I only have my personal anecdotes of this being the case, and thanks for pointing to blogpost about the reasoning. I also noticed the points where the new version includes minor but crucial edits in terms of softening some of the imperative language and I appreciate that. With minor edits in mind, I do have a further suggestion for the newest version. In the sentence "Further, as a large and ever-evolving project, policy compliance also can shift over time," it uses the word "compliance." I would suggest that the more accurate framing would be that Wikipedians engage in "policy interpretation" rather than policy compliance. In fact, the new version uses this language elsewhere when it describes the fact that Wikipedians "interpret and apply" policy. So being consistent here would make sense.

I also understand the constraints of genre and the outcomes you are trying to work towards (so one text cannot be everything for everyone). That being said, perhaps the white paper would benefit by being complemented by something that is intended to do the work of "modelling" what contextual research on NPOV looks like in practice. I think there are many possible ways to do this, but some ideas might be to have panels dedicated to how different researchers address these issues (either at Wikiresearch workshop or ICWSM), a special issue in a journal or perhaps another publication format that allows for cross-disciplinary work to be gathered in a single space. Perhaps WMF research can work as a facilitator in this respect. I'll think about this as well. Textaural (talk) 21:21, 13 October 2025 (UTC)

1. response_3. your_name_or_signature
2. response_4. your_name_or_signature
3. response_5. your_name_or_signature
4. ...

If you are a Wikipedia contributor

• Is there an important piece of information that you wished to be communicated to researchers of NPOV on Wikipedia that is currently missing? If you propose an addition, please help us understand why including this information is important from your perspective.
  1. I really liked the document as a whole, although I feel like it currently relies too much on the English Wikipedia as a reference what is being done regarding NPOV. While there is some mention of how the definition of NPOV varies across projects, it could help to include more concrete examples from other Wikipedia editions, or references to the UCoC (specifically paragraph § 3.3.2). This is especially true since the question of NPOV on smaller projects has also been studied.
     Besides that, while I do agree with making an explicit distinction between bias and reliability in 4.1.1, it misses the hard-to-assess point that reliability can be very context-dependent. Broad evaluations (like Media Bias Fact Check, used by Yang and Colavizza, 2024) can be helpful as a baseline, but the nuances of how sources can be more or less reliable in different contexts is only briefly touched on in 4.2.1, which risks confusing it with the amount of attention given to a particular citation. Many sources are considered reliable in one domain (say, entertainment) but not in another (say, politics), and a mostly politics-focused analysis could give the impression that we over-rely on an unreliable source.
     It also isn't clear what is meant by Baigutanova et al. (2023) assessing the proportion of articles citing a source from the perennial sources list. This list is meant to represent common sources, with various degrees of reliability (which, for some of them, depend more specifically on the context). It could be good to specify whether this was measuring articles citing a source deemed reliable by the list, or unreliable, or both – some sources on the list may also either have no consensus on reliability, or be considered marginally reliable. Chaotic Enby (talk) 09:42, 1 August 2025 (UTC)

     Thanks @Chaotic Enby.

     • Regarding Baigutanova, that's a good clarification to ask for. The research splits between the different categories but the 23% number I was reporting was any source (reliable or not or uncertain) just to show that no list has 100% or even close to 100% coverage of the sources used on Wikipedia. I updated the text to clarify that point.
     • Regarding nuance in how sources are cited: we indeed tried to make that distinction clear but I think your point about political bias being very different from other topic areas is a good one that I had not thought of but would be useful to include. I added a few more examples to Section 4.2.1. around CNET (time of publication), Forbes (bylines), and HuffPost (topics).
     • Regarding a reliance on English Wikipedia. This is being written in parallel with the cross-wiki analysis of NPOV. Unfortunately we don't have a clear sense yet of what specific differences exist between English Wikipedia's policy and how other language editions enact NPOV. We have added a positionality statement to the Guidance to be more clear about these limitations though.

     Isaac (WMF) (talk) 21:39, 1 October 2025 (UTC)

     Thanks a lot @Isaac (WMF)! Still on the topic of cross-wiki policies, the "five pillars" expression varies between language editions. Most languages keep these same pillars (e.g. it:Wikipedia:Cinque pilastri, es:Wikipedia:Los cinco pilares, ca:Viquipèdia:Els cinc pilars), while French Wikipedia calls them "founding principles" (although with a drawing of a pillar at the top), and Persian Wikipedia calls them "five foundations". German Wikipedia has four "ground principles" (the missing one being "no firm rules").

     So, while "five pillars" is clearly the most widespread name, maybe an explanatory note could be added to clarify that it isn't universal (and that NPOV is still present even in languages that don't have "five pillars")? Chaotic Enby (talk) 22:05, 1 October 2025 (UTC)

     Interesting - I was not aware of the variation in pillars. I'm gathering a few other potential changes before doing another pass on the draft but I'll leave myself a note to see about addressing this. Thanks! Isaac (WMF) (talk) 20:10, 2 October 2025 (UTC)

  2. As a Wikipedia editor particularly concerned with identifying and reducing gender bias, I believe the draft could strengthen how it incorporates this dimension within the study of NPOV. Neutrality is not only affected by the selection of sources and content, but also by systematic absences: topics, biographies, and perspectives of women or non-binary people that are missing from the encyclopedic corpus.
     I suggest: Explicitly including gender bias in section 4.1 (baselines) as a measurable element, both in terms of sources (e.g., proportion of female authors cited) and content (proportion of female biographies or coverage of topics relevant to women). Incorporating methods to detect absences in section 4.2, since the lack of articles or perspectives is also a form of bias that affects neutrality. Recommending that research findings on NPOV make such biases visible and propose strategies to reduce them, for example through community editing projects or collaborations with groups working for gender equality.Tiputini (talk) 10:37, 13 August 2025 (UTC)

     Thanks @Tiputini for the work around gender bias and this feedback. This was a facet that we discussed as well – what's the line between content gaps and NPOV. Clearly they're related in many cases but we also did not want to overgeneralize. We currently note in Section 4.2 that "At the Wikipedia language edition level, NPOV applies to an entire topic area: are there major gaps in representation in terms of what articles exist and how thorough they are? Given the scale of content studied at this level, methods like those from the Knowledge Gaps literature for tracking the prevalence of various sub-topics and their associated quality/framing or their trajectory in article-for-deletion discussions may capture some of the dynamics here." Both of those citations discuss the gender gap literature and I was able to incorporate a few more like Park et al. Isaac (WMF) (talk) 21:43, 1 October 2025 (UTC)

  3. Having found this page via an article on The Conversation [3], I'd like to chime in and say that, although I find that article makes unfounded assumptions and is meant to rattle and rage, it does successfully point out in my opinion that the way the draft is written could come across (at least from the outside - I'm not versed in humanities research and how communication is like in these spheres) as paternalistic and as the Foundation trying to direct researchers' methodologies. I think another equivalent formulation would communicate the intent better; for instance instead of saying "avoid doing X" or "consider Y", it might be better to frame things as "here are a few things that we think are pitfalls and would make your research biased or incomplete or easy to refute with the following argument"- then, if they still do it, you just have to point at the sign to rebut their argument (instead of the conversation risking being framed afterwards as "the foundation is criticizing me because i didnt follow their rules! free speech! subscribe to my substack!"). Just my two cents from a communication perspective! Thanks for your work Mule hollandaise (talk) 02:52, 26 August 2025 (UTC)

     Thanks @Mule hollandaise for sharing. That indeed seems like a reasonable strategy and we tried to update the language throughout to clarify that these are intended to help researchers avoid pitfalls but not guidelines that are being enforced or believed to be the only way to do this research. Isaac (WMF) (talk) 21:46, 1 October 2025 (UTC)

  4. Can you add commentary on words which shouldn;t be used like describing someone as a "terrorist"? Guz13 (talk) 01:59, 6 August 2026 (UTC)
  5. response_5. your_name_or_signature
  6. ...

• Do you find factual inaccuracies in the whitepaper? If so, please point them out and ideally provide a reference where we can learn more if relevant.
  1. response_1. your_name_or_signature
  2. response_2. your_name_or_signature
  3. response_3. your_name_or_signature
  4. ...