User:Johannes Richter (WMDE)/Sandbox
| Reusing references | |
|---|---|
 | |
| Status | In progress |
| Duration | 2022 – 2024 |
| Origin | Technical Wishes Survey 2022 |
| Phabricator | project board |
| Responsible | Technical Wishes Team |
In the 2022 Technical Wishes survey on German Wikipedia, community members selected „make reusing references easier“ as the next focus area for Wikimedia Deutschland’s Technical Wishes team. This page documents our research and development progress on this topic. Comments and questions are welcome on the talk page!
Focus: Reusing references within an article[edit]
Through our research and input from community members, we found many problems in different workflows around reusing references: problems with reusing them from external sources, from another wiki, from another article, or from within the same article. To get more work done in the timeframe that we have for this project, we are focusing on the issues around reusing references within the same article. Our research has shown that this is where most problems are. Working on this area, we will also be able to tackle the wish that was brought up the most: reusing a reference with different details (e.g. a different page number) on the same article.
Our team has worked on this project before, but could not finish it. In the past years, we gained experiences on Visual Editor and references, e.g. by working on templates and Reference Previews, allowing us to deliver a solution for this project.
Projects[edit]
Confirmed projects[edit]
| Project | On some wikis | On all wikis |
|---|---|---|
| Sub-referencing
Reusing a reference with different details (e.g. page number) in wikitext and VisualEditor |
 In development |
In development |
| Bug fixes |
 Done |
Done |
Main project: Sub-referencing[edit]
Our research has shown that one of the most pressing issues when it comes to reusing references is that it is not possible to reuse references with different details within an article, in a simple way. More reasons why this is our main project:
- There is currently no solution in MediaWiki for reusing references with different details. There are community-built solutions via templates, but those solutions do not exist across all wikis and cannot cover all use cases of references with different details.
- A solution is highly desired by the wiki communities, regardless of editing style, editor preference or level of experience.
- We want to improve feature parity: reusing references with different details is significantly more complicated and more error-prone in Visual Editor.
- Many of the other problems identified in our research are caused by the lack of a solution for this problem. Some other problems might be addressed as a byproduct of an improved Visual Editor citation workflow.
- Readers will also benefit from a solution to this problem, as citations with different details will be grouped, similar to reused identical references, making it easier to assess the overall quality of an article's sources.
At this point, it is unclear if we’re going to solve other problems in this area. Read more about our work on this problem.
History and research[edit]
The original plan was to start working on this focus area shortly after the survey in February 2022 and finish our work towards the end of 2023. Due to several longer, unplanned absences in key roles in the Technical Wishes team, we are currently behind schedule and plan to be working on this focus area until the end of 2024.
We’ve kicked off our research with a call for problems in 2022. We analyzed those problems – as well as lots of open Phabricator tickets around reusing references, which had been submitted over the years –, sketched user workflows and tested referencing in different wikis ourselves, resulting in this problem collection in 2023. After taking a break due to several absences, we conducted user interviews in early 2024 which confirmed many of our assumptions.
Summary of user interviews in early 2024[edit]
A total of 9 participants were interviewed to help us understand their challenges with reusing references. The participants were selected based on diverse backgrounds with a final selection of:
- Experience
- Editor level (self-reported): 1 beginner, 5 medium, 4 expert
- Editing hours per week: 0-2 (3), 3-10 (2), 11-20 (2), 21+ (2)
- Experience working with references (multiple answers possible): adding (9), reusing (8), editing (8), template maintenance (2), automation (2), policy (2)
- Tech-savviness (self-reported): 0 beginner, 2 intermediate, 3 proficient, 4 expert
- Regularly used editors (multiple answers possible): Visual Editor (6), wikitext 2010 (5), wikitext 2017 (2)
- Comfortable with Visual Editor: Not comfortable (2), somewhat (3) very (4)
- Comfortable with wikitext: Not comfortable (3), very (6)
- Active on projects (multiple answers possible): Wikipedia (9), Wiktionary (2), Wikiquote (1), Wikisource (3), Wikiversity (1), Wikivoyage (1), Commons (6), Wikidata (6), Wikispecies (1), Metawiki (1)
- Editors on 8 different language wikis: German, Italian, Spanish, English, Kannada, Tulu, Gujarati, French
- 6 from the Global North (4 from German-speaking areas), 3 from the Global South
- 4 women and 5 men
- Age: <25 (2), 25-40 (3) 40-60 (1), 60+ (3)
The results have shown the following:
- Adding verifiable and reliable references requires a time investment and is a cumbersome task, often leading to high cognitive load.
- There are multiple workflows for adding references. Wikimedians use a combination of wikitext and Visual Editor to add references. They employ combinations of copy-and-paste and user interface elements. Even experienced wikitext 2010 editor users switch to Visual Editor in order to use the Automatic Reference Maker (Citoid), as they find it very handy and easy to use.
- Inconsistent experience across editors
- There is a lack of feature parity across all three editor types (Visual Editor, 2010 wikitext and 2017 wikitext editor). Some workflows are easily achieved in wikitext but are disadvantageous in Visual Editor.
- One notable issue is that wikitext users can use copy-and-paste to create a new reference with different details, while Visual Editor users need to resort to clicking through footnote markers or moving between the footnote marker and the reference area to manually copy the information piece by piece.
- Duplicate references: Resolving duplicates is a cumbersome task. Among other things, because duplicates are difficult to find in Visual Editor.
- Named references: The Visual Editor auto-generation of named references (:0) poses a challenge for wikitext users, because it does not convey information about the source, making it harder to identify a reference.
- Simple insertion via Automatic Reference Maker (Citoid) doesn’t always work well
- The Automatic Reference Maker is used by most of the interviewees.
- It has issues, such as not always finding enough metadata or any metadata for the entered identifier.
- Citing the same reference but with different details
- Most notably for reusing references with different details, such as page numbers, there is currently no clean way to do so without using templates.
- All workarounds known to us can result in errors or duplication.
- Many users just add duplicates instead of using existing solutions like {{sfn}} and rely on others for wikitext cleanup.
- Template-based workarounds are not very compatible with Visual Editor at the moment. Overall, Visual Editor users are mostly in a disadvantageous position, with regard to existing solutions for reusing references with different details.
Thank you to everyone who participated in tests and interviews or offered to participate, as well as everyone who took the time to comment on our talk pages over the years. It is deeply appreciated! If you have further suggestions, ideas or feedback, please leave it on the talk page.
Where a solution to this problem was requested – examples[edit]
- Via Bugzilla as early as 2008: T15127 Page number attribute for <ref> tags
- WMDE Technical Wishes survey on German Wikipedia
- 2013 (10
Support; top 20 wish) - 2015 (17 Support; top 20 wish)
- 2022: German wiki communities vote „Make reusing references easier“ as the new focus area (468 Support; No. 1 wish).
In a call for problems around reusing references in 2022, a majority of dewiki users mentioned this problem specifically. Our research in 2023 confirmed that this was the most pressing issue in the focus area, as did additional user research with a group of international users in 2024 ((LINK #Focus area page – reasearch section)).
- 2013 (10
- Wikimedia Foundation’s Community Wishlist Survey
History of our work[edit]
Overall, our work on this problem was less straightforward than previous projects. Some general factors that make it very challenging to work on this:
- The Cite and Citoid extensions are two separate code bases serving largely the same purpose. This required duplicate work, code clean-up and removal of technical debt.
- Due to the large number of extensions (e.g. Cite, Citoid, Visual Editor) and areas of the wikis affected, a lot of coordination with different Wikimedia Foundation teams was necessary.
- Unlike most other features we’ve introduced in the past, it’s not possible to release this solution as an opt-in or opt-out feature: Once it’s deployed and used within articles, it’s affecting everyone. And un-deploying the change (in case unforeseen problems arise) would be harder because all the articles where sub-referencing was used would need to be cleaned. Because of this, we want to make extra sure that our solution meets the communities’ standards and needs. We are still thankful for your feedback ((LINK #Feedback)).
- We cannot deploy sub-referencing as long as the feature is not working in Visual Editor. For a long time, we did not have the confidence nor tools necessary to implement the Visual Editor solution ourselves. After successfully working on the Visual-Editor-related focus area “make working with templates easier”, the team now does have the necessary knowledge to build a solution for Visual Editor.
Timeline[edit]
2016
- First research, e.g. in community discussions at Wikimania.
- The Technical Wishes team makes a first proposal for a wikitext solution. Discussions with various stakeholders at the Wikimedia Foundation show: the proposal is feasible from a technical standpoint.
- Feedback round on German Wikipedia: 22 people are in favor of the solution, 8 are against it, 2 are neutral.
2017
- Development gets postponed because the work on another wish takes up more time than anticipated.
- Approval from stakeholders at the Wikimedia Foundation: We can move on with implementing the first proposal.
- The proposal is presented to an international audience in a lightning talk at WikiCite 2017 (talk #28), and is perceived very well.
2018
- Feedback round on Meta: Editors from all wikis are invited to give feedback. Result: 46% of participants are in favor of the proposed solution. 28% oppose it. 25% can’t be clearly associated with support or oppose.
- The same feedback round asks for opinions on the name of the new attribute, but there is no clear majority for any of the suggestions. The Technical Wishes team decides to use the attribute name “extends” for now ((LINK #sub-referencing – name section)).
- A call for feedback with the technical community shows that there is no opposition to go ahead with the proposal.
- The Technical Wishes team waits for progress on the Wikimedia Foundation’s Parser Unification Project because we wanted to make these changes in only one parser instead of two. While waiting, the team works on another community wish.
2019
- The Technical Wishes team lays the groundwork for the wikitext solution by rewriting the Cite extension’s code, which at this point is 14 years old.
- The team tests the feasibility of the wikitext solution – while also working on other community wishes.
- December: The wikitext implementation becomes available on the beta cluster – it has been open for testing ever since.
2020
- No work on this project. The Technical Wishes team deploys a feature from another community wish and starts working on the new focus area „Make working with templates easier“.
2021
- The Technical Wishes team cancels further work on sub-referencing. Main reasons: Visual Editor support has not yet been implemented, and the Technical Wishes team does not have resources for it. We want to avoid contributors using wikitext having a vastly different experience than Visual Editor users. This would create confusion and even bigger discrepancies between wikitext and Visual Editor workflows. Besides, it is unclear how much more work would be required to finish the wikitext implementation.
2022
- German-language wiki communities select „Make reusing references easier“ as the new focus area for the Technical Wishes project (468 votes). Research starts with a call for problems on the wikis.
- The Technical Wishes team finishes work on the previous focus area, “Better support for geoinformation”.
2023
- The Technical Wishes team starts working on this focus area, by diving deeper into the problems submitted on the wikis, as well as looking at Phabricator, where lots of technical problems around reusing references had been reported over the years. The team sketches user workflows around reusing references, and also tests the referencing system themselves, bringing more problems to light.
- Decision to focus on improvements for reusing references within an article. This is where most problems have been found, and the wish that was mentioned most often also falls into this category: reusing a reference with different details (e.g. a different page number) on the same article.
- Due to several longer absences in the project team, work on the focus area is put on hold for most of 2023 and gets extended to 2024.
2024
- The team is complete again and continues working on the focus area.
- User interviews are conducted.
- Goals:
- find out whether new, more important problems have surfaced around reusing references since our last research, or changes in user behavior and user needs;
- understand the workflows and problems around reusing references within an article better; especially for new team members;
- diversify the group of people we interview to learn about those problems from users with different experience levels, languages, generations, genders, cultures and from sister projects.
- Result ((Link: Focus area section results)): There are several problems in this area, but reusing references with different details within an article is confirmed as one the most pressing issues. Our proposed solution is positively perceived.
- Decision to implement our solution for reusing a reference with different details (sub-referencing). ((Link:Subref-page))
- Goals:
- The existing wikitext solution is tested with community members in order to evaluate its usability and desirability. Anyone can still test the solution on the beta wikis and give feedback ((Link: Sub-ref page feedback section)).
- Because of the delays in 2023, the new plan is to finish our work on this focus area by the end of 2024. The survey to determine our next focus area is scheduled for the beginning of 2025.