Community Wishlist Survey 2023/Edit-recovery feature

Edit-recovery feature
Recovering unsaved edits that are lost while editing.
Group:	Community Tech
Team members:	Joydeep Sengupta, Dayllan Maza, Harumi Monroy, MusikAnimal, Sam Wilson, Karolin Siebert, Sandister Tei, Sammy Tarling
Lead:	Joydeep Sengupta, Dayllan Maza and Karolin Siebert (Product Owners)

The Edit Recovery feature (previously known as the Auto-save feature) was the #8 wish in the Community Wishlist Survey 2023. This feature saves wikitext and other edit form information while typing, and allows for restoring it after the browser has been accidentally closed, or a power or network outage or browser crash has happened. Requests for this functionality have been made more than once.

This page outlines Community Tech's approach to addressing the needs expressed in this wish's proposal, as well as considering previous discussions around the request.

You can try out the current state of the new feature now on Test Wikipedia. Please leave feedback on the talk page here. The feature will be enabled on more wikis from February 2024.

Useful links:

• Community Wishlist Survey 2023/Editing/Auto-save feature — original wishlist proposal
• mw:Help:Edit Recovery — end-user help page
• mw:Manual:Edit Recovery — technical documentation

Background & problem space

History

Edit-recovery functionality was requested in the 2016 survey and then the 2022 edition.

Additionally, in 2012 a ticket was submitted for this auto-save feature in the localStorage and a patch was started. localStorage size was limited to about 5MB and could only contain only strings; the patch was abandoned in 2017.

Later in 2014 a ticket was submitted for this auto-save feature with an attempt to add auto save using IndexedDB. This patch was also abandoned in 2016.

There has been a discussion that sessionStorage is lost when you switch to a new tab so that localStorage would be an improvement. The trouble with localStorage is that there's a size limit, and multiple autosaves would be likely to exceed it.

Another option is to store the autosave data server-side, where there would be no size limit and recovering data could also be done on a separate device.

In 2022, Community Tech opened an investigation to investigate the StashEdit API as a possible solution. This would mean an autosave lifetime of somewhere in the order of 5 minutes to half an hour. Alternatively, a new database table could be created, in which case the lifetime could be up to 90 days.

Implementation options

For this wish, we looked at a few different ways of implementing it. A primary distinction in these was the location of the auto-save data: if it's stored only on the user's device (what we're calling 'client-side'), or transferred to the Wikimedia servers ('server-side'). See below for the legal concerns relating to the former.

Building upon edit stash —  time limit insufficient

While a user is editing a wiki page, every three seconds the page text is sent to the server as an 'edit stash'. This feature exists so that when the page is finally saved it is quicker to view because the wikitext has already been rendered to HTML. We wondered if we could leverage this feature by making it possible to retrieve the stashed wikitext when re-opening an edit session. This would involve a) increasing the time that the stashed edits are stored for (currently only five minutes); b) changing the stashedit API so that it'd return the wikitext given the unique hash of the content; and c) storing that hash in localStorage in the browser. Apart from not being a great idea to hack an existing feature to do something quite unrelated, this method would be limited to storing the autosave data for something around half an hour, which is not enough.

New database table and API —  complicated, privacy concerns

A whole new database schema and API could be created to store the autosave data. This would be the most flexible and powerful, allowing autosaves to be linked to users and stored indefinitely (much as Phabricator does it). However, with that comes a lot of complexity in devlopment, as well as concerns about data access (see below for the legal issues), and the worry that this would become a 'drafts' feature by another name. The data retention time would be limited to 90 days.

Fully client-side —  private, guards against connectivity loss, simpler

Browsers have three main mechanisms for storing data: sessionStorage, localStorage, and IndexedDB. The first has a reasonably large storage quota but applies only to the 'page session' so is deleted after you close a tab. The second has a much smaller quota which is shared between all of a wiki's usages of localStorage and so also isn't suitable. The last, indexedDB, is most useful, as it allows for lots of data to be stored and persists even if the browser is closed or crashes etc. This is the most promising system to use for autosave, although there are still issues to be considered (such as how to reliably delete the data after it's no longer required).

The other aspect of implementation that we considered was where to build the feature: in MediaWiki core, or in an extension (either new or existing). We settled on Core, for a number of reasons:

Extension  

There was no existing extension that was an obvious good fit for Edit Recovery, so the option was to create a new one. An extension would be suitable if the codebase will become large or have other dependencies, but we didn't think this very likely. It could also make deployment more complicated, as it'd have to undergo security review and the full new-deployment process.

MediaWiki Core  

Features added to core should be applicable to every MediaWiki installation (including non-Wikimedia ones), regardless of what extensions it has installed. They may also be used by extensions, and so having them in core means that those extensions don't need to have extra dependencies on other extensions.

Legal considerations

In previous work around this topic, concerns have been raised about the possibility of people using a private data storage feature for undesirable purposes (for example, sharing an account username and password and then passing data via the autosave feature).

Community Tech asked WMF Legal to weigh in on this, and their analysis was that there are not any strict legal reasons to not build such a feature, but that a few points need to be kept in mind:

• A time limit on the time that the data is stored, with a maximum of 90 days (in line with other private data retention within Wikimedia).
• A way for Trust and Safety to access the data for a given user, if a court requests it (this is common in other systems with this sort of feature, such as email drafts).
• For all of a user's data to be able to be deleted if they request it.

These concerns do not apply if the data is stored client-side, because the data stays on the user's device and is their own responsibility.

Not a 'drafts' feature

One aspect we're very clear that we want to avoid is that it end up being a system for maintaining private drafts of pages. There is a long history around that topic, but the autosave wish is specifically around shorter-term recovery of in-progress edits.

Feature details

Summary: It is possible to close an in-progress edit form (via closing the window, the browser crashing, etc.) and have all data restored when you re-open that same page for editing again (at any later time).

• A page is identified by its title (so if a page is moved during an editing session, the recovery happens at the old name).
• A section of a page is identified by its number (so if a section is added or removed by a different user, the recovery might happen for the wrong section, or not happen at all). It's not possible to edit a section of an old revision.
• While editing, all edit form values are saved to the browser's IndexedDB storage. This is done after every one-second pause in typing, or any time a non-typing field (checkbox etc.) is changed. All core content types are supported (Wikitext, JavaScript, JSON, & CSS), and extensions may provide or remove support for edit recovery as they see fit.
• If a page (or section) is opened for editing and there is saved recovery data, it is loaded into the form.
• The edit-recovery data (for a page and all of its sections) is deleted in a few different situations:
  • after a page is published;
  • when the Cancel link is clicked;
  • when the user logs out;
  • when the user switches from wikitext to visual editing (Visual Editor has its own edit-recovery system);
• After edit-recovery data is loaded the page will show the normal 'are you sure' warning when the user tries to navigate away (even if no further changes have been made).
• If an editing session is underway for a page, and the user navigates away (or opens a separate tab) to edit an old version of the page, then the recovery data is not loaded and no changes to old revisions will be stored.
• Still to be decided: What happens when someone edits the same page at the same time in two different tabs?

Updates

October 25, 2023 – Edit-Recovery is now available for testing in Beta

Hello community, we have some updates. Edit-Recovery wish (formerly known as Auto-save feature) is now available on Beta Cluster, and you are invited to test it.

Start editing any page on any Beta site, for example simple.wikipedia.beta.wmflabs.org, but don't publish your change. Wait 5 seconds and close the tab. Reopen the tab. Your edit should be recovered!

We are working to make the feature more visible with an element like a toast notification when restoring edit data, with the option of discarding the recovered data.

Questions and feedback are welcome.

Thank you, Community Tech Team ––– STei (WMF) (talk) 17:28, 25 October 2023 (UTC)

July 4, 2023 – Investigations and your input

The CommTech team is reviewing any investigations, discussions, patches that have happened around this wish to determine what is next.

We have would like you to answer some questions as follows:

• How long do we need to save the data for the auto-save functionality (Keeping in mind legal matters, as legal implications are reduced by reducing the amount of time we store edits)?
• What should we store in the database to be able to make autosave functionality work? I.e:
  • Edits
  • Revisions
  • Pages
  • Text

Please leave your responses on the talkpage if you have any.

Pinging more voters/users about the feedback round. ––– ​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​ STei (WMF) (talk) 10:50, 4 July 2023 (UTC)

Pinging more voters/users again about the feedback round. ––– ​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​ STei (WMF) (talk) 16:33, 5 July 2023 (UTC)