Learning patterns/Writing a new MediaWiki extension for deployment on a Wikimedia project

From Meta, a Wikimedia project coordination wiki
A learning pattern foronline engagement
Writing a new MediaWiki extension for deployment on a Wikimedia project
problemYou want to improve the software that runs Wikipedia and the other Wikimedia projects
solutionIdentify your users, start with a prototype, and have a plan for implementation
creatorHarej
endorse
created on21:15, 4 May 2016 (UTC)
status:DRAFT


What problem does this solve?[edit]

You want to make MediaWiki do something it does not currently do.

There are many processes on Wikipedia and the other wikis that rely on copying and pasting text and doing all manner of inefficient things. This makes the work of volunteers much harder. But with software, you can make things easier.

What is the solution?[edit]

After you have a general idea of what you are interested in working on, identify your community. While you may have an impression of how your new extension should work, there may be factors you are not considering. See Designing and developing for an active, existing project.

Before writing the extension, considering a prototype system such as a JavaScript gadget or an offsite tool on Toolforge. These are much easier to set up and may in fact solve the problem you were seeking to address. If this is not adequate, however, it will strengthen your case for writing an extension.

The key documentation for writing an extension for WMF deployment is mw:Writing an extension for deployment. The key thing here is that what you write needs to be internationalized so that it can support multiple languages, and that it will need to undergo multiple levels of review. This review process is lengthy but ensures that the code you are running is secure and efficient—in other words, it won't bring down the servers.

Once the extension is ready, give your users an opportunity to test the extension. Your goal is to get it deployed. In doing so, be open to feedback—you want your work to be useful to others, after all. Additionally, do not try to "force" your extension on people; this can lead to a huge backlash that will prevent anyone from using it, even people who want to.

Things to consider[edit]

  • Staffing: Who is going to work with you on this extension? If you are the sole person scoping, designing, coding, and testing the extension, that makes managing the project easier. However, with multiple people working on the extension, you benefit from having varied experiences and skillsets that should lead to a better work product. At the least, you should have someone who is responsible for deciding what is and isn't in scope for the project (this role is typically served by a product manager), someone responsible for designing initial prototypes and wireframes, someone to write the code, and someone to test it once it's written. (One person can take on multiple roles, of course.)
  • Scope: A poorly defined project scope can kill a project. It is very tempting to include additional features to make a specific person or group of people happy, or to decide that your work isn't good enough and should be made better. However, a perfect software feature is impossible and also very subjective. It is better to put out something small and basic than nothing at all. Based on the underlying goal you are trying to solve, define the minimum viable product, the absolute smallest thing that can be shipped.
  • Ensure you have a shared vision: If each member on the team has a different idea of what implementing the extension might look like, it's very hard for everyone to work effectively together. The CollaborationKit team addressed this issue by convening a three-day in-person meeting to discuss the extension's feature set and approach to implementation. This meeting was absolutely worth the cost because otherwise we would've never gotten anywhere. It is worth including in your budget.
  • Track progress: Use a kanban board to track what's in your backlog, what's in progress, and what's done. Phabricator lets you do this easily; see the CollaborationKit board as an example.
  • Security and code review: Security review by the Wikimedia Foundation's security team is a mandatory prerequisite for deployment to the production systems. Ask for a review as far in advance as possible so that they know to expect you. Hint: this is only required for the first deployment. So if you follow best practice and release a minimum viable product, the amount of code to review should be less, the review should be easier, and the extension should be deployed sooner.
  • Testing sites: Consider setting up your own wiki on a Cloud VPS so you can rapidly test your code as you write it. See the WPX staging wiki as an example. This is a good start for testing, but not sufficient for checking how well the code will work in production. There is a very large difference between how a standalone MediaWiki works on a single machine and how MediaWiki works in a massive production setting like the Wikimedia projects. After clearing security review you will need to test on the Beta Cluster and then on Test Wikipedia (or Test Wikidata) to identify bugs that may only be visible in more sophisticated setups. One example of a complication that may occur is race conditions that come from the replica DBs not being in sync with the master DB. Enabling use of the extension with the PatchDemo tool can make testing individual changes and integration with other extensions easier as well.
  • Community consensus: Once you've worked out all the bugs, you will need the approval of the community you are deploying the extension to. Make sure your extension avoids creating extra work for the community or otherwise disrupts their user experience.

Endorsements[edit]

  • In addition to the gadget, another convenient way to create a prototype for an extension is to write a Lua module. If the extension is associated with the wiki syntax, and not with the interface, it is Lua that makes the implementation, which is closer to the final result. At the same time, you can make rapid changes and immediately receive feedback from the community. For example, in the Wikidata module project it all started with the module, and only after a lot of iterations and changes, when it became finally clear how everything should work, work on the MediaWiki extension began. — putnik 10:28, 22 March 2017 (UTC)[reply]
  • This is exactly what happened in the d:Wikidata:Primary_sources_tool lifecycle. The first version was a gadget, while the StrepHit project brought it to the next level: a MediaWiki extension. Hjfocs (talk) 10:56, 30 May 2018 (UTC)[reply]
  • Workboards and sorting them by status is probably one of the best things I got out of this project, as a developer. Applies to whatever in whatever role, though. I may write a learning pattern about that later as part of a report on a followup project to the project this was from... -— Isarra 16:39, 31 August 2019 (UTC)[reply]
  • This is what I'm currently doing to create mw:Extension:GlobalWatchlist. I added a note about using PatchDemo to the considerations, since that has been a tremendous aid in testing and development. --DannyS712 (talk) 01:34, 6 October 2020 (UTC)[reply]

See also[edit]

Related patterns[edit]

External links[edit]

References[edit]