User:Johan (WMF)/Creating Wikipedia

From Meta, a Wikimedia project coordination wiki

This text is a shorter version of the presentation ”Creating Wikipedia: Cooperation and decision-making in the software development process” that I gave at the Swedish Software Technology Exchange Workshop in Malmö 2018. This interpretation of our software development process is my own, and doesn't necessary reflect the position of the Wikimedia Foundation. This was written with a non-Wikimedia audience in mind.

Creating Wikipedia: Cooperation and decision-making in the software development process[edit]

Wikipedia is an encyclopedia that everyone can edit. It’s free – both gratis and libre, as we say. Free as in beer, free as in speech. You can read it for free, but you can also use it elsewhere, copy it, make your own version. Text and illustrations on Wikipedia are typically published under a Creative Commons license, but the software is published under the GNU General Public License. It’s run by pseudonymous volunteer editing communities who together form the larger part of the Wikimedia movement – the Wikipedia communities, the Wikimedia organisations, Wikipedia’s sister projects such as Wikimedia Commons (a collection of images, videos and sounds and other media files), Wikidata (a database of machine-readable facts), Wiktionary (a dictionary) and others.

From a technical perspective, Wikipedia is a MediaWiki wiki. MediaWiki is the software that has been developed for Wikipedia since 2002. There’s an entire infrastructure around Wikipedia, of course, but a simplified explanation could focus on MediaWiki core, MediaWiki extensions and on-wiki scripts and bots. MediaWiki core is the central functionality of the wiki, but extensions in MediaWiki aren’t nice-to-have extras, fancy frills on top of the main product – they’re a fundamental part. We even build our editing tools as extensions, and a wiki without editing is a pretty useless things. You want new functionality? The general rule is that you build or improve an extension, rather than messing around with MediaWiki core. And this is important.

MediaWiki is developed for the Wikimedia communities and tailored to their needs. It’s used elsewhere – spread not least because of the popularity of Wikipedia – but the software is optimised to handle large projects. It does this very well, but if one wants to have a small internal wiki for twenty persons, MediaWiki might not necessarily be the obvious choice.

The idea is that everyone uses MediaWiki core as it is and then tailor their wikis to their needs using MediaWiki extensions. Even the different Wikimedia wikis vary a lot when it comes to how they work and what purpose and workflows they have. When third-party users edit MediaWiki core, they might end up forking without necessarily meaning to. Unless they want to take on a growing maintenance burden, they end up being unable to use new versions and new extensions might not work for them. This could have been fine, if it had meant an ecosystem of MediaWiki development, with different branches on the tree having a healthy, living developer communities. However, as the Wikimedia movement dominates MediaWiki development, the risk is that they will simply end up carrying the burden on their own.

The Wikimedia Foundation is the non-profit that owns and operates Wikipedia. Many of the active MediaWiki developers are employed by the Foundation. There is a number of Wikimedia affiliates, typically local organisations. They tend to focus on things like supporting editors in their region or working with galleries, libraries, archives or museums to upload material to Wikimedia Commons or Wikidata, but a few do some software development of their own, and Wikimedia Germany – a behemoth among the Wikimedia affiliates, by far bigger than any of the others – take responsibility for some development that is at the core of the future strategies of the Wikimedia movement. MediaWiki being an open source project and the Wikimedia universe overwhelmingly a volunteer effort, the technical MediaWiki community has a number of volunteer developers.

The editing communities on the Wikimedia wikis – who consider themselves the rightful owners, or at least stewards, of the wikis – have a strong voice in software development, convinced that software development should focus on what they see as their current needs. They don’t speak with one voice: not only do individual editors disagree on the course, depending on how they interact with the wikis and what kind of work they typically take on, but even when there’s a rough consensus on their particular wiki, English Wikipedia and Swedish Wikipedia might not have the same needs, and Italian Wikisource will definitely have a completely different set of priorities.

They make their voices heard in a number of different places – in conversations on their local Wikimedia wikis, on Wikimedia mailing lists, in the task management system where the specifics of technical changes are discussed and bugs are reported, as well as in processes set up to make sure they are involved in the software development, both around specific bigger changes and annual wishlists focused on improving tools they use for their workflows. Many of these take place on the wikis, not necessarily because a wiki is a fantastic place for hosting large-scale discussions and decision-making processes but because that’s where Wikimedians feel comfortable and in control.

There are also external stakeholders – readers, donors and third-party MediaWiki users. It is, of course, important for the Wikimedia Foundation that donors can’t decide on the content of the wikis or how the Foundation will work, but it still occasionally accepts restricted grants to develop certain features that are high on its priority list. While readers make up the overwhelming majority of the actual users of Wikipedia, they don’t have much of a voice in the technical development. Careful not to collect too much information about its visitors, Wikipedia knows far less about their readers' behaviour than other large websites. Readers don’t participate in the discussions. They’re rarely invited to give their opinions on technical development. This means that others have to do their best to advocate on readers' behalf – this often falls to those who are paid to think about how to make the content more accessible to readers, or Wikimedians who worry about how the content they produce will be presented to their audience. In comparison to their numbers, readers typically don’t have much of a voice in MediaWiki development. Nor can the Wikimedia developers typically afford to spend much time on the needs of third-party users. They’re not ignored, but in the balance between focusing on Wikimedia editors or third-party users, the Wikimedians will almost always win – an unsurprising effect of the fact that MediaWiki development is largely controlled by a movement focused on making better tools for the Wikimedia wikis.

There are many complications in the technical decision-making process around Wikipedia. Even when just focusing on the needs of the editing community, there are 800 wikis in roughly 200 languages. Some of these are better at making their voices heard than others, and some small communities can’t communicate in languages understood by the larger movement. English Wikipedia is the one dominating wiki, size-wise, and best at making noise when it’s unhappy – but precisely because it’s bigger than all the other wikis, many of its workflows are atypical and have no bearing on most Wikimedia wikis.

As with many other organisations, the Wikimedia movement suffers from survivorship bias creating inertia in the development process: those who stayed around were comfortable enough with how the wiki works now, but those who couldn’t handle the tools, who were turned off and would have benefited from more radical changes, were turned away. Development needs to balance paying attention to current workflows and not disrupting the work already done by the existing community, and at the same time make editing more accessible to those who would have material to add but who currently don’t because of technical barriers.

Because of this, the decision-making process is often a patchwork. It all comes down to the needs of the Wikimedia wikis. What those are, however, isn’t always obvious.