WMDE Technical Wishes/Geoinformation/Research findings

The focus area “Better support for Geoinformation” was the winner of the Technical Wishes survey 2020. From December 2020 to February 2021, problems and needs were collected on the German-language Wikipedia and on Meta. More than 30 people actively participated in the discussion. In April, the first technical investigations and interviews with community members took place. A summary of the research results is presented below.

Discovery phase[edit]

In December 2020 the discovery phase was launched.

Onwiki problem collection[edit]

~32 individual users were active in discussions on the problem collection page and the project discussion page
22 official problems posted and many others in discussions and interview sign-ups

Problems reviewed and summary posted on german Wikipedia and here on Meta.

Interviews[edit]

4 background interviews with designers, engineers and PMs from previous maps work on Wikipedia and Wikidata
11 user interviews with members of the community

Project involvement:

Geoinformation sits at the intersection of multiple projects, which can be linked. We talked to people with overlapping experience who could talk to the intersections. Darker colors represent intensive involvement while light colors are less involved. All participants edit Wikipedia, most edit Commons as a main part of their work. Wikidata and OSM were both common, but more peripheral.

Findings: Geoinformation and its importance for the wikis[edit]

In the course of the research, the Technical Wishes team learned some basic facts about the topic of geoinformation: What kind of geoinformation is actually used in the wikis? Where and why is it important?

To the best of our knowledge, such in-depth research on the topic of geoinformation has not been done before. For the results of the research can be of use to others working with geoinformation in the wikis in the future, they are presented here.

There is a huge quantity of geoinfo, in many forms, creating a giant maintenance task (done by relatively small number of people)
Differences between wikis are large and it’s unlikely to make something which makes all wikis happy, esp. in relation to maps
Lack of standardization causes a wide range of issues and confusion
Big split between the tools needed by specialist users who focus on geoinfo and those needed to make it more accessible to others who may come across it
Linking to Wikidata is not always done and there is a lot of double work caused by data living in multiple locations (Wikipedias, Commons, Wikivoyage, OSM)

Users edit coordinates more often than maps. Adding and correcting coordinates is a large area of ongoing work, while maps are updated more infrequently and many of the needed maps already exist in some form.
Many different coordinate workflows
Coordinates require less time and expertise to edit than maps, lower barrier to entry.
Coordinate work is linked to maps, but those who do it did not necessarily create the map it is displayed on.
Interactive maps have benefits but there are many barriers to using them. Some want to but can’t, while others are indifferent or satisfied with current solutions.

What is geoinformation?[edit]

Where can geoinformation be found in the wikis? We addressed this question mainly from the perspective of what and where contributors edit. We have understood that geo-information roughly covers maps, coordinates and geoshapes.

We have understood that geoinformation falls broadly into the areas of maps, coordinates and geoshapes.

Coordinates[edit]

Coordinates are geographic data points that are created, stored, linked and displayed in many different places.

Geoinformation in the form of coordinates is linked in the wikis with various other forms of knowledge - for example articles, images or Wikidata objects. And making this connection can also be done differently depending on the form.

- Articles: When coordinates are added to an article, they are often displayed at the top right. How this looks and what happens when you click on it, however, varies greatly from wiki to wiki. Coordinates are also often displayed in info boxes. Adding coordinates at the top of the article and in infoboxes is mostly done via templates.
- Images: Coordinates can also be linked to images via templates. They then appear in the description together with other metadata. They can also be added via Structured Data on Commons which creates machine-readable information on Commons.
- Structured Data is mainly about adding information about what is represented in an image to improve search results. But other information, such as location, can also be added.
- Wikidata object: Various properties can also be added to Wikidata objects to insert geographical information or coordinates. The most commonly used property is geographic coordinates. For example, there is also one property to indicate the location of the photographer. The Wikidata objects can then in turn be linked to articles or images on Commons via their IDs, which then automatically displays their geographical coordinates, e.g. in an infobox.

Geoshapes[edit]

Geoshapes live at the interface between coordinates and maps, as they are created from a set of coordinates but only become useful on a map.

As far as we have learned, there are three ways in which geoshapes can be saved:

WIWOSM: Geoshapes can be displayed in various tools (e.g. OSM-Gadget or WikiMiniAtlas; maybe there are more). In this case, they are not stored in the wikis, but imported from OpenStreetMap, using the WIWOSM tool as a cache.
.map file on Commons: Alternatively, geoshapes are uploaded to Commons. For this, geoJSON is used to create a .map file in the Commons data namespace.
Wikidata: Geoshapes can also be linked to a Wikidata object. A Commons .map file can be linked to an object, via the Wikidata property geographic shape. For example, the Wikidata object for Berlin has a property that specifies the geographic shape of Berlin, and this then links to the Commons file.

Maps[edit]

Maps are visual representations of geographic data and get added value, among other things, by adding coordinates and geoshapes.

When it comes to maps, we have understood that there are roughly two categories in terms of their creation and interaction: interactive (autogenerated) and static (manually or semi-manually generated) maps. Perhaps these are not the most established terms; if not, feel free to comment on the discussion page.

We have defined interactive (autogenerated) maps as maps,

that can be displayed in full screen mode,
where you can zoom in and out, maybe click on markers or links to articles,
but most importantly: as you move around the map, the map loads more or less information based on what is currently in view.

As far as we know, there are three commonly used types of interactive maps:

Two are pop-ups: the OSM Gadget and WikiMiniAtlas. They were both developed by community members of the German-language Wikipedia and are linked on article pages via various icons that appear next to the coordinates.
Also there are Kartographer generated maps embedded in a page. Such maps are often used on e.g. the English Wikipedia and on dewikivoyage.

We defined static (manually or semi-manually generated) maps as maps

which are an image file or based on one,
in which the zoom level cannot be adjusted,
which require programmes outside the wiki to edit them.

We found two types of static maps:

"semi-interactive" maps: maps generated via templates. The maps are built into the templates and you can add geographic data points (coordinates) to the map via the template. Sometimes you can click on these points and you will be redirected to the ♁GeoHack tool, but sometimes not.
purely static maps: images in .jpeg, .png or .svg formats. These are created by community members with the necessary expertise and programmes. Requests for changes and creations come through the map workshops.

How is geoinformation linked across projects?[edit]

The broken lines show items that can be linked but also might not. It is very hard to understand the whole system, but the most important insight we got is that geoinformation is not only collected in one place. E.g. OSM contains a lot of geodata, but it does not exist only there. OSM is used as a basis for the interactive maps, but the coordinates or geoshapes are not always imported from there, they may come from Commons as well. The same could be said about Wikidata: a lot of data is collected there, but it is not always linked to other projects (Wikipedia or Commons).

The system is very complex and has grown a lot over the years.

Why is geoinformation important?[edit]

Beyond being the top voted topic in the 2020 survey, why is geoinformation important to the wiki projects?

Geoinformation is extremely widespread across articles, items, and images
- Lots of types of content are inherently tied to geoinfo: physical objects (buildings, art, monuments, etc), places (cities, states, countries, etc.), people’s birthplaces, even species
- in 80-90% of wikipedia articles according to one interviewee, in wikidata items, in images on commons, in all wikivoyage articles (one of the main aspects of creating a new article). This creates a huge amount of data and also of maintenance effort.
Geoinformation visualizes existing vs. missing information
- Ex. editors map wikidata items missing pictures in a certain location
- Allows the community to look at the big picture, not just indiv. articles

Interactive maps and interfaces make information more visually rich
- There is some information that can only be presented in a map, bringing things to light that might not otherwise: spatial relationships
- Can make articles “more three-dimensional” (P15)
- Allows information from around the world to be organized and structured in a clear way
- Facilitated through Wikidata map query (entry point)
Maps can communicate when words sometimes can’t
- Though not universal, maps can be visually meaningful across some cultural and language barriers

Geoinformation leads to exploration and discovery
- Many people use the “Show nearby” feature and want it to be improved because it’s an important way to explore the currently available knowledge on various platforms
- Users can visit easily, including photographers
Geoinformation is a link between projects (internal and external)
- A map can show images from commons, articles from wikipedia and items from wikidata
- Coordinates also allow other maps services to integrate wikipedia articles; re-use of information increases value of work
Coordinates differentiate between objects with same name
- Ex. two churches with the same name in the same village can be identified by location
Geoinformation helps to catch and correct mistakes
- Ex. Adding photographer location coordinates means others can check if it’s actually the object the author claims it is
- Ex. Obviously out of place markers show on maps (middle of ocean)

Problem statements[edit]

Where and when do problems arise? What are the effects of these problems? Who is affected by them? What is the impact? We have identified many overarching problems that do not fall into a specific area or workflow, but also specific problems in the areas of maps and coordinates.

General: What are the major problems?[edit]

Geoinformation as a field is complicated[edit]

Large amount to learn before editing related information
- Need to learn both about how geoinformation works generally, then how it works onwiki
Often involves specialized knowledge and technical skills
- Sometimes both as with GeoJSON or with coordinate precision
- When dealing with the backend, it can get even more complex, requiring significant effort to create the community created solutions which currently exist
The field itself is not always standardized
- Ex. Many ways of formatting coordinates

Lack of standardization has led to diverse workflows and complex workarounds[edit]

Inconsistent, decentralized solutions
- Many different templates are used, functioning differently across wikis
- Variety of community-generated and external tools support work
Hard to learn
- Difficult for a general or new editor to understand
- Many of the community built tools are useful but are not user friendly
- Rules about when to use what are not always transparent or might not exist (partly documentation issue, partly coordination issue)
Visual representation of geo-information is different per wiki
- Not just way of working differs, but also what the result looks like
  - For example, adding coordinates to an article usually results in them being displayed at the top right of the article, but how they are displayed or what happens when you click on them varies from wiki to wiki.
- Many of the current representation don’t work on mobile well and can create odd formatting bugs
  - Ex. The marker for Miami is not shown on the map in the Android app.

dewiki
frwiki
jawiki
ruwiki

Community tools provide essential but incomplete support 🔗[edit]

Large variety of tools are integral parts of existing workflows
- Many have been around for a long time and are well loved
- Some are specific to wikis, while others are totally external
Multiple tools do similar things
- Many map items, articles, images or SPARQL queries (ex. wikishootme)
- Many help in formatting coordinates, esp. for different templates (ex. GeoLocator)
Many tools need maintenance, while others are fully broken
- Some of even the frequently used tools need significant attention

No singular workflow > No singular topic everyone cares about[edit]

While some patterns emerged, each user we spoke to had a different workflow in relation to geoinformation
Wanting a better way to input coordinates was the most consistent request, but beyond that editors have a wide variety of focuses, motivations and ways of working

Geodata is not linked enough yet[edit]

Coordinates are not always stored in one central location, creating duplicate work and possible mismatches
Wikidata geoinfo is not being used to its full potential
- Wikidata is not universally used to store geoinfo
- Some users have a distrust of Wikidata, while others are limited because of the technical complexity
- Linking is often done using lua templates, which can be brittle and difficult to troubleshoot
Linking between Wikidata and OSM needs improvement
- Current linking is one directional: “Wikidata item doesn’t have a link to OSM but OSM has a link to Wikidata” (P4)
- Limit to how strong the link can be because OSM doesn’t have persistent identifiers like wikidata

Geoinformation is not static across time[edit]

Static map of Bavaria in the 10th century — Static maps are used to show historical borders and border changes

Borders and coordinates change, requiring updating across projects
- Ex. if a city acquires a new county, the center coordinate shifts
Keeping geoinfo up-to-date is a big part of the maintenance work
- Maintaining correctness is important editing work, alongside adding new coordinates or maps
Not currently possible to easily visualize important historical geographic shifts, particularly in an interactive way

Problem statement: Coordinates[edit]

Coordinates are difficult to enter and edit[edit]

Different projects require different formats and entry methods
- Editors have to use outside tools to find the coordinate, and then often reformat coordinates
- Even within the same project, there may be multiple coordinate templates in use
Hard to know if coordinate is correct and to adjust if wrong
- Most inputs across the wiki projects require users to enter coordinates in numbers and letters, without being able to check on a map if it’s correct
- Commons and Wikidata have maps, but it is not possible to drag the marker to adjust and fix incorrect coordinates
Users want a map-based picker with satellite/aerial images integrated
- Satellite/aerial images are important to verify if it’s the correct location
- Currently, editors must use external map services to figure out their location and then copy+paste (and possibly reformat)

Incorrect coordinates are difficult to find[edit]

Poor quality or conflicting geodata is a common problem
- Incorrect coordinates can be worse than having no coordinate at all
- Not always obvious at a glance that something is wrong
- Many coordinates come from public databases where coordinates are too strongly rounded (creates a grid pattern on maps)
- A single object in Wikidata might have multiple entries from different language Wikipedias
- Coordinates from mass import from 500px or Flickr to commons are often wrong or too general (central point of a country)
- Lat/long are sometimes swapped on accident, putting the coordinates in a completely wrong location
Community wants better tools to automatically identify coordinates needing fixing
- Ex. analyze wikidata objects to see if coord matches “is in”

Special:Nearby has limited features[edit]

Search field requested
- Want to be able to search by location by place name, coordinate, or article name instead of only being able to see ‘nearby’
Want to be able to filter what is displayed
- For example, by importance if looking at a whole city
Many commonly community-created tools do similar things, clearly demonstrating a need
Special:Nearby is used on all of the sister sites, including Commons, Wikidata and Wikivoyage

Problem statement: Maps[edit]

Each type of map creates different barriers to edit[edit]

Static maps need outside tools for editing and are not collaborative
- Require significant time and expertise to create and update
- svg, png, jpg all need to be edited off-wiki, while coordination and requests happen in the map workshops
- Leads to lots of maintenance and outdated maps

Semi-interactive maps (semi-manually generated) maps can be buggy and don’t work well on mobile
- Maps created using templates are seen as more collaborate and can integrate multiple markers, but their display across devices does not always work

Interactive maps can be more easily edited by many but it is difficult to learn how, especially for non-technical users

Difficult to show the ‘right’ information[edit]

A fundamental difficulty of maps is figuring out which information to show and the options onwiki are relatively limited
- Community members want more options to configure which information is shown in a map
- Beyond filtering the available information in OSM, it can also be important to filter between articles or wikidata items because the large number of them could make a map unreadable

Geoshapes are hard to show and edit[edit]

Ability to add/edit markers, geoshapes and lines is critical but currently difficult to accomplish
- Working with markers and shapes is possible but can be confusing
- This is a key feature to making interactive maps useful. Many geographic entities cannot be easily displayed with only a single point.
Multiple ways to create and link this type of information
- Easiest way using the Kartographer VisualEditor interface seems mostly unknown
- Pulling in shapes from other sources requires many steps

Kartographer[edit]

Kartograher is problematic[edit]

Kartographer was developed by WMF to display maps in the wikis. While the feature has some of the desired functionality, it also has its own problems. Since we work on MediaWiki extensions, it was important for us to look at this feature to see what possibilities it offers. Not available on all wikis because of FlaggedRevisions

Current functionality is not widely known or understood
- Many users know it exists, but not what features it has or how to use it
- Sometimes misunderstandings were quite extreme and features were requested which already exist
- Many users tried it but gave up on it quickly, deciding it didn’t fit their needs or that it was too technically advanced for them
- Not a single user we interviewed knew about the existing visual interface
- Difficult to learn and use
  - Maps are often embedded in templates, so must use wikitext to edit
  - Templates have been created as alternatives to <mapframe>, which can also be finicky

But…Kartographer might solve some problems[edit]

Makes showing geoshapes easier, either importing from Commons, Wikidata (incl. OSM), geoJSON or by simply drawing directly on a map
Current location maps created by templates are displayed incorrectly on mobile, in the apps, Mediaviewer, Page Previews and some skins. Kartographer-based maps work throughout.
Easy linking to external map services without needing to open a new tab
Creating color-coded maps as svg files is cumbersome manual work, requiring special skills and off wiki programs. Potentially easier using geoshapes
Show multiple markers from within the page, incl. with icons and images
Allows for collaboration
Also includes <maplink> so that clicking on a coordinate opens it in a full screen map

Tools and resources[edit]

Coordinates[edit]

GeoLocator: important for finding and formatting coordinates (currently partially defective: OSM is no longer usable as a background map)http://tools.freeside.sk/geolocator/geolocator.html