Information about Google Search result changes due to Project Toledo

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search

Other languages:
English • ‎العربية

Google search result changes due to Project Toledo - General information[edit]

Google Inc. recently started piloting for some languages, changes in the way Google presents machine translated content in search results. Due to this, users searching for content in these languages may get access to a machine translated version of an English Wikipedia article. This can happen if there is no article for that topic in Wikipedia of that language, or the article is substantially inadequate to provide any information. This project was rolled out for Indonesian in late 2018. Google Inc. plans to make this gradually available for more languages in the near future. This particular initiative has been named Project Toledo.

Explanation of Project Toledo[edit]

Google Inc.’s Project Toledo is a pilot that changes the way Google presents machine translated content in search results for some languages. As Wikipedia pages are shown significantly in search results, Google invited Wikimedia Foundation to provide feedback prior to the launch.

Expanding access and availability of content in all the languages of the world is critical to Wikimedia’s mission. For languages where the content gap is wide, we see this pilot as an opportunity to extend access to knowledge and information. By making translations more visible, speakers of the selected languages have access to knowledge areas that have not been covered yet in their Wikipedias. Besides providing an interim method to meet the immediate reading needs of the users of those languages, this will also make them aware of the process to contribute such content back to Wikipedia. In this way, Wikipedia editors accessing automatically translated content are encouraged to start a community-created version for the topic. As more editor driven content is added, other users will be able find the information about those topics much more easily in future. More details about this process can be found in the Making contributions to Wikipedia pages section.

Project pilot[edit]

Currently, when someone uses Google search in a language other than English, Google will often surface an English Wikipedia article in the search results with a preview in English and an option to translate the article.

In this example screenshot (Figure 1), a search in Indonesian is showing the first result with contents in English:

Figure 1 - search result with English content.

In this pilot, when someone searches for a topic in a language where the project is running, and there is no existing Wikipedia article in that language with information about that topic, Google will provide a machine-translated version of the English Wikipedia article in the search results.

You can see a screenshot of this new functionality in Figure 2:

Figure 2 - search result with machine translated version

Users will also see:

  • a message stating that the contents of that particular search result is machine translated, and
  • the option to read the original article in English Wikipedia.

When there is a quality article in Wikipedia, Google’s ranking algorithm will favour the Wikipedia article in that language over the machine translated version.

Please note: This new functionality does not alter anything on the Wikipedia website, and only applies to searches for content conducted on Google Search.

Getting to translated Wikipedia pages[edit]

There are three ways to access a Wikipedia translated page:

Google Search Results[edit]

Google search results clearly labeled as translations that will take the user directly to a translated page. These are provided by Project Toledo. You can see an example of this in the screenshot in Figure 3:

Figure 3 - Google search results clearly labeled as translations

Translate Action on Search Results[edit]

When search results exist in a different language where Toledo is not available, Google shows an option to translate from Google. In the example in Figure 4 the second result shows the English Wikipedia result with the translate option:

Figure 4 - Search result showing option to translate the content

Note at the bottom-left corner of the image how the destination URL (as indicated by the pointer) is coming from Google Translate.

Google Translate[edit]

Users can go to Google Translate and paste any URL in order to access a translated version of the page.

Making contributions to Wikipedia pages[edit]

We mentioned earlier that users will have an option to contribute to Wikipedia from the machine translated page which they may choose to read from the Google search results. This set of options, provided by the External Guidance extension, were built specifically by the WMF team, to ensure users can get back to the Wikipedia contribution work-flow to help improve the Wikipedia articles in their language for those topic. The extension presents different options to contribute and redirects users to the regular editing tools on the corresponding Wikipedia. These changes were designed after reviewing the Project Toledo proposal. An illustration of this workflow can be seen in the flow-chart shown in Figure 5:

Figure 5 - Flow chart of workflow with the External Guidance extension

Wikipedia users can contribute to Wikipedia by writing a new article either from scratch or by expanding it from content shown in the automatic translation page from their smartphone or desktop computer. The workflow for editing Wikipedia from a smartphone is slightly different than a desktop, especially for the articles that do not exist in Wikipedia. Please read the descriptions in the section below for more details:

New articles[edit]

If a new article needs to be created, there are different options that can be used from desktop and smartphone. If the user would like to create a new article on the desktop, they have two options:

  1. Using Visual Editor to write a completely new article, or
  2. Using Content Translation to translate from the translation result (only available for logged-in users).

If using a smartphone, then the only option is to write the article from scratch, as there is no option currently available on mobile to create a new page from the translation result.


Existing articles[edit]

For articles that already exist on Wikipedia, the contribution workflow would be the same for smartphone and desktop. Users will be redirected to the existing Wikipedia page when they make a choice to improve the article.

More information and illustrations of this the editing process are available on workflow guidance document.

Impact of this change[edit]

Activity happening through External Guidance is captured in the analytics report as well as Recent Changes on Wikipedia to give an overall picture of how much content is read and created from externally translated pages. These are the aspects measured:

  • Reading to contribution funnel - How many people are accessing translated pages, how many of them decide to contribute, and how many of those users have completed their contribution?
  • Content creation - How many edits and pages were published as a result of people coming from an externally translated page.
  • Content survival - How many contributions have been reverted or not, which gives an idea of the quality of those contributions.

These measurements can be compared to the same activity of regular wiki pages.

How did the Wikimedia Foundation work with Google on this pilot?[edit]

Prior to the rollout Google Inc. reached out to WMF informing us about this project and asked for feedback. These conversations have happened as part of regular sync-ups between Wikimedia Foundation, and Google Inc. As Wikipedia articles feature prominently on Google Search Results, the WMF teams reviewed the proposal and suggested adaptations on how Wikipedia was attributed on the new functionality of the search results feature. Additionally, the External Guidance extension was also created so that users could get an easy entry point through which they could improve articles for other users.

As Google rolls out this functionality to more languages, WMF will continue to speak with them, share feedback on the pilot, and ensure suggestions from Wikipedia users are recorded. The Wikimedia Foundation staff in partnership with volunteers from the relevant Wikipedia communities will maintain communication with Google to share any feedback as the project continues.

If you have questions about this pilot, please ask them on the Talk page. Thank you!