Geonotice/Survey results

Results of the Cristian's (Wiki) Geolocation Survey

In the following we discuss the results of the survey about the geolocation tools which can be used on it.wiki. The survey has been conducted from 2 to 7 november 2011. 118 person have participated.

Aim of the survey was to compare the precision of the IP-based geolocation used by geoiplookup.wikimedia.org and other geolocation tools embedded in modern browsers. To participate to the survey it was needed to visit the following pages: http://geoiplookup.wikimedia.org
http://placepuppy.it/geo.html

Introduction

Data gathering method

Answers have been collected through a Google Docs form, available at http://bit.ly/GeoSurvey (direct link).

The first page, geoiplookup.wikimedia.org, returned a string similar the following:
Geo = {"city":"Cameri","country":"IT","lat":"45.500000","lon":"8.650000","IP":"2.32.43.196","netmask":"19"}

The second, placepuppy.it/geo.html, was based on the code presented in Appendix A and returns a string like:
Latitude: 0.00 Longitude: 0.00

To find the coordinates of the "true location" (see questions 5-6-7 below) the proposed method was the following:

[…] use Google Maps (http://maps.google.it), write the name of the inserted city, click with the right button and choose "What's here?", the two numbers which appear are the latitude and the longitude.

Different methods has been proposed as well (with a reference to the page: http://en.wikipedia.org/wiki/Wikipedia:Obtaining_geographic_coordinates)

Questions

These questions were asked:

1. May you tell me the latitude where you are positioned using geoiplookup.wikimedia.org?
2. May you tell me the longitude where you are positioned using geoiplookup.wikimedia.org?
3. May you tell me the latitude where you are positioned using placepuppy.it/geo.html?
4. May you tell me the longitude where you are positioned using placepuppy.it/geo.html?
5. May you tell me where's the city where you are now (where the pc you are using is located?)
6. Will you be so kind to indicate the latitude of the city you write above here?
7. Will you be so kind to indicate the longitude of the city you write above here?
8. Evaluate the result you obtained using geoiplookup.wikimedia.org [Nation]
9. Evaluate the result you obtained using geoiplookup.wikimedia.org [Region]
10. Evaluate the result you obtained using geoiplookup.wikimedia.org [Province]
11. Evaluate the result you obtained using geoiplookup.wikimedia.org [Town]
12. Evaluate the result you obtained using placepuppy.it/geo.html [Nation]
13. Evaluate the result you obtained using placepuppy.it/geo.html [Region]
14. Evaluate the result you obtained using placepuppy.it/geo.html [Province]
15. Evaluate the result you obtained using placepuppy.it/geo.html [Town]
16. What browser are you using to surf the web?
17. May you tell me the version of your browser?
18. Does your browser support geolocation, as far as you can tell?

Then for each subsimision the following data were available:

• form sending timestamp;
• coordinates detected by geoiplookup.wikimedia.org and their evaluation
• coordinates detected by placepuppy.it/geo.html and their evaluation
• name and coordinates of the city where the person who submitted the form was.
• information about the browser (type, version, localization support)

Data collection

Unfortunately due to an error in choosing Google Doc form settings some data (22 records) were corrupted and resulted unusable, in fact both tools returned answers (coordinate pairs) following a USA locale and in particular using the dot (".") as radix point, while the form was set to use the IT locale and then numbers inserted with the dot have been interpreted as dates and converted. It was not possible, even after some attempts, to recover the original data so these records have been discarded.

So we had 96 record available. Figure 1 shows the progress of user submission where negative numbers indicates discarded data:

Figure 1: form submissions vs time

The regional distribution of the submission is visualized by the following:

In particular for Lombardy (the region with most submission) the distribution in provinces is:

Figure 2: Form submission per Italian region

8 submissions have been sent from foreign countries.

Figure 3: Form sent from Lombardy

All data and their analysis are available as OSD (LibreOffice, OpenOffice) file for download here: http://wiki.wikimedia.it/wiki/File:Cristian,_geonotice_survey.ods

Furthermore, for people who have indicated only their "town" (real location) and not the coordinates these data have been completed using the method described above.

Data Analysis

Using data about the real location provided by the users the root mean square (rms) has been computed for each value of latitude and longitude revealed. Using the mean quadratic radius of Earth as a constant equal to 6373 km the various area indicated have been computed.

The mean of real position collected is the following:
lat = 44.61147757
long = 9.72832298
located near Bardi (PR)

Maximum and minimum values found where:
(B) lat_max = 52.355788, long_min = -8.617963
(A) long_max = 16.16666667, lat_min = 37.520619

Methodological annotation

With RMS values for latitude and longitude expresses in degrees, they have been combined in quadrature, the error due to this formula instead of using the correct one (since these are angles on an approximate sphere and not on a plane) is negligible.

Analysis of the geolocalization results obtained via geoiplookup.wikimedia.org

For geoiplookup.wikimedia.org results are:
rms_lat = 5.0°
rms_long = 2.29°

From the data we calculate that the position detected are on average in an area of radius 97 km from the real position. For comparison, here is a map which shows which cities would include an area with the same radius centered around Milan:

Localization error of geoiplookup.wikimedia.org, the circle is centered on Milan

Evaluation of the localization by the users

In the following we report the results of the evaluation performed by the users about the quality of the localization. Please note that results should be read as “It has been correctly identified at leat the [town/province/region]” so values grow from left to right since every cell on the right contains the previous values, in the first row absolute values are presented, in the second percentages are calculated.

The tools identifies correctly at least the region in more than the 60 % of cases. The state is almost always correctly identified.

Correct town	Bordering town	Correct province	Bordering province	Correct region	Bordering region	Correct state
27	32	37	56	60	68	94
28.13%	33.33%	38.54%	58.33%	62.50%	70.83%	97.92%

Analysis of the geolocalization results obtained via browser

For the geolocalization via browser it's important to note that in 10 cases out of the 96 examined (10.4 %) the localization procedure did not work, generating an error or the null result “0. 0.”.

This can be due to a number of factors including the type of browser and version or connection-related factor (in fact it has been demonstrated that, when using the same browser and PC connected from different location with different provider, in some cases the geolocalization information was not available).

The analysis has then been limited to the 86 cases which returned a result, obtaining: rms_lat = 0.43° rms_long = 0.66°

from this result we can calculate that the positions detected are on average in an area of radius 14 km from the real position. For comparison, here is a map which shows which cities would include an area with the same radius centered around Milan:

Browser localization error, the circle is centered around Milan

Evaluation of the localization by the users

Due to a problem with the form used to collect data the labels of the different answers have been messed, then results had to be reconstruted from available data. What obtained seems compatible with the data of the previous section. The 10 cases in which the tool did not work have been classified as “Incorrect state”.

As it can be seen from the table, using localization via browser the ability to identify correctly the city is much higher (58 % versus 28 % by geoiplookup.wikimedia.org) and the correct region is identified in more that the 80 % of cases (83 % versus 22 % by geoiplookup.wikimedia.org), but the likelihood of failure limits the usefulness of the tool.

Correct town	Bordering town	Correct province	Bordering province	Correct region	Bordering region	Correct state
56	60	67	69	80	82	84
58.33%	62.50%	69.79%	71.88%	83.33%	85.42%	87.50%

Browser informations

Since the sample was not prepared to be statistically significant, a predominance of Mozilla Firefox and Google Chrome browsers has been registered, Internet Explorer being significantly underrepresented. For the former two browsers a prevalence of versions 3.6.X (7) and 7.00.01 (40) for Firefox and 15.X (19) for Chrome is registered.

Browser used

From these answer, however, it is possible to see than a vast part of users does not know that his browser, very likely, supports geolocalization.

Conclusions

We have analysed 96 cases of use of two different tools of geolocalization: IP-based geolocalization (using geoiplookup.wikimedia.org) and using web browsers, for users mostly located in northern Italy. IP-based localization has an acceptable precision, positioning a user in the correct region in the 62.5 % of cases, but the gap between the percentage of towns correctly placed and correct region identification is high (~ 34 %). Furthermore the system is realiable since the state is correctly identified in almost all the cases. Web-browser localization instead can correctly identify the user region 83 % of the times, furthermore the gap is narrower (25 %) between of the region and of the town where a precision of 58 % is reached. The tool didn't work in 10.4 % of the cases, though.

A combination of the two tools seems then the best solution since it is possible to recognize when browser geolocalization data aren't available; in this way it is possible to merge the reliability of the first method and the greater precision of the second.

After the first publication of the results of the survey a widget with an implementation of the geonotice has been implemented on it.wiki using localization from geoiplooku.wikimedia.org . The testing, although very limited, confirmed the poor utility of a system based on geoiplookup.wikimedia.org.

The implementation of a browser localization system would require to show users a notice from the browser prompting for transmssion of their location to wikipedia.org servers.

Acknowledgements

I would like to thank Elitre, for her support with the idea and helping with the translation, Salvatore Ingala for implementing the widget and other technical help, Nemo_bis for helping with the translation and all the other Wikipedians which participated at the discussions.

Appendix A

Codice for http://placepuppy.it/geo.html

<!DOCTYPE html> 
<html> 
  <head> 
    <title>Geolocation Demo</title> 
  </head> 
  <body> 
    <h1>Geolocation Demo</h1> 
    code from: <a id="greenhughes" href=http://www.greenhughes.com/content/where-are-you-find-out-geolocation-javascript>greenhughes.com</a> 
    <p>Latitude: <span id="lat">0.00</span> Longitude: <span id="lon">0.00</span></p> 
    <p><a id="gmaps_link" href="http://maps.google.co.uk/" target="_blank">View on Google Maps</a></p> 

    <script language="javascript"> 
       // show the position on the page and make a google maps link 
       function showPosition(position) { 
         var lat = position.coords.latitude; 
         var lon = position.coords.longitude; 
         document.getElementById("lat").innerHTML = lat; 
         document.getElementById("lon").innerHTML = lon; 
         var gmaps_url = "http://maps.google.co.uk/maps?f=q&source=s_q&hl=en&geocode=&q=" + lat + "+" + lon; 
         document.getElementById("gmaps_link").href = gmaps_url; 
       } 
       // report errors to user 
       function errorHandler(error) { 
         switch (error.code) { 
          case error.PERMISSION_DENIED: 
            alert("Could not get position as permission was denied."); 
            break; 
          case error.POSITION_UNAVAILABLE: 
            alert("Could not get position as this information is not available at this time."); 
            break; 
           case error.TIMEOUT: 
             alert("Attempt to get position timed out."); 
            break; 
           default: 
            alert("Sorry, an error occurred. Code: " + error.code + " Message: " + error.message); 
            break; 
           } 
       } 
       // check browser can support geolocation, if so get the current position 
       if (navigator.geolocation) { 
         navigator.geolocation.getCurrentPosition(showPosition, errorHandler); 
       } 
       else { 
         alert("Sorry, your browser does not support geolocation services."); 
       } 
    </script> 
  </body> 
</html>