Jump to content

Image server overload 2005-03

From Meta, a Wikimedia project coordination wiki

In March 2005 the main Wikimedia image server became overloaded, seeing too many read requests to keep up. As a result the images were temporarily removed from several very-frequently-used templates to decrease the load and assist with site performance. In addition to the temporary image removals the following work is happening:

  1. A new server to be used for this if necessary, and as a database server if not, is being purchased. This will have 1.2TB of disk space using six 400GB SATA drives and 8 or 16GB of RAM. The current machine has just under 1TB in RAID 5 and 1GB of RAM.
  2. Some of the image-size checking which is done by loading images off the image server is going to be moved to database fields so the image server doesn't need to read the images to get the information with each page view.
  3. One developer is writing a distributed image/blob tool which will let us spread the images around a large number of machines.
  4. A cache to help reduce the amount of image loading has been added but didn't make sufficient difference.

All projects are affected and it would be helpful, and make page loading faster, if images in templates used by more than about 500 articles were temporarily removed. Please add an entry on this page so we can easily undo this when it is no longer necessary.

Images temporarily hidden or removed

[edit]

en wikipedia

[edit]

To reduce server load, I have commented out images on the following pages. Feel free to revert any or all of these edits using the rollback feature. I promise I won't be offended. - Mark Ryan

Many templates on this list centrally located at en:Wikipedia:Template_messages/Image_namespace -- en:User:dbroadwell

Admin needed, done

[edit]

Admin needed pending

[edit]

Images which are candidates for temporary removal

[edit]

Use count and image name for the most used en images, many of which are probably used in templates and easy to temporarily remove.

en Wikipedia

[edit]

First group are believed done and showing up only because they don't get removed from the list until the articles using the templates are changed. Second group is pending.

+-------+------------------------------------+------------------------------------------------+
| cnt   | example_page_using                 | image_name                                     |
+-------+------------------------------------+------------------------------------------------+
| 12972 | GNU_Free_Documentation_License     | Heckert_GNU_white.png                          |
| 12370 | Analog_Brothers                    | David_face.png                                 |
|  9999 | Andr▒-Marie_Amp▒re                 | Wiki_letter_w.png                              |
|  5945 | Ascorbin.gif                       | Gnu-head-sm.jpeg                               |
|  3461 | Alfonso_Leng                       | Musicalnotes.png                               |
|  3327 | Karen_Johnson                      | Cc-by_white.png                                |
|  2959 | WikiEditsAug2002.JPG               | Somerights20.png                               |
|  2848 | American_football                  | Evolution-tasks.png                            |
|  2638 | La_Perouse_Strait                  | Japangeo.png                                   |
|  2274 | Aviemore                           | UK_map_icon.png                                |
|  1980 | Accessible_computing               | Gnome-system.png                               |
|  1948 | Abraham_Lincoln                    | Wikiquote.png                                  |
|  1753 | Alberta                            | Canada_flag_large.png                          |
|  1566 | USS_Indianapolis_(CA-35)           | Usnjack.png                                    |
|  1538 | Carlo_Goldoni                      | Quill_and_ink.png                              |
|  1362 | GNU                                | The_GNU_logo.png                               |
|  1353 | The_Plague                         | FreeCiv_B.great_library.png                    |
|  1321 | Amber_Road                         | Castlesketch.gif                               |
|  1319 | Central_Pacific_Railroad           | Factoryclipart.gif                             |
|  1154 | Abu_Dhabi                          | CommonsLogo.png                                |
|  1097 | Damian_Yerrick                     | Somerights20.gif                               |
|  1039 | Batman_(1966_movie)                | FilmRoll-small.gif                             |
|  1011 | Gareth_Owen                        | Ball2738-20px.png                              |
|  1011 | Coleco_Telstar                     | Gamepad_Icon.png                               |
|  1008 | Damian_Yerrick                     | Somerights.gif                                 |
|  1004 | Gothenburg_University              | Mortarboard.jpg                                |
|   996 | Banacek                            | Televisionset.jpg                              |
|   976 | Barcelonnette                      | Francestubmap.png                              |
|   912 | Klondike                           | Canadamap-stub.jpg                             |
|   902 | Altruism                           | Stop_hand.png                                  |
|   897 | The_Cider_House_Rules              | Movie-stub.jpg                                 |
+-------+------------------------------------+------------------------------------------------+
+-------+------------------------------------+------------------------------------------------+
| cnt   | example_page_using                 | image_name                                     |
+-------+------------------------------------+------------------------------------------------+
|   873 | Politics_of_Australia              | Australia_flag_large.png                       |
|   843 | Ask_and_Embla                      | Ddraig.png                                     |
|   843 | Antacid                            | Star_of_life2.png                              |
|   802 | Falknerisland.jpg                  | United_States_Geological_Survey_logo.png       |
|   801 | Abatis                             | M4_Sherman.jpg                                 |
|   785 | Anguilla                           | Uk_flag_large.png                              |
|   747 | Itaipu.jpg                         | Cc-sa_white.png                                |
|   747 | American_Samoa                     | Us_flag_large.png                              |
|   734 | Copyright                          | Copyright.png                                  |
|   731 | Total_bases                        | Soccerballplain.gif                            |
|   721 | Chocolate_mousse                   | Foodlogo.png                                   |
|   712 | Chechnya                           | Russia_flag_large.png                          |
|   703 | Andhra_Pradesh                     | India_flag_large.png                           |
|   694 | Arsenal_F.C.                       | France_flag_large.png                          |
|   684 | Casablanca                         | Africastubmap.png                              |
|   683 | Annual_plant                       | Dahlia_redoute.JPG                             |
|   672 | Calf                               | Coyote_icon.jpg                                |
|   645 | Baud                               | Telecom-icon.jpg                               |
|   644 | Harald_Tveit_Alvestrand            | Norway_flag_large.png                          |
|   644 | Long_Island_City,_Queens,_New_York | Usaoutlinemaprwb.jpg                           |
|   621 | Apus                               | Sun_SOHO_image.jpg                             |
|   613 | John_Lee_Hooker                    | Musician-stub.gif                              |
|   611 | Batman                             | Wordballoon.png                                |
|   608 | The_Ashes                          | England_flag_large.png                         |
|   603 | Carol_Kane                         | Oscar-stub.gif                                 |
|   571 | Arsenal_F.C.                       | Kit_shorts.png                                 |
|   571 | Arsenal_F.C.                       | Kit_socks.png                                  |
|   566 | Arsenal_F.C.                       | Kit_left_arm.png                               |
|   564 | Arsenal_F.C.                       | Kit_right_arm.png                              |
|   555 | Aircraft_hijacking                 | US_Department_of_Justice_Scales_Of_Justice.gif |
|   543 | Abraham_Lincoln                    | Sourceberg.jpg                                 |
|   532 | Arsenal_F.C.                       | Kit_body.png                                   |
|   531 | Anode                              | Hydrogen_orbital_3p.JPG                        |
|   517 | Pearl_Harbour,_New_Zealand         | SimpleNZtiny.png                               |
|   506 | Ajax_Amsterdam                     | South_africa_flag_large.png                    |
|   480 | Despotism                          | Society.png                                    |
|   455 | NASA                               | Logo-nasa-800px.png                            |
|   450 | Christchurch                       | Grutness.jpg                                   |
|   445 | Australia                          | Austmap.png                                    |
|   443 | Arsenal_F.C.                       | Sweden_flag_large.png                          |
|   440 | Knute_Rockne                       | Sportsman-stub.jpg                             |
|   438 | Donnybrook,_Dublin                 | Ireland_smaller.png                            |
|   435 | Arsenal_F.C.                       | Germany_flag_large.png                         |
|   434 | Chloroplast                        | Blue_morpho_butterfly_300x271.jpg              |
|   434 | Economy_of_Latvia                  | Flag_of_Europe.png                             |
|   434 | Arsenal_F.C.                       | Netherlands_flag_large.png                     |
|   431 | Conjunction_introduction           | E-to-the-i-pi.png                              |
|   423 | Karen_Johnson                      | Somerights.png                                 |
|   420 | Arsenal_F.C.                       | Italy_flag_large.png                           |
|   415 | HMS_Beagle                         | Rnensign.png                                   |
|   408 | Olympic_Games                      | Olympic-rings.png                              |
|   407 | Ajax_Amsterdam                     | Romania_flag_large.png                         |
|   403 | Arsenal_F.C.                       | Denmark_flag_large.png                         |
+-------+------------------------------------+------------------------------------------------+

de wikipedia

[edit]
+-------+-------------------------+-------------------------------------+
| cnt   | example_page_using      | image_name                          |
+-------+-------------------------+-------------------------------------+
| 11754 | GNU-Projekt             | Heckert_GNU_white.png               |
|  6853 | Amarant                 | Pfeil-rechts_2.png                  |
|  5379 | Tunisia_flag_medium.png | Gnu-head.png                        |
|  3882 | Hermann_August_Korff    | Baustelle-mittel.png                |
|  3054 | AP                      | Logo_Begriffsklärung.png            |
|  1544 | Hawaii                  | Us_flag_large.png                   |
|  1065 | Olympische_Spiele       | Olympische_Ringe.png                |
|   777 | Alexandria              | Baustelle.png                       |
+-------+-------------------------+-------------------------------------+
+-------+-------------------------+-------------------------------------+
| cnt   | example_page_using      | image_name                          |
+-------+-------------------------+-------------------------------------+
|  3191 | Bielefeld               | Wappen_fehlt.jpg   (leider subst)   | 
|   977 | Seite_bearbeiten        | Smile.png                           |
|   918 | Hera                    | Meyerskonvlexikon.jpg               |
|   735 | Vinci                   | Somerights.gif                      |
|   681 | Alexander_der_Große     | CommonsLogo.png                     |
|   674 | Pierre_de_Coubertin     | Olympische_Ringe.jpg                |
|   613 | Arsen                   | Banner_qualitaetsoffensive_quer.png |
|   533 | Andy_Warhol             | Wikiquote-trans_135px_ohne_text.png |
|   511 | Atmosphäre              | Stop_hand.png                       |
|   454 | Arsen                   | Qsicon_exzellent.png                |
|   445 | Arbeitsrecht            | Germany_flag_mittel.png             |
+-------+-------------------------+-------------------------------------+

fr wikipedia

[edit]
+------+----------------------------+----------------------------------+
| cnt  | example_page_using         | image_name                       |
+------+----------------------------+----------------------------------+
| 9041 | Aka                        | Fairytale_waring.png             |
| 6047 | Avignon                    | France_flag_large.png            |
| 3240 | Aïkido                     | WikiLettreMini.png               |
| 2009 | Avignon                    | Localite.jpg                     |
| 1936 | Egypt_Abou_Simbel4.jpg     | Gnu-head.png                     |
| 1887 | Limousin                   | Marianne_Buste.jpg               |
| 1776 | Cinéma                     | Icone_cine.jpg                   |
| 1470 | Egypt_Abou_Simbel6.jpg     | Heckert_GNU_white.png            |
| 1192 | Peuples_gaulois            | Panneau_attention_40.png         |
| 1080 | Hashar                     | Smiley.png                       |
| 1052 | Abréviations_des_monnaies  | Switzerland_flag_large.png       |
|  887 | Arsène_Lupin               | Logo_litt.jpg                    |
|  731 | Aphidolète                 | Defaut3.png                      |
|  715 | Arvo_Pärt                  | Logo_music.PNG                   |
|  614 | Caducée                    | Cartouche_mythologie_grecque.jpg |
|  588 | Amphétamine                | Médecine.png                     |
|  547 | Espèce_extirpée            | Symbole-faune.jpg                |
|  531 | Alejandro_Toledo_Manrique  | Einstein.jpg                     |
|  518 | Buffy_contre_les_vampires  | Panneau_attention_20.png         |
|  517 | Sion                       | Wappen_fehlt.jpg                 |
|  500 | Myanmar                    | Geographylogo.png                |
|  443 | Jules_César                | Hist-Mil3.png                    |
|  433 | Akseli_Gallen-Kallela      | Icone_peinture.jpg               |
|  419 | Lausanne                   | Flag-ch-vd.gif                   |
|  412 | Bâle                       | CH_Carte_manquante.png           |
|  410 | Cecil_Blount_DeMille       | Film.png                         |
+------+----------------------------+----------------------------------+

commons

[edit]
+-------+------------------------+-----------------------+
| cnt   | example_page_using     | image_name            |
+-------+------------------------+-----------------------+
| 14682 | GFDL                   | Heckert_GNU_white.png |
|  2182 | Upload_log             | Somerights20.png      |
|  1835 | Cc-by-sa               | Somerights.png        |
|  1035 | Yasser-arafat-1999.jpg | The_GNU_logo.png      |
+-------+------------------------+-----------------------+