Wikimedia servers
From Meta
(Redirected from Hardware)
 |
Warning: Do not rely on any information on this page being up-to-date or correct. The information at https://wikitech.leuksman.com/ is likely to be more accurate. |
Other languages: de, eo, es, fr, it, ja, ko, zh
Wikipedia and the other Wikimedia projects are run from several racks full of servers. The names of the Florida servers were primarily based on famous historical encyclopedists, while the Kennisnet and Yahoo! machines have been named after various types of plants. (See the discussion page for name suggestions.) Recently there have been so many new servers that many have just been given numerical names.
See also:
|
|
Overall system architecture
- Note: much of the following is out of date as configurations change quickly and frequently. Server roles may be more up-to-date.
- About 300 machines in Florida, 26 in Amsterdam, 23 in Yahoo!'s Korean hosting facility.
- The master database servers run MySQL and store the article metadata.
- The databases are subdivided over 3 clusters. See database clusters for more information
- Text is stored on separate database instances running on Apache servers, to avoid consuming expensive database disk space.
- Which PHP cache is used? APC or Turck MMcache?
- The Apaches are running identically-configured Apache web servers. The Apache servers accept requests from users, get data from the database if necessary, and format the requests back to the users, by running the MediaWiki software implemented in PHP with the APC PHP cache (our experience). They share their work directories by NFS, so uploads etc. should remain quite in sync.
- The Squid systems maintain large caches of pages, so that common or repeated requests don't need to touch the Apache or database servers. They serve most page requests made by visitors who aren't logged in. They are currently running at a hit-rate of approximately 75%, effectively quadrupling the capacity of the Apache servers behind them. This is particularly noticeable when a large surge of traffic arrives directed to a particular page via a web link from another site, as the caching efficiency for that page will be nearly 100%. See cache strategy for more details.
The system is designed to failover to backup configurations at both the squid and Apache levels, and backup database support is in place, but not in an automatic failover.
(Details on database replication in MySQL: [1])
Hosting
- See Wikimedia partners and hosts for more
At present all Database servers, and most Apaches and Squids, are hosted at the Florida Power Medium data center. From the start of the project until September 2004 Bomis.com paid for all bandwidth.
Kennisnet in the Netherlands is providing hosting and bandwidth for several servers since June 2005. They are installed at SARA in Amsterdam and provide service for European regions. Also the Toolserver cluster is installed here.
Yahoo! is providing servers, hosting, and bandwidth for twenty-three servers in South Korea.
Yarrow was bought by Wikimedia Deutschland, while Zedler and Hemlock were donated by Sun Microsystems to provide hosting for miscellaneous tools written by users, and are not part of the main cluster.
Cluster Nomenclature:
- pmtpa - Powermedium / Tampa, Florida, US
- knams - KennisNet / Amsterdam, Netherlands
- yaseo - Yahoo! / Seoul, South Korea
More equipment, hosting and bandwidth offers for squid cache clusters are welcome, see this page for requirements.
Orders and detailed hardware descriptions
See also: Hardware orders
2007
Quarter 1
- Hardware ordered March 2007 (~ $280,000)
2006
- Hardware purchase Sep-06 (part II not bought mid December) (~ $300,000)
- Hardware Purchase Jul-06 (~ $61,440)
- Hardware Purchase June-06 (~ $60,000)
- Hardware ordered February 06, 2006 (~ $138,000)
2005 - 129 new servers
- Hardware ordered November 15, 2005
- Hardware ordered October 18, 2005
- Hardware ordered October 6, 2005
- Hardware ordered September 14, 2005
- Hardware ordered August 30, 2005
- Donated: 23 multi-purpose servers in South Korea
- Hardware ordered May 2005: 2 db servers.
- Hardware deployed May 9, 2005: 20 apache servers (10 with 1GB, 10 with 3GB for memcached use).
- Hardware ordered January 2005: 10 new servers.
- Donated: 2 squid servers in Paris (florence, sophie)
- Donated: 8 squid servers and three multipurpose servers in Amsterdam
2004 - 39 new servers
- Hardware ordered December 2004: 5 new servers: all 3GB apache/memcached/squid type.
- Hardware ordered October 2004: 7 new servers: 2 database, 5 apache
- Hardware ordered August 2004: 10 new servers. Search database server (bacon), NFS storage server (albert), 8 3.0GHz P4 web servers (diderot, goeje, avicenna, dalembert, tingxi, alrazi, friedrich, harris), gigabit ethernet switch, 146GB SCSI drive for Suda.
- Hardware ordered May 2004: 5 new servers. Replacement for Geoffrin database server (ariel), four 2.8GHz P4 general purpose machines (maurus, rabanus, yongle) and a pair of 250GB ATA drives. Based on upgrade discussion April 2004.
- Hardware ordered January 2004: 9 new servers: 8 multipurpose machines (bart, bayle, browne, coronelli, isidore, moreri, vincent, zwinger) and 1 database server (suda).
- Donated: three for Paris squids, extra RAM for them purchased
Server list
The popularity of the Wikimedia projects necessitates the use of many servers, all but two run the GNU/Linux operating system. Below may be the currentconfiguration, Server roles is likely more up-to-date.
For IP addresses, see here.
| Name | Date | Tasks | OS | Hardware | Named after | |
| 12 | MySQL Database | |||||
| samuel | 05/2005 | S3 database slave | Fedora Core | 2U, 2 x Opteron 250 2.4GHz, 16 GB (8x2G) PC3200/DDR Registered ECC - Interleaved RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Samuel Johnson | |
| ariel | 05/2004 | S1 database slave | Ubuntu 6.10 | 2U, 2 x Opteron 248 2.2GHz, 8 GB (8x1G) PC2700 registered ECC RAM, 6x73GB 15K SCA SCSI drives (4 RAID 1+0 (146GB), 2 RAID 1 (72GB)) | Ariel Durant | |
| holbach | 10/2004 | S2a database slave | Fedora Core | 2U, 2xOpteron 242 1.6GHz, 4 GB (4x1G) PC2700 registered ECC RAM, 6x74GB WD Raptor 10K SATA RAID 0 | Baron d'Holbach | |
| ixia | 09/2005 | S2 database slave | Fedora Core | 2U, 2 x Opteron 275 (dual-core) 2.2GHz, 16 GB RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Ixia | |
| lomaria | 09/2005 | S2 database master | Fedora Core | 2U, 2 x Opteron 275 (dual-core) 2.2GHz, 16 GB RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Lomaria | |
| adler | 05/2005 | S3 database slave | Fedora x86_64 | 2U, 2 x Opteron 250 2.4GHz, 16 GB (8x2G) PC3200/DDR400 Registered ECC - Interleaved RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Mortimer Adler | |
| thistle | 09/2005 | S3 database slave | Fedora Core | >2U, 2 x Opteron 275 (dual-core) 2.2GHz, 16 GB RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Thistle | |
| webster | 10/2004 | S2 database slave | Fedora Core | 2U, 2xOpteron 242 1.6GHz, 4 GB (4x1G) PC2700 registered ECC RAM, 6x73GB Seagate Cheetah 10K.6 U320 SCSI drives RAID 0 | Noah Webster | |
| db1 .. db10 | 02/2006 | db4 S1 master; db2 .. db3, db6, db7, db9 S1 slave; db8 S2 slave; db5 S3 master, db1 S3 slave |
Fedora Core | 2U, 2 x Opteron 248 (dual-core) 2.2GHz, 16 GB RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | - | |
| 14 | Miscellaneous | |||||
| albert | 08/2004 | NFS storage server | SUSE Lin. 9.1 | 2U, 2xOpteron 242 1.6GHz, 1 GB (2x512M) PC2700 registered ECC RAM, 6x250GB SATA (RAID 5) (details) | Albertus Magnus | |
| amane | 11/2005 | Image server | Fedora Core | 3U, 2x Xeon 3.4GHz, 8GB RAM, 16 x 400GB SATA disk | Nishi Amane | |
| zwinger | 01/2004 | SSH, NTP, ganglia | Red Hat Lin. 9 | 1U, 1 x Pentium 4 2.6 GHz, 1 GB (cap?) unbuf DDR400 non-ECC RAM, 250 GB SATA (details) | Theodor Zwinger | |
| bacon | 08/2004 | Thumbnail server | Fedora Core | 2U, 2xOpteron 242 1.6GHz, 4 GB (4x1G) PC2700 registered ECC RAM, 6x200GB SATA RAID 10 | Francis Bacon | |
| suda | 01/2004 | NFS storage server | Fedora Core | 2U, 2 x Opteron 246 2.0GHz, 4 GB (4x1G) PC2700 registered ECC RAM, 4x146GB SCSI RAID 1+0 (292GB) and 2x36GB SCA SCSI RAID 1 (36GB) | Suda | |
| coronelli | 01/2004 | Search server (s2/3 non main namespace) | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 3 GB (2x1G? +?) unbuf non-ECC DDR400 RAM, 80 GB SATA | Vincenzo Coronelli | |
| maurus | 05/2004 | Search server (s1 non main namespace) | Fedora Core | 1U, 1 x Pentium 4 2.8 GHz, 4 GB (4x1G) unbuf DDR 400 ECC RAM, 80 GB SATA | Rabanus Maurus | |
| avicenna | 08/2004 | LVS load balancer (text squid) | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 RAM, 200 GB SATA | Avicenna | |
| alrazi | 08/2004 | LVS load balancer (upload) | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 RAM, 200 GB SATA | Al-Razi | |
| isidore | 01/2004 | down | FreeBSD5.3-REL | 1U, 1 x Pentium 4 2.6 GHz, 1 GB (cap?) unbuf DDR400 non-ECC RAM, 80 GB SATA | Isidore of Seville | |
| khaldun | 10/2004 | APT repository, Ubuntu mirror, Squid gateway | Ubuntu 6.10 | 1U, 1x Pentium 4 3.0 GHz, 1 GB RAM, 2x250 GB SATA RAID 0 | Ibn Khaldun | |
| browne | 01/2004 | IRC, sqltunnel, udpmcast | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 4 GB (4x1G) unbuf non-ECC DDR400 RAM, 80 GB SATA | Thomas Browne | |
| 18 | Squid | |||||
| srv5 .. srv10 | 01/2005 | 6x Squid | Fedora Core / Ubuntu | 1U, 1x Pentium 4 3.2GHz, 3GB RAM, 2x 36GB SATA | - | |
| sq1 .. sq10 | 11/2005 | 10x Upload/Image Squids | Ubuntu 6.10 | 1U, 2xOpteron 248 (2.2Ghz), 4x1GB DDR ECC RAM, 4x 36GB 10k SATA | - | |
| bayle | 01/2004 | Apache, 1120MB memcached, switched between Squid and Apache as needed | Ubuntu 6.10 | 1U, 1 x Pentium 4 2.6 GHz, 2 GB (2x1G) unbuf DDR400 non-ECC RAM, 80 GB SATA | Pierre Bayle | |
| will | 05/2004 | Squid | Fedora Core | 1U, 1 x Pentium 4 2.8 GHz, 4 GB RAM, 2 x 200GB SATA in RAID 1 (details) | Will Durant | |
| 158 | Apache | |||||
| srv0 | 10/2004 | Apache, DB backup slave | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 1 GB RAM, 2x 250GB SATA RAID 0 | Averroes | |
| srv1 .. srv4 | 01/2005 | 4x Apache | Fedora Core | 1U, 1x Pentium 4 3.2GHz, 1GB RAM, 1x 200GB SATA | - | |
| srv11 .. srv20 | 05/2005 | 10x Apache | Fedora Core | 1U, 1x Pentium 4 3.4GHz, 1GB RAM, 1x 80GB SATA | - | |
| srv21 .. srv27 | 05/2005 | 7x Apache | Fedora Core | 1U, 1x Pentium 4 3.4GHz, 3GB RAM, 1x 80GB SATA | - | |
| srv28 .. srv30 | 05/2005 | 4x Apache, external storage | Fedora Core | 1U, 1x Pentium 4 3.4GHz, 3GB RAM, 1x 80GB SATA | - | |
| srv31 .. srv50 | 09/2005 | 10x Apache | Fedora Core | 1U, 2x Opteron 248 (2.2GHz), 2x1GB DDR ECC RAM, 1x 250GB 7.2k SATA | - | |
| srv51 .. srv70 | 10/2005 | 20x Apache | Fedora Core | 1U, 2xOpteron 248 (2.2Ghz), 4x1GB DDR ECC RAM, 1x 7.2k 250GB SATA | - | |
| srv71 .. srv80 | 11/2005 | 10x Apache | Fedora Core | 1U, 2xOpteron 248 (2.2Ghz), 4x1GB DDR ECC RAM, 1x 250GB 7.2k SATA | - | |
| srv81 .. srv120 | 02/2006 | 40x Apache | Fedora Core | 1U, 2xOpteron 265 (1.8Ghz), 4x1GB DDR ECC RAM, 2x 250GB 7.2k SATA | - | |
| anthony | 12/2004 | Apache/ Memcached | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 2x1 GB + 2x512 MB DDRII-533 RAM, 1x200 GB SATA RAID 0 | Anthony Winkler Prins | |
| bart | 01/2004 | Apache | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 4 GB (4x1G) unbuf DDR400 non-ECC RAM, 80 GB SATA | Bartholomeus de Glanvilla | |
| benet | 12/2004 | Apache/ Memcached/ Squid. Database slave, test production use. | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 2x1 GB + 2x512 MB DDRII-533 RAM, 2x250 GB SATA RAID 0 | William Rose Benet | |
| biruni | 12/2004 | Apache/ Memcached/ Squid. | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 2x1 GB + 2x512 MB DDRII-533 RAM, 1x200 GB SATA | Al-Biruni | |
| dalembert | 08/2004 | LVS | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 RAM, 200 GB SATA | Jean le Rond d'Alembert | |
| diderot, friedrich, goeje, harris | 08/2004 | 4x Apache. | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 ECC RAM, 200 GB SATA | Denis Diderot, Friedrich Arnold Brockhaus, Michael Jan de Goeje, John Harris | |
| hypatia | 10/2004 | Apache, IRC bots | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 1 GB RAM, 1x 200GB SATA | Hypatia of Alexandria | |
| humboldt, kluge | 10/2004 | 2x Apache | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 1 GB RAM, 1x 200GB SATA | Alexander von Humboldt, Friedrich Kluge | |
| moreri | 01/2004 | Apache, 280MB memcached | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 1 GB (cap?) unbuf DDR400 non-ECC RAM, 80 GB SATA | Louis Moréri | |
| rabanus | 05/2004 | Apache, DHCP boot server | Fedora Core | 1U, 1 x Pentium 4 2.8 GHz, 4 GB (4x1G) unbuf DDR 400 ECC RAM, 80 GB SATA | Rabanus Maurus | |
| vincent | 01/2004 | Apache | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 1 GB (cap?) unbuf DDR400 non-ECC RAM, 80 GB SATA | Vincent of Beauvais | |
| rose, smellie | 12/2004 | 2x Apache / Memcached. | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 2x1 GB + 2x512 MB DDRII-533 RAM, 1x200 GB SATA | William Rose Benet, William Smellie | |
| tingxi | 08/2004 | Apache, IRC bots. Currently malfunctioning and in need of developer attention | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 RAM, 200 GB SATA | Jiang Tingxi | |
| yongle | 05/2004 | Apache, 1960MB memcached | Fedora Core | 1U, 1 x Pentium 4 2.8 GHz, 4 GB (4x1G) unbuf DDR 400 ECC RAM, 80 GB SATA | Emperor Yongle | |
| 12 | Kennisnet cluster | |||||
| pascal | 5/2005 | WWW, DNS, NFS | Fedora Core | Sun V40z, 2 x 1.8GHz Opteron, 2GB RAM, 2x 70GB SCSI | Blaise Pascal | |
| vandale | 5/2005 | Toolserver user databases | Solaris 10 11/06 | Sun V40z, 2 x 1.8Ghz Opteron, 8GB RAM, 6x 146GB SCSI | Johan Hendrik van Dale | |
| clematis | 5/2005 | Text database backup, toolserver infrastructure | Solaris 10 11/06 | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| fuchsia | 5/2005 | Unused | Solaris 10 11/06 | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | ||
| hawtorn | 5/2005 | Ubuntu 6.10 build host | Ubuntu 6.10 | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| lily | 5/2005 | Mailing lists server | Ubuntu 7.04 | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| iris | 5/2005 | Unused | Ubuntu 6.10 | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| mayflower | 5/2005 | svn.wikimedia.org | Ubuntu 7.04 | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| mint, sage | 5/2005 | idle | Ubuntu 7.04 | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| ragweed | 5/2005 | Backup server | Ubuntu 6.10 | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| zedler | 8/2005 | Toolserver | Solaris 10 | Sun V40z, 2 x 2.2GHz Opteron, 8GB RAM, 6x 146GB SCSI | Johann Heinrich Zedler | |
| hemlock | 4/2006 | Toolserver | Debian Linux | Sun X4200, 2 x Opteron 254 (2.8GHz), 8GB RAM, 2x73 GB SAS | - | |
| 23 | Korea Yahoo! cluster | |||||
| amaryllis | 9/2005 | NFS | Fedora Core | HP DL385, 2x Opteron 250 (2.4GHz), 8GB RAM, 6x 146GB disk | w:amaryllis | |
| dryas | 9/2005 | down | Fedora Core | HP DL385, 2x Opteron 250 (2.4GHz), 8GB RAM, 6x 146GB disk | dryas | |
| henbane | 9/2005 | logging | Fedora Core | HP DL385, 2x Opteron 250 (2.4GHz), 8GB RAM, 6x 146GB disk | henbane | |
| yf1000 ... yf1004 | 9/2005 | Text squids | Ubuntu 6.10 | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| yf1005 ... yf1009 | 9/2005 | Upload squids | Fedora Core / Ubuntu | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| yf1010 ... yf1014, yf1017 | 9/2005 | down | Fedora Core / Ubuntu | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| yf1015 | 9/2005 | log/stats experiments | Fedora Core / Ubuntu | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| yf1015 | 9/2005 | wikimaps | Fedora Core / Ubuntu | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| yf1018 | 9/2005 | LVS load balancer | Fedora Core | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| yf1019 | 9/2005 | DNS | Fedora Core | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| 240 | Total | |||||
Old servers
The old web servers, which are currently not in service, were also named after historical encyclopedists:
- pliny (Pliny the Elder)
- geoffrin (Marie Thérèse Rodet Geoffrin)
- larousse (Pierre Larousse)
Donations
While Wikipedia is free in both the "free speech" and "no charge" senses of the word, running the web site does cost money. You can help with purchasing new server hardware by donating to the non-profit Wikimedia Foundation: http://wikimediafoundation.org/wiki/Donate
Status and problems
You can check one of the following sites if you want to know if the Wikimedia servers are overloaded, or if you just want to see how they are doing.
If you are seeing errors in real time, visit #wikimedia-tech on irc.freenode.net. Check the topic to see if someone is already looking into the problem you are having. If not, please report your problem to the channel. It would be helpful if you could report specific symptoms, including the exact text of any error messages, what you were doing right before the error, and what server(s) are generating the error, if you can tell. The #wikipedia channel may be better populated and the topic there may contain more updated information about the status of the problem. (But this channel is for general conversation about Wikipedia.)
If you're wondering if it's only you experiencing this problem, you can check the following sites. Unfortunately, the Wikimedia administrators do not monitor these sites for problems.
If you are getting a "connection refused" error, that is a squid problem. Determine which IP address you are trying to connect to, and ask someone to look at that host.
See also
 |
Wikimedia Commons has media related to: Category:Wikimedia servers |
More hardware info
- Recent hardware orders, accounting summary
- Wikimedia partners and hosts
- Technical FAQ - How about the hardware?
Admin logs
- Server admin log - Documents server changes (especially software changes)
Offsite traffic pages
Long-term planning
Useful information about other sites
- Evolution of LiveJournal systems:
- 04/2004 MySQLCon 2004 PDF/SXI
- 07/2004 OSCON 2004 PDF/SXI
- 11/2004 LISA 2004 PDF/SXI
- 04/2005 MySQLCon 2005 PDF/PPT/SXI
- journals to watch for system details: Brad (Fitzpatrick) lj_backend lj_maintenance
- Google cluster architecture (PDF)
- MySQL User's Conference 2004 blog highlights
