XFF project

Introduction

Wikipedia's radical openness sounds too good to be true -- how can you let anyone edit a website, without even logging in? Won't it be swamped by obsessive teenagers, trying to spray their graffiti all over it in a misguided need to feel important? Won't people just set up bots to constantly revert an article to their desired version, 24 hours per day?

In a way it is too good to be true. Wikipedia relies for its radical openness on IP-based access control -- the ability to block individuals from editing by blocking their IP address, and the difficulty for individuals to obtain more IP addresses.

There are threats to this model. For example, open proxies allow technically capable users to edit from a pool of thousands of IP addresses. Even more capable users can hack into remote computers and use them as "zombies" to attack Wikipedia. In both cases however, we can block the proxies and the zombies as they appear, with very little inconvenience to other users. Thus the attacker is forced to constantly search for new hosts to compromise, and that is the best way we have to slow down their attack.

A more serious threat is the use of HTTP proxies by Internet service providers. Poorly configured proxies can make it impossible to block one user without blocking thousands of others -- they obscure the identity of the attacker by mixing their traffic in with the traffic of other customers of the same ISP. This is where the XFF project comes in.

The X-Forwarded-For (XFF) header is a de facto standard supported by most HTTP proxy software. It appends the IP address of the client to an HTTP header which is passed on to the server. Thus, the server can determine the client IP. Because this header can be forged, Wikipedia and other Wikimedia websites will only accept XFF headers which come from sources which are known to be trusted. The aims of the XFF project are to:

• Create a catalogue of ISP proxy addresses and their XFF status
• Encourage ISPs to configure their proxies to give XFF headers

Configuration

How to configure your proxy to send XFF headers:

Squid

Put this in your squid.conf:

forwarded_for on

This is the default, so really it's only necessary to remove forwarded_for off if it is present.

Reference: Squid Configuration Manual.

Cisco Cache Engine

Switch to configure mode and run:

http append x-forwarded-for-header

Reference: Cisco Cache Engine User Guide, Version 2.1.0, Appendix A

NetApp NetCache

In the Network Settings page of the web-based administration interface, click "off" next to "Privacy". This only affects the X-Forwarded-For header, as far as we know.

Reference: ISP-Planet Cache Review Series - NetApp NetCache C720, Tuning Your Web Cache

Cacheflow Client Accelerator

In configure mode:

reveal http

http add-header x-forwarded-for

Trusted XFF list

If a proxy is on our trusted XFF list, users editing Wikipedia from behind the proxy will appear to be editing from their client IP, not from the proxy IP. Wikipedia administrators will be able to block clients individually, the whole proxy will never be blocked due to the actions of a single vandal.

A copy of the list can be found at http://wikimedia.org/trusted-xff.html

Proxies will be added to the trusted XFF list when they come to our attention if they:

• Have a reverse DNS entry that implies their dedicated use as an ISP proxy, and
• Primarily forward for public IP addresses, not RFC 1918. See /RFC 1918 for a list of proxies which forward for these addresses.

Contact tstarling at wikimedia.org for listing and delisting inquiries.

Listed ISPs

• America Online (not sending XFF headers consistently)
• Cable & Wireless (Energis proxies *.cache.pol.co.uk)
• EscapeNet
• Internode
• LTT Broadband
• NetCabo
• Netspace
• NTL
• Oulu Telephone Company
• OzEmail
• Qatar Telecom
• SAIX
• Saudi Arabia national proxy service
• Shaw cable
• SingNet
• Telefonica
• TPG Internet
• United Online
• University College Cork