Benefits of identifying clients based upon the "X-Forwarded-For" or similar HTTP header in addition to Client IP

Question

The X-Forwarded-For header is used by some HTTP proxies to identify the IP address of the client. The wiki page (linked above) mentions that ISPs may use this header as well.

In addition there are a variety additional headers that may be used to identify a client. Some examples include:

• HTTP_CLIENT_IP
• HTTP_X_FORWARDED_FOR can be comma delimited list of IPs
• HTTP_X_FORWARDED
• HTTP_X_CLUSTER_CLIENT_IP
• HTTP_FORWARDED_FOR
• HTTP_FORWARDED

1. Can I use any of these headers to for access control, to block or allow access my site?
   e.g. Block in an escalating manner: HTTP_Header_X first then, client IP?

2. What are the known usages of these headers? e.g. ISPs, proxy software like Squid, etc.

I was then considering that some web servers may dynamically alter their response based on the presence (or lack of) these header. Suppose a server banned users by IP address. Should it ban a user based upon the received IP or the one specified in this header? I then considered the possibility of spoofing this header as a way to work around an IP-ban that the server may have imposed.

1. What security concerns may exist around these headers, what is the appropriate way to address it?
   e.g. include that is relevant for logging, etc.

Answer 1

Any of those HTTP headers can be used for access-control purposes, however I would not rely on either the presence of those headers, or the validity of their data.

The most I would ever use them for is logging. There are no guarantees that any of those headers will be present, or even if they are, that they will contain valid data. I know there are X-Forwarded-For spoof plugins for Firefox, and probably for other browsers, too. This makes the information useful only at a statistical level - trending, pattern analysis, etc, there can definitely be some value in collecting and logging those headers. Just don't do any thing that relies on them

Any sort of reliance on HTTP headers for any kind of authentication or access control would have to be classified as worst-practice behavior.

There are much better ways to control access and authentication.

Answer 2

When hackers break into your site, they will try to go through proxies in order to hide their origin. Most of these proxies add one of these headers. By logging these headers, you are more likely to discover the origin.

For example, two years ago a hacker used open proxies in order to post stolen emails from climate researchers to skeptics websites. If those websites had tracked those headers, they might have been able to discover the identity of the hacker.

Answer 3

The X-Forwarded-For: header can be added by the client, whether it has gone through a proxy or not. A simple curl -H "X-Forwarded-For: 127.0.0.1" http://example.com will achieve this.

Proxies can be configured to remove this header in order to provide extra privacy for the end user. Many publicly available proxies advertise that they do this.

Although you could check all of the IP addresses found in the comma-delimited list and block or allow access based on any of them, the two above points mean that it is not difficult for any client to completely alter those headers. The access control would not be reliable.

Maintaining a list of all known anonymous proxy servers is a difficult and ultimately futile task.

Would you gain any advantage from doing this? Possibly. You may frustrate an attacker for a short while until he finds an anonymous proxy or spoofs his own header. This frustration may be enough to make him move on to an easier target.

Are there any downsides? A spoofed header could be quite long and could be malformed and not contain any IP addresses at all. There could also be multiple copies of the same header. Adding the code to do all this checking could introduce new bugs which could contain security flaws. It could also cause significant overhead on malformed, long or multiple headers.

Side note: If you are using Apache, mod_rpaf will take the last IP address from one of several headers and replace the internal REMOTE_HOST variable with it. This is mostly useful if your web servers are behind a reverse proxy.

This means that if you are using mod_rpaf, access control within Apache by IP address is not reliable and can be easily circumvented.

There is no guarantee that the last IP address in the list is the correct one or in fact that any of them are. The only guaranteed known value is the IP address you are actually connected to.

Answer 4

Mostly agree with Ladadada - this is user supplied data, although it's not guaranteed to be accurate, the client address at least is significantly more difficult to spoof.

But even where you can be reasonably confident of the end-point address, authenticating a request based on it's IP address is just a bad idea.

Can I use any of these headers to for access control, to block or allow access my site?

It definitely shouldn't be a criteria for allowing access - but that does not mean it's not a valid criteria for blocking access. After all someone whose intention is to abuse your site is unlikely to gain by acting in a way which prevents them accessing your site.

Indeed, if you want to be able to respond effectively to denial of service attacks, then a feedback system based on attributes of suspected attacks is the way to go - this is what fail2ban does.

Answer 5

Value of the X-Forwarded-For header field can be set at the client's side - this can also be termed as X-Forwarded-For spoofing. However, when the web request is made via a proxy server (a non-elite proxy server with low anonymity level), the proxy server modifies the X-Forwarded-For field by appending the IP address of the client (user). This will result in 2 comma separated IP addresses in the X-Forwarded-For field. Hence, the web server, if needed, may detect the use of a proxy server and most likely detect the spoofing. The following article gives an explanation of this with a Python code sample X-Forwarded-For Spoofing.